justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 252557.0@dunegpschedd01.fnal.gov

Jobsub ID252557.0@dunegpschedd01.fnal.gov
Workflow ID10245
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-16 04:28:33
SiteUS_FNAL-T1
EntryCMSHTPC_T1_US_FNAL_condce_opp1_whole
Last heartbeat2025-11-16 06:24:44
From worker nodeHostnamedunegli-47944-0-cmswn4057.fnal.gov
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-16 04:29:27
Input filesmonte-carlo-010245-000001
JobscriptExit code0
Real time1h (6902s)
CPU time1h (6851s = 99%)
Max RSS bytes4078981120 (3890 MiB)
Outputting started2025-11-16 06:24:29
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/10245/1/001/reco_stage1.root
Finished2025-11-16 06:24:44
Saved logsjustin-logs:252557.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

KeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 161691 traces, 0 summary
Retagger: tagging trace set: gauss with 121976 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 121976 traces tagged "gauss"
FrameSaver: q=4.40171e+06 n=2151233 tag=gauss
wclsFrameSaver: saving 161691 traces tagged "wiener"
FrameSaver: q=4.66298e+06 n=1900537 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 26928 XUs and 28861 XVs -> 373 XUVs
C:0 T:1 52740 XUs and 52016 XVs -> 5376 XUVs
C:0 T:2 139558 XUs and 195643 XVs -> 20030 XUVs
C:0 T:3 56943 XUs and 54000 XVs -> 345 XUVs
C:0 T:4 14157 XUs and 16221 XVs -> 169 XUVs
C:0 T:5 66805 XUs and 65712 XVs -> 5967 XUVs
C:0 T:6 126012 XUs and 175591 XVs -> 11558 XUVs
C:0 T:7 36589 XUs and 50929 XVs -> 725 XUVs
C:0 T:8 11725 XUs and 11524 XVs -> 115 XUVs
C:0 T:9 80239 XUs and 97486 XVs -> 6351 XUVs
C:0 T:10 63084 XUs and 47749 XVs -> 9786 XUVs
C:0 T:11 232 XUs and 245 XVs -> 31 XUVs
60826 XUVs total
14530 collection wire objects
56113 potential space points
Neighbour search...
9651549 tests to find 4110318 neighbours
Iterating with no regularization...
Begin: 9.85005e+07
0 7.49582e+07
1 7.30546e+07
2 7.27542e+07
3 7.26669e+07
4 7.26341e+07
Now with regularization...
Begin: 5.89008e+07
0 5.88063e+07
1 5.87496e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410542562
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410542562
fLifetime: 30000 [us]
16-Nov-2025 06:24:29 UTC  Closed output file "reco_stage1.root"
16-Nov-2025 06:24:29 UTC  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          137.703       199.549       254.202       197.053       32.0439        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000929696   0.00243088    0.00344235    0.00260504    0.00086114       10     
reco:ophit:OpHitFinder                             0.156504      0.194308      0.212977       0.19835      0.0160264       10     
reco:opflash:OpFlashFinder                         0.0719503     0.110492      0.141412      0.112992      0.0178561       10     
reco:opslicer:OpSlicer                             0.324418       0.42109      0.505674      0.414633      0.0538839       10     
reco:caldata:DataPrepModule                         3.31975       3.41505       3.6201        3.39451      0.0769176       10     
reco:wclsdatasp:WireCellToolkit                     22.6671       23.0328       23.5395       23.0497      0.272354        10     
reco:gaushit:GausHitFinder                         0.562575      0.697792      0.856749      0.681689      0.079644        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000228902   0.000355618   0.000519782   0.000324012   9.47424e-05      10     
reco:reco3d:SpacePointSolver                        5.41786       10.0604       15.7554       9.64418       4.01517        10     
reco:hitpdune:DisambigFromSpacePoints              0.603232      0.902952       1.28077      0.810677      0.199828        10     
reco:pandora:StandardPandora                        8.52305       19.2547       31.6299       18.1635       6.93346        10     
reco:pandoraWriter:StandardPandora                 0.166089      0.246432      0.307311       0.23919      0.0396212       10     
reco:pandoraTrack:LArPandoraTrackCreation          0.939061       2.00883       2.93787       1.89936      0.543935        10     
reco:pandoraShower:LArPandoraShowerCreation        0.886182       2.06299       3.26253       2.01833      0.578722        10     
reco:pandoracalo:Calorimetry                       0.598376       1.23867       1.75925       1.18947      0.312085        10     
reco:pandoracalonosce:Calorimetry                  0.533199       1.14803       1.64745       1.10323      0.297927        10     
reco:pandorapid:Chi2ParticleID                    0.00220207    0.00327956     0.0046337    0.00314118    0.000655634      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0237021     0.0301499     0.0353067     0.0293005    0.00412376       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00213433     0.0029391    0.00356871    0.00298739    0.000408183      10     
reco:pandoraShowercalo:ShowerCalorimetry           0.923264       2.38902       3.68635       2.3667       0.693007        10     
reco:pandoraShowercalonosce:ShowerCalorimetry      0.892549       2.32935       3.62267       2.31276      0.680078        10     
reco:emtrkmichelid:EmTrackMichelId                  80.1389       106.311       130.696       104.142       13.4249        10     
reco:crttag:SingleCRTMatchingProducer             0.00458783    0.00728741     0.0109164    0.00709448    0.00179382       10     
reco:crtreco:TwoCRTMatchingProducer               0.00355378      0.00467     0.00576912    0.00476687    0.000715668      10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.484832      0.585034      0.724451       0.57426      0.0743347       10     
reco:pandora2Track:LArPandoraTrackCreation          2.69056       6.16443       9.49389       5.99039       1.62726        10     
reco:pandora2calo:Calorimetry                       1.45569       3.45468        5.16         3.47495      0.905032        10     
reco:pandora2calonosce:Calorimetry                  1.36475       3.30264       4.9493        3.33092       0.87541        10     
reco:pandora2pid:Chi2ParticleID                   0.00471673     0.0073509    0.00944466     0.0074017    0.00111949       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0273328     0.0373416     0.0453159     0.0365281    0.00537862       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00463885    0.00718982    0.00904016    0.00723703    0.00106469       10     
reco:pandora2Shower:LArPandoraShowerCreation        1.0646        2.3344        3.49389       2.28745      0.587198        10     
reco:pandora2Showercalo:ShowerCalorimetry           1.24195       3.07227       4.58688       3.08017      0.821874        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      1.16372       2.86477       4.25393       2.87562      0.766439        10     
[art]:TriggerResults:TriggerResultInserter        2.3946e-05    4.48569e-05   0.000165012   3.25215e-05   4.02781e-05      10     
end_path:out1:RootOutput                           9.007e-06    1.16469e-05   1.8325e-05    1.10555e-05   2.38145e-06      10     
end_path:out1:RootOutput(write)                     1.36359       1.79279       2.25185       1.76616      0.235604        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4898.43 MB
  Peak resident set size usage (VmHWM): 4078.98 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.15 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.15 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9729412
-rw-r--r-- 1 dunegli fnalgrid 2300131971 Nov 16 05:50 detsim_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 2187090549 Nov 16 05:49 detsim_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 1981252549 Nov 16 06:24 reco_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 1735538757 Nov 16 04:39 g4_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 1684136389 Nov 16 04:32 g4_stage1.root
-rw-r--r-- 1 dunegli fnalgrid   73957200 Nov 16 06:21 Pandora_Events.pndr
-rw-r--r-- 1 dunegli fnalgrid     242886 Nov 16 06:24 jobscript.log
-rw-r--r-- 1 dunegli fnalgrid     217088 Nov 16 06:24 mem.db
-rw-r--r-- 1 dunegli fnalgrid      62376 Nov 16 06:24 Pandora_Geometry.xml
-rw-r--r-- 1 dunegli fnalgrid      36864 Nov 16 06:24 time.db
-rw-r--r-- 1 dunegli fnalgrid        519 Nov 16 05:50 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid        519 Nov 16 04:39 g4_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid        519 Nov 16 06:24 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dunegli fnalgrid         52 Nov 16 04:29 all-input-dids.txt
-rw-r--r-- 1 dunegli fnalgrid          7 Nov 16 06:24 justin-processed-pfns.txt
-rw-r--r-- 1 dunegli fnalgrid          0 Nov 16 05:53 ProtoDUNE.log
-rw-r--r-- 1 dunegli fnalgrid          0 Nov 16 05:50 debugprod.log
justIN time: 2026-02-09 10:52:27 UTC       justIN version: 01.06.00