Jobsub ID 296189.0@dunegpschedd02.fnal.gov
| Jobsub ID | 296189.0@dunegpschedd02.fnal.gov |
| Workflow ID | 12455 |
| Stage ID | 1 |
| User name | pmelas@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 8388608000 (8000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2026-01-27 21:01:50 |
| Site | US_FNAL-FermiGrid |
| Entry | FNAL_GPGrid_ce04_mcore_op_duneonly |
| Last heartbeat | 2026-01-28 02:20:02 |
| From worker node | Hostname | dunegli-8449287-0-fnpc18007.fnal.gov |
| cpuinfo | AMD EPYC 7551 32-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 10485760000 (10000 MiB) |
| Wall seconds limit | 172800 (48 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2026-01-27 21:03:18 |
| Input files | monte-carlo-012455-000001
|
| Jobscript | Exit code | 0 |
| Real time | 5h (18983s) |
| CPU time | 4h (16162s = 85%) |
| Max RSS bytes | 4217012224 (4021 MiB) |
| Outputting started | 2026-01-28 02:19:42 |
| Output files | https://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12455/1/001/reco_stage1.root
|
| Finished | 2026-01-28 02:20:02 |
| Saved logs | justin-logs:296189.0-dunegpschedd02.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
vice:prepare: Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare: Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 130578 traces, 0 summary
Retagger: tagging trace set: gauss with 103392 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 103392 traces tagged "gauss"
FrameSaver: q=3.70492e+06 n=1895326 tag=gauss
wclsFrameSaver: saving 130578 traces tagged "wiener"
FrameSaver: q=3.89167e+06 n=1697769 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 56033 XUs and 58751 XVs -> 997 XUVs
C:0 T:1 110071 XUs and 106910 XVs -> 8126 XUVs
C:0 T:2 113979 XUs and 230409 XVs -> 14718 XUVs
C:0 T:3 45829 XUs and 63823 XVs -> 447 XUVs
C:0 T:4 8321 XUs and 8212 XVs -> 44 XUVs
C:0 T:5 26396 XUs and 18291 XVs -> 1904 XUVs
C:0 T:6 75751 XUs and 109153 XVs -> 7237 XUVs
C:0 T:7 26261 XUs and 33545 XVs -> 313 XUVs
C:0 T:8 8324 XUs and 5860 XVs -> 43 XUVs
C:0 T:9 47782 XUs and 45329 XVs -> 2938 XUVs
C:0 T:10 21544 XUs and 20173 XVs -> 3921 XUVs
C:0 T:11 23 XUs and 16 XVs -> 5 XUVs
40693 XUVs total
12388 collection wire objects
36978 potential space points
Neighbour search...
4588158 tests to find 2184790 neighbours
Iterating with no regularization...
Begin: 8.0789e+07
0 7.07604e+07
1 7.00526e+07
2 6.99445e+07
3 6.99121e+07
Now with regularization...
Begin: 5.84263e+07
0 5.83896e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 412004795
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 412004795
fLifetime: 30000 [us]
28-Jan-2026 02:19:36 UTC Closed output file "reco_stage1.root"
28-Jan-2026 02:19:36 UTC Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob: # events skipped: 0
====================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
====================================================================================================================================
Full event 442.75 513.939 617.401 516.234 49.1956 10
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.0020577 0.0038683 0.0056712 0.00384235 0.00137541 10
reco:ophit:OpHitFinder 0.293229 0.323233 0.360063 0.321753 0.0241717 10
reco:opflash:OpFlashFinder 0.155879 0.194524 0.238092 0.194008 0.0241229 10
reco:opslicer:OpSlicer 0.760924 0.854131 1.00577 0.850628 0.0798264 10
reco:caldata:DataPrepModule 5.86703 6.43828 7.00243 6.37165 0.31803 10
reco:wclsdatasp:WireCellToolkit 39.1962 43.7137 47.6482 44.2233 2.77853 10
reco:gaushit:GausHitFinder 1.41299 1.61171 1.98576 1.58411 0.161823 10
reco:nhitsfilter:NumberOfHitsFilter 0.000610083 0.000747347 0.00116566 0.000710032 0.00015386 10
reco:reco3d:SpacePointSolver 15.4388 28.6826 97.7772 20.9191 23.4463 10
reco:hitpdune:DisambigFromSpacePoints 1.97933 2.44614 3.88562 2.26253 0.546673 10
reco:pandora:StandardPandora 41.7439 62.0349 79.8858 65.1671 12.538 10
reco:pandoraWriter:StandardPandora 0.565836 1.56803 2.44727 1.66419 0.750575 10
reco:pandoraTrack:LArPandoraTrackCreation 5.80116 7.75608 11.18 7.38421 1.78229 10
reco:pandoraShower:LArPandoraShowerCreation 6.9694 10.2519 14.7169 10.3782 2.16125 10
reco:pandoracalo:Calorimetry 3.08406 4.2901 6.07555 4.11367 0.992438 10
reco:pandoracalonosce:Calorimetry 3.05361 4.10736 5.56747 4.06528 0.892124 10
reco:pandorapid:Chi2ParticleID 0.00658827 0.0079967 0.0112457 0.0075331 0.00151575 10
reco:pandoracalinoxyzt:CalibrationdEdXPDSP 0.0557129 0.0652069 0.0786836 0.0633451 0.00685682 10
reco:pandoracalipidnoxyzt:Chi2ParticleID 0.00518567 0.00733154 0.01091 0.0070463 0.0014902 10
reco:pandoraShowercalo:ShowerCalorimetry 7.03499 10.4042 14.8761 10.4643 2.19884 10
reco:pandoraShowercalonosce:ShowerCalorimetry 6.6961 10.11 15.0819 10.2489 2.31977 10
reco:emtrkmichelid:EmTrackMichelId 196.946 227.816 277.101 221.119 21.8063 10
reco:crttag:SingleCRTMatchingProducer 0.0171997 0.0245718 0.0373378 0.0233895 0.00512242 10
reco:crtreco:TwoCRTMatchingProducer 0.0130384 0.0156131 0.0191096 0.0151987 0.0019098 10
reco:anodepiercerst0:T0RecoAnodePiercers 0.877436 1.1517 1.65233 1.06411 0.232561 10
reco:pandora2Track:LArPandoraTrackCreation 19.4085 27.0818 36.6482 26.8291 4.88317 10
reco:pandora2calo:Calorimetry 9.70192 13.404 17.7265 13.691 2.2618 10
reco:pandora2calonosce:Calorimetry 9.5712 12.6997 17.0729 12.6743 2.10427 10
reco:pandora2pid:Chi2ParticleID 0.0156299 0.0215002 0.0275553 0.0203798 0.00362967 10
reco:pandora2calinoxyzt:CalibrationdEdXPDSP 0.0778871 0.0894512 0.102273 0.0885785 0.00712834 10
reco:pandora2calipidnoxyzt:Chi2ParticleID 0.0172053 0.0210554 0.0276628 0.0201433 0.00306358 10
reco:pandora2Shower:LArPandoraShowerCreation 7.32884 9.50465 12.0496 9.53481 1.35218 10
reco:pandora2Showercalo:ShowerCalorimetry 7.82149 10.6765 14.5404 10.7097 1.71414 10
reco:pandora2Showercalonosce:ShowerCalorimetry 6.56799 9.50516 12.5 9.56573 1.49229 10
[art]:TriggerResults:TriggerResultInserter 5.831e-05 8.02619e-05 0.000180672 6.7703e-05 3.46133e-05 10
end_path:out1:RootOutput 1.034e-05 1.28292e-05 1.9728e-05 1.20525e-05 2.7187e-06 10
end_path:out1:RootOutput(write) 3.66086 6.87372 11.3936 7.76282 2.57898 10
====================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5033.78 MB
Peak resident set size usage (VmHWM): 4217.01 MB
Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor: Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor: digitReader: 0.08 sec/event
ToolBasedRawDigitPrepService:dtor: adcSampleFiller: 0.24 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleAdcToKe: 0.06 sec/event
ToolBasedRawDigitPrepService:dtor: pdspTailPedRemovalZKe: 4.18 sec/event
ToolBasedRawDigitPrepService:dtor: adcKeepAllSignalFinder: 0.13 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleKeToAdc: 0.06 sec/event
ToolBasedRawDigitPrepService:dtor: pdsp_RemoveBadChannels: 0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10243464
-rw-r--r-- 1 dunegli fnalgrid 2417238991 Jan 28 00:43 detsim_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 2297579283 Jan 28 00:24 detsim_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 2136589204 Jan 28 02:19 reco_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 1812366118 Jan 27 21:42 g4_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 1741830324 Jan 27 21:14 g4_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 83069600 Jan 28 02:14 Pandora_Events.pndr
-rw-r--r-- 1 dunegli fnalgrid 243145 Jan 28 02:19 jobscript.log
-rw-r--r-- 1 dunegli fnalgrid 217088 Jan 28 02:19 mem.db
-rw-r--r-- 1 dunegli fnalgrid 62376 Jan 28 02:19 Pandora_Geometry.xml
-rw-r--r-- 1 dunegli fnalgrid 36864 Jan 28 02:19 time.db
-rw-r--r-- 1 dunegli fnalgrid 519 Jan 28 00:43 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid 519 Jan 27 21:42 g4_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid 519 Jan 28 02:19 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dunegli fnalgrid 52 Jan 27 21:03 all-input-dids.txt
-rw-r--r-- 1 dunegli fnalgrid 7 Jan 28 02:19 justin-processed-pfns.txt
-rw-r--r-- 1 dunegli fnalgrid 0 Jan 28 00:56 ProtoDUNE.log
-rw-r--r-- 1 dunegli fnalgrid 0 Jan 28 00:43 debugprod.log
