Jobsub ID 288878.1@dunegpschedd02.fnal.gov
| Jobsub ID | 288878.1@dunegpschedd02.fnal.gov |
| Workflow ID | 12111 |
| Stage ID | 1 |
| User name | pmelas@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 8388608000 (8000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2026-01-20 15:48:50 |
| Site | US_FNAL-FermiGrid |
| Entry | FNAL_GPGrid_ce03_mcore_op_duneonly |
| Last heartbeat | 2026-01-20 18:10:19 |
| From worker node | Hostname | dunegli-8229250-0-fnpc23019.fnal.gov |
| cpuinfo | AMD EPYC 7543 32-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 10485760000 (10000 MiB) |
| Wall seconds limit | 172800 (48 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2026-01-20 15:49:15 |
| Input files | monte-carlo-012111-000001
|
| Jobscript | Exit code | 0 |
| Real time | 2h (8450s) |
| CPU time | 2h (8348s = 98%) |
| Max RSS bytes | 4508704768 (4299 MiB) |
| Outputting started | 2026-01-20 18:10:06 |
| Output files | https://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12111/1/001/reco_stage1.root
|
| Finished | 2026-01-20 18:10:19 |
| Saved logs | justin-logs:288878.1-dunegpschedd02.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare: Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 116099 traces, 0 summary
Retagger: tagging trace set: gauss with 92596 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 92596 traces tagged "gauss"
FrameSaver: q=3.45239e+06 n=1671642 tag=gauss
wclsFrameSaver: saving 116099 traces tagged "wiener"
FrameSaver: q=3.67154e+06 n=1504654 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 28606 XUs and 26542 XVs -> 433 XUVs
C:0 T:1 49125 XUs and 46302 XVs -> 2590 XUVs
C:0 T:2 174963 XUs and 179126 XVs -> 10134 XUVs
C:0 T:3 61439 XUs and 49433 XVs -> 1174 XUVs
C:0 T:4 15321 XUs and 13149 XVs -> 93 XUVs
C:0 T:5 61228 XUs and 46314 XVs -> 4857 XUVs
C:0 T:6 54013 XUs and 51831 XVs -> 2058 XUVs
C:0 T:7 25494 XUs and 23136 XVs -> 171 XUVs
C:0 T:8 10932 XUs and 8736 XVs -> 154 XUVs
C:0 T:9 74864 XUs and 82705 XVs -> 6112 XUVs
C:0 T:10 12484 XUs and 8243 XVs -> 1642 XUVs
C:0 T:11 388 XUs and 294 XVs -> 44 XUVs
29462 XUVs total
10869 collection wire objects
25452 potential space points
Neighbour search...
2003702 tests to find 686102 neighbours
Iterating with no regularization...
Begin: 1.40669e+08
0 1.06712e+08
1 1.0391e+08
2 1.03647e+08
3 1.03593e+08
Now with regularization...
Begin: 8.87485e+07
0 8.8632e+07
1 8.86029e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411853240
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411853240
fLifetime: 30000 [us]
20-Jan-2026 18:10:04 UTC Closed output file "reco_stage1.root"
20-Jan-2026 18:10:04 UTC Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob: # events skipped: 0
====================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
====================================================================================================================================
Full event 202.723 255.393 342.361 254.91 43.0904 10
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00082698 0.00316915 0.00491048 0.00324443 0.00131296 10
reco:ophit:OpHitFinder 0.1981 0.227312 0.264053 0.224516 0.0192545 10
reco:opflash:OpFlashFinder 0.130037 0.154639 0.178026 0.14928 0.0157495 10
reco:opslicer:OpSlicer 0.333464 0.483337 0.598531 0.47759 0.0850888 10
reco:caldata:DataPrepModule 3.69569 3.78464 4.05754 3.76542 0.0979534 10
reco:wclsdatasp:WireCellToolkit 26.0023 26.8486 28.0145 26.7121 0.634901 10
reco:gaushit:GausHitFinder 0.749074 0.919736 1.27902 0.862237 0.161055 10
reco:nhitsfilter:NumberOfHitsFilter 0.000430588 0.000669482 0.00110649 0.000599719 0.00022549 10
reco:reco3d:SpacePointSolver 6.06192 12.5188 23.2704 10.7432 5.04367 10
reco:hitpdune:DisambigFromSpacePoints 0.924813 1.26932 1.93043 1.2209 0.29481 10
reco:pandora:StandardPandora 15.5365 26.8015 45.9148 25.6374 9.87186 10
reco:pandoraWriter:StandardPandora 0.272285 0.355204 0.456924 0.340861 0.0588835 10
reco:pandoraTrack:LArPandoraTrackCreation 2.20157 3.01367 4.27631 3.04747 0.605177 10
reco:pandoraShower:LArPandoraShowerCreation 1.94965 3.19687 4.7435 3.069 0.945005 10
reco:pandoracalo:Calorimetry 1.30118 1.78415 2.50725 1.78141 0.361649 10
reco:pandoracalonosce:Calorimetry 1.21126 1.6328 2.33492 1.66463 0.324989 10
reco:pandorapid:Chi2ParticleID 0.00345047 0.00474048 0.0108139 0.00399505 0.00210314 10
reco:pandoracalinoxyzt:CalibrationdEdXPDSP 0.0318386 0.0371911 0.0425485 0.0353155 0.00391799 10
reco:pandoracalipidnoxyzt:Chi2ParticleID 0.00342012 0.00400578 0.00510161 0.00412294 0.000477224 10
reco:pandoraShowercalo:ShowerCalorimetry 2.21061 3.76181 5.73401 3.71529 1.14654 10
reco:pandoraShowercalonosce:ShowerCalorimetry 2.16843 3.66252 5.49054 3.62427 1.09863 10
reco:emtrkmichelid:EmTrackMichelId 109.59 128.854 159.522 125.968 16.4965 10
reco:crttag:SingleCRTMatchingProducer 0.00750475 0.0107724 0.0193598 0.0096565 0.00337342 10
reco:crtreco:TwoCRTMatchingProducer 0.00462965 0.00641791 0.00874622 0.0061734 0.00126449 10
reco:anodepiercerst0:T0RecoAnodePiercers 0.439615 0.639206 0.824605 0.662186 0.100043 10
reco:pandora2Track:LArPandoraTrackCreation 6.03842 10.5317 16.89 9.47869 3.7049 10
reco:pandora2calo:Calorimetry 3.30252 5.59916 9.05146 5.17401 1.87383 10
reco:pandora2calonosce:Calorimetry 3.1655 5.22684 8.3134 4.8273 1.73376 10
reco:pandora2pid:Chi2ParticleID 0.00761982 0.0102366 0.0174365 0.00940891 0.0029457 10
reco:pandora2calinoxyzt:CalibrationdEdXPDSP 0.0405934 0.0507545 0.0891781 0.0450356 0.0137543 10
reco:pandora2calipidnoxyzt:Chi2ParticleID 0.0075714 0.0100787 0.0170258 0.00935272 0.00271347 10
reco:pandora2Shower:LArPandoraShowerCreation 2.22007 3.51275 5.87445 3.06392 1.22314 10
reco:pandora2Showercalo:ShowerCalorimetry 2.91602 4.36542 6.97556 4.01912 1.18042 10
reco:pandora2Showercalonosce:ShowerCalorimetry 2.64234 3.79872 5.70164 3.54627 0.89763 10
[art]:TriggerResults:TriggerResultInserter 2.7282e-05 4.78919e-05 0.000113345 4.04465e-05 2.3189e-05 10
end_path:out1:RootOutput 6.583e-06 9.5702e-06 1.3485e-05 9.5735e-06 1.65787e-06 10
end_path:out1:RootOutput(write) 1.86605 2.18617 2.81514 2.1552 0.281029 10
====================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5326.47 MB
Peak resident set size usage (VmHWM): 4508.7 MB
Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor: Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor: digitReader: 0.05 sec/event
ToolBasedRawDigitPrepService:dtor: adcSampleFiller: 0.17 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleAdcToKe: 0.04 sec/event
ToolBasedRawDigitPrepService:dtor: pdspTailPedRemovalZKe: 2.36 sec/event
ToolBasedRawDigitPrepService:dtor: adcKeepAllSignalFinder: 0.09 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleKeToAdc: 0.04 sec/event
ToolBasedRawDigitPrepService:dtor: pdsp_RemoveBadChannels: 0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10810712
-rw-r--r-- 1 dunegli fnalgrid 2525182371 Jan 20 17:26 detsim_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 2400337120 Jan 20 17:24 detsim_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 2229117722 Jan 20 18:10 reco_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 1962152822 Jan 20 16:01 g4_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 1873385352 Jan 20 15:52 g4_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 79303860 Jan 20 18:07 Pandora_Events.pndr
-rw-r--r-- 1 dunegli fnalgrid 242724 Jan 20 18:10 jobscript.log
-rw-r--r-- 1 dunegli fnalgrid 217088 Jan 20 18:10 mem.db
-rw-r--r-- 1 dunegli fnalgrid 62376 Jan 20 18:10 Pandora_Geometry.xml
-rw-r--r-- 1 dunegli fnalgrid 36864 Jan 20 18:10 time.db
-rw-r--r-- 1 dunegli fnalgrid 519 Jan 20 17:26 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid 519 Jan 20 16:01 g4_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid 519 Jan 20 18:10 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dunegli fnalgrid 52 Jan 20 15:49 all-input-dids.txt
-rw-r--r-- 1 dunegli fnalgrid 7 Jan 20 18:10 justin-processed-pfns.txt
-rw-r--r-- 1 dunegli fnalgrid 0 Jan 20 17:31 ProtoDUNE.log
-rw-r--r-- 1 dunegli fnalgrid 0 Jan 20 17:26 debugprod.log
