Jobsub ID 265337.1@dunegpschedd02.fnal.gov
| Jobsub ID | 265337.1@dunegpschedd02.fnal.gov |
| Workflow ID | 11448 |
| Stage ID | 1 |
| User name | pmelas@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 8388608000 (8000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2025-12-31 23:20:29 |
| Site | US_FNAL-FermiGrid |
| Entry | FNAL_GPGrid_ce04_mcore_op_duneonly |
| Last heartbeat | 2026-01-01 01:08:22 |
| From worker node | Hostname | dunegli-8206725-0-fnpc22005.fnal.gov |
| cpuinfo | AMD EPYC 7543 32-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 10485760000 (10000 MiB) |
| Wall seconds limit | 172800 (48 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2025-12-31 23:21:08 |
| Input files | monte-carlo-011448-000001
|
| Jobscript | Exit code | 0 |
| Real time | 1h (6423s) |
| CPU time | 1h (6359s = 99%) |
| Max RSS bytes | 4229586944 (4033 MiB) |
| Outputting started | 2026-01-01 01:08:11 |
| Output files | https://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11448/1/001/reco_stage1.root
|
| Finished | 2026-01-01 01:08:22 |
| Saved logs | justin-logs:265337.1-dunegpschedd02.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
g run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 163302 traces, 0 summary
Retagger: tagging trace set: gauss with 119789 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 119789 traces tagged "gauss"
FrameSaver: q=4.31469e+06 n=2068216 tag=gauss
wclsFrameSaver: saving 163302 traces tagged "wiener"
FrameSaver: q=4.60972e+06 n=1817406 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 30129 XUs and 23652 XVs -> 222 XUVs
C:0 T:1 51782 XUs and 43932 XVs -> 5499 XUVs
C:0 T:2 72518 XUs and 89445 XVs -> 4333 XUVs
C:0 T:3 38358 XUs and 40413 XVs -> 394 XUVs
C:0 T:4 12127 XUs and 10270 XVs -> 109 XUVs
C:0 T:5 44049 XUs and 32788 XVs -> 3687 XUVs
C:0 T:6 143647 XUs and 159629 XVs -> 10857 XUVs
C:0 T:7 53374 XUs and 57263 XVs -> 1489 XUVs
C:0 T:8 11621 XUs and 11379 XVs -> 607 XUVs
C:0 T:9 160764 XUs and 149278 XVs -> 41698 XUVs
C:0 T:10 149155 XUs and 127570 XVs -> 32986 XUVs
C:0 T:11 157 XUs and 154 XVs -> 16 XUVs
101897 XUVs total
13758 collection wire objects
95522 potential space points
Neighbour search...
27858384 tests to find 11126254 neighbours
Iterating with no regularization...
Begin: 6.8986e+07
0 5.3693e+07
1 5.2267e+07
2 5.20216e+07
3 5.19456e+07
4 5.19096e+07
Now with regularization...
Begin: 3.92639e+07
0 3.91658e+07
1 3.90887e+07
2 3.90265e+07
3 3.89738e+07
4 3.8929e+07
5 3.88907e+07
Failure in algorithm Alg0061, LArCosmicRayTrackRecovery, STATUS_CODE_FAILURE
run: 1 ; subrun: 0 ; event: 10
evttime: 411463661
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411463661
fLifetime: 30000 [us]
01-Jan-2026 01:08:11 UTC Closed output file "reco_stage1.root"
01-Jan-2026 01:08:11 UTC Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob: # events skipped: 0
====================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
====================================================================================================================================
Full event 134.958 183.472 263.079 170.819 39.3674 10
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000666945 0.00218995 0.0035108 0.00246176 0.000861924 10
reco:ophit:OpHitFinder 0.165941 0.191691 0.222712 0.18682 0.0179168 10
reco:opflash:OpFlashFinder 0.0839046 0.120764 0.160263 0.117656 0.021216 10
reco:opslicer:OpSlicer 0.315932 0.393668 0.491528 0.388222 0.0569023 10
reco:caldata:DataPrepModule 3.33133 3.41275 3.54191 3.39234 0.0742413 10
reco:wclsdatasp:WireCellToolkit 21.4711 22.3621 23.2616 22.2604 0.547161 10
reco:gaushit:GausHitFinder 0.483145 0.634548 0.823384 0.618316 0.100147 10
reco:nhitsfilter:NumberOfHitsFilter 0.000259132 0.000300132 0.000375402 0.000288222 3.94076e-05 10
reco:reco3d:SpacePointSolver 3.85724 8.84871 25.6347 6.36409 6.01879 10
reco:hitpdune:DisambigFromSpacePoints 0.464244 0.818692 1.31224 0.724237 0.258089 10
reco:pandora:StandardPandora 9.4152 14.9979 25.9754 13.5904 5.1965 10
reco:pandoraWriter:StandardPandora 0.174591 0.228001 0.307369 0.215869 0.0429478 10
reco:pandoraTrack:LArPandoraTrackCreation 1.11137 1.73723 2.50974 1.65977 0.469876 10
reco:pandoraShower:LArPandoraShowerCreation 0.989271 1.96285 3.35978 1.76206 0.763979 10
reco:pandoracalo:Calorimetry 0.635823 1.03456 1.53441 1.01061 0.283958 10
reco:pandoracalonosce:Calorimetry 0.570354 0.945066 1.41222 0.914477 0.267868 10
reco:pandorapid:Chi2ParticleID 0.0026581 0.00301463 0.00332174 0.00301528 0.000231762 10
reco:pandoracalinoxyzt:CalibrationdEdXPDSP 0.0240752 0.0289336 0.0388836 0.027227 0.00448193 10
reco:pandoracalipidnoxyzt:Chi2ParticleID 0.00222097 0.00277374 0.00317273 0.0027876 0.000261864 10
reco:pandoraShowercalo:ShowerCalorimetry 1.03078 2.16597 3.84243 1.94527 0.914443 10
reco:pandoraShowercalonosce:ShowerCalorimetry 1.00147 2.10666 3.79964 1.88071 0.907417 10
reco:emtrkmichelid:EmTrackMichelId 77.7258 100.1 130.079 94.7831 16.9962 10
reco:crttag:SingleCRTMatchingProducer 0.00516995 0.00747877 0.00952813 0.00786297 0.00138157 10
reco:crtreco:TwoCRTMatchingProducer 0.00349867 0.00434879 0.00570612 0.00405111 0.000740192 10
reco:anodepiercerst0:T0RecoAnodePiercers 0.346201 0.514359 0.677722 0.493312 0.100565 10
reco:pandora2Track:LArPandoraTrackCreation 3.02908 5.68609 9.45079 5.17272 2.08517 10
reco:pandora2calo:Calorimetry 1.57188 3.06563 5.17093 2.8255 1.18204 10
reco:pandora2calonosce:Calorimetry 1.47344 2.91628 4.93787 2.67195 1.13849 10
reco:pandora2pid:Chi2ParticleID 0.00541302 0.00735364 0.00931421 0.00728885 0.00124368 10
reco:pandora2calinoxyzt:CalibrationdEdXPDSP 0.0286318 0.0362325 0.0471511 0.0342669 0.00622538 10
reco:pandora2calipidnoxyzt:Chi2ParticleID 0.00522739 0.00716897 0.00901073 0.00712625 0.00118229 10
reco:pandora2Shower:LArPandoraShowerCreation 1.15495 2.14853 3.46195 1.95844 0.758491 10
reco:pandora2Showercalo:ShowerCalorimetry 1.37614 2.71098 4.56632 2.46207 1.06478 10
reco:pandora2Showercalonosce:ShowerCalorimetry 1.24565 2.51648 4.19995 2.29349 0.993906 10
[art]:TriggerResults:TriggerResultInserter 2.7803e-05 3.99308e-05 8.944e-05 3.4681e-05 1.69526e-05 10
end_path:out1:RootOutput 7.544e-06 8.1474e-06 1.1171e-05 7.86e-06 1.02946e-06 10
end_path:out1:RootOutput(write) 1.24433 1.70293 2.33354 1.57581 0.352056 10
====================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5037.25 MB
Peak resident set size usage (VmHWM): 4229.59 MB
Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor: Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor: digitReader: 0.04 sec/event
ToolBasedRawDigitPrepService:dtor: adcSampleFiller: 0.15 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleAdcToKe: 0.03 sec/event
ToolBasedRawDigitPrepService:dtor: pdspTailPedRemovalZKe: 2.14 sec/event
ToolBasedRawDigitPrepService:dtor: adcKeepAllSignalFinder: 0.08 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleKeToAdc: 0.03 sec/event
ToolBasedRawDigitPrepService:dtor: pdsp_RemoveBadChannels: 0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9582308
-rw-r--r-- 1 dunegli fnalgrid 2277986060 Jan 1 00:36 detsim_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 2165149354 Jan 1 00:35 detsim_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 1937419925 Jan 1 01:08 reco_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 1716527150 Dec 31 23:30 g4_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 1645911491 Dec 31 23:24 g4_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 68667520 Jan 1 01:05 Pandora_Events.pndr
-rw-r--r-- 1 dunegli fnalgrid 243273 Jan 1 01:08 jobscript.log
-rw-r--r-- 1 dunegli fnalgrid 217088 Jan 1 01:08 mem.db
-rw-r--r-- 1 dunegli fnalgrid 62376 Jan 1 01:08 Pandora_Geometry.xml
-rw-r--r-- 1 dunegli fnalgrid 36864 Jan 1 01:08 time.db
-rw-r--r-- 1 dunegli fnalgrid 519 Jan 1 00:36 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid 519 Dec 31 23:30 g4_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid 519 Jan 1 01:08 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dunegli fnalgrid 52 Dec 31 23:21 all-input-dids.txt
-rw-r--r-- 1 dunegli fnalgrid 7 Jan 1 01:08 justin-processed-pfns.txt
-rw-r--r-- 1 dunegli fnalgrid 0 Jan 1 00:38 ProtoDUNE.log
-rw-r--r-- 1 dunegli fnalgrid 0 Jan 1 00:36 debugprod.log
