Jobsub ID 288574.0@dunegpschedd02.fnal.gov
| Jobsub ID | 288574.0@dunegpschedd02.fnal.gov |
| Workflow ID | 12115 |
| Stage ID | 1 |
| User name | pmelas@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 8388608000 (8000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2026-01-20 14:50:46 |
| Site | UK_Lancaster |
| Entry | UBoone_UK_Lancaster_HEC_grendel_ce02 |
| Last heartbeat | 2026-01-20 17:29:28 |
| From worker node | Hostname | comp17-00 |
| cpuinfo | Intel(R) Xeon(R) 6767P |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 10485760000 (10000 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Started | 2026-01-20 14:53:28 |
| Input files | monte-carlo-012115-000001
|
| Jobscript | Exit code | 0 |
| Real time | 2h (9310s) |
| CPU time | 2h (9017s = 96%) |
| Max RSS bytes | 4139261952 (3947 MiB) |
| Outputting started | 2026-01-20 17:28:39 |
| Output files | |
| Finished | 2026-01-20 17:29:28 |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
vice:prepare: Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 169518 traces, 0 summary
Retagger: tagging trace set: gauss with 123908 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 123908 traces tagged "gauss"
FrameSaver: q=4.58238e+06 n=2182022 tag=gauss
wclsFrameSaver: saving 169518 traces tagged "wiener"
FrameSaver: q=4.8964e+06 n=1925273 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 22891 XUs and 26734 XVs -> 342 XUVs
C:0 T:1 44174 XUs and 52286 XVs -> 3762 XUVs
C:0 T:2 135871 XUs and 234193 XVs -> 10193 XUVs
C:0 T:3 55610 XUs and 83424 XVs -> 848 XUVs
C:0 T:4 16754 XUs and 17674 XVs -> 210 XUVs
C:0 T:5 70194 XUs and 55259 XVs -> 12011 XUVs
C:0 T:6 149128 XUs and 219922 XVs -> 13343 XUVs
C:0 T:7 49250 XUs and 68986 XVs -> 590 XUVs
C:0 T:8 9028 XUs and 7028 XVs -> 107 XUVs
C:0 T:9 98985 XUs and 77898 XVs -> 12305 XUVs
C:0 T:10 54716 XUs and 49157 XVs -> 6957 XUVs
C:0 T:11 539 XUs and 451 XVs -> 71 XUVs
60739 XUVs total
15099 collection wire objects
56120 potential space points
Neighbour search...
7844478 tests to find 3701054 neighbours
Iterating with no regularization...
Begin: 1.0324e+08
0 8.27327e+07
1 8.11041e+07
2 8.08349e+07
3 8.07447e+07
4 8.07036e+07
Now with regularization...
Begin: 6.69395e+07
0 6.68315e+07
1 6.6767e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411853263
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411853263
fLifetime: 30000 [us]
20-Jan-2026 17:28:29 GMT Closed output file "reco_stage1.root"
20-Jan-2026 17:28:29 GMT Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob: # events skipped: 0
====================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
====================================================================================================================================
Full event 242.164 343.313 489.073 341.616 66.9845 10
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000766679 0.00402498 0.00699137 0.00390576 0.00171575 10
reco:ophit:OpHitFinder 0.265641 0.296976 0.364251 0.286289 0.0305671 10
reco:opflash:OpFlashFinder 0.10602 0.142593 0.198143 0.140627 0.0286645 10
reco:opslicer:OpSlicer 0.47743 0.556855 0.680123 0.542949 0.068799 10
reco:caldata:DataPrepModule 5.02258 5.21407 5.39242 5.20361 0.125163 10
reco:wclsdatasp:WireCellToolkit 43.6481 49.6533 54.8251 49.4238 3.24956 10
reco:gaushit:GausHitFinder 0.822142 1.09441 1.36942 1.06738 0.169886 10
reco:nhitsfilter:NumberOfHitsFilter 0.000461373 0.000613553 0.000959419 0.000577011 0.000149926 10
reco:reco3d:SpacePointSolver 5.3491 11.8892 22.7322 10.5298 4.77457 10
reco:hitpdune:DisambigFromSpacePoints 0.876829 1.26165 1.89673 1.2466 0.282707 10
reco:pandora:StandardPandora 18.2971 33.8205 67.066 30.1279 13.5802 10
reco:pandoraWriter:StandardPandora 0.308425 0.408958 0.566232 0.392708 0.0817434 10
reco:pandoraTrack:LArPandoraTrackCreation 4.05543 6.33498 8.82378 6.14491 1.49129 10
reco:pandoraShower:LArPandoraShowerCreation 3.97016 7.72323 13.9886 7.84016 2.53918 10
reco:pandoracalo:Calorimetry 1.88379 3.04881 4.72962 3.02025 0.814311 10
reco:pandoracalonosce:Calorimetry 1.48028 2.79824 4.17379 2.87205 0.745066 10
reco:pandorapid:Chi2ParticleID 0.00362615 0.00473251 0.00654306 0.00457911 0.000820043 10
reco:pandoracalinoxyzt:CalibrationdEdXPDSP 0.03371 0.0438613 0.0564727 0.0425448 0.00678838 10
reco:pandoracalipidnoxyzt:Chi2ParticleID 0.00345098 0.00431284 0.00514249 0.00441498 0.00044261 10
reco:pandoraShowercalo:ShowerCalorimetry 2.5727 6.6342 12.5673 6.66063 2.59187 10
reco:pandoraShowercalonosce:ShowerCalorimetry 2.48391 6.64533 11.6297 6.80536 2.4653 10
reco:emtrkmichelid:EmTrackMichelId 112.959 141.186 182.353 138.93 21.7921 10
reco:crttag:SingleCRTMatchingProducer 0.0130509 0.0207637 0.0368977 0.0191217 0.00699192 10
reco:crtreco:TwoCRTMatchingProducer 0.00935922 0.0133977 0.0170256 0.0133408 0.00234843 10
reco:anodepiercerst0:T0RecoAnodePiercers 0.644682 0.965153 1.36318 0.935943 0.199693 10
reco:pandora2Track:LArPandoraTrackCreation 9.26877 19.2307 30.426 19.3704 5.46755 10
reco:pandora2calo:Calorimetry 4.89408 9.38911 15.7002 10.0353 2.95706 10
reco:pandora2calonosce:Calorimetry 4.3697 9.01011 14.1444 9.35277 2.67725 10
reco:pandora2pid:Chi2ParticleID 0.00823785 0.011433 0.015043 0.0116193 0.00170249 10
reco:pandora2calinoxyzt:CalibrationdEdXPDSP 0.044104 0.0571411 0.074205 0.0556017 0.00809474 10
reco:pandora2calipidnoxyzt:Chi2ParticleID 0.00812888 0.0108627 0.0142589 0.0109522 0.00149565 10
reco:pandora2Shower:LArPandoraShowerCreation 4.36446 7.43376 11.8245 7.42067 1.93044 10
reco:pandora2Showercalo:ShowerCalorimetry 3.45923 8.30338 13.7974 8.89828 2.8448 10
reco:pandora2Showercalonosce:ShowerCalorimetry 3.22995 7.49444 11.9274 7.68052 2.37557 10
[art]:TriggerResults:TriggerResultInserter 4.5145e-05 6.17107e-05 0.000108962 5.6683e-05 1.70743e-05 10
end_path:out1:RootOutput 1.1521e-05 1.37605e-05 1.6132e-05 1.40955e-05 1.71943e-06 10
end_path:out1:RootOutput(write) 1.9789 2.47216 3.12224 2.39509 0.370535 10
====================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4952.94 MB
Peak resident set size usage (VmHWM): 4139.26 MB
Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor: Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor: digitReader: 0.08 sec/event
ToolBasedRawDigitPrepService:dtor: adcSampleFiller: 0.21 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleAdcToKe: 0.04 sec/event
ToolBasedRawDigitPrepService:dtor: pdspTailPedRemovalZKe: 3.12 sec/event
ToolBasedRawDigitPrepService:dtor: adcKeepAllSignalFinder: 0.13 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleKeToAdc: 0.04 sec/event
ToolBasedRawDigitPrepService:dtor: pdsp_RemoveBadChannels: 0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10240149
-rw-r--r-- 1 pltdune004 pltdune 2407287251 Jan 20 16:29 detsim_stage2.root
-rw-r--r-- 1 pltdune004 pltdune 2286555393 Jan 20 16:27 detsim_stage1.root
-rw-r--r-- 1 pltdune004 pltdune 2101512766 Jan 20 17:28 reco_stage1.root
-rw-r--r-- 1 pltdune004 pltdune 1850287391 Jan 20 15:11 g4_stage2.root
-rw-r--r-- 1 pltdune004 pltdune 1763140092 Jan 20 14:59 g4_stage1.root
-rw-r--r-- 1 pltdune004 pltdune 76564420 Jan 20 17:22 Pandora_Events.pndr
-rw-r--r-- 1 pltdune004 pltdune 242834 Jan 20 17:28 jobscript.log
-rw-r--r-- 1 pltdune004 pltdune 217088 Jan 20 17:28 mem.db
-rw-r--r-- 1 pltdune004 pltdune 62376 Jan 20 17:28 Pandora_Geometry.xml
-rw-r--r-- 1 pltdune004 pltdune 36864 Jan 20 17:28 time.db
-rw-r--r-- 1 pltdune004 pltdune 519 Jan 20 16:29 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pltdune004 pltdune 519 Jan 20 15:11 g4_protoDUNE_hist.root
-rw-r--r-- 1 pltdune004 pltdune 519 Jan 20 17:28 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pltdune004 pltdune 52 Jan 20 14:53 all-input-dids.txt
-rw-r--r-- 1 pltdune004 pltdune 7 Jan 20 17:28 justin-processed-pfns.txt
-rw-r--r-- 1 pltdune004 pltdune 0 Jan 20 16:35 ProtoDUNE.log
-rw-r--r-- 1 pltdune004 pltdune 0 Jan 20 16:29 debugprod.log
