Jobsub ID 288561.0@dunegpschedd02.fnal.gov
| Jobsub ID | 288561.0@dunegpschedd02.fnal.gov |
| Workflow ID | 12114 |
| Stage ID | 1 |
| User name | pmelas@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 8388608000 (8000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2026-01-20 14:48:55 |
| Site | UK_RAL-PPD |
| Entry | CMSHTPC_T2_UK_SGrid_RALPP_hep208 |
| Last heartbeat | 2026-01-20 18:29:09 |
| From worker node | Hostname | heplnc170.pp.rl.ac.uk |
| cpuinfo | AMD EPYC 7763 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 10485760000 (10000 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Started | 2026-01-20 14:52:16 |
| Input files | monte-carlo-012114-000001
|
| Jobscript | Exit code | 0 |
| Real time | 3h (12971s) |
| CPU time | 3h (12504s = 96%) |
| Max RSS bytes | 4129927168 (3938 MiB) |
| Outputting started | 2026-01-20 18:28:28 |
| Output files | |
| Finished | 2026-01-20 18:29:09 |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare: Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 126426 traces, 0 summary
Retagger: tagging trace set: gauss with 96581 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 96581 traces tagged "gauss"
FrameSaver: q=3.38747e+06 n=1732191 tag=gauss
wclsFrameSaver: saving 126426 traces tagged "wiener"
FrameSaver: q=3.58871e+06 n=1548443 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 39136 XUs and 55921 XVs -> 553 XUVs
C:0 T:1 80899 XUs and 99234 XVs -> 6097 XUVs
C:0 T:2 46612 XUs and 104570 XVs -> 6024 XUVs
C:0 T:3 23156 XUs and 44042 XVs -> 186 XUVs
C:0 T:4 11483 XUs and 12539 XVs -> 72 XUVs
C:0 T:5 37066 XUs and 33898 XVs -> 3025 XUVs
C:0 T:6 165217 XUs and 175994 XVs -> 16767 XUVs
C:0 T:7 53494 XUs and 64390 XVs -> 687 XUVs
C:0 T:8 5463 XUs and 4272 XVs -> 60 XUVs
C:0 T:9 26695 XUs and 31979 XVs -> 2356 XUVs
C:0 T:10 16917 XUs and 10840 XVs -> 2140 XUVs
C:0 T:11 92 XUs and 46 XVs -> 11 XUVs
37978 XUVs total
10239 collection wire objects
35138 potential space points
Neighbour search...
4896628 tests to find 2219622 neighbours
Iterating with no regularization...
Begin: 6.54846e+07
0 5.46002e+07
1 5.39035e+07
2 5.37974e+07
3 5.37645e+07
Now with regularization...
Begin: 4.4267e+07
0 4.42251e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411853275
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411853275
fLifetime: 30000 [us]
20-Jan-2026 18:28:26 GMT Closed output file "reco_stage1.root"
20-Jan-2026 18:28:26 GMT Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob: # events skipped: 0
====================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
====================================================================================================================================
Full event 233.207 392.174 608.496 396.346 92.7066 10
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00121795 0.005325 0.0113251 0.00525929 0.00264131 10
reco:ophit:OpHitFinder 0.224484 0.347061 0.427373 0.37509 0.0661393 10
reco:opflash:OpFlashFinder 0.139364 0.193294 0.211782 0.203401 0.0220273 10
reco:opslicer:OpSlicer 0.38963 0.80591 1.07897 0.859336 0.190277 10
reco:caldata:DataPrepModule 4.68008 6.74553 8.14278 6.81801 1.11542 10
reco:wclsdatasp:WireCellToolkit 30.8216 38.4209 43.8466 39.0369 4.1799 10
reco:gaushit:GausHitFinder 0.76193 1.47714 2.03336 1.62173 0.377085 10
reco:nhitsfilter:NumberOfHitsFilter 0.000284542 0.000621667 0.000895353 0.000609068 0.000181306 10
reco:reco3d:SpacePointSolver 6.97691 16.7731 33.3756 15.4851 7.0876 10
reco:hitpdune:DisambigFromSpacePoints 1.23237 1.92039 3.01941 1.83448 0.494582 10
reco:pandora:StandardPandora 13.933 40.4226 112.915 34.1372 25.6343 10
reco:pandoraWriter:StandardPandora 0.239434 0.508024 0.817689 0.511764 0.160805 10
reco:pandoraTrack:LArPandoraTrackCreation 1.85138 4.38374 7.09204 4.30297 1.36295 10
reco:pandoraShower:LArPandoraShowerCreation 1.4061 4.76798 8.32422 4.1959 1.98927 10
reco:pandoracalo:Calorimetry 1.12617 2.57258 3.85147 2.52412 0.740254 10
reco:pandoracalonosce:Calorimetry 1.10434 2.24506 3.40773 2.08884 0.61058 10
reco:pandorapid:Chi2ParticleID 0.0044268 0.00722781 0.00973081 0.00808323 0.00190667 10
reco:pandoracalinoxyzt:CalibrationdEdXPDSP 0.0389784 0.061358 0.09054 0.0574817 0.0175846 10
reco:pandoracalipidnoxyzt:Chi2ParticleID 0.00481686 0.00700961 0.00950527 0.00687855 0.00187876 10
reco:pandoraShowercalo:ShowerCalorimetry 1.68863 5.08546 8.21745 4.70668 1.95758 10
reco:pandoraShowercalonosce:ShowerCalorimetry 1.50074 4.82062 7.89345 4.39047 1.89553 10
reco:emtrkmichelid:EmTrackMichelId 142.297 212.14 278.216 216.258 37.942 10
reco:crttag:SingleCRTMatchingProducer 0.00807039 0.0168088 0.0275001 0.0177463 0.00496866 10
reco:crtreco:TwoCRTMatchingProducer 0.00485944 0.0101596 0.0141807 0.0103332 0.00260967 10
reco:anodepiercerst0:T0RecoAnodePiercers 0.344289 0.972404 1.82503 0.967738 0.353447 10
reco:pandora2Track:LArPandoraTrackCreation 4.35882 14.0904 24.4248 14.4746 5.22301 10
reco:pandora2calo:Calorimetry 2.42927 7.06542 13.8857 6.83253 2.76949 10
reco:pandora2calonosce:Calorimetry 2.82884 6.6693 12.6299 6.38244 2.48047 10
reco:pandora2pid:Chi2ParticleID 0.0119811 0.0181979 0.0304059 0.0178815 0.00494835 10
reco:pandora2calinoxyzt:CalibrationdEdXPDSP 0.0590565 0.0884298 0.133097 0.0894686 0.0200609 10
reco:pandora2calipidnoxyzt:Chi2ParticleID 0.0116639 0.018443 0.0282411 0.0182042 0.00448229 10
reco:pandora2Shower:LArPandoraShowerCreation 2.20363 4.79039 8.48368 4.63019 1.54485 10
reco:pandora2Showercalo:ShowerCalorimetry 2.67438 5.90679 10.5237 5.70442 1.93048 10
reco:pandora2Showercalonosce:ShowerCalorimetry 2.37569 5.5074 9.67773 5.3542 1.82703 10
[art]:TriggerResults:TriggerResultInserter 4.4133e-05 7.19327e-05 0.0001553 6.3499e-05 2.96024e-05 10
end_path:out1:RootOutput 7.134e-06 1.25034e-05 1.8344e-05 1.2388e-05 2.90548e-06 10
end_path:out1:RootOutput(write) 2.079 3.17744 4.73129 3.11549 0.671871 10
====================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4990.1 MB
Peak resident set size usage (VmHWM): 4129.93 MB
Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor: Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor: digitReader: 0.08 sec/event
ToolBasedRawDigitPrepService:dtor: adcSampleFiller: 0.31 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleAdcToKe: 0.05 sec/event
ToolBasedRawDigitPrepService:dtor: pdspTailPedRemovalZKe: 4.25 sec/event
ToolBasedRawDigitPrepService:dtor: adcKeepAllSignalFinder: 0.18 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleKeToAdc: 0.05 sec/event
ToolBasedRawDigitPrepService:dtor: pdsp_RemoveBadChannels: 0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9912268
-rw-r--r-- 1 pltdune04 pltdune 2350831901 Jan 20 17:20 detsim_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 2234690305 Jan 20 17:18 detsim_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 2018180169 Jan 20 18:28 reco_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 1767270437 Jan 20 15:12 g4_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 1707781731 Jan 20 14:59 g4_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 70730240 Jan 20 18:23 Pandora_Events.pndr
-rw-r--r-- 1 pltdune04 pltdune 242836 Jan 20 18:28 jobscript.log
-rw-r--r-- 1 pltdune04 pltdune 217088 Jan 20 18:28 mem.db
-rw-r--r-- 1 pltdune04 pltdune 62376 Jan 20 18:28 Pandora_Geometry.xml
-rw-r--r-- 1 pltdune04 pltdune 36864 Jan 20 18:28 time.db
-rw-r--r-- 1 pltdune04 pltdune 519 Jan 20 17:20 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune 519 Jan 20 15:12 g4_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune 519 Jan 20 18:28 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pltdune04 pltdune 52 Jan 20 14:52 all-input-dids.txt
-rw-r--r-- 1 pltdune04 pltdune 7 Jan 20 18:28 justin-processed-pfns.txt
-rw-r--r-- 1 pltdune04 pltdune 0 Jan 20 17:28 ProtoDUNE.log
-rw-r--r-- 1 pltdune04 pltdune 0 Jan 20 17:20 debugprod.log
