Jobsub ID 294676.0@dunegpschedd02.fnal.gov
| Jobsub ID | 294676.0@dunegpschedd02.fnal.gov |
| Workflow ID | 12384 |
| Stage ID | 1 |
| User name | pmelas@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 8388608000 (8000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2026-01-24 21:42:03 |
| Site | UK_RAL-PPD |
| Entry | CMSHTPC_T2_UK_SGrid_RALPP_hep208 |
| Last heartbeat | 2026-01-25 01:57:00 |
| From worker node | Hostname | heplnc157.pp.rl.ac.uk |
| cpuinfo | AMD EPYC 7763 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 10485760000 (10000 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Started | 2026-01-24 21:43:03 |
| Input files | monte-carlo-012384-000001
|
| Jobscript | Exit code | 0 |
| Real time | 4h (15191s) |
| CPU time | 3h (14323s = 94%) |
| Max RSS bytes | 4191846400 (3997 MiB) |
| Outputting started | 2026-01-25 01:56:15 |
| Output files | |
| Finished | 2026-01-25 01:57:00 |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
oAdc
ToolBasedRawDigitPrepService:prepare: Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 169270 traces, 0 summary
Retagger: tagging trace set: gauss with 133102 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 133102 traces tagged "gauss"
FrameSaver: q=4.82419e+06 n=2424907 tag=gauss
wclsFrameSaver: saving 169270 traces tagged "wiener"
FrameSaver: q=5.08651e+06 n=2171226 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 42908 XUs and 35979 XVs -> 347 XUVs
C:0 T:1 80679 XUs and 70102 XVs -> 5419 XUVs
C:0 T:2 104745 XUs and 176263 XVs -> 5753 XUVs
C:0 T:3 51984 XUs and 70571 XVs -> 501 XUVs
C:0 T:4 15259 XUs and 19706 XVs -> 218 XUVs
C:0 T:5 108158 XUs and 98675 XVs -> 8735 XUVs
C:0 T:6 159362 XUs and 144546 XVs -> 10513 XUVs
C:0 T:7 52439 XUs and 51654 XVs -> 781 XUVs
C:0 T:8 10618 XUs and 8586 XVs -> 159 XUVs
C:0 T:9 61453 XUs and 71995 XVs -> 5401 XUVs
C:0 T:10 121406 XUs and 53763 XVs -> 13241 XUVs
C:0 T:11 181 XUs and 120 XVs -> 12 XUVs
51080 XUVs total
15882 collection wire objects
47035 potential space points
Neighbour search...
6285589 tests to find 2780548 neighbours
Iterating with no regularization...
Begin: 1.29779e+08
0 1.0794e+08
1 1.06211e+08
2 1.0596e+08
3 1.05893e+08
Now with regularization...
Begin: 9.00829e+07
0 8.99846e+07
1 8.99429e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411945059
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411945059
fLifetime: 30000 [us]
25-Jan-2026 01:56:12 GMT Closed output file "reco_stage1.root"
25-Jan-2026 01:56:12 GMT Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob: # events skipped: 0
====================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
====================================================================================================================================
Full event 385.667 466.379 668.004 449.483 77.8461 10
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00174815 0.00554876 0.0161034 0.00474157 0.00393571 10
reco:ophit:OpHitFinder 0.388929 0.426176 0.496591 0.422049 0.0338102 10
reco:opflash:OpFlashFinder 0.198783 0.254034 0.300852 0.253179 0.025731 10
reco:opslicer:OpSlicer 0.925519 1.02231 1.19393 1.01601 0.0898745 10
reco:caldata:DataPrepModule 6.62993 7.80704 8.39347 7.84965 0.449076 10
reco:wclsdatasp:WireCellToolkit 41.3336 43.8208 45.3088 44.0107 1.11805 10
reco:gaushit:GausHitFinder 1.4649 1.71942 2.154 1.57645 0.233317 10
reco:nhitsfilter:NumberOfHitsFilter 0.000598965 0.000750207 0.00119842 0.000719 0.00016127 10
reco:reco3d:SpacePointSolver 14.4546 20.7112 27.985 20.5151 4.47455 10
reco:hitpdune:DisambigFromSpacePoints 1.81076 2.32218 3.70005 2.14272 0.57292 10
reco:pandora:StandardPandora 27.7855 49.0655 95.0733 45.8062 18.5267 10
reco:pandoraWriter:StandardPandora 0.519334 0.676367 0.814117 0.663636 0.0950475 10
reco:pandoraTrack:LArPandoraTrackCreation 4.44402 5.66996 7.6269 5.59829 1.012 10
reco:pandoraShower:LArPandoraShowerCreation 4.18443 6.33063 12.7256 5.37237 2.4464 10
reco:pandoracalo:Calorimetry 2.36998 3.18987 4.91996 2.93162 0.694896 10
reco:pandoracalonosce:Calorimetry 2.32987 2.96153 4.644 2.72333 0.649714 10
reco:pandorapid:Chi2ParticleID 0.00726034 0.00862505 0.0105924 0.00841301 0.000843912 10
reco:pandoracalinoxyzt:CalibrationdEdXPDSP 0.0681653 0.0805585 0.101723 0.0793659 0.00989463 10
reco:pandoracalipidnoxyzt:Chi2ParticleID 0.00537253 0.00789978 0.00947553 0.00816452 0.00112055 10
reco:pandoraShowercalo:ShowerCalorimetry 4.01074 6.93015 13.3859 6.3054 2.43724 10
reco:pandoraShowercalonosce:ShowerCalorimetry 3.90093 6.45083 12.3283 6.04255 2.14322 10
reco:emtrkmichelid:EmTrackMichelId 204.556 242.874 321.528 235.173 31.2715 10
reco:crttag:SingleCRTMatchingProducer 0.0158421 0.0243951 0.0424601 0.0241565 0.00735555 10
reco:crtreco:TwoCRTMatchingProducer 0.00968473 0.012661 0.0160911 0.0125686 0.00213534 10
reco:anodepiercerst0:T0RecoAnodePiercers 0.857697 1.14335 1.55926 1.08626 0.19818 10
reco:pandora2Track:LArPandoraTrackCreation 12.9519 18.7381 33.8852 17.9566 5.57631 10
reco:pandora2calo:Calorimetry 6.3305 9.56066 16.4308 9.33291 2.70425 10
reco:pandora2calonosce:Calorimetry 6.44863 9.08226 16.011 8.43333 2.63227 10
reco:pandora2pid:Chi2ParticleID 0.015965 0.0217279 0.0312832 0.0207508 0.00384721 10
reco:pandora2calinoxyzt:CalibrationdEdXPDSP 0.0912182 0.110461 0.161615 0.103526 0.0187285 10
reco:pandora2calipidnoxyzt:Chi2ParticleID 0.0154658 0.0213088 0.0299063 0.02085 0.00354896 10
reco:pandora2Shower:LArPandoraShowerCreation 4.17262 6.39256 10.5249 6.05942 1.61162 10
reco:pandora2Showercalo:ShowerCalorimetry 5.45465 7.81586 11.5465 7.75703 1.68907 10
reco:pandora2Showercalonosce:ShowerCalorimetry 4.81615 6.9832 11.4082 6.91691 1.77567 10
[art]:TriggerResults:TriggerResultInserter 8.0631e-05 0.000104867 0.000219923 9.309e-05 3.90455e-05 10
end_path:out1:RootOutput 1.1241e-05 1.43928e-05 2.1901e-05 1.4011e-05 2.69081e-06 10
end_path:out1:RootOutput(write) 3.44102 3.98192 5.08839 3.90401 0.47598 10
====================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5091.41 MB
Peak resident set size usage (VmHWM): 4191.85 MB
Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor: Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor: digitReader: 0.10 sec/event
ToolBasedRawDigitPrepService:dtor: adcSampleFiller: 0.36 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleAdcToKe: 0.06 sec/event
ToolBasedRawDigitPrepService:dtor: pdspTailPedRemovalZKe: 4.91 sec/event
ToolBasedRawDigitPrepService:dtor: adcKeepAllSignalFinder: 0.21 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleKeToAdc: 0.06 sec/event
ToolBasedRawDigitPrepService:dtor: pdsp_RemoveBadChannels: 0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10754388
-rw-r--r-- 1 pltdune04 pltdune 2508673578 Jan 25 00:35 detsim_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 2378668263 Jan 25 00:32 detsim_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 2202242986 Jan 25 01:56 reco_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 1963048237 Jan 24 22:11 g4_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 1882794792 Jan 24 21:55 g4_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 76455190 Jan 25 01:48 Pandora_Events.pndr
-rw-r--r-- 1 pltdune04 pltdune 243030 Jan 25 01:56 jobscript.log
-rw-r--r-- 1 pltdune04 pltdune 217088 Jan 25 01:56 mem.db
-rw-r--r-- 1 pltdune04 pltdune 62376 Jan 25 01:56 Pandora_Geometry.xml
-rw-r--r-- 1 pltdune04 pltdune 36864 Jan 25 01:56 time.db
-rw-r--r-- 1 pltdune04 pltdune 519 Jan 25 00:35 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune 519 Jan 24 22:11 g4_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune 519 Jan 25 01:56 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pltdune04 pltdune 52 Jan 24 21:43 all-input-dids.txt
-rw-r--r-- 1 pltdune04 pltdune 7 Jan 25 01:56 justin-processed-pfns.txt
-rw-r--r-- 1 pltdune04 pltdune 0 Jan 25 00:44 ProtoDUNE.log
-rw-r--r-- 1 pltdune04 pltdune 0 Jan 25 00:35 debugprod.log
