Jobsub ID 294238.1@dunegpschedd01.fnal.gov
| Jobsub ID | 294238.1@dunegpschedd01.fnal.gov |
| Workflow ID | 12109 |
| Stage ID | 1 |
| User name | pmelas@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 8388608000 (8000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2026-01-20 14:48:55 |
| Site | UK_RAL-PPD |
| Entry | CMSHTPC_T2_UK_SGrid_RALPP_hep208 |
| Last heartbeat | 2026-01-20 18:35:28 |
| From worker node | Hostname | heplnc157.pp.rl.ac.uk |
| cpuinfo | AMD EPYC 7763 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 10485760000 (10000 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Started | 2026-01-20 14:49:38 |
| Input files | monte-carlo-012109-000001
|
| Jobscript | Exit code | 0 |
| Real time | 3h (13506s) |
| CPU time | 3h (13015s = 96%) |
| Max RSS bytes | 4221313024 (4025 MiB) |
| Outputting started | 2026-01-20 18:34:45 |
| Output files | |
| Finished | 2026-01-20 18:35:28 |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
rvice:prepare: Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 169511 traces, 0 summary
Retagger: tagging trace set: gauss with 124226 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 124226 traces tagged "gauss"
FrameSaver: q=4.58742e+06 n=2185133 tag=gauss
wclsFrameSaver: saving 169511 traces tagged "wiener"
FrameSaver: q=4.9018e+06 n=1928083 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 22638 XUs and 26989 XVs -> 344 XUVs
C:0 T:1 44296 XUs and 52783 XVs -> 3804 XUVs
C:0 T:2 137115 XUs and 233476 XVs -> 10232 XUVs
C:0 T:3 55899 XUs and 82841 XVs -> 835 XUVs
C:0 T:4 16827 XUs and 18049 XVs -> 211 XUVs
C:0 T:5 70944 XUs and 56153 XVs -> 12156 XUVs
C:0 T:6 147413 XUs and 219408 XVs -> 13272 XUVs
C:0 T:7 48625 XUs and 68881 XVs -> 573 XUVs
C:0 T:8 9058 XUs and 7099 XVs -> 114 XUVs
C:0 T:9 99619 XUs and 78731 XVs -> 12307 XUVs
C:0 T:10 54879 XUs and 48924 XVs -> 7056 XUVs
C:0 T:11 463 XUs and 422 XVs -> 64 XUVs
60968 XUVs total
15213 collection wire objects
56361 potential space points
Neighbour search...
7844947 tests to find 3709938 neighbours
Iterating with no regularization...
Begin: 1.02318e+08
0 8.18181e+07
1 8.01744e+07
2 7.99018e+07
3 7.98174e+07
4 7.97785e+07
Now with regularization...
Begin: 6.59026e+07
0 6.57873e+07
1 6.57206e+07
2 6.56698e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411853263
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411853263
fLifetime: 30000 [us]
20-Jan-2026 18:34:43 GMT Closed output file "reco_stage1.root"
20-Jan-2026 18:34:43 GMT Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob: # events skipped: 0
====================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
====================================================================================================================================
Full event 323.073 423.875 565.872 424.614 73.3812 10
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00150936 0.00437205 0.00770569 0.00417605 0.00198082 10
reco:ophit:OpHitFinder 0.285669 0.340879 0.416321 0.336077 0.0453053 10
reco:opflash:OpFlashFinder 0.159858 0.201962 0.248549 0.194797 0.0342889 10
reco:opslicer:OpSlicer 0.572533 0.836637 1.09338 0.854484 0.173977 10
reco:caldata:DataPrepModule 5.32938 6.70571 7.71181 6.99065 0.844883 10
reco:wclsdatasp:WireCellToolkit 35.6237 40.0377 45.7021 40.9884 2.88117 10
reco:gaushit:GausHitFinder 0.862879 1.46219 2.00938 1.4563 0.341914 10
reco:nhitsfilter:NumberOfHitsFilter 0.000435517 0.000659323 0.000966224 0.000639896 0.000136301 10
reco:reco3d:SpacePointSolver 12.7545 22.9332 37.831 21.9915 7.24767 10
reco:hitpdune:DisambigFromSpacePoints 1.12899 1.95175 2.50148 1.97721 0.473763 10
reco:pandora:StandardPandora 21.7921 40.5112 87.5886 37.8395 17.5713 10
reco:pandoraWriter:StandardPandora 0.373413 0.564289 0.731882 0.555551 0.0966875 10
reco:pandoraTrack:LArPandoraTrackCreation 3.30752 5.30638 6.89481 5.15912 1.12851 10
reco:pandoraShower:LArPandoraShowerCreation 2.9665 5.19291 7.29543 5.32992 1.2344 10
reco:pandoracalo:Calorimetry 1.70852 2.98774 3.91571 3.19435 0.645475 10
reco:pandoracalonosce:Calorimetry 1.85742 2.73475 3.84737 2.82727 0.56892 10
reco:pandorapid:Chi2ParticleID 0.00422121 0.00745975 0.0101317 0.00789651 0.00177468 10
reco:pandoracalinoxyzt:CalibrationdEdXPDSP 0.0508951 0.071919 0.0959974 0.0737749 0.0144536 10
reco:pandoracalipidnoxyzt:Chi2ParticleID 0.00426278 0.00729282 0.00996276 0.00777137 0.00168858 10
reco:pandoraShowercalo:ShowerCalorimetry 2.82663 5.66255 10.4724 5.3776 1.90246 10
reco:pandoraShowercalonosce:ShowerCalorimetry 2.74239 5.33717 8.27292 5.33578 1.67355 10
reco:emtrkmichelid:EmTrackMichelId 182.83 224.219 274.769 220.944 31.0933 10
reco:crttag:SingleCRTMatchingProducer 0.0116815 0.0204466 0.0438062 0.0176036 0.00836998 10
reco:crtreco:TwoCRTMatchingProducer 0.00626337 0.0129406 0.0237143 0.0111681 0.00478014 10
reco:anodepiercerst0:T0RecoAnodePiercers 0.573171 0.994973 1.47316 0.944622 0.257852 10
reco:pandora2Track:LArPandoraTrackCreation 7.8528 16.2739 27.318 16.9138 5.71613 10
reco:pandora2calo:Calorimetry 3.72491 8.15649 14.4728 8.66585 2.93801 10
reco:pandora2calonosce:Calorimetry 3.03165 7.78029 11.8023 7.6126 2.75233 10
reco:pandora2pid:Chi2ParticleID 0.00839568 0.0173366 0.0215147 0.0187966 0.00405874 10
reco:pandora2calinoxyzt:CalibrationdEdXPDSP 0.0491087 0.0888287 0.122113 0.0942684 0.021881 10
reco:pandora2calipidnoxyzt:Chi2ParticleID 0.00837254 0.0168008 0.0214825 0.0182831 0.00392183 10
reco:pandora2Shower:LArPandoraShowerCreation 3.25301 5.92245 8.90341 5.71919 1.9271 10
reco:pandora2Showercalo:ShowerCalorimetry 3.93804 7.29166 10.4971 7.01322 2.23361 10
reco:pandora2Showercalonosce:ShowerCalorimetry 3.83163 6.51834 9.50492 7.00534 1.91956 10
[art]:TriggerResults:TriggerResultInserter 7.6212e-05 0.000112456 0.00023961 9.54395e-05 4.54501e-05 10
end_path:out1:RootOutput 1.4728e-05 1.67534e-05 2.5077e-05 1.59345e-05 2.85754e-06 10
end_path:out1:RootOutput(write) 2.90489 3.54959 5.0389 3.46972 0.600618 10
====================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5086.41 MB
Peak resident set size usage (VmHWM): 4221.31 MB
Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor: Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor: digitReader: 0.09 sec/event
ToolBasedRawDigitPrepService:dtor: adcSampleFiller: 0.31 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleAdcToKe: 0.05 sec/event
ToolBasedRawDigitPrepService:dtor: pdspTailPedRemovalZKe: 4.20 sec/event
ToolBasedRawDigitPrepService:dtor: adcKeepAllSignalFinder: 0.18 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleKeToAdc: 0.05 sec/event
ToolBasedRawDigitPrepService:dtor: pdsp_RemoveBadChannels: 0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10239316
-rw-r--r-- 1 pltdune04 pltdune 2407276081 Jan 20 17:21 detsim_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 2286561670 Jan 20 17:19 detsim_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 2100579913 Jan 20 18:34 reco_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 1850287391 Jan 20 15:10 g4_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 1763140092 Jan 20 14:56 g4_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 76602700 Jan 20 18:28 Pandora_Events.pndr
-rw-r--r-- 1 pltdune04 pltdune 242806 Jan 20 18:34 jobscript.log
-rw-r--r-- 1 pltdune04 pltdune 217088 Jan 20 18:34 mem.db
-rw-r--r-- 1 pltdune04 pltdune 62376 Jan 20 18:34 Pandora_Geometry.xml
-rw-r--r-- 1 pltdune04 pltdune 36864 Jan 20 18:34 time.db
-rw-r--r-- 1 pltdune04 pltdune 519 Jan 20 17:21 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune 519 Jan 20 15:10 g4_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune 519 Jan 20 18:34 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pltdune04 pltdune 52 Jan 20 14:49 all-input-dids.txt
-rw-r--r-- 1 pltdune04 pltdune 7 Jan 20 18:34 justin-processed-pfns.txt
-rw-r--r-- 1 pltdune04 pltdune 0 Jan 20 17:29 ProtoDUNE.log
-rw-r--r-- 1 pltdune04 pltdune 0 Jan 20 17:22 debugprod.log
