Jobsub ID 254065.1@dunegpschedd01.fnal.gov
| Jobsub ID | 254065.1@dunegpschedd01.fnal.gov |
| Workflow ID | 10373 |
| Stage ID | 1 |
| User name | pmelas@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 8388608000 (8000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2025-11-19 09:33:44 |
| Site | UK_Manchester |
| Entry | UBoone_T2_UK_Manchester_ce01 |
| Last heartbeat | 2025-11-19 16:20:10 |
| From worker node | Hostname | wn5917301.tier2.hep.manchester.ac.uk |
| cpuinfo | AMD EPYC 7702P 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 10485760000 (10000 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Started | 2025-11-19 09:35:00 |
| Input files | monte-carlo-010373-000001
|
| Jobscript | Exit code | 0 |
| Real time | 6h (23910s) |
| CPU time | 6h (22897s = 95%) |
| Max RSS bytes | 4016066560 (3830 MiB) |
| Outputting started | 2025-11-19 16:13:31 |
| Output files | |
| Finished | 2025-11-19 16:20:10 |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare: Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare: Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 76266 traces, 0 summary
Retagger: tagging trace set: gauss with 60324 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 60324 traces tagged "gauss"
FrameSaver: q=2.14238e+06 n=1079257 tag=gauss
wclsFrameSaver: saving 76266 traces tagged "wiener"
FrameSaver: q=2.2625e+06 n=969411 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 33574 XUs and 31717 XVs -> 346 XUVs
C:0 T:1 56041 XUs and 76747 XVs -> 6909 XUVs
C:0 T:2 34734 XUs and 30857 XVs -> 1142 XUVs
C:0 T:3 19456 XUs and 17408 XVs -> 50 XUVs
C:0 T:4 7006 XUs and 8878 XVs -> 148 XUVs
C:0 T:5 32672 XUs and 86302 XVs -> 7735 XUVs
C:0 T:6 24323 XUs and 29412 XVs -> 798 XUVs
C:0 T:7 12283 XUs and 14331 XVs -> 61 XUVs
C:0 T:8 4507 XUs and 4212 XVs -> 53 XUVs
C:0 T:9 21997 XUs and 27060 XVs -> 1603 XUVs
C:0 T:10 48132 XUs and 34705 XVs -> 19022 XUVs
C:0 T:11 76 XUs and 44 XVs -> 8 XUVs
37875 XUVs total
6042 collection wire objects
36365 potential space points
Neighbour search...
9003895 tests to find 3856046 neighbours
Iterating with no regularization...
Begin: 4.51907e+07
0 3.8917e+07
1 3.84985e+07
2 3.84259e+07
3 3.84023e+07
Now with regularization...
Begin: 3.28549e+07
0 3.28153e+07
1 3.27834e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410607214
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410607214
fLifetime: 30000 [us]
19-Nov-2025 16:13:27 GMT Closed output file "reco_stage1.root"
19-Nov-2025 16:13:27 GMT Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob: # events skipped: 0
====================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
====================================================================================================================================
Full event 545.02 912.723 1339.72 889.765 219.956 10
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00324882 0.0136172 0.0607467 0.00817226 0.0159598 10
reco:ophit:OpHitFinder 0.37294 0.449558 0.520046 0.44116 0.0443075 10
reco:opflash:OpFlashFinder 0.203641 0.253016 0.302648 0.240517 0.0346435 10
reco:opslicer:OpSlicer 0.944432 1.23868 1.61111 1.20346 0.208341 10
reco:caldata:DataPrepModule 9.53433 10.4311 11.5397 10.2921 0.699777 10
reco:wclsdatasp:WireCellToolkit 82.9949 92.6258 104.401 92.4977 6.25143 10
reco:gaushit:GausHitFinder 1.89389 2.53386 3.10889 2.53969 0.369887 10
reco:nhitsfilter:NumberOfHitsFilter 0.00171291 0.00268117 0.00387952 0.00255926 0.000613513 10
reco:reco3d:SpacePointSolver 16.3641 29.292 43.4995 30.2067 9.12914 10
reco:hitpdune:DisambigFromSpacePoints 1.51089 2.98543 4.9507 2.75541 1.0265 10
reco:pandora:StandardPandora 29.7045 75.1348 132.468 72.9471 29.3101 10
reco:pandoraWriter:StandardPandora 0.785693 1.08135 1.42868 1.09291 0.185972 10
reco:pandoraTrack:LArPandoraTrackCreation 7.85212 16.0227 25.3994 17.2457 5.24782 10
reco:pandoraShower:LArPandoraShowerCreation 7.52052 20.2468 39.4246 18.592 9.01251 10
reco:pandoracalo:Calorimetry 4.90733 9.73445 17.8808 9.70516 3.98951 10
reco:pandoracalonosce:Calorimetry 3.84554 9.00543 15.3934 9.35908 3.6309 10
reco:pandorapid:Chi2ParticleID 0.00883656 0.0147508 0.0255737 0.0138741 0.00466421 10
reco:pandoracalinoxyzt:CalibrationdEdXPDSP 0.0827788 0.128648 0.166219 0.1289 0.026495 10
reco:pandoracalipidnoxyzt:Chi2ParticleID 0.00889617 0.0128079 0.0176137 0.0130192 0.00224428 10
reco:pandoraShowercalo:ShowerCalorimetry 7.2833 23.2299 45.4501 21.9003 11.6943 10
reco:pandoraShowercalonosce:ShowerCalorimetry 6.53766 22.2099 44.6194 20.7127 10.3827 10
reco:emtrkmichelid:EmTrackMichelId 261.351 403.903 512.668 406.442 64.9057 10
reco:crttag:SingleCRTMatchingProducer 0.0254971 0.0662052 0.112531 0.0620411 0.0286954 10
reco:crtreco:TwoCRTMatchingProducer 0.0233278 0.0451297 0.0702211 0.0454207 0.0129299 10
reco:anodepiercerst0:T0RecoAnodePiercers 1.02645 1.53128 2.46601 1.43508 0.428866 10
reco:pandora2Track:LArPandoraTrackCreation 25.2351 53.8378 95.075 49.6079 19.6388 10
reco:pandora2calo:Calorimetry 13.6116 31.2916 62.1092 27.3564 14.3052 10
reco:pandora2calonosce:Calorimetry 13.3936 29.7355 58.7242 26.3856 14.4839 10
reco:pandora2pid:Chi2ParticleID 0.0172052 0.0295668 0.0433688 0.029641 0.00859258 10
reco:pandora2calinoxyzt:CalibrationdEdXPDSP 0.07602 0.152852 0.215236 0.147935 0.0413985 10
reco:pandora2calipidnoxyzt:Chi2ParticleID 0.0163311 0.0304241 0.0476601 0.0296986 0.00960729 10
reco:pandora2Shower:LArPandoraShowerCreation 9.34672 19.3137 30.2444 19.2316 6.35784 10
reco:pandora2Showercalo:ShowerCalorimetry 11.9545 26.3372 46.7733 23.7268 11.3493 10
reco:pandora2Showercalonosce:ShowerCalorimetry 9.21951 24.063 41.3289 22.483 10.5665 10
[art]:TriggerResults:TriggerResultInserter 5.2139e-05 0.000103307 0.000316858 7.7396e-05 7.34795e-05 10
end_path:out1:RootOutput 8.846e-06 1.24373e-05 3.0818e-05 9.738e-06 6.36637e-06 10
end_path:out1:RootOutput(write) 3.449 5.27526 7.26909 5.17276 1.13013 10
====================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4871.27 MB
Peak resident set size usage (VmHWM): 4016.07 MB
Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor: Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor: digitReader: 0.25 sec/event
ToolBasedRawDigitPrepService:dtor: adcSampleFiller: 0.53 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleAdcToKe: 0.18 sec/event
ToolBasedRawDigitPrepService:dtor: pdspTailPedRemovalZKe: 6.17 sec/event
ToolBasedRawDigitPrepService:dtor: adcKeepAllSignalFinder: 0.23 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleKeToAdc: 0.17 sec/event
ToolBasedRawDigitPrepService:dtor: pdsp_RemoveBadChannels: 0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9110020
-rw-r--r-- 1 dune01 dune01 2176981108 Nov 19 13:36 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2071885455 Nov 19 13:32 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 1826232282 Nov 19 16:13 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 1623072767 Nov 19 10:10 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1564496733 Nov 19 09:47 g4_stage1.root
-rw-r--r-- 1 dune01 dune01 65363670 Nov 19 16:06 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01 242889 Nov 19 16:13 jobscript.log
-rw-r--r-- 1 dune01 dune01 217088 Nov 19 16:13 mem.db
-rw-r--r-- 1 dune01 dune01 62376 Nov 19 16:13 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01 36864 Nov 19 16:13 time.db
-rw-r--r-- 1 dune01 dune01 519 Nov 19 13:36 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01 519 Nov 19 10:10 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01 519 Nov 19 16:13 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01 52 Nov 19 09:35 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01 7 Nov 19 16:13 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01 0 Nov 19 13:50 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01 0 Nov 19 13:36 debugprod.log
