Jobsub ID 254067.0@dunegpschedd01.fnal.gov
| Jobsub ID | 254067.0@dunegpschedd01.fnal.gov |
| Workflow ID | 10375 |
| Stage ID | 1 |
| User name | pmelas@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 8388608000 (8000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2025-11-19 09:33:45 |
| Site | UK_Manchester |
| Entry | UBoone_T2_UK_Manchester_ce01 |
| Last heartbeat | 2025-11-19 16:05:11 |
| From worker node | Hostname | wn1205251.tier2.hep.manchester.ac.uk |
| cpuinfo | AMD EPYC 7702P 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 10485760000 (10000 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Started | 2025-11-19 09:34:42 |
| Input files | monte-carlo-010375-000001
|
| Jobscript | Exit code | 0 |
| Real time | 6h (23008s) |
| CPU time | 6h (22144s = 96%) |
| Max RSS bytes | 3950358528 (3767 MiB) |
| Outputting started | 2025-11-19 15:58:11 |
| Output files | |
| Finished | 2025-11-19 16:05:11 |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
l adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare: Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare: Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 123881 traces, 0 summary
Retagger: tagging trace set: gauss with 96994 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 96994 traces tagged "gauss"
FrameSaver: q=3.33489e+06 n=1718527 tag=gauss
wclsFrameSaver: saving 123881 traces tagged "wiener"
FrameSaver: q=3.52378e+06 n=1530781 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 23251 XUs and 21536 XVs -> 341 XUVs
C:0 T:1 62455 XUs and 51863 XVs -> 6529 XUVs
C:0 T:2 82947 XUs and 96429 XVs -> 6398 XUVs
C:0 T:3 35608 XUs and 35727 XVs -> 496 XUVs
C:0 T:4 11625 XUs and 11540 XVs -> 150 XUVs
C:0 T:5 52483 XUs and 47144 XVs -> 6585 XUVs
C:0 T:6 140840 XUs and 104819 XVs -> 13609 XUVs
C:0 T:7 49349 XUs and 37797 XVs -> 543 XUVs
C:0 T:8 10359 XUs and 8907 XVs -> 173 XUVs
C:0 T:9 95845 XUs and 81607 XVs -> 16206 XUVs
C:0 T:10 23985 XUs and 11776 XVs -> 2088 XUVs
C:0 T:11 252 XUs and 153 XVs -> 34 XUVs
53152 XUVs total
10750 collection wire objects
48875 potential space points
Neighbour search...
12051651 tests to find 4921798 neighbours
Iterating with no regularization...
Begin: 5.56583e+07
0 4.80308e+07
1 4.74282e+07
2 4.7341e+07
3 4.73159e+07
Now with regularization...
Begin: 3.8578e+07
0 3.8544e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410607200
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410607200
fLifetime: 30000 [us]
19-Nov-2025 15:58:07 GMT Closed output file "reco_stage1.root"
19-Nov-2025 15:58:07 GMT Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob: # events skipped: 0
====================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
====================================================================================================================================
Full event 535.508 812.992 1061.88 810.55 161.319 10
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00251639 0.00682083 0.0120219 0.00570891 0.00268749 10
reco:ophit:OpHitFinder 0.303007 0.469016 0.567287 0.481535 0.0740733 10
reco:opflash:OpFlashFinder 0.171337 0.269392 0.320208 0.276895 0.0468193 10
reco:opslicer:OpSlicer 0.723944 1.27874 1.63264 1.35432 0.268259 10
reco:caldata:DataPrepModule 6.19884 9.59951 11.0793 9.64961 1.34516 10
reco:wclsdatasp:WireCellToolkit 49.0293 76.2163 94.8034 78.2609 12.7328 10
reco:gaushit:GausHitFinder 1.42681 2.2164 3.18022 2.26567 0.516777 10
reco:nhitsfilter:NumberOfHitsFilter 0.000745537 0.0017052 0.00268024 0.00173308 0.000614427 10
reco:reco3d:SpacePointSolver 12.6079 28.3662 51.8858 28.718 12.1025 10
reco:hitpdune:DisambigFromSpacePoints 1.60471 2.83909 4.17826 2.86022 0.854228 10
reco:pandora:StandardPandora 29.6664 58.5513 95.53 59.2634 23.3616 10
reco:pandoraWriter:StandardPandora 0.504242 0.92069 1.29447 0.981229 0.265537 10
reco:pandoraTrack:LArPandoraTrackCreation 6.65102 12.8044 18.6704 13.8489 3.8008 10
reco:pandoraShower:LArPandoraShowerCreation 6.43019 15.2274 23.1252 16.1195 5.03175 10
reco:pandoracalo:Calorimetry 3.68083 7.70697 11.0985 7.56314 2.5007 10
reco:pandoracalonosce:Calorimetry 3.01901 7.6669 12.375 7.571 2.70823 10
reco:pandorapid:Chi2ParticleID 0.00747898 0.0123789 0.0181265 0.0123607 0.0032867 10
reco:pandoracalinoxyzt:CalibrationdEdXPDSP 0.068725 0.117485 0.222122 0.103181 0.0424403 10
reco:pandoracalipidnoxyzt:Chi2ParticleID 0.00828573 0.013638 0.0271471 0.0121032 0.00523347 10
reco:pandoraShowercalo:ShowerCalorimetry 6.70253 17.8325 28.799 18.1774 6.8862 10
reco:pandoraShowercalonosce:ShowerCalorimetry 7.30052 17.0854 26.1387 16.4164 5.98621 10
reco:emtrkmichelid:EmTrackMichelId 317.617 400.789 497.211 386.833 50.9496 10
reco:crttag:SingleCRTMatchingProducer 0.0219051 0.046627 0.0975238 0.0417001 0.0248098 10
reco:crtreco:TwoCRTMatchingProducer 0.0165875 0.0353971 0.0646883 0.0342044 0.0141316 10
reco:anodepiercerst0:T0RecoAnodePiercers 1.25359 1.52128 1.958 1.44527 0.216802 10
reco:pandora2Track:LArPandoraTrackCreation 26.5454 42.7897 53.4445 48.0532 10.2016 10
reco:pandora2calo:Calorimetry 9.77877 25.0375 35.9456 29.5285 8.95972 10
reco:pandora2calonosce:Calorimetry 8.53116 23.139 32.9284 27.2419 8.44229 10
reco:pandora2pid:Chi2ParticleID 0.0168109 0.0261329 0.0356712 0.027663 0.00635557 10
reco:pandora2calinoxyzt:CalibrationdEdXPDSP 0.0804231 0.140324 0.200242 0.153749 0.0402861 10
reco:pandora2calipidnoxyzt:Chi2ParticleID 0.0166847 0.0275214 0.0432556 0.0289436 0.00784417 10
reco:pandora2Shower:LArPandoraShowerCreation 6.28338 14.6878 19.4313 16.2767 4.0826 10
reco:pandora2Showercalo:ShowerCalorimetry 6.80594 20.9065 28.7259 22.8876 7.62397 10
reco:pandora2Showercalonosce:ShowerCalorimetry 5.82381 19.4144 32.8794 20.9471 7.87245 10
[art]:TriggerResults:TriggerResultInserter 3.5116e-05 9.02302e-05 0.000268036 5.859e-05 7.02213e-05 10
end_path:out1:RootOutput 6.042e-06 1.03777e-05 2.3755e-05 9.368e-06 4.65542e-06 10
end_path:out1:RootOutput(write) 3.37994 4.89367 5.7379 5.16378 0.739023 10
====================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4795.92 MB
Peak resident set size usage (VmHWM): 3950.36 MB
Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor: Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor: digitReader: 0.15 sec/event
ToolBasedRawDigitPrepService:dtor: adcSampleFiller: 0.43 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleAdcToKe: 0.12 sec/event
ToolBasedRawDigitPrepService:dtor: pdspTailPedRemovalZKe: 5.97 sec/event
ToolBasedRawDigitPrepService:dtor: adcKeepAllSignalFinder: 0.23 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleKeToAdc: 0.11 sec/event
ToolBasedRawDigitPrepService:dtor: pdsp_RemoveBadChannels: 0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9457024
-rw-r--r-- 1 dune01 dune01 2246184624 Nov 19 13:38 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2128813973 Nov 19 13:34 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 1914664990 Nov 19 15:58 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 1693308923 Nov 19 10:07 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1630137969 Nov 19 09:45 g4_stage1.root
-rw-r--r-- 1 dune01 dune01 70254820 Nov 19 15:47 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01 243042 Nov 19 15:58 jobscript.log
-rw-r--r-- 1 dune01 dune01 217088 Nov 19 15:58 mem.db
-rw-r--r-- 1 dune01 dune01 62376 Nov 19 15:58 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01 36864 Nov 19 15:58 time.db
-rw-r--r-- 1 dune01 dune01 519 Nov 19 13:38 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01 519 Nov 19 10:07 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01 519 Nov 19 15:58 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01 52 Nov 19 09:34 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01 7 Nov 19 15:58 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01 0 Nov 19 13:50 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01 0 Nov 19 13:38 debugprod.log
