Jobsub ID 279034.0@dunegpschedd01.fnal.gov
| Jobsub ID | 279034.0@dunegpschedd01.fnal.gov |
| Workflow ID | 11959 |
| Stage ID | 1 |
| User name | pmelas@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 8388608000 (8000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2026-01-15 10:55:47 |
| Site | UK_QMUL |
| Entry | DUNE_UK_London_QMUL_arcce03 |
| Last heartbeat | 2026-01-15 18:22:11 |
| From worker node | Hostname | cn538.htc.esc.qmul |
| cpuinfo | Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 10485760000 (10000 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Started | 2026-01-15 10:57:10 |
| Input files | monte-carlo-011959-000001
|
| Jobscript | Exit code | 0 |
| Real time | 6h (23138s) |
| CPU time | 6h (22840s = 98%) |
| Max RSS bytes | 4482379776 (4274 MiB) |
| Outputting started | 2026-01-15 17:22:50 |
| Output files | |
| Finished | 2026-01-15 18:22:11 |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 132007 traces, 0 summary
Retagger: tagging trace set: gauss with 103627 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 103627 traces tagged "gauss"
FrameSaver: q=3.60214e+06 n=1863000 tag=gauss
wclsFrameSaver: saving 132007 traces tagged "wiener"
FrameSaver: q=3.78173e+06 n=1657343 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 27709 XUs and 34135 XVs -> 251 XUVs
C:0 T:1 54033 XUs and 61825 XVs -> 3362 XUVs
C:0 T:2 62369 XUs and 174580 XVs -> 9858 XUVs
C:0 T:3 28337 XUs and 35412 XVs -> 335 XUVs
C:0 T:4 10188 XUs and 14974 XVs -> 106 XUVs
C:0 T:5 39846 XUs and 47655 XVs -> 3351 XUVs
C:0 T:6 101023 XUs and 133404 XVs -> 6335 XUVs
C:0 T:7 41129 XUs and 49233 XVs -> 417 XUVs
C:0 T:8 8048 XUs and 5918 XVs -> 62 XUVs
C:0 T:9 54456 XUs and 47424 XVs -> 3861 XUVs
C:0 T:10 45289 XUs and 39854 XVs -> 5771 XUVs
C:0 T:11 94 XUs and 89 XVs -> 14 XUVs
33723 XUVs total
11542 collection wire objects
31219 potential space points
Neighbour search...
3257037 tests to find 1529026 neighbours
Iterating with no regularization...
Begin: 6.77189e+07
0 5.42072e+07
1 5.33675e+07
2 5.32543e+07
3 5.32201e+07
Now with regularization...
Begin: 4.34955e+07
0 4.34667e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 411754979
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411754979
fLifetime: 30000 [us]
15-Jan-2026 17:22:46 GMT Closed output file "reco_stage1.root"
15-Jan-2026 17:22:46 GMT Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob: # events skipped: 0
====================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
====================================================================================================================================
Full event 867.218 1000.15 1210.48 970.536 110.348 10
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.0018739 0.00544448 0.00925713 0.0057885 0.00214259 10
reco:ophit:OpHitFinder 0.54573 0.612998 0.692075 0.609054 0.0497331 10
reco:opflash:OpFlashFinder 0.348453 0.428064 0.508982 0.432705 0.0513143 10
reco:opslicer:OpSlicer 1.24328 1.45315 1.71941 1.40266 0.14548 10
reco:caldata:DataPrepModule 10.9582 11.6566 12.9302 11.3979 0.597759 10
reco:wclsdatasp:WireCellToolkit 73.8874 81.8789 89.4434 81.1348 4.19946 10
reco:gaushit:GausHitFinder 2.60143 3.03937 3.71973 2.99069 0.339301 10
reco:nhitsfilter:NumberOfHitsFilter 0.00102849 0.00128272 0.00185821 0.00119609 0.000256653 10
reco:reco3d:SpacePointSolver 22.568 48.3858 115.311 35.5431 30.4246 10
reco:hitpdune:DisambigFromSpacePoints 4.67668 5.8916 7.89529 5.30435 1.22597 10
reco:pandora:StandardPandora 72.5146 98.4107 141.722 96.1863 19.6466 10
reco:pandoraWriter:StandardPandora 0.707979 0.998414 1.1715 1.00016 0.127225 10
reco:pandoraTrack:LArPandoraTrackCreation 13.8587 18.8725 25.5698 17.672 4.0833 10
reco:pandoraShower:LArPandoraShowerCreation 16.8773 20.6159 27.8481 18.53 3.64432 10
reco:pandoracalo:Calorimetry 6.61578 8.8949 11.905 8.36691 1.77069 10
reco:pandoracalonosce:Calorimetry 5.93118 8.35207 11.0635 7.85419 1.73547 10
reco:pandorapid:Chi2ParticleID 0.00765565 0.0137396 0.0186854 0.0133569 0.00281576 10
reco:pandoracalinoxyzt:CalibrationdEdXPDSP 0.0952933 0.136534 0.167647 0.133895 0.0198615 10
reco:pandoracalipidnoxyzt:Chi2ParticleID 0.00736635 0.0131686 0.0175873 0.0130775 0.0027447 10
reco:pandoraShowercalo:ShowerCalorimetry 16.5413 20.3814 28.4907 18.8524 3.77017 10
reco:pandoraShowercalonosce:ShowerCalorimetry 15.3697 19.8792 28.1803 18.1307 3.81683 10
reco:emtrkmichelid:EmTrackMichelId 379.997 455.527 541.974 447.474 46.7669 10
reco:crttag:SingleCRTMatchingProducer 0.0389104 0.0476786 0.065045 0.0460681 0.00730019 10
reco:crtreco:TwoCRTMatchingProducer 0.0328056 0.0386886 0.0459064 0.0378941 0.00423498 10
reco:anodepiercerst0:T0RecoAnodePiercers 1.92532 2.75563 3.89791 2.69842 0.492528 10
reco:pandora2Track:LArPandoraTrackCreation 54.2706 65.1597 82.6654 61.7664 10.7205 10
reco:pandora2calo:Calorimetry 23.4466 28.5113 34.9823 27.4149 4.06839 10
reco:pandora2calonosce:Calorimetry 22.124 27.2336 34.5591 26.9842 4.43439 10
reco:pandora2pid:Chi2ParticleID 0.0294443 0.0338677 0.0412608 0.033809 0.00343471 10
reco:pandora2calinoxyzt:CalibrationdEdXPDSP 0.153144 0.1785 0.247036 0.172809 0.0264704 10
reco:pandora2calipidnoxyzt:Chi2ParticleID 0.0285582 0.0334072 0.0474108 0.032105 0.00521162 10
reco:pandora2Shower:LArPandoraShowerCreation 14.3353 19.5666 24.8067 18.8308 3.12994 10
reco:pandora2Showercalo:ShowerCalorimetry 18.6088 23.8279 30.2369 22.913 3.57626 10
reco:pandora2Showercalonosce:ShowerCalorimetry 16.7127 21.1011 27.6817 19.9906 3.75838 10
[art]:TriggerResults:TriggerResultInserter 9.2126e-05 0.000116281 0.000239007 0.000104551 4.14181e-05 10
end_path:out1:RootOutput 1.1436e-05 1.54593e-05 2.3433e-05 1.5422e-05 2.99403e-06 10
end_path:out1:RootOutput(write) 5.1646 5.98995 6.98712 5.84369 0.616782 10
====================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5289.17 MB
Peak resident set size usage (VmHWM): 4482.38 MB
Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor: Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor: digitReader: 0.15 sec/event
ToolBasedRawDigitPrepService:dtor: adcSampleFiller: 0.45 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleAdcToKe: 0.07 sec/event
ToolBasedRawDigitPrepService:dtor: pdspTailPedRemovalZKe: 7.40 sec/event
ToolBasedRawDigitPrepService:dtor: adcKeepAllSignalFinder: 0.41 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleKeToAdc: 0.09 sec/event
ToolBasedRawDigitPrepService:dtor: pdsp_RemoveBadChannels: 0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11305864
-rw-r--r-- 1 pildune32 pildune 2611914878 Jan 15 14:32 detsim_stage2.root
-rw-r--r-- 1 pildune32 pildune 2476153671 Jan 15 14:27 detsim_stage1.root
-rw-r--r-- 1 pildune32 pildune 2347396523 Jan 15 17:22 reco_stage1.root
-rw-r--r-- 1 pildune32 pildune 2067396605 Jan 15 11:35 g4_stage2.root
-rw-r--r-- 1 pildune32 pildune 1987267639 Jan 15 11:08 g4_stage1.root
-rw-r--r-- 1 pildune32 pildune 86458370 Jan 15 17:11 Pandora_Events.pndr
-rw-r--r-- 1 pildune32 pildune 243126 Jan 15 17:22 jobscript.log
-rw-r--r-- 1 pildune32 pildune 217088 Jan 15 17:22 mem.db
-rw-r--r-- 1 pildune32 pildune 62376 Jan 15 17:22 Pandora_Geometry.xml
-rw-r--r-- 1 pildune32 pildune 36864 Jan 15 17:22 time.db
-rw-r--r-- 1 pildune32 pildune 519 Jan 15 14:32 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune 519 Jan 15 11:35 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune 519 Jan 15 17:22 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune32 pildune 52 Jan 15 10:57 all-input-dids.txt
-rw-r--r-- 1 pildune32 pildune 7 Jan 15 17:22 justin-processed-pfns.txt
-rw-r--r-- 1 pildune32 pildune 0 Jan 15 14:46 ProtoDUNE.log
-rw-r--r-- 1 pildune32 pildune 0 Jan 15 14:32 debugprod.log
