Jobsub ID 294239.0@dunegpschedd01.fnal.gov
| Jobsub ID | 294239.0@dunegpschedd01.fnal.gov |
| Workflow ID | 12110 |
| Stage ID | 1 |
| User name | pmelas@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 8388608000 (8000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2026-01-20 14:48:55 |
| Site | UK_QMUL |
| Entry | DUNE_UK_London_QMUL_arcce03 |
| Last heartbeat | 2026-01-20 17:45:20 |
| From worker node | Hostname | cn027.htc.esc.qmul |
| cpuinfo | Intel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 10485760000 (10000 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Started | 2026-01-20 14:49:58 |
| Input files | monte-carlo-012110-000001
|
| Jobscript | Exit code | 0 |
| Real time | 2h (10097s) |
| CPU time | 2h (9917s = 98%) |
| Max RSS bytes | 4096843776 (3907 MiB) |
| Outputting started | 2026-01-20 17:38:16 |
| Output files | |
| Finished | 2026-01-20 17:45:20 |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
igitPrepService:prepare: Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 131693 traces, 0 summary
Retagger: tagging trace set: gauss with 102593 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 102593 traces tagged "gauss"
FrameSaver: q=3.55944e+06 n=1832871 tag=gauss
wclsFrameSaver: saving 131693 traces tagged "wiener"
FrameSaver: q=3.76005e+06 n=1630171 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:1 219752 XUs and 53746 XVs -> 18813 XUVs
C:0 T:2 133735 XUs and 146623 XVs -> 12145 XUVs
C:0 T:3 59761 XUs and 45013 XVs -> 532 XUVs
C:0 T:4 7840 XUs and 8293 XVs -> 82 XUVs
C:0 T:5 47511 XUs and 22849 XVs -> 4988 XUVs
C:0 T:6 116035 XUs and 122798 XVs -> 6157 XUVs
C:0 T:7 44418 XUs and 42060 XVs -> 473 XUVs
C:0 T:8 10316 XUs and 10251 XVs -> 121 XUVs
C:0 T:9 56165 XUs and 70508 XVs -> 5165 XUVs
C:0 T:10 68873 XUs and 69944 XVs -> 8746 XUVs
C:0 T:11 146 XUs and 139 XVs -> 15 XUVs
57237 XUVs total
12112 collection wire objects
52468 potential space points
Neighbour search...
9980538 tests to find 4332260 neighbours
Iterating with no regularization...
Begin: 7.18971e+07
0 6.08968e+07
1 6.0275e+07
2 6.01823e+07
3 6.01559e+07
Now with regularization...
Begin: 5.01799e+07
0 5.01378e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 411853252
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411853252
fLifetime: 30000 [us]
20-Jan-2026 17:38:13 GMT Closed output file "reco_stage1.root"
20-Jan-2026 17:38:13 GMT Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob: # events skipped: 0
====================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
====================================================================================================================================
Full event 281.043 425.417 665.136 444.258 108.07 10
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000909323 0.00346015 0.00590709 0.00344641 0.00150399 10
reco:ophit:OpHitFinder 0.230475 0.270674 0.351862 0.259966 0.0355894 10
reco:opflash:OpFlashFinder 0.154505 0.190409 0.233686 0.184297 0.020739 10
reco:opslicer:OpSlicer 0.533774 0.62049 0.860969 0.577145 0.0980849 10
reco:caldata:DataPrepModule 5.46531 5.8669 8.09847 5.58947 0.765238 10
reco:wclsdatasp:WireCellToolkit 45.2817 48.4361 55.965 47.9194 2.80744 10
reco:gaushit:GausHitFinder 0.959421 1.30117 1.61834 1.32171 0.206347 10
reco:nhitsfilter:NumberOfHitsFilter 0.000626412 0.000838031 0.00103437 0.000861665 0.000131896 10
reco:reco3d:SpacePointSolver 7.13459 15.1621 21.1016 15.9982 4.20641 10
reco:hitpdune:DisambigFromSpacePoints 1.18007 1.8975 2.58793 2.01608 0.45135 10
reco:pandora:StandardPandora 18.5641 37.2516 64.6569 35.3579 14.7255 10
reco:pandoraWriter:StandardPandora 0.33432 0.459018 0.640521 0.458903 0.081783 10
reco:pandoraTrack:LArPandoraTrackCreation 3.88245 8.79928 13.9517 9.00056 2.61302 10
reco:pandoraShower:LArPandoraShowerCreation 3.9149 9.68388 15.2604 9.84369 3.66843 10
reco:pandoracalo:Calorimetry 1.72809 3.88726 6.02164 4.17701 1.17835 10
reco:pandoracalonosce:Calorimetry 1.84325 3.58772 5.75224 3.78256 1.066 10
reco:pandorapid:Chi2ParticleID 0.00405639 0.00538357 0.0068335 0.00541481 0.000876793 10
reco:pandoracalinoxyzt:CalibrationdEdXPDSP 0.0485949 0.0615899 0.0769014 0.0624134 0.00926318 10
reco:pandoracalipidnoxyzt:Chi2ParticleID 0.00401606 0.00527364 0.00720656 0.00543002 0.000844356 10
reco:pandoraShowercalo:ShowerCalorimetry 3.08128 8.4097 13.296 9.18632 3.11682 10
reco:pandoraShowercalonosce:ShowerCalorimetry 2.86695 7.7517 10.9647 7.89304 2.6939 10
reco:emtrkmichelid:EmTrackMichelId 130.183 184.241 297.755 181.287 45.4228 10
reco:crttag:SingleCRTMatchingProducer 0.0174979 0.028411 0.0390489 0.0285867 0.00643318 10
reco:crtreco:TwoCRTMatchingProducer 0.0111318 0.0230653 0.0320746 0.0236088 0.00554638 10
reco:anodepiercerst0:T0RecoAnodePiercers 0.795264 1.25611 1.59396 1.32296 0.269799 10
reco:pandora2Track:LArPandoraTrackCreation 13.2114 30.6268 55.4835 31.6444 12.161 10
reco:pandora2calo:Calorimetry 4.92853 12.735 24.5281 13.3257 5.55931 10
reco:pandora2calonosce:Calorimetry 3.68752 12.0907 22.8969 12.4058 5.65247 10
reco:pandora2pid:Chi2ParticleID 0.00865378 0.0145416 0.0259783 0.013034 0.00557109 10
reco:pandora2calinoxyzt:CalibrationdEdXPDSP 0.0581708 0.0835669 0.133531 0.0776832 0.0244983 10
reco:pandora2calipidnoxyzt:Chi2ParticleID 0.00864172 0.0143808 0.02538 0.0130858 0.00538151 10
reco:pandora2Shower:LArPandoraShowerCreation 3.15487 8.32989 15.8691 7.66837 3.65356 10
reco:pandora2Showercalo:ShowerCalorimetry 3.68107 10.4495 18.7183 9.98718 4.6463 10
reco:pandora2Showercalonosce:ShowerCalorimetry 3.19621 8.95454 16.6898 8.28331 4.26415 10
[art]:TriggerResults:TriggerResultInserter 4.1757e-05 5.69067e-05 0.000103354 4.7427e-05 1.76482e-05 10
end_path:out1:RootOutput 6.413e-06 8.3366e-06 1.067e-05 7.7695e-06 1.35765e-06 10
end_path:out1:RootOutput(write) 1.84181 2.75605 4.51397 2.59055 0.869601 10
====================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4909.94 MB
Peak resident set size usage (VmHWM): 4096.84 MB
Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor: Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor: digitReader: 0.09 sec/event
ToolBasedRawDigitPrepService:dtor: adcSampleFiller: 0.23 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleAdcToKe: 0.05 sec/event
ToolBasedRawDigitPrepService:dtor: pdspTailPedRemovalZKe: 3.59 sec/event
ToolBasedRawDigitPrepService:dtor: adcKeepAllSignalFinder: 0.20 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleKeToAdc: 0.05 sec/event
ToolBasedRawDigitPrepService:dtor: pdsp_RemoveBadChannels: 0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9589772
-rw-r--r-- 1 pildune32 pildune 2276186206 Jan 20 16:25 detsim_stage2.root
-rw-r--r-- 1 pildune32 pildune 2163320728 Jan 20 16:23 detsim_stage1.root
-rw-r--r-- 1 pildune32 pildune 1947238372 Jan 20 17:38 reco_stage1.root
-rw-r--r-- 1 pildune32 pildune 1714077566 Jan 20 15:08 g4_stage2.root
-rw-r--r-- 1 pildune32 pildune 1647140905 Jan 20 14:56 g4_stage1.root
-rw-r--r-- 1 pildune32 pildune 71351080 Jan 20 17:32 Pandora_Events.pndr
-rw-r--r-- 1 pildune32 pildune 242803 Jan 20 17:38 jobscript.log
-rw-r--r-- 1 pildune32 pildune 217088 Jan 20 17:38 mem.db
-rw-r--r-- 1 pildune32 pildune 62376 Jan 20 17:38 Pandora_Geometry.xml
-rw-r--r-- 1 pildune32 pildune 36864 Jan 20 17:38 time.db
-rw-r--r-- 1 pildune32 pildune 519 Jan 20 16:25 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune 519 Jan 20 15:08 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune 519 Jan 20 17:38 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune32 pildune 52 Jan 20 14:49 all-input-dids.txt
-rw-r--r-- 1 pildune32 pildune 7 Jan 20 17:38 justin-processed-pfns.txt
-rw-r--r-- 1 pildune32 pildune 0 Jan 20 16:32 ProtoDUNE.log
-rw-r--r-- 1 pildune32 pildune 0 Jan 20 16:25 debugprod.log
