Jobsub ID 263931.0@dunegpschedd02.fnal.gov
| Jobsub ID | 263931.0@dunegpschedd02.fnal.gov |
| Workflow ID | 11312 |
| Stage ID | 1 |
| User name | pmelas@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 8388608000 (8000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2025-12-23 08:25:34 |
| Site | NL_SURFsara |
| Entry | DUNE_SurfSARA_arc03 |
| Last heartbeat | 2025-12-23 12:17:15 |
| From worker node | Hostname | wn-la-20.gina.surf.nl |
| cpuinfo | AMD EPYC 9754 128-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 10485760000 (10000 MiB) |
| Wall seconds limit | 129600 (36 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Started | 2025-12-23 08:26:57 |
| Input files | monte-carlo-011312-000001
|
| Jobscript | Exit code | 0 |
| Real time | 3h (13774s) |
| CPU time | 3h (13658s = 99%) |
| Max RSS bytes | 4844756992 (4620 MiB) |
| Outputting started | 2025-12-23 12:16:32 |
| Output files | |
| Finished | 2025-12-23 12:17:15 |
| List job events (HTCondor job logs unavailable) |
Jobscript log (last 10,000 characters)
ol adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare: Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare: Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 149159 traces, 0 summary
Retagger: tagging trace set: gauss with 111980 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 111980 traces tagged "gauss"
FrameSaver: q=3.77233e+06 n=1946904 tag=gauss
wclsFrameSaver: saving 149159 traces tagged "wiener"
FrameSaver: q=3.99745e+06 n=1703205 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 37156 XUs and 41607 XVs -> 505 XUVs
C:0 T:1 76075 XUs and 72533 XVs -> 6380 XUVs
C:0 T:2 79620 XUs and 96569 XVs -> 3810 XUVs
C:0 T:3 37227 XUs and 36661 XVs -> 432 XUVs
C:0 T:4 11252 XUs and 13654 XVs -> 101 XUVs
C:0 T:5 49230 XUs and 42406 XVs -> 3427 XUVs
C:0 T:6 171173 XUs and 186164 XVs -> 13071 XUVs
C:0 T:7 52007 XUs and 61140 XVs -> 601 XUVs
C:0 T:8 6421 XUs and 6807 XVs -> 104 XUVs
C:0 T:9 40804 XUs and 52323 XVs -> 2794 XUVs
C:0 T:10 52404 XUs and 33016 XVs -> 6484 XUVs
C:0 T:11 308 XUs and 154 XVs -> 25 XUVs
37734 XUVs total
13946 collection wire objects
34819 potential space points
Neighbour search...
3548913 tests to find 1650788 neighbours
Iterating with no regularization...
Begin: 7.12026e+07
0 5.86795e+07
1 5.78815e+07
2 5.777e+07
3 5.77381e+07
Now with regularization...
Begin: 4.73701e+07
0 4.7325e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411290249
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411290249
fLifetime: 30000 [us]
23-Dec-2025 13:16:30 CET Closed output file "reco_stage1.root"
23-Dec-2025 13:16:30 CET Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob: # events skipped: 0
====================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
====================================================================================================================================
Full event 233.642 380.225 510.035 376.894 74.3011 10
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000936571 0.00525406 0.00901319 0.00520769 0.00262752 10
reco:ophit:OpHitFinder 0.286513 0.352432 0.438906 0.344716 0.0383053 10
reco:opflash:OpFlashFinder 0.171713 0.272291 0.444129 0.252377 0.0753983 10
reco:opslicer:OpSlicer 0.587894 0.695371 0.898421 0.673582 0.0874554 10
reco:caldata:DataPrepModule 4.69035 4.92004 5.21288 4.91474 0.129582 10
reco:wclsdatasp:WireCellToolkit 33.7234 41.4543 46.5642 42.1075 4.3358 10
reco:gaushit:GausHitFinder 0.768894 1.17804 1.45605 1.14969 0.200324 10
reco:nhitsfilter:NumberOfHitsFilter 0.000273439 0.000586965 0.00114789 0.000464345 0.000250345 10
reco:reco3d:SpacePointSolver 12.5939 18.8736 27.257 19.2917 4.3491 10
reco:hitpdune:DisambigFromSpacePoints 0.777547 1.69021 2.56358 1.56559 0.527922 10
reco:pandora:StandardPandora 15.7573 37.4152 61.7953 37.0661 11.5772 10
reco:pandoraWriter:StandardPandora 0.306889 0.46806 0.582493 0.478167 0.0754966 10
reco:pandoraTrack:LArPandoraTrackCreation 3.4275 7.46415 10.9499 7.2455 2.02212 10
reco:pandoraShower:LArPandoraShowerCreation 3.30338 8.95169 14.761 9.00247 2.99193 10
reco:pandoracalo:Calorimetry 1.72398 3.77929 5.72022 3.73881 0.983945 10
reco:pandoracalonosce:Calorimetry 1.52838 3.56947 5.12287 3.57992 0.911992 10
reco:pandorapid:Chi2ParticleID 0.00409149 0.00559852 0.00837653 0.00537586 0.00106816 10
reco:pandoracalinoxyzt:CalibrationdEdXPDSP 0.0397672 0.0474354 0.05607 0.0459402 0.00522274 10
reco:pandoracalipidnoxyzt:Chi2ParticleID 0.00391994 0.00534734 0.00605143 0.00548875 0.000609308 10
reco:pandoraShowercalo:ShowerCalorimetry 2.75709 7.95198 13.7728 7.80985 2.78722 10
reco:pandoraShowercalonosce:ShowerCalorimetry 2.57538 7.76709 13.4802 7.88694 2.79899 10
reco:emtrkmichelid:EmTrackMichelId 112.246 169.634 209.403 170.004 27.521 10
reco:crttag:SingleCRTMatchingProducer 0.00892954 0.0152294 0.0201146 0.0150056 0.00286272 10
reco:crtreco:TwoCRTMatchingProducer 0.0074076 0.013255 0.0176937 0.0139961 0.00344877 10
reco:anodepiercerst0:T0RecoAnodePiercers 0.541055 0.881301 1.04515 0.902132 0.1413 10
reco:pandora2Track:LArPandoraTrackCreation 9.12234 20.6374 31.8012 20.8938 6.49277 10
reco:pandora2calo:Calorimetry 3.90872 9.68468 14.4864 10.1346 2.97172 10
reco:pandora2calonosce:Calorimetry 3.65616 9.12261 13.3288 9.14896 2.82759 10
reco:pandora2pid:Chi2ParticleID 0.00856856 0.0124014 0.015253 0.0121632 0.00194764 10
reco:pandora2calinoxyzt:CalibrationdEdXPDSP 0.0437157 0.0605916 0.0744708 0.0592818 0.00954612 10
reco:pandora2calipidnoxyzt:Chi2ParticleID 0.00905415 0.0123717 0.0153132 0.0122257 0.00171277 10
reco:pandora2Shower:LArPandoraShowerCreation 3.38267 6.65811 9.72597 7.04609 1.9258 10
reco:pandora2Showercalo:ShowerCalorimetry 3.37658 7.31943 10.7592 7.46697 2.1471 10
reco:pandora2Showercalonosce:ShowerCalorimetry 2.83466 6.16962 9.22427 6.55509 1.8379 10
[art]:TriggerResults:TriggerResultInserter 2.8332e-05 4.31646e-05 0.000101562 3.6295e-05 2.05317e-05 10
end_path:out1:RootOutput 7.02e-06 8.2394e-06 1.1778e-05 7.7815e-06 1.35061e-06 10
end_path:out1:RootOutput(write) 1.93506 2.90836 3.57323 2.89697 0.450191 10
====================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5435.63 MB
Peak resident set size usage (VmHWM): 4634.6 MB
Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor: Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor: digitReader: 0.06 sec/event
ToolBasedRawDigitPrepService:dtor: adcSampleFiller: 0.22 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleAdcToKe: 0.03 sec/event
ToolBasedRawDigitPrepService:dtor: pdspTailPedRemovalZKe: 3.05 sec/event
ToolBasedRawDigitPrepService:dtor: adcKeepAllSignalFinder: 0.12 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleKeToAdc: 0.03 sec/event
ToolBasedRawDigitPrepService:dtor: pdsp_RemoveBadChannels: 0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11160488
-rw-r--r--. 1 dune009 dune 2579346912 Dec 23 12:11 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2443345535 Dec 23 12:09 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 2305373162 Dec 23 13:16 reco_stage1.root
-rw-r--r--. 1 dune009 dune 2038910166 Dec 23 09:47 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1976335638 Dec 23 09:32 g4_stage1.root
-rw-r--r--. 1 dune009 dune 84417980 Dec 23 13:11 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune 243169 Dec 23 13:16 jobscript.log
-rw-r--r--. 1 dune009 dune 217088 Dec 23 13:16 mem.db
-rw-r--r--. 1 dune009 dune 62376 Dec 23 13:16 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune 36864 Dec 23 13:16 time.db
-rw-r--r--. 1 dune009 dune 519 Dec 23 12:11 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune 519 Dec 23 09:47 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune 519 Dec 23 13:16 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune 52 Dec 23 09:26 all-input-dids.txt
-rw-r--r--. 1 dune009 dune 7 Dec 23 13:16 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune 0 Dec 23 12:16 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune 0 Dec 23 12:11 debugprod.log
