Jobsub ID 273835.0@dunegpschedd02.fnal.gov
| Jobsub ID | 273835.0@dunegpschedd02.fnal.gov |
| Workflow ID | 11989 |
| Stage ID | 1 |
| User name | pmelas@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 8388608000 (8000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2026-01-16 21:23:36 |
| Site | NL_SURFsara |
| Entry | DUNE_SurfSARA_arc03 |
| Last heartbeat | 2026-01-17 02:07:26 |
| From worker node | Hostname | wn-la-06.gina.surf.nl |
| cpuinfo | AMD EPYC 9754 128-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 10485760000 (10000 MiB) |
| Wall seconds limit | 129600 (36 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2026-01-16 21:24:36 |
| Input files | monte-carlo-011989-000001
|
| Jobscript | Exit code | 0 |
| Real time | 4h (16864s) |
| CPU time | 4h (16754s = 99%) |
| Max RSS bytes | 4522328064 (4312 MiB) |
| Outputting started | 2026-01-17 02:05:41 |
| Output files | https://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11989/1/001/reco_stage1.root
|
| Finished | 2026-01-17 02:07:26 |
| Saved logs | justin-logs:273835.0-dunegpschedd02.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
lBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 141815 traces, 0 summary
Retagger: tagging trace set: gauss with 106327 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 106327 traces tagged "gauss"
FrameSaver: q=3.65262e+06 n=1839271 tag=gauss
wclsFrameSaver: saving 141815 traces tagged "wiener"
FrameSaver: q=3.8758e+06 n=1606580 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 28005 XUs and 24104 XVs -> 264 XUVs
C:0 T:1 45748 XUs and 44720 XVs -> 2733 XUVs
C:0 T:2 153143 XUs and 138375 XVs -> 9295 XUVs
C:0 T:3 65291 XUs and 53597 XVs -> 546 XUVs
C:0 T:4 12556 XUs and 15384 XVs -> 131 XUVs
C:0 T:5 40908 XUs and 40915 XVs -> 3702 XUVs
C:0 T:6 139553 XUs and 100058 XVs -> 7171 XUVs
C:0 T:7 44192 XUs and 34211 XVs -> 369 XUVs
C:0 T:8 6680 XUs and 9791 XVs -> 245 XUVs
C:0 T:9 37873 XUs and 59688 XVs -> 3585 XUVs
C:0 T:10 88932 XUs and 52062 XVs -> 12063 XUVs
C:0 T:11 106 XUs and 85 XVs -> 12 XUVs
40116 XUVs total
13331 collection wire objects
36965 potential space points
Neighbour search...
3743347 tests to find 1711286 neighbours
Iterating with no regularization...
Begin: 6.55773e+07
0 5.46037e+07
1 5.36864e+07
2 5.35334e+07
3 5.34863e+07
Now with regularization...
Begin: 4.23728e+07
0 4.23194e+07
1 4.22879e+07
Failure in algorithm Alg0061, LArCosmicRayTrackRecovery, STATUS_CODE_FAILURE
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 411783880
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411783880
fLifetime: 30000 [us]
17-Jan-2026 03:05:39 CET Closed output file "reco_stage1.root"
17-Jan-2026 03:05:39 CET Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob: # events skipped: 0
====================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
====================================================================================================================================
Full event 337.21 505.612 785.961 486.219 115.941 10
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00114481 0.00528462 0.00854226 0.00559009 0.00248905 10
reco:ophit:OpHitFinder 0.394081 0.451693 0.580779 0.434694 0.0581366 10
reco:opflash:OpFlashFinder 0.255607 0.344161 0.485193 0.329929 0.0711904 10
reco:opslicer:OpSlicer 0.719545 0.92367 1.18667 0.917571 0.138188 10
reco:caldata:DataPrepModule 6.24932 6.45923 7.19082 6.41211 0.253179 10
reco:wclsdatasp:WireCellToolkit 44.0568 62.4695 72.8642 68.6206 11.3222 10
reco:gaushit:GausHitFinder 1.00123 1.45078 1.99551 1.38744 0.280077 10
reco:nhitsfilter:NumberOfHitsFilter 0.000397936 0.000730624 0.00153952 0.000580829 0.000325181 10
reco:reco3d:SpacePointSolver 12.0517 21.46 56.3932 17.0691 12.1834 10
reco:hitpdune:DisambigFromSpacePoints 1.10309 2.03735 3.54357 1.83151 0.713606 10
reco:pandora:StandardPandora 20.5557 47.9826 101.426 42.042 22.1432 10
reco:pandoraWriter:StandardPandora 0.391995 0.536229 0.822427 0.534872 0.113728 10
reco:pandoraTrack:LArPandoraTrackCreation 5.13953 8.57824 20.2031 6.96771 4.10688 10
reco:pandoraShower:LArPandoraShowerCreation 4.06447 10.3999 27.8653 7.96647 6.51115 10
reco:pandoracalo:Calorimetry 2.49442 4.446 10.8521 3.51209 2.28928 10
reco:pandoracalonosce:Calorimetry 2.29584 4.22476 10.2041 3.28831 2.16566 10
reco:pandorapid:Chi2ParticleID 0.00573254 0.00684576 0.0120368 0.0061993 0.00177297 10
reco:pandoracalinoxyzt:CalibrationdEdXPDSP 0.0442988 0.0587526 0.0844969 0.0582127 0.0107522 10
reco:pandoracalipidnoxyzt:Chi2ParticleID 0.0054705 0.00655882 0.00957635 0.00614423 0.00110095 10
reco:pandoraShowercalo:ShowerCalorimetry 3.04757 10.0733 26.9107 7.81415 6.36806 10
reco:pandoraShowercalonosce:ShowerCalorimetry 3.40896 9.6125 25.0036 7.47569 5.80828 10
reco:emtrkmichelid:EmTrackMichelId 171.129 236.689 331.152 228.677 43.1149 10
reco:crttag:SingleCRTMatchingProducer 0.0150654 0.020206 0.0280379 0.0194463 0.00444196 10
reco:crtreco:TwoCRTMatchingProducer 0.00839489 0.0154579 0.0250988 0.0143127 0.00457394 10
reco:anodepiercerst0:T0RecoAnodePiercers 0.828765 1.10845 1.47441 1.06663 0.185089 10
reco:pandora2Track:LArPandoraTrackCreation 9.17721 24.4465 38.4596 22.6962 8.27167 10
reco:pandora2calo:Calorimetry 4.10731 11.4232 16.6877 10.7142 3.891 10
reco:pandora2calonosce:Calorimetry 3.74515 10.778 15.3749 9.91883 3.68267 10
reco:pandora2pid:Chi2ParticleID 0.0109079 0.015336 0.022311 0.0149511 0.00318694 10
reco:pandora2calinoxyzt:CalibrationdEdXPDSP 0.0543012 0.075301 0.102796 0.0751252 0.0137377 10
reco:pandora2calipidnoxyzt:Chi2ParticleID 0.0107973 0.0154886 0.0225646 0.0150274 0.00325302 10
reco:pandora2Shower:LArPandoraShowerCreation 3.40191 8.43576 13.39 7.83273 2.8589 10
reco:pandora2Showercalo:ShowerCalorimetry 3.39385 9.16374 13.0618 9.09005 2.84228 10
reco:pandora2Showercalonosce:ShowerCalorimetry 3.08663 7.9741 11.2922 8.00949 2.35255 10
[art]:TriggerResults:TriggerResultInserter 3.6704e-05 5.88363e-05 0.000118188 4.7907e-05 2.33082e-05 10
end_path:out1:RootOutput 7.251e-06 8.5646e-06 1.349e-05 8.0215e-06 1.6912e-06 10
end_path:out1:RootOutput(write) 2.65259 3.67236 5.35563 3.54167 0.720744 10
====================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5327.22 MB
Peak resident set size usage (VmHWM): 4522.33 MB
Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor: Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor: digitReader: 0.07 sec/event
ToolBasedRawDigitPrepService:dtor: adcSampleFiller: 0.29 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleAdcToKe: 0.04 sec/event
ToolBasedRawDigitPrepService:dtor: pdspTailPedRemovalZKe: 4.03 sec/event
ToolBasedRawDigitPrepService:dtor: adcKeepAllSignalFinder: 0.17 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleKeToAdc: 0.04 sec/event
ToolBasedRawDigitPrepService:dtor: pdsp_RemoveBadChannels: 0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10999420
-rw-r--r--. 1 dune009 dune 2547655075 Jan 17 01:39 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2412637738 Jan 17 01:36 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 2255553334 Jan 17 03:05 reco_stage1.root
-rw-r--r--. 1 dune009 dune 2030688003 Jan 16 22:49 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1936129904 Jan 16 22:32 g4_stage1.root
-rw-r--r--. 1 dune009 dune 80126880 Jan 17 02:59 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune 243146 Jan 17 03:05 jobscript.log
-rw-r--r--. 1 dune009 dune 217088 Jan 17 03:05 mem.db
-rw-r--r--. 1 dune009 dune 62376 Jan 17 03:05 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune 36864 Jan 17 03:05 time.db
-rw-r--r--. 1 dune009 dune 519 Jan 17 01:39 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune 519 Jan 16 22:49 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune 519 Jan 17 03:05 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune 52 Jan 16 22:24 all-input-dids.txt
-rw-r--r--. 1 dune009 dune 7 Jan 17 03:05 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune 0 Jan 17 01:51 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune 0 Jan 17 01:39 debugprod.log
