Jobsub ID 257419.1@dunegpschedd01.fnal.gov
| Jobsub ID | 257419.1@dunegpschedd01.fnal.gov |
| Workflow ID | 10686 |
| Stage ID | 1 |
| User name | pmelas@fnal.gov |
| HTCondor Group | group_dune |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 8388608000 (8000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2025-11-25 15:08:27 |
| Site | NL_SURFsara |
| Entry | DUNE_SurfSARA_arc01 |
| Last heartbeat | 2025-11-25 19:05:09 |
| From worker node | Hostname | wn-lb-14.gina.surf.nl |
| cpuinfo | AMD EPYC 9754 128-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 10485760000 (10000 MiB) |
| Wall seconds limit | 129600 (36 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Started | 2025-11-25 15:09:37 |
| Input files | monte-carlo-010686-000001
|
| Jobscript | Exit code | 0 |
| Real time | 3h (14080s) |
| CPU time | 3h (13905s = 98%) |
| Max RSS bytes | 4236115968 (4039 MiB) |
| Outputting started | 2025-11-25 19:04:18 |
| Output files | |
| Finished | 2025-11-25 19:05:09 |
| List job events (HTCondor job logs unavailable) |
Jobscript log (last 10,000 characters)
awDigitPrepService:prepare: Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 135686 traces, 0 summary
Retagger: tagging trace set: gauss with 99244 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 99244 traces tagged "gauss"
FrameSaver: q=3.47696e+06 n=1685155 tag=gauss
wclsFrameSaver: saving 135686 traces tagged "wiener"
FrameSaver: q=3.70138e+06 n=1456927 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 38152 XUs and 38232 XVs -> 952 XUVs
C:0 T:1 185942 XUs and 203161 XVs -> 35032 XUVs
C:0 T:2 42988 XUs and 56984 XVs -> 1837 XUVs
C:0 T:3 23727 XUs and 26305 XVs -> 121 XUVs
C:0 T:4 11610 XUs and 10820 XVs -> 195 XUVs
C:0 T:5 154028 XUs and 134883 XVs -> 40223 XUVs
C:0 T:6 81536 XUs and 94657 XVs -> 3845 XUVs
C:0 T:7 35762 XUs and 39888 XVs -> 313 XUVs
C:0 T:8 6991 XUs and 6210 XVs -> 69 XUVs
C:0 T:9 164779 XUs and 127674 XVs -> 41023 XUVs
C:0 T:10 42442 XUs and 24001 XVs -> 5587 XUVs
C:0 T:11 59 XUs and 81 XVs -> 12 XUVs
129209 XUVs total
12884 collection wire objects
120810 potential space points
Neighbour search...
40018984 tests to find 15541646 neighbours
Iterating with no regularization...
Begin: 7.70194e+07
0 5.53925e+07
1 5.43903e+07
2 5.42316e+07
3 5.41767e+07
4 5.41474e+07
Now with regularization...
Begin: 4.40508e+07
0 4.39582e+07
1 4.38701e+07
2 4.37919e+07
3 4.37217e+07
4 4.36611e+07
5 4.36124e+07
6 4.3573e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410732551
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410732551
fLifetime: 30000 [us]
25-Nov-2025 20:04:15 CET Closed output file "reco_stage1.root"
25-Nov-2025 20:04:15 CET Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob: # events skipped: 0
====================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
====================================================================================================================================
Full event 267.864 361.421 452.927 360.739 55.6865 10
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00131788 0.00553285 0.0106528 0.00548901 0.00268823 10
reco:ophit:OpHitFinder 0.324955 0.386735 0.43993 0.38817 0.0280062 10
reco:opflash:OpFlashFinder 0.181579 0.236681 0.306201 0.231543 0.0371146 10
reco:opslicer:OpSlicer 0.638048 0.802025 0.967974 0.809714 0.103235 10
reco:caldata:DataPrepModule 6.15399 6.44963 7.04272 6.45024 0.240492 10
reco:wclsdatasp:WireCellToolkit 42.2083 56.3847 63.4724 59.7392 6.66451 10
reco:gaushit:GausHitFinder 0.907789 1.18628 1.42892 1.19583 0.162607 10
reco:nhitsfilter:NumberOfHitsFilter 0.000323744 0.000537763 0.000963193 0.000430595 0.000198293 10
reco:reco3d:SpacePointSolver 7.67843 20.4601 48.7114 14.2059 13.0503 10
reco:hitpdune:DisambigFromSpacePoints 0.96745 1.41587 1.81988 1.52068 0.274847 10
reco:pandora:StandardPandora 13.326 27.9298 37.9774 27.7494 7.7772 10
reco:pandoraWriter:StandardPandora 0.342294 0.436315 0.527387 0.430073 0.0592132 10
reco:pandoraTrack:LArPandoraTrackCreation 2.96795 4.92781 6.95018 5.01472 1.22905 10
reco:pandoraShower:LArPandoraShowerCreation 2.67638 5.20031 7.16525 5.54209 1.48597 10
reco:pandoracalo:Calorimetry 1.57147 2.66102 3.7831 2.67077 0.683659 10
reco:pandoracalonosce:Calorimetry 1.43432 2.50468 3.8052 2.41174 0.706695 10
reco:pandorapid:Chi2ParticleID 0.00462426 0.00560613 0.00730882 0.00534591 0.000747866 10
reco:pandoracalinoxyzt:CalibrationdEdXPDSP 0.0390704 0.0478207 0.0624216 0.0478106 0.00619439 10
reco:pandoracalipidnoxyzt:Chi2ParticleID 0.00502157 0.00545433 0.00622844 0.00532241 0.000379254 10
reco:pandoraShowercalo:ShowerCalorimetry 2.30966 5.17948 7.79553 5.7745 1.79231 10
reco:pandoraShowercalonosce:ShowerCalorimetry 2.20759 5.02272 7.4736 5.44238 1.82548 10
reco:emtrkmichelid:EmTrackMichelId 141.528 172.708 199.48 173.755 18.6681 10
reco:crttag:SingleCRTMatchingProducer 0.00905243 0.0179914 0.0281664 0.0180571 0.00498118 10
reco:crtreco:TwoCRTMatchingProducer 0.00831653 0.0111811 0.0138509 0.0109315 0.00166383 10
reco:anodepiercerst0:T0RecoAnodePiercers 0.661443 0.906556 1.17915 0.864526 0.169984 10
reco:pandora2Track:LArPandoraTrackCreation 7.64441 13.8604 19.3487 14.8465 3.3007 10
reco:pandora2calo:Calorimetry 3.68707 6.888 10.2265 7.56373 1.78051 10
reco:pandora2calonosce:Calorimetry 3.66469 6.51647 9.85703 7.07097 1.74785 10
reco:pandora2pid:Chi2ParticleID 0.0104041 0.0125479 0.0153962 0.0125345 0.00146057 10
reco:pandora2calinoxyzt:CalibrationdEdXPDSP 0.0493708 0.0604768 0.0720148 0.0602961 0.00736931 10
reco:pandora2calipidnoxyzt:Chi2ParticleID 0.0100396 0.0122976 0.0143453 0.0125292 0.0013604 10
reco:pandora2Shower:LArPandoraShowerCreation 2.93729 5.04296 7.49093 5.29172 1.24877 10
reco:pandora2Showercalo:ShowerCalorimetry 3.00675 5.73865 8.90946 6.16983 1.65347 10
reco:pandora2Showercalonosce:ShowerCalorimetry 2.6995 5.18805 7.99681 5.36038 1.49563 10
[art]:TriggerResults:TriggerResultInserter 4.015e-05 5.6651e-05 0.000132067 4.42265e-05 2.71163e-05 10
end_path:out1:RootOutput 7.421e-06 8.8582e-06 1.5733e-05 8.1275e-06 2.32599e-06 10
end_path:out1:RootOutput(write) 2.44485 3.01697 3.5705 3.00859 0.37628 10
====================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5036.92 MB
Peak resident set size usage (VmHWM): 4236.12 MB
Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor: Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor: digitReader: 0.06 sec/event
ToolBasedRawDigitPrepService:dtor: adcSampleFiller: 0.29 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleAdcToKe: 0.04 sec/event
ToolBasedRawDigitPrepService:dtor: pdspTailPedRemovalZKe: 3.98 sec/event
ToolBasedRawDigitPrepService:dtor: adcKeepAllSignalFinder: 0.17 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleKeToAdc: 0.04 sec/event
ToolBasedRawDigitPrepService:dtor: pdsp_RemoveBadChannels: 0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9594168
-rw-r--r--. 1 dune009 dune 2271953859 Nov 25 19:01 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2157576896 Nov 25 18:59 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 1917991476 Nov 25 20:04 reco_stage1.root
-rw-r--r--. 1 dune009 dune 1742381640 Nov 25 16:30 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1669041124 Nov 25 16:16 g4_stage1.root
-rw-r--r--. 1 dune009 dune 64864380 Nov 25 19:59 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune 242818 Nov 25 20:04 jobscript.log
-rw-r--r--. 1 dune009 dune 217088 Nov 25 20:04 mem.db
-rw-r--r--. 1 dune009 dune 62376 Nov 25 20:04 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune 36864 Nov 25 20:04 time.db
-rw-r--r--. 1 dune009 dune 519 Nov 25 19:01 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune 519 Nov 25 16:30 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune 519 Nov 25 20:04 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune 52 Nov 25 16:09 all-input-dids.txt
-rw-r--r--. 1 dune009 dune 7 Nov 25 20:04 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune 0 Nov 25 19:08 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune 0 Nov 25 19:01 debugprod.log
