Jobsub ID 40385.78@dunegpschedd02.fnal.gov
| Jobsub ID | 40385.78@dunegpschedd02.fnal.gov |
| Workflow ID | 2650 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2025-09-16 11:46:54 |
| Site | NL_SURFsara |
| Entry | DUNE_SurfSARA_arc02 |
| Last heartbeat | 2025-09-16 13:40:10 |
| From worker node | Hostname | wn-lb-01.gina.surf.nl |
| cpuinfo | AMD EPYC 9754 128-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 129600 (36 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Started | 2025-09-16 12:22:28 |
| Input files | vd-protodune:np02vd_raw_run039275_0323_df-s02-d2_dw_0_20250902T020139.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Max RSS bytes | 0 (0 MiB) |
| Outputting started | |
| Output files | |
| Finished | 2025-09-16 13:40:10 |
| Saved logs | justin-logs:40385.78-dunegpschedd02.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
channel 10415
wclsFrameSaver: no samples within desired window for channel 10415
wclsFrameSaver: no samples within desired window for channel 10526
wclsFrameSaver: no samples within desired window for channel 10628
wclsFrameSaver: no samples within desired window for channel 10948
wclsFrameSaver: no samples within desired window for channel 10969
wclsFrameSaver: no samples within desired window for channel 11003
wclsFrameSaver: no samples within desired window for channel 11038
wclsFrameSaver: no samples within desired window for channel 11062
wclsFrameSaver: no samples within desired window for channel 11196
wclsFrameSaver: no samples within desired window for channel 11197
wclsFrameSaver: no samples within desired window for channel 12086
wclsFrameSaver: no samples within desired window for channel 12086
wclsFrameSaver: no samples within desired window for channel 12086
wclsFrameSaver: no samples within desired window for channel 12088
FrameSaver: q=7.38228e+06 n=966907 tag=gauss
wclsFrameSaver: saving 60443 traces tagged "wiener"
wclsFrameSaver: no samples within desired window for channel 6224
wclsFrameSaver: no samples within desired window for channel 6308
wclsFrameSaver: no samples within desired window for channel 6311
wclsFrameSaver: no samples within desired window for channel 6312
wclsFrameSaver: no samples within desired window for channel 6312
wclsFrameSaver: no samples within desired window for channel 6313
wclsFrameSaver: no samples within desired window for channel 6314
wclsFrameSaver: no samples within desired window for channel 6314
wclsFrameSaver: no samples within desired window for channel 6314
wclsFrameSaver: no samples within desired window for channel 6315
wclsFrameSaver: no samples within desired window for channel 6315
wclsFrameSaver: no samples within desired window for channel 6316
wclsFrameSaver: no samples within desired window for channel 6317
wclsFrameSaver: no samples within desired window for channel 6656
wclsFrameSaver: no samples within desired window for channel 6778
wclsFrameSaver: no samples within desired window for channel 7143
wclsFrameSaver: no samples within desired window for channel 7265
wclsFrameSaver: no samples within desired window for channel 7267
wclsFrameSaver: no samples within desired window for channel 7268
wclsFrameSaver: no samples within desired window for channel 7666
wclsFrameSaver: no samples within desired window for channel 7880
wclsFrameSaver: no samples within desired window for channel 7880
wclsFrameSaver: no samples within desired window for channel 7891
wclsFrameSaver: no samples within desired window for channel 7891
wclsFrameSaver: no samples within desired window for channel 8187
wclsFrameSaver: no samples within desired window for channel 8191
wclsFrameSaver: no samples within desired window for channel 8192
wclsFrameSaver: no samples within desired window for channel 8193
wclsFrameSaver: no samples within desired window for channel 9136
wclsFrameSaver: no samples within desired window for channel 9144
wclsFrameSaver: no samples within desired window for channel 9145
wclsFrameSaver: no samples within desired window for channel 9145
wclsFrameSaver: no samples within desired window for channel 9146
wclsFrameSaver: no samples within desired window for channel 9844
wclsFrameSaver: no samples within desired window for channel 10415
wclsFrameSaver: no samples within desired window for channel 10415
wclsFrameSaver: no samples within desired window for channel 10526
wclsFrameSaver: no samples within desired window for channel 10628
wclsFrameSaver: no samples within desired window for channel 10948
wclsFrameSaver: no samples within desired window for channel 10969
wclsFrameSaver: no samples within desired window for channel 11003
wclsFrameSaver: no samples within desired window for channel 11038
wclsFrameSaver: no samples within desired window for channel 11062
wclsFrameSaver: no samples within desired window for channel 11196
wclsFrameSaver: no samples within desired window for channel 11197
wclsFrameSaver: no samples within desired window for channel 12086
wclsFrameSaver: no samples within desired window for channel 12086
wclsFrameSaver: no samples within desired window for channel 12087
wclsFrameSaver: no samples within desired window for channel 12088
FrameSaver: q=7.8118e+06 n=923659 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 946 XUs and 1073 XVs -> 25 XUVs
C:0 T:1 1703 XUs and 1789 XVs -> 107 XUVs
C:0 T:2 3673 XUs and 3663 XVs -> 256 XUVs
C:0 T:3 679 XUs and 904 XVs -> 49 XUVs
C:0 T:4 317 XUs and 389 XVs -> 14 XUVs
C:0 T:5 1672 XUs and 1812 XVs -> 64 XUVs
C:0 T:6 630 XUs and 897 XVs -> 56 XUVs
C:0 T:7 1699 XUs and 1777 XVs -> 61 XUVs
C:0 T:8 656 XUs and 639 XVs -> 73 XUVs
C:0 T:9 1128 XUs and 1957 XVs -> 67 XUVs
C:0 T:10 806 XUs and 796 XVs -> 85 XUVs
C:0 T:11 2116 XUs and 2527 XVs -> 103 XUVs
C:0 T:12 688 XUs and 1017 XVs -> 76 XUVs
C:0 T:13 1934 XUs and 1849 XVs -> 108 XUVs
C:0 T:14 3208 XUs and 2997 XVs -> 143 XUVs
C:0 T:15 2091 XUs and 2454 XVs -> 128 XUVs
1415 XUVs total
1173 collection wire objects
1415 potential space points
Neighbour search...
11587 tests to find 5514 neighbours
Iterating with no regularization...
Begin: 3.25298e+08
0 3.15059e+08
1 3.14998e+08
Now with regularization...
Begin: 3.0844e+08
0 3.08436e+08
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
16-Sep-2025 15:39:10 CEST Closed output file "np02vd_raw_run039275_0323_df-s02-d2_dw_0_20250902T020139_reco_stage1_20250916T133910_keepup.root"
==================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
==================================================================================================================================
Full event 160.814 252.615 430.713 213.036 85.4072 18
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 6.9195e-05 0.000104084 0.00026652 8.12465e-05 4.91153e-05 18
produce:tpcrawdecoder:PDVDTPCReader 13.7239 27.751 57.0554 25.6217 12.9122 18
produce:triggerrawdecoder:PDVDTriggerReader4 0.0339332 0.0390316 0.0461981 0.0380923 0.0032645 18
produce:pdvddaphne:DAPHNEReaderPDVD 6.55969 7.72317 9.26595 7.66563 0.696927 18
produce:ophit:OpHitFinder 0.0446245 0.061342 0.0801309 0.0593705 0.00813873 18
produce:opflash:OpFlashFinderVerticalDrift 0.00609176 0.0134211 0.0208338 0.013059 0.00393289 18
produce:wclsdatavd:WireCellToolkit 75.5966 86.4955 94.3835 87.4519 6.0503 18
produce:gaushit:GausHitFinder 0.925412 1.43531 2.06565 1.36487 0.344568 18
produce:nhitsfilter:NumberOfHitsFilter 0.000973473 0.00138808 0.00217686 0.00135416 0.000312779 18
produce:reco3d:SpacePointSolver 9.88004 21.0663 34.6756 19.3019 6.48218 18
produce:hitpdune:DisambigFromSpacePoints 0.137132 0.335239 0.660082 0.310824 0.14264 18
produce:pandora:StandardPandora 24.8238 97.6699 243.843 63.5574 74.3543 18
produce:pandoraTrack:LArPandoraTrackCreation 1.29137 4.59049 9.85614 4.20609 2.03656 18
produce:pandoraGnocalo:GnocchiCalorimetry 0.0233091 0.0466427 0.0803881 0.0464087 0.0130976 18
[art]:TriggerResults:TriggerResultInserter 3.8097e-05 6.51733e-05 0.000143415 5.16875e-05 2.99152e-05 18
end_path:out1:RootOutput 6.039e-06 1.13646e-05 3.0485e-05 8.763e-06 6.84823e-06 18
end_path:out1:RootOutput(write) 4.90184 5.26133 5.69811 5.22681 0.220869 18
==================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5172.62 MB
Peak resident set size usage (VmHWM): 3287.51 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039275_0323_df-s02-d2_dw_0_20250902T020139_reco_stage1_20250916T133910_keepup.root
\tHists: np02vd_raw_run039275_0323_df-s02-d2_dw_0_20250902T020139_reco_stage1_20250916T133911_keepup_hists.root
Forming reco metadata
Traceback (most recent call last):
File "/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/cvmfs/fifeuser2.opensciencegrid.org/sw/dune/20943968d0063a8f11ac6a4062b5ad0c8392b138/meta_maker.py", line 1, in <module>
File "/cvmfs/larsoft.opensciencegrid.org/products/root/v6_28_12/Linux64bit+3.10-2.17-e26-p3915-prof/lib/ROOT/_facade.py", line 154, in _importhook
return _orig_ihook(name, *args, **kwds)
ModuleNotFoundError: No module named 'inherit_metadata'
Error in reco metadata 