Jobsub ID 46968.76@dunegpschedd01.fnal.gov
| Jobsub ID | 46968.76@dunegpschedd01.fnal.gov |
| Workflow ID | 2650 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2025-09-16 11:42:55 |
| Site | NL_SURFsara |
| Entry | DUNE_SurfSARA_arc01 |
| Last heartbeat | 2025-09-16 11:55:44 |
| From worker node | Hostname | wn-lb-01.gina.surf.nl |
| cpuinfo | AMD EPYC 9754 128-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 129600 (36 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Started | 2025-09-16 11:54:29 |
| Input files | vd-protodune:np02vd_raw_run039275_0358_df-s03-d1_dw_0_20250902T025116.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Max RSS bytes | 0 (0 MiB) |
| Outputting started | |
| Output files | |
| Finished | 2025-09-16 11:55:44 |
| Saved logs | justin-logs:46968.76-dunegpschedd01.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
inputs:[ WireCell::IFrame ] outputs:[ WireCell::IFrame ]:(WireCell::IFrame:0) -> <Node type:WireCell::Gen::FrameFanin cat:7 sig:WireCell::IFrameFanin inputs:[ WireCell::IFrame WireCell::IFrame WireCell::IFrame WireCell::IFrame WireCell::IFrame WireCell::IFrame WireCell::IFrame WireCell::IFrame ] outputs:[ WireCell::IFrame ](WireCell::IFrame:5)
[13:55:08.627] D [ pgraph ] connect <Node type:WireCell::SigProc::OmnibusSigProc cat:3 sig:WireCell::IFrameFilter inputs:[ WireCell::IFrame ] outputs:[ WireCell::IFrame ]:(WireCell::IFrame:0) -> <Node type:WireCell::Gen::FrameFanin cat:7 sig:WireCell::IFrameFanin inputs:[ WireCell::IFrame WireCell::IFrame WireCell::IFrame WireCell::IFrame WireCell::IFrame WireCell::IFrame WireCell::IFrame WireCell::IFrame ] outputs:[ WireCell::IFrame ](WireCell::IFrame:6)
[13:55:08.627] D [ pgraph ] connect <Node type:WireCell::SigProc::OmnibusSigProc cat:3 sig:WireCell::IFrameFilter inputs:[ WireCell::IFrame ] outputs:[ WireCell::IFrame ]:(WireCell::IFrame:0) -> <Node type:WireCell::Gen::FrameFanin cat:7 sig:WireCell::IFrameFanin inputs:[ WireCell::IFrame WireCell::IFrame WireCell::IFrame WireCell::IFrame WireCell::IFrame WireCell::IFrame WireCell::IFrame WireCell::IFrame ] outputs:[ WireCell::IFrame ](WireCell::IFrame:7)
[13:55:08.627] D [ pgraph ] connect <Node type:WireCell::SigProc::ChannelSelector cat:3 sig:WireCell::IFrameFilter inputs:[ WireCell::IFrame ] outputs:[ WireCell::IFrame ]:(WireCell::IFrame:0) -> <Node type:WireCell::Aux::Resampler cat:3 sig:WireCell::IFrameFilter inputs:[ WireCell::IFrame ] outputs:[ WireCell::IFrame ](WireCell::IFrame:0)
[13:55:08.628] D [ pgraph ] connect <Node type:WireCell::Aux::Resampler cat:3 sig:WireCell::IFrameFilter inputs:[ WireCell::IFrame ] outputs:[ WireCell::IFrame ]:(WireCell::IFrame:0) -> <Node type:WireCell::SigProc::OmnibusSigProc cat:3 sig:WireCell::IFrameFilter inputs:[ WireCell::IFrame ] outputs:[ WireCell::IFrame ](WireCell::IFrame:0)
[13:55:08.628] D [ pgraph ] connect <Node type:WireCell::SigProc::ChannelSelector cat:3 sig:WireCell::IFrameFilter inputs:[ WireCell::IFrame ] outputs:[ WireCell::IFrame ]:(WireCell::IFrame:0) -> <Node type:WireCell::Aux::Resampler cat:3 sig:WireCell::IFrameFilter inputs:[ WireCell::IFrame ] outputs:[ WireCell::IFrame ](WireCell::IFrame:0)
[13:55:08.628] D [ pgraph ] connect <Node type:WireCell::Aux::Resampler cat:3 sig:WireCell::IFrameFilter inputs:[ WireCell::IFrame ] outputs:[ WireCell::IFrame ]:(WireCell::IFrame:0) -> <Node type:WireCell::SigProc::OmnibusSigProc cat:3 sig:WireCell::IFrameFilter inputs:[ WireCell::IFrame ] outputs:[ WireCell::IFrame ](WireCell::IFrame:0)
[13:55:08.628] D [ pgraph ] connect <Node type:WireCell::SigProc::ChannelSelector cat:3 sig:WireCell::IFrameFilter inputs:[ WireCell::IFrame ] outputs:[ WireCell::IFrame ]:(WireCell::IFrame:0) -> <Node type:WireCell::Aux::Resampler cat:3 sig:WireCell::IFrameFilter inputs:[ WireCell::IFrame ] outputs:[ WireCell::IFrame ](WireCell::IFrame:0)
[13:55:08.628] D [ pgraph ] connect <Node type:WireCell::Aux::Resampler cat:3 sig:WireCell::IFrameFilter inputs:[ WireCell::IFrame ] outputs:[ WireCell::IFrame ]:(WireCell::IFrame:0) -> <Node type:WireCell::SigProc::OmnibusSigProc cat:3 sig:WireCell::IFrameFilter inputs:[ WireCell::IFrame ] outputs:[ WireCell::IFrame ](WireCell::IFrame:0)
[13:55:08.628] D [ pgraph ] connect <Node type:WireCell::SigProc::ChannelSelector cat:3 sig:WireCell::IFrameFilter inputs:[ WireCell::IFrame ] outputs:[ WireCell::IFrame ]:(WireCell::IFrame:0) -> <Node type:WireCell::Aux::Resampler cat:3 sig:WireCell::IFrameFilter inputs:[ WireCell::IFrame ] outputs:[ WireCell::IFrame ](WireCell::IFrame:0)
[13:55:08.628] D [ pgraph ] connect <Node type:WireCell::Aux::Resampler cat:3 sig:WireCell::IFrameFilter inputs:[ WireCell::IFrame ] outputs:[ WireCell::IFrame ]:(WireCell::IFrame:0) -> <Node type:WireCell::SigProc::OmnibusSigProc cat:3 sig:WireCell::IFrameFilter inputs:[ WireCell::IFrame ] outputs:[ WireCell::IFrame ](WireCell::IFrame:0)
[13:55:08.628] D [ pgraph ] connect <Node type:WireCell::SigProc::ChannelSelector cat:3 sig:WireCell::IFrameFilter inputs:[ WireCell::IFrame ] outputs:[ WireCell::IFrame ]:(WireCell::IFrame:0) -> <Node type:WireCell::SigProc::OmnibusSigProc cat:3 sig:WireCell::IFrameFilter inputs:[ WireCell::IFrame ] outputs:[ WireCell::IFrame ](WireCell::IFrame:0)
[13:55:08.628] D [ pgraph ] connect <Node type:WireCell::SigProc::ChannelSelector cat:3 sig:WireCell::IFrameFilter inputs:[ WireCell::IFrame ] outputs:[ WireCell::IFrame ]:(WireCell::IFrame:0) -> <Node type:WireCell::SigProc::OmnibusSigProc cat:3 sig:WireCell::IFrameFilter inputs:[ WireCell::IFrame ] outputs:[ WireCell::IFrame ](WireCell::IFrame:0)
[13:55:08.628] D [ pgraph ] connect <Node type:WireCell::SigProc::ChannelSelector cat:3 sig:WireCell::IFrameFilter inputs:[ WireCell::IFrame ] outputs:[ WireCell::IFrame ]:(WireCell::IFrame:0) -> <Node type:WireCell::SigProc::OmnibusSigProc cat:3 sig:WireCell::IFrameFilter inputs:[ WireCell::IFrame ] outputs:[ WireCell::IFrame ](WireCell::IFrame:0)
[13:55:08.628] D [ pgraph ] connect <Node type:WireCell::SigProc::ChannelSelector cat:3 sig:WireCell::IFrameFilter inputs:[ WireCell::IFrame ] outputs:[ WireCell::IFrame ]:(WireCell::IFrame:0) -> <Node type:WireCell::SigProc::OmnibusSigProc cat:3 sig:WireCell::IFrameFilter inputs:[ WireCell::IFrame ] outputs:[ WireCell::IFrame ](WireCell::IFrame:0)
Inputer: "wclsRawFrameSource"
Outputer: "wclsFrameSaver:spsaver"
wclsFrameSaver: promising to produce recob::Wires named "gauss"
wclsFrameSaver: promising to produce recob::Wires named "wiener"
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 0:
#000: /scratch/workspace/build-single/BUILDTYPE/prof/QUAL/e26/label1/swarm/label2/SLF7/build/hdf5/v1_12_2a/source/hdf5-1.12.2/src/H5F.c line 620 in H5Fopen(): unable to open file
major: File accessibility
minor: Unable to open file
#001: /scratch/workspace/build-single/BUILDTYPE/prof/QUAL/e26/label1/swarm/label2/SLF7/build/hdf5/v1_12_2a/source/hdf5-1.12.2/src/H5VLcallback.c line 3501 in H5VL_file_open(): failed to iterate over available VOL connector plugins
major: Virtual Object Layer
minor: Iteration failed
#002: /scratch/workspace/build-single/BUILDTYPE/prof/QUAL/e26/label1/swarm/label2/SLF7/build/hdf5/v1_12_2a/source/hdf5-1.12.2/src/H5PLpath.c line 578 in H5PL__path_table_iterate(): can't iterate over plugins in plugin path '(null)'
major: Plugin for dynamically loaded library
minor: Iteration failed
#003: /scratch/workspace/build-single/BUILDTYPE/prof/QUAL/e26/label1/swarm/label2/SLF7/build/hdf5/v1_12_2a/source/hdf5-1.12.2/src/H5PLpath.c line 620 in H5PL__path_table_iterate_process_path(): can't open directory: /usr/local/hdf5/lib/plugin
major: Plugin for dynamically loaded library
minor: Can't open directory or file
#004: /scratch/workspace/build-single/BUILDTYPE/prof/QUAL/e26/label1/swarm/label2/SLF7/build/hdf5/v1_12_2a/source/hdf5-1.12.2/src/H5VLcallback.c line 3351 in H5VL__file_open(): open failed
major: Virtual Object Layer
minor: Can't open object
#005: /scratch/workspace/build-single/BUILDTYPE/prof/QUAL/e26/label1/swarm/label2/SLF7/build/hdf5/v1_12_2a/source/hdf5-1.12.2/src/H5VLnative_file.c line 97 in H5VL__native_file_open(): unable to open file
major: File accessibility
minor: Unable to open file
#006: /scratch/workspace/build-single/BUILDTYPE/prof/QUAL/e26/label1/swarm/label2/SLF7/build/hdf5/v1_12_2a/source/hdf5-1.12.2/src/H5Fint.c line 1834 in H5F_open(): unable to open file: name = 'root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/vd-protodune/4d/53/np02vd_raw_run039275_0358_df-s03-d1_dw_0_20250902T025116.hdf5', tent_flags = 0
major: File accessibility
minor: Unable to open file
#007: /scratch/workspace/build-single/BUILDTYPE/prof/QUAL/e26/label1/swarm/label2/SLF7/build/hdf5/v1_12_2a/source/hdf5-1.12.2/src/H5FD.c line 723 in H5FD_open(): open failed
major: Virtual File Layer
minor: Unable to initialize object
#008: /scratch/workspace/build-single/BUILDTYPE/prof/QUAL/e26/label1/swarm/label2/SLF7/build/hdf5/v1_12_2a/source/hdf5-1.12.2/src/H5FDsec2.c line 352 in H5FD__sec2_open(): unable to open file: name = 'root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/vd-protodune/4d/53/np02vd_raw_run039275_0358_df-s03-d1_dw_0_20250902T025116.hdf5', errno = 113, error message = 'No route to host', flags = 0, o_flags = 0
major: File accessibility
minor: Unable to open file
====================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
====================================================================================================================
[ No processed events ]
====================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 2686.57 MB
Peak resident set size usage (VmHWM): 710.64 MB
Details saved in: 'mem.db'
====================================================================================================
%MSG-s ArtException: PostEndJob 16-Sep-2025 13:55:09 CEST ModuleEndJob
---- HDF5RawDataFile BEGIN
File open failure: root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/vd-protodune/4d/53/np02vd_raw_run039275_0358_df-s03-d1_dw_0_20250902T025116.hdf5 Unable to open file root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/vd-protodune/4d/53/np02vd_raw_run039275_0358_df-s03-d1_dw_0_20250902T025116.hdf5 (File accessibility) Unable to open file
---- HDF5RawDataFile END
%MSG
Art has completed and will exit with status 1.
Error in reco1 