justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 228838.42@dunegpschedd02.fnal.gov

Jobsub ID228838.42@dunegpschedd02.fnal.gov
Workflow ID9021
Stage ID1
User namecalcuttj@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-10-15 16:11:35
SiteUK_RAL-Tier1
EntryLIGO_UK_RAL_arc_ce02
Last heartbeat2025-10-15 16:20:08
From worker nodeHostnamedune004-8503877.0-lcg2469.gridpp.rl.ac.uk
cpuinfoAMD EPYC 7452 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job stateoutputting_failed
Started2025-10-15 16:13:06
Input fileshd-protodune:np04hd_raw_run028891_0860_dataflow4_datawriter_0_20240824T131021.hdf5
hd-protodune:np04hd_raw_run028891_0862_dataflow6_datawriter_0_20240824T131320.hdf5
hd-protodune:np04hd_raw_run028891_0879_dataflow0_datawriter_0_20240824T133704.hdf5
hd-protodune:np04hd_raw_run028891_0879_dataflow6_datawriter_0_20240824T133704.hdf5
hd-protodune:np04hd_raw_run028891_0865_dataflow6_datawriter_0_20240824T131720.hdf5
hd-protodune:np04hd_raw_run028891_0865_dataflow4_datawriter_0_20240824T131720.hdf5
JobscriptExit code0
Real time6m (395s)
CPU time0m (44s = 11%)
Max RSS bytes778059776 (742 MiB)
Outputting started2025-10-15 16:19:41
Output files
Finished2025-10-15 16:20:08
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
%MSG-w BeamEvent:  BeamEvent:beamevent@BeginModule 15-Oct-2025 16:19:38 UTC  run: 28891 subRun: 1 event: 214756 BeamEvent_module.cc:1828
Could not get XCET1 info

%MSG
%MSG-w BeamEvent:  BeamEvent:beamevent@BeginModule 15-Oct-2025 16:19:38 UTC  run: 28891 subRun: 1 event: 214756 BeamEvent_module.cc:1846
Could not get XCET2 info

%MSG
Timing trigger: 12
Matched: 1
CKovs: 1 0
TOF, P: 95.4883 -1
%MSG-w HDF5FileLayout.cpp:  PostProcessEvent 15-Oct-2025 16:19:38 UTC  run: 28891 subRun: 1 event: 214756 HDF5FileLayout.cpp:70
 File Layout Not Enough Digits For Path 214764 5
%MSG
Timing trigger: 12
Matched: 1
CKovs: 1 0
TOF, P: -1 5.44795
15-Oct-2025 16:19:41 UTC  Closed output file "pdhd_228838_42_1_20251015T161317Z_beam.root"
15-Oct-2025 16:19:41 UTC  Closed output file "pdhd_228838_42_1_20251015T161317Z_beam.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                        0.699995       1.58829       24.2758       1.19926       2.65258        186    
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                        1.634e-05    2.23214e-05   0.00020095    2.0624e-05    1.37324e-05      186    
produce:triggerrawdecoder:PDHDTriggerReader3       0.49714      0.921033       1.82089      0.897977      0.326991        186    
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0870081      0.10359      0.182493      0.0965567     0.0165184       186    
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0175673     0.0182361     0.0356142     0.0180373    0.00138254       186    
produce:beamevent:BeamEvent                      0.000306229    0.439946       22.9548       0.01039       2.62075        186    
[art]:TriggerResults:TriggerResultInserter       1.2865e-05    2.44787e-05   7.2918e-05    2.4381e-05     6.444e-06       186    
end_path:out1:RootOutput                          3.336e-06    4.2643e-06    3.0587e-05     4.063e-06    1.98017e-06      186    
end_path:out1:RootOutput(write)                   0.0513465     0.105072      0.177982      0.103008      0.0219489       186    
===================================================================================================================================

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                        0.699995       1.58829       24.2758       1.19926       2.65258        186    
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                        1.634e-05    2.23214e-05   0.00020095    2.0624e-05    1.37324e-05      186    
produce:triggerrawdecoder:PDHDTriggerReader3       0.49714      0.921033       1.82089      0.897977      0.326991        186    
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0870081      0.10359      0.182493      0.0965567     0.0165184       186    
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0175673     0.0182361     0.0356142     0.0180373    0.00138254       186    
produce:beamevent:BeamEvent                      0.000306229    0.439946       22.9548       0.01039       2.62075        186    
[art]:TriggerResults:TriggerResultInserter       1.2865e-05    2.44787e-05   7.2918e-05    2.4381e-05     6.444e-06       186    
end_path:out1:RootOutput                          3.336e-06    4.2643e-06    3.0587e-05     4.063e-06    1.98017e-06      186    
end_path:out1:RootOutput(write)                   0.0513465     0.105072      0.177982      0.103008      0.0219489       186    
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1690.56 MB
  Peak resident set size usage (VmHWM): 808.313 MB
====================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1690.56 MB
  Peak resident set size usage (VmHWM): 808.313 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 186 passed = 186 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport        186        186          0 out1

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 36.834975 Real = 378.489848

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 1690.56 VmHWM = 808.313

Art has completed and will exit with status 0.
gen lar returns 0
==== Start *.json ====
cat: *.json: No such file or directory
==== End *.json ====
.:
total 68104
-rw-r--r-- 1 dune004 dune  7015986 Oct 15 16:19 jobscript.log
-rw-r--r-- 1 dune004 dune      978 Oct 15 16:19 justin-processed-pfns.txt
-rw-r--r-- 1 dune004 dune 19587295 Oct 15 16:19 pdhd_228838_42_1_20251015T161317Z_beam_hist.root
-rw-r--r-- 1 dune004 dune  6868920 Oct 15 16:19 debug.log
-rw-r--r-- 1 dune004 dune 34878296 Oct 15 16:19 pdhd_228838_42_1_20251015T161317Z_beam.root
-rw-r--r-- 1 dune004 dune      498 Oct 15 16:13 all-input-dids.txt
justIN time: 2025-11-03 18:40:25 UTC       justIN version: 01.05.01