justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 228838.147@dunegpschedd02.fnal.gov

Jobsub ID228838.147@dunegpschedd02.fnal.gov
Workflow ID9021
Stage ID1
User namecalcuttj@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-10-15 16:11:35
SiteUK_Sheffield
EntryDUNE_UK_Sheffield_lcgce2
Last heartbeat2025-10-15 16:20:11
From worker nodeHostnamewn040.hep
cpuinfoAMD EPYC 7402 24-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job stateoutputting_failed
Started2025-10-15 16:13:09
Input fileshd-protodune:np04hd_raw_run028891_0870_dataflow7_datawriter_0_20240824T132419.hdf5
hd-protodune:np04hd_raw_run028891_0882_dataflow7_datawriter_0_20240824T134143.hdf5
hd-protodune:np04hd_raw_run028891_0887_dataflow4_datawriter_0_20240824T134839.hdf5
hd-protodune:np04hd_raw_run028891_0865_dataflow5_datawriter_0_20240824T131720.hdf5
JobscriptExit code0
Real time6m (400s)
CPU time0m (36s = 8%)
Max RSS bytes797417472 (760 MiB)
Outputting started2025-10-15 16:19:50
Output files
Finished2025-10-15 16:20:11
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
%MSG-w BeamEvent:  BeamEvent:beamevent@BeginModule 15-Oct-2025 17:19:43 BST  run: 28891 subRun: 1 event: 214749 BeamEvent_module.cc:1828
Could not get XCET1 info

%MSG
%MSG-w BeamEvent:  BeamEvent:beamevent@BeginModule 15-Oct-2025 17:19:43 BST  run: 28891 subRun: 1 event: 214749 BeamEvent_module.cc:1846
Could not get XCET2 info

%MSG
Timing trigger: 12
Matched: 1
CKovs: 0 0
TOF, P: 88.3574 4.54809
%MSG-w HDF5FileLayout.cpp:  PostProcessEvent 15-Oct-2025 17:19:43 BST  run: 28891 subRun: 1 event: 214749 HDF5FileLayout.cpp:70
 File Layout Not Enough Digits For Path 214757 5
%MSG
Timing trigger: 12
Matched: 1
CKovs: 1 0
TOF, P: -1 -1
%MSG-w HDF5FileLayout.cpp:  PostProcessEvent 15-Oct-2025 17:19:45 BST  run: 28891 subRun: 1 event: 214757 HDF5FileLayout.cpp:70
 File Layout Not Enough Digits For Path 214765 5
%MSG
Timing trigger: 12
Matched: 1
CKovs: 1 0
TOF, P: -1 5.07747
15-Oct-2025 17:19:49 BST  Closed output file "pdhd_228838_147_1_20251015T161317Z_beam.root"
15-Oct-2025 17:19:49 BST  Closed output file "pdhd_228838_147_1_20251015T161317Z_beam.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         1.01991       2.53746       30.6076       1.8062        3.92903        124    
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       1.6411e-05    2.4766e-05    0.000150664   2.1731e-05    1.32276e-05      124    
produce:triggerrawdecoder:PDHDTriggerReader3      0.789127       1.52445       2.64029       1.48086      0.367136        124    
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.124069      0.139902      0.411491      0.125773      0.0293418       124    
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0248599     0.0266995     0.0843439     0.0251824    0.00736046       124    
produce:beamevent:BeamEvent                      0.000289015    0.738563       29.5016     0.00994685      3.89005        124    
[art]:TriggerResults:TriggerResultInserter       1.1942e-05    2.46056e-05   0.000112302   2.30435e-05   1.03496e-05      124    
end_path:out1:RootOutput                          3.016e-06    4.65306e-06   3.0818e-05     4.293e-06    2.51423e-06      124    
end_path:out1:RootOutput(write)                   0.0688866     0.107409      0.165692      0.107425      0.0182014       124    
===================================================================================================================================

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         1.01991       2.53746       30.6076       1.8062        3.92903        124    
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       1.6411e-05    2.4766e-05    0.000150664   2.1731e-05    1.32276e-05      124    
produce:triggerrawdecoder:PDHDTriggerReader3      0.789127       1.52445       2.64029       1.48086      0.367136        124    
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.124069      0.139902      0.411491      0.125773      0.0293418       124    
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0248599     0.0266995     0.0843439     0.0251824    0.00736046       124    
produce:beamevent:BeamEvent                      0.000289015    0.738563       29.5016     0.00994685      3.89005        124    
[art]:TriggerResults:TriggerResultInserter       1.1942e-05    2.46056e-05   0.000112302   2.30435e-05   1.03496e-05      124    
end_path:out1:RootOutput                          3.016e-06    4.65306e-06   3.0818e-05     4.293e-06    2.51423e-06      124    
end_path:out1:RootOutput(write)                   0.0688866     0.107409      0.165692      0.107425      0.0182014       124    
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1629.63 MB
  Peak resident set size usage (VmHWM): 797.417 MB
====================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1629.63 MB
  Peak resident set size usage (VmHWM): 797.417 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 124 passed = 124 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport        124        124          0 out1

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 29.348834 Real = 387.704214

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 1629.63 VmHWM = 797.417

Art has completed and will exit with status 0.
gen lar returns 0
==== Start *.json ====
cat: *.json: No such file or directory
==== End *.json ====
.:
total 45292
-rw-r--r--. 1 dune004 dune  4792778 Oct 15 17:19 jobscript.log
-rw-r--r--. 1 dune004 dune      652 Oct 15 17:19 justin-processed-pfns.txt
-rw-r--r--. 1 dune004 dune  9592270 Oct 15 17:19 pdhd_228838_147_1_20251015T161317Z_beam_hist.root
-rw-r--r--. 1 dune004 dune  4691994 Oct 15 17:19 debug.log
-rw-r--r--. 1 dune004 dune 23694711 Oct 15 17:19 pdhd_228838_147_1_20251015T161317Z_beam.root
-rw-r--r--. 1 dune004 dune      332 Oct 15 17:13 all-input-dids.txt
justIN time: 2025-11-04 01:55:10 UTC       justIN version: 01.05.01