justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 234367.30@dunegpschedd01.fnal.gov

Jobsub ID234367.30@dunegpschedd01.fnal.gov
Workflow ID9021
Stage ID1
User namecalcuttj@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-10-15 16:33:36
SiteUK_Sheffield
EntryDUNE_UK_Sheffield_lcgce2
Last heartbeat2025-10-15 17:33:58
From worker nodeHostnamewn035.hep
cpuinfoAMD EPYC 7352 24-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job stateoutputting_failed
Started2025-10-15 16:35:27
Input fileshd-protodune:np04hd_raw_run028891_0591_dataflow0_datawriter_0_20240824T061426.hdf5
hd-protodune:np04hd_raw_run028891_0598_dataflow7_datawriter_0_20240824T062516.hdf5
hd-protodune:np04hd_raw_run028891_0601_dataflow0_datawriter_0_20240824T063000.hdf5
hd-protodune:np04hd_raw_run028891_0609_dataflow6_datawriter_0_20240824T064202.hdf5
JobscriptExit code0
Real time58m (3485s)
CPU time0m (35s = 1%)
Max RSS bytes817856512 (779 MiB)
Outputting started2025-10-15 17:33:33
Output files
Finished2025-10-15 17:33:58
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

3 276 Breaking 2A Gen -4.38409e+07

%MSG
%MSG-i BeamEvent:  BeamEvent:beamevent@BeginModule 15-Oct-2025 18:33:28 BST  run: 28891 subRun: 1 event: 151270 BeamEvent_module.cc:1625
604 276 Breaking 2A Gen -1.58488e+07

%MSG
%MSG-i BeamEvent:  BeamEvent:beamevent@BeginModule 15-Oct-2025 18:33:28 BST  run: 28891 subRun: 1 event: 151270 BeamEvent_module.cc:1625
605 276 Breaking 2A Gen -1.07424e+07

%MSG
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
%MSG-w BeamEvent:  BeamEvent:beamevent@BeginModule 15-Oct-2025 18:33:28 BST  run: 28891 subRun: 1 event: 151270 BeamEvent_module.cc:1828
Could not get XCET1 info

%MSG
%MSG-w BeamEvent:  BeamEvent:beamevent@BeginModule 15-Oct-2025 18:33:28 BST  run: 28891 subRun: 1 event: 151270 BeamEvent_module.cc:1846
Could not get XCET2 info

%MSG
Timing trigger: 12
Matched: 1
CKovs: 0 0
TOF, P: 97.8788 5.04076
%MSG-w HDF5FileLayout.cpp:  PostProcessEvent 15-Oct-2025 18:33:28 BST  run: 28891 subRun: 1 event: 151270 HDF5FileLayout.cpp:70
 File Layout Not Enough Digits For Path 151278 5
%MSG
Timing trigger: 8
Matched: 0
CKovs: 0 0
TOF, P: 0 -1
15-Oct-2025 18:33:32 BST  Closed output file "pdhd_234367_30_1_20251015T163536Z_beam.root"
15-Oct-2025 18:33:32 BST  Closed output file "pdhd_234367_30_1_20251015T163536Z_beam.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         1.03719       27.4041       1367.77       1.61053       161.581        124    
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       1.3907e-05    1.94917e-05   0.000117103   1.80545e-05   9.76986e-06      124    
produce:triggerrawdecoder:PDHDTriggerReader3      0.814588       1.29075       2.30608       1.27636      0.257935        124    
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.124247      0.138197      0.170565      0.133202      0.0150332       124    
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0249557     0.0251945     0.0263392     0.0251758    0.000148746      124    
produce:beamevent:BeamEvent                      0.000208317     25.8366       1366.53     0.00692769      161.631        124    
[art]:TriggerResults:TriggerResultInserter       1.1171e-05    1.84022e-05   5.0075e-05    1.7814e-05    4.58955e-06      124    
end_path:out1:RootOutput                          2.745e-06    3.66921e-06   1.8275e-05     3.527e-06    1.34711e-06      124    
end_path:out1:RootOutput(write)                   0.0568644     0.113032      0.169027      0.113469      0.021172        124    
===================================================================================================================================

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         1.03719       27.4041       1367.77       1.61053       161.581        124    
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       1.3907e-05    1.94917e-05   0.000117103   1.80545e-05   9.76986e-06      124    
produce:triggerrawdecoder:PDHDTriggerReader3      0.814588       1.29075       2.30608       1.27636      0.257935        124    
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.124247      0.138197      0.170565      0.133202      0.0150332       124    
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0249557     0.0251945     0.0263392     0.0251758    0.000148746      124    
produce:beamevent:BeamEvent                      0.000208317     25.8366       1366.53     0.00692769      161.631        124    
[art]:TriggerResults:TriggerResultInserter       1.1171e-05    1.84022e-05   5.0075e-05    1.7814e-05    4.58955e-06      124    
end_path:out1:RootOutput                          2.745e-06    3.66921e-06   1.8275e-05     3.527e-06    1.34711e-06      124    
end_path:out1:RootOutput(write)                   0.0568644     0.113032      0.169027      0.113469      0.021172        124    
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1640.72 MB
  Peak resident set size usage (VmHWM): 817.857 MB
====================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1640.72 MB
  Peak resident set size usage (VmHWM): 817.857 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 124 passed = 124 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport        124        124          0 out1

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 28.880834 Real = 3472.049618

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 1640.72 VmHWM = 817.857

Art has completed and will exit with status 0.
gen lar returns 0
==== Start *.json ====
cat: *.json: No such file or directory
==== End *.json ====
.:
total 40436
-rw-r--r--. 1 dune004 dune  4464491 Oct 15 18:33 jobscript.log
-rw-r--r--. 1 dune004 dune      652 Oct 15 18:33 justin-processed-pfns.txt
-rw-r--r--. 1 dune004 dune  8780164 Oct 15 18:33 pdhd_234367_30_1_20251015T163536Z_beam_hist.root
-rw-r--r--. 1 dune004 dune  4366992 Oct 15 18:33 debug.log
-rw-r--r--. 1 dune004 dune 23522513 Oct 15 18:33 pdhd_234367_30_1_20251015T163536Z_beam.root
-rw-r--r--. 1 dune004 dune      332 Oct 15 17:35 all-input-dids.txt
justIN time: 2025-11-04 01:57:42 UTC       justIN version: 01.05.01