justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 228842.40@dunegpschedd02.fnal.gov

Jobsub ID228842.40@dunegpschedd02.fnal.gov
Workflow ID9021
Stage ID1
User namecalcuttj@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-10-15 16:17:36
SiteUK_Sheffield
EntryDUNE_UK_Sheffield_lcgce2
Last heartbeat2025-10-15 17:08:46
From worker nodeHostnamewn029.hep
cpuinfoIntel(R) Xeon(R) Silver 4216 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job stateoutputting_failed
Started2025-10-15 16:18:42
Input fileshd-protodune:np04hd_raw_run028891_0070_dataflow2_datawriter_0_20240823T165824.hdf5
hd-protodune:np04hd_raw_run028891_0069_dataflow6_datawriter_0_20240823T165656.hdf5
hd-protodune:np04hd_raw_run028891_0072_dataflow1_datawriter_0_20240823T170125.hdf5
hd-protodune:np04hd_raw_run028891_0078_dataflow0_datawriter_0_20240823T171052.hdf5
JobscriptExit code0
Real time49m (2982s)
CPU time0m (47s = 1%)
Max RSS bytes789098496 (752 MiB)
Outputting started2025-10-15 17:08:25
Output files
Finished2025-10-15 17:08:46
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
%MSG-w BeamEvent:  BeamEvent:beamevent@BeginModule 15-Oct-2025 18:08:14 BST  run: 28891 subRun: 1 event: 19552 BeamEvent_module.cc:1828
Could not get XCET1 info

%MSG
%MSG-w BeamEvent:  BeamEvent:beamevent@BeginModule 15-Oct-2025 18:08:14 BST  run: 28891 subRun: 1 event: 19552 BeamEvent_module.cc:1846
Could not get XCET2 info

%MSG
Timing trigger: 12
Matched: 1
CKovs: 0 0
TOF, P: 386.21 4.50139
Timing trigger: 12
Matched: 1
CKovs: 0 0
TOF, P: 97.5283 5.22174
Timing trigger: 12
Matched: 1
CKovs: 1 0
TOF, P: 94.4377 5.06303
Timing trigger: 12
Matched: 1
CKovs: 0 0
TOF, P: -24.4746 5.397
Timing trigger: 8
Matched: 0
CKovs: 0 0
TOF, P: 0 -1
Timing trigger: 8
Matched: 0
CKovs: 0 0
TOF, P: 0 -1
15-Oct-2025 18:08:24 BST  Closed output file "pdhd_228842_40_1_20251015T161852Z_beam.root"
15-Oct-2025 18:08:24 BST  Closed output file "pdhd_228842_40_1_20251015T161852Z_beam.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                        0.983448       22.7682        2357         1.40765       210.936        124    
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       1.6389e-05    2.2546e-05    0.000123584   2.0445e-05    1.10221e-05      124    
produce:triggerrawdecoder:PDHDTriggerReader3      0.701106       1.28246       4.34038       1.01771      0.655959        124    
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.124194      0.146977      0.381771      0.127546      0.0443964       124    
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0249722      0.02567      0.0494605     0.0251163    0.00258294       124    
produce:beamevent:BeamEvent                      0.000745075     21.1645       2355.7       0.0103657      210.958        124    
[art]:TriggerResults:TriggerResultInserter       1.4455e-05    2.30835e-05   6.2107e-05    2.26595e-05   4.40886e-06      124    
end_path:out1:RootOutput                          2.969e-06    3.75674e-06   1.9009e-05     3.628e-06    1.4091e-06       124    
end_path:out1:RootOutput(write)                   0.0629861     0.148248      0.237898      0.149771      0.0272514       124    
===================================================================================================================================

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                        0.983448       22.7682        2357         1.40765       210.936        124    
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       1.6389e-05    2.2546e-05    0.000123584   2.0445e-05    1.10221e-05      124    
produce:triggerrawdecoder:PDHDTriggerReader3      0.701106       1.28246       4.34038       1.01771      0.655959        124    
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.124194      0.146977      0.381771      0.127546      0.0443964       124    
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0249722      0.02567      0.0494605     0.0251163    0.00258294       124    
produce:beamevent:BeamEvent                      0.000745075     21.1645       2355.7       0.0103657      210.958        124    
[art]:TriggerResults:TriggerResultInserter       1.4455e-05    2.30835e-05   6.2107e-05    2.26595e-05   4.40886e-06      124    
end_path:out1:RootOutput                          2.969e-06    3.75674e-06   1.9009e-05     3.628e-06    1.4091e-06       124    
end_path:out1:RootOutput(write)                   0.0629861     0.148248      0.237898      0.149771      0.0272514       124    
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1619.88 MB
  Peak resident set size usage (VmHWM): 792.044 MB
====================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1619.88 MB
  Peak resident set size usage (VmHWM): 792.044 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 124 passed = 124 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport        124        124          0 out1

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 40.869511 Real = 2969.076546

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 1619.88 VmHWM = 792.044

Art has completed and will exit with status 0.
gen lar returns 0
==== Start *.json ====
cat: *.json: No such file or directory
==== End *.json ====
.:
total 39244
-rw-r--r--. 1 dune004 dune  4123303 Oct 15 18:08 jobscript.log
-rw-r--r--. 1 dune004 dune      652 Oct 15 18:08 justin-processed-pfns.txt
-rw-r--r--. 1 dune004 dune  8059489 Oct 15 18:08 pdhd_228842_40_1_20251015T161852Z_beam_hist.root
-rw-r--r--. 1 dune004 dune  4033086 Oct 15 18:08 debug.log
-rw-r--r--. 1 dune004 dune 23884047 Oct 15 18:08 pdhd_228842_40_1_20251015T161852Z_beam.root
-rw-r--r--. 1 dune004 dune      332 Oct 15 17:18 all-input-dids.txt
justIN time: 2025-11-04 01:54:59 UTC       justIN version: 01.05.01