justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 228842.71@dunegpschedd02.fnal.gov

Jobsub ID228842.71@dunegpschedd02.fnal.gov
Workflow ID9021
Stage ID1
User namecalcuttj@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-10-15 16:17:36
SiteUK_Lancaster
EntryUBoone_UK_Lancaster_HEC_grendel_ce02
Last heartbeat2025-10-15 16:40:58
From worker nodeHostnamecomp05-24
cpuinfoIntel(R) Xeon(R) Platinum 8352Y CPU @ 2.20GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job stateoutputting_failed
Started2025-10-15 16:18:51
Input fileshd-protodune:np04hd_raw_run028891_0108_dataflow1_datawriter_0_20240823T182009.hdf5
hd-protodune:np04hd_raw_run028891_0119_dataflow1_datawriter_0_20240823T183842.hdf5
hd-protodune:np04hd_raw_run028891_0125_dataflow7_datawriter_0_20240823T184754.hdf5
hd-protodune:np04hd_raw_run028891_0126_dataflow6_datawriter_0_20240823T184930.hdf5
JobscriptExit code0
Real time21m (1313s)
CPU time0m (23s = 1%)
Max RSS bytes774975488 (739 MiB)
Outputting started2025-10-15 16:40:44
Output files
Finished2025-10-15 16:40:58
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
%MSG-w BeamEvent:  BeamEvent:beamevent@BeginModule 15-Oct-2025 17:40:33 BST  run: 28891 subRun: 1 event: 31462 BeamEvent_module.cc:1828
Could not get XCET1 info

%MSG
%MSG-w BeamEvent:  BeamEvent:beamevent@BeginModule 15-Oct-2025 17:40:33 BST  run: 28891 subRun: 1 event: 31462 BeamEvent_module.cc:1846
Could not get XCET2 info

%MSG
Timing trigger: 12
Matched: 1
CKovs: 1 0
TOF, P: 96.1348 4.91581
Timing trigger: 12
Matched: 1
CKovs: 0 0
TOF, P: 97.6885 5.35743
Timing trigger: 12
Matched: 1
CKovs: 1 0
TOF, P: 93.623 6.09746
Timing trigger: 12
Matched: 1
CKovs: 1 0
TOF, P: -1 4.75383
Timing trigger: 8
Matched: 0
CKovs: 0 0
TOF, P: 0 -1
15-Oct-2025 17:40:43 BST  Closed output file "pdhd_228842_71_1_20251015T161856Z_beam.root"
15-Oct-2025 17:40:43 BST  Closed output file "pdhd_228842_71_1_20251015T161856Z_beam.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                        0.991851       9.91849       629.771       1.70318       59.8977        124    
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       1.2372e-05    1.71547e-05   0.000102454   1.5901e-05    7.97171e-06      124    
produce:triggerrawdecoder:PDHDTriggerReader3      0.710548       1.40579       2.95927       1.38106      0.408153        124    
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.122534      0.144696      0.355952      0.138696      0.0283563       124    
produce:timingrawdecoder:PDHDTimingRawDecoder     0.024286      0.0261473     0.0783682     0.0249753    0.00616011       124    
produce:beamevent:BeamEvent                      0.000191297     8.2548        628.766     0.00542943      59.9707        124    
[art]:TriggerResults:TriggerResultInserter       1.1287e-05    2.04908e-05   4.6934e-05    2.0727e-05    4.39759e-06      124    
end_path:out1:RootOutput                          2.579e-06    3.44531e-06   1.3211e-05     3.251e-06    1.27897e-06      124    
end_path:out1:RootOutput(write)                   0.0513044     0.086746      0.202702      0.0852489     0.0174169       124    
===================================================================================================================================

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                        0.991851       9.91849       629.771       1.70318       59.8977        124    
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       1.2372e-05    1.71547e-05   0.000102454   1.5901e-05    7.97171e-06      124    
produce:triggerrawdecoder:PDHDTriggerReader3      0.710548       1.40579       2.95927       1.38106      0.408153        124    
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.122534      0.144696      0.355952      0.138696      0.0283563       124    
produce:timingrawdecoder:PDHDTimingRawDecoder     0.024286      0.0261473     0.0783682     0.0249753    0.00616011       124    
produce:beamevent:BeamEvent                      0.000191297     8.2548        628.766     0.00542943      59.9707        124    
[art]:TriggerResults:TriggerResultInserter       1.1287e-05    2.04908e-05   4.6934e-05    2.0727e-05    4.39759e-06      124    
end_path:out1:RootOutput                          2.579e-06    3.44531e-06   1.3211e-05     3.251e-06    1.27897e-06      124    
end_path:out1:RootOutput(write)                   0.0513044     0.086746      0.202702      0.0852489     0.0174169       124    
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1640.82 MB
  Peak resident set size usage (VmHWM): 774.975 MB
====================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1640.82 MB
  Peak resident set size usage (VmHWM): 774.975 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 124 passed = 124 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport        124        124          0 out1

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 20.180638 Real = 1305.380182

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 1640.82 VmHWM = 774.975

Art has completed and will exit with status 0.
gen lar returns 0
==== Start *.json ====
cat: *.json: No such file or directory
==== End *.json ====
.:
total 37169
-rw-r--r-- 1 pltdune004 pltdune  3892998 Oct 15 17:40 jobscript.log
-rw-r--r-- 1 pltdune004 pltdune      652 Oct 15 17:40 justin-processed-pfns.txt
-rw-r--r-- 1 pltdune004 pltdune  7442742 Oct 15 17:40 pdhd_228842_71_1_20251015T161856Z_beam_hist.root
-rw-r--r-- 1 pltdune004 pltdune  3808906 Oct 15 17:40 debug.log
-rw-r--r-- 1 pltdune004 pltdune 22913660 Oct 15 17:40 pdhd_228842_71_1_20251015T161856Z_beam.root
-rw-r--r-- 1 pltdune004 pltdune      332 Oct 15 17:18 all-input-dids.txt
justIN time: 2025-11-04 01:54:58 UTC       justIN version: 01.05.01