justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 228838.36@dunegpschedd02.fnal.gov

Jobsub ID228838.36@dunegpschedd02.fnal.gov
Workflow ID9021
Stage ID1
User namecalcuttj@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-10-15 16:11:35
SiteUK_RAL-Tier1
EntryLIGO_UK_RAL_arc_ce01
Last heartbeat2025-10-15 16:15:28
From worker nodeHostnamedune004-9567449.0-lcg2703.gridpp.rl.ac.uk
cpuinfoAMD EPYC 9654 96-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job stateoutputting_failed
Started2025-10-15 16:13:11
Input fileshd-protodune:np04hd_raw_run028891_0883_dataflow7_datawriter_0_20240824T134311.hdf5
JobscriptExit code0
Real time1m (117s)
CPU time0m (14s = 11%)
Max RSS bytes701341696 (668 MiB)
Outputting started2025-10-15 16:15:09
Output files
Finished2025-10-15 16:15:28
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
WARNING: NONZERO HALF
%MSG-w BeamEvent:  BeamEvent:beamevent@BeginModule 15-Oct-2025 16:15:07 UTC  run: 28891 subRun: 1 event: 219247 BeamEvent_module.cc:1828
Could not get XCET1 info

%MSG
%MSG-w BeamEvent:  BeamEvent:beamevent@BeginModule 15-Oct-2025 16:15:07 UTC  run: 28891 subRun: 1 event: 219247 BeamEvent_module.cc:1846
Could not get XCET2 info

%MSG
Timing trigger: 12
Matched: 1
CKovs: 1 0
TOF, P: 97.4444 5.14991
15-Oct-2025 16:15:08 UTC  Closed output file "pdhd_228838_36_1_20251015T161317Z_beam.root"
15-Oct-2025 16:15:08 UTC  Closed output file "pdhd_228838_36_1_20251015T161317Z_beam.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                        0.887201       2.85055       58.1292      0.986331       10.0928        31     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       1.7747e-05    2.74534e-05   0.000164628    1.998e-05    2.73887e-05      31     
produce:triggerrawdecoder:PDHDTriggerReader3       0.64584      0.706016      0.827946      0.698155      0.0407287       31     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.113844      0.127847      0.173441      0.115068       0.01643        31     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0228581     0.0231286     0.0238776     0.0231051    0.000182812      31     
produce:beamevent:BeamEvent                       0.0011116      1.87675       57.0119     0.00480383      10.0665        31     
[art]:TriggerResults:TriggerResultInserter       1.6855e-05    2.52808e-05   6.4317e-05    2.1392e-05    1.10523e-05      31     
end_path:out1:RootOutput                          3.776e-06    5.01306e-06   2.4387e-05     4.387e-06    3.55453e-06      31     
end_path:out1:RootOutput(write)                   0.0640262     0.116356      0.163673      0.114243      0.0224164       31     
===================================================================================================================================

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                        0.887201       2.85055       58.1292      0.986331       10.0928        31     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       1.7747e-05    2.74534e-05   0.000164628    1.998e-05    2.73887e-05      31     
produce:triggerrawdecoder:PDHDTriggerReader3       0.64584      0.706016      0.827946      0.698155      0.0407287       31     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.113844      0.127847      0.173441      0.115068       0.01643        31     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0228581     0.0231286     0.0238776     0.0231051    0.000182812      31     
produce:beamevent:BeamEvent                       0.0011116      1.87675       57.0119     0.00480383      10.0665        31     
[art]:TriggerResults:TriggerResultInserter       1.6855e-05    2.52808e-05   6.4317e-05    2.1392e-05    1.10523e-05      31     
end_path:out1:RootOutput                          3.776e-06    5.01306e-06   2.4387e-05     4.387e-06    3.55453e-06      31     
end_path:out1:RootOutput(write)                   0.0640262     0.116356      0.163673      0.114243      0.0224164       31     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1677.96 MB
  Peak resident set size usage (VmHWM): 811.614 MB
====================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1677.96 MB
  Peak resident set size usage (VmHWM): 811.614 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 31 passed = 31 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport         31         31          0 out1

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 8.004306 Real = 107.323369

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 1677.96 VmHWM = 811.614

Art has completed and will exit with status 0.
gen lar returns 0
==== Start *.json ====
cat: *.json: No such file or directory
==== End *.json ====
.:
total 9632
-rw-r--r-- 1 dune004 dune 1243359 Oct 15 16:15 jobscript.log
-rw-r--r-- 1 dune004 dune     163 Oct 15 16:15 justin-processed-pfns.txt
-rw-r--r-- 1 dune004 dune  717982 Oct 15 16:15 pdhd_228838_36_1_20251015T161317Z_beam_hist.root
-rw-r--r-- 1 dune004 dune 1213080 Oct 15 16:15 debug.log
-rw-r--r-- 1 dune004 dune 5819738 Oct 15 16:15 pdhd_228838_36_1_20251015T161317Z_beam.root
-rw-r--r-- 1 dune004 dune      83 Oct 15 16:13 all-input-dids.txt
justIN time: 2025-11-03 18:44:37 UTC       justIN version: 01.05.01