Jobsub ID 235452.3@dunegpschedd02.fnal.gov

Jobsub ID	235452.3@dunegpschedd02.fnal.gov
Workflow ID	9479
Stage ID	1
User name	osbiston@fnal.gov
Requested	Processors	1
	GPU	No
	RSS bytes	2097152000 (2000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-10-30 17:11:58
Site	US_FNAL-FermiGrid
Entry	FNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat	2025-10-30 18:25:32
From worker node	Hostname	dunegli-7579803-0-fnpc9003.fnal.gov
	cpuinfo	Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	2097152000 (2000 MiB)
	Wall seconds limit	172800 (48 hours)
	GPU
	Inner Apptainer?	True
Job state	stalled
Started	2025-10-30 18:20:27
Input files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250210T155438Z_gen_000946_supernova_g4_halfActiveVol_detsim_reco_20250725T050227Z_reco2.root
Outputting started	2025-10-30 18:25:32
Output files
Finished	2025-10-30 18:44:41
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

cp: cannot stat '/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/7e4d8a7884851e80797d7d3c5855496e8acfe48e/*.xml': No such file or directory
Input PFN = root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/a3/25/prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250210T155438Z_gen_000946_supernova_g4_halfActiveVol_detsim_reco_20250725T050227Z_reco2.root
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/

MRB_PROJECT=larsoft
MRB_PROJECT_VERSION=v09_91_03
MRB_QUALS=e26:prof
MRB_TOP=/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/7e4d8a7884851e80797d7d3c5855496e8acfe48e
MRB_SOURCE=/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/7e4d8a7884851e80797d7d3c5855496e8acfe48e/srcs
MRB_BUILDDIR=/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/7e4d8a7884851e80797d7d3c5855496e8acfe48e/build_slf7.x86_64
MRB_INSTALL=/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/7e4d8a7884851e80797d7d3c5855496e8acfe48e/v09_91_03.prof/localProducts_larsoft_v09_91_03_e26_prof

PRODUCTS=/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/7e4d8a7884851e80797d7d3c5855496e8acfe48e/v09_91_03.prof/localProducts_larsoft_v09_91_03_e26_prof:/cvmfs/dune.opensciencegrid.org/products/dune:/cvmfs/larsoft.opensciencegrid.org/products:/cvmfs/larsoft.opensciencegrid.org/packages:/cvmfs/fermilab.opensciencegrid.org/products/common/db/
CETPKG_INSTALL=/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/7e4d8a7884851e80797d7d3c5855496e8acfe48e/v09_91_03.prof/localProducts_larsoft_v09_91_03_e26_prof

local product directory is /cvmfs/fifeuser1.opensciencegrid.org/sw/dune/7e4d8a7884851e80797d7d3c5855496e8acfe48e/v09_91_03.prof/localProducts_larsoft_v09_91_03_e26_prof
----------- this block should be empty ------------------
---------------------------------------------------------
/cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_5_5a/Linux64bit+3.10-2.17-e26-p3915-prof/lib/libXrdPosixPreload.so
=== Start last 100 lines of lar log file ===
hit initial : 163.823 | hit correction : 1.21291
hit initial : 58.8364 | hit correction : 1.2129
hit initial : 117.459 | hit correction : 1.21115
hit initial : 126.778 | hit correction : 1.21121
hit initial : 110.291 | hit correction : 1.21128
hit initial : 109.119 | hit correction : 1.21134
hit initial : 120.837 | hit correction : 1.21141
hit initial : 117.835 | hit correction : 1.21151
hit initial : 120.435 | hit correction : 1.21162
hit initial : 119.948 | hit correction : 1.21168
hit initial : 114.218 | hit correction : 1.21172
hit initial : 117.646 | hit correction : 1.21173
hit initial : 117.664 | hit correction : 1.21175
hit initial : 122.372 | hit correction : 1.21186
hit initial : 135.113 | hit correction : 1.21179
hit initial : 44.3605 | hit correction : 1.21202
 *** Total Charge : 5359.17
 Looking at view: 2 | Number of hits in view: 32
Begin processing the 9th record. run: 5117 subRun: 0 event: 47259 at 30-Oct-2025 18:20:45 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
This pfp has size:  41
Added hit list length: 41 | Max was set to: -3.40282e+38
True Electron Energy: 25.4535
Completed MCParticle Loop
True Neutrino Energy: 30.317
 Size of all hits: 41
hit initial : 269.86 | hit correction : 1.04623
hit initial : 183.535 | hit correction : 1.04606
hit initial : 67.7192 | hit correction : 1.04621
hit initial : 136.883 | hit correction : 1.04548
hit initial : 71.32 | hit correction : 1.04546
hit initial : 69.6796 | hit correction : 1.04629
hit initial : 528.956 | hit correction : 1.0463
hit initial : 687.516 | hit correction : 1.04642
hit initial : 641.828 | hit correction : 1.04659
hit initial : 160.781 | hit correction : 1.04675
 *** Total Charge : 2948.66
 Looking at view: 0 | Number of hits in view: 10
hit initial : 58.6146 | hit correction : 1.04668
hit initial : 58.7578 | hit correction : 1.04665
hit initial : 54.2381 | hit correction : 1.04696
hit initial : 158.87 | hit correction : 1.04692
hit initial : 133.741 | hit correction : 1.0468
hit initial : 88.7951 | hit correction : 1.04558
hit initial : 169.159 | hit correction : 1.04603
hit initial : 267.001 | hit correction : 1.04557
hit initial : 89.1795 | hit correction : 1.04548
hit initial : 76.1558 | hit correction : 1.04633
hit initial : 219.617 | hit correction : 1.04623
hit initial : 253.962 | hit correction : 1.04612
hit initial : 56.3388 | hit correction : 1.04606
hit initial : 188.254 | hit correction : 1.04623
hit initial : 262.407 | hit correction : 1.04629
hit initial : 244.055 | hit correction : 1.0464
hit initial : 309.112 | hit correction : 1.0465
hit initial : 275.396 | hit correction : 1.04645
hit initial : 171.429 | hit correction : 1.04642
hit initial : 152.496 | hit correction : 1.04644
hit initial : 162.931 | hit correction : 1.04657
 *** Total Charge : 3610.26
 Looking at view: 1 | Number of hits in view: 21
hit initial : 114.33 | hit correction : 1.04696
hit initial : 366.911 | hit correction : 1.04673
hit initial : 318.132 | hit correction : 1.04619
hit initial : 228.248 | hit correction : 1.04575
hit initial : 223.676 | hit correction : 1.04548
hit initial : 207.635 | hit correction : 1.04631
hit initial : 356.189 | hit correction : 1.04606
hit initial : 360.221 | hit correction : 1.04642
hit initial : 808.558 | hit correction : 1.04643
hit initial : 326.175 | hit correction : 1.04644
 *** Total Charge : 3463.33
 Looking at view: 2 | Number of hits in view: 10
Begin processing the 10th record. run: 5117 subRun: 0 event: 47260 at 30-Oct-2025 18:20:45 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.568
True Electron Energy: 2.4591
Completed MCParticle Loop
True Neutrino Energy: 5.74306
 Size of all hits: 0
30-Oct-2025 18:20:45 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/a3/25/prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250210T155438Z_gen_000946_supernova_g4_halfActiveVol_detsim_reco_20250725T050227Z_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0206992     0.0255391     0.0411557     0.0246183    0.00554583       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.00120649    0.00147514    0.00196203    0.00135511    0.000250837      10     
end_path:ana:AnalyzeEvents         0.0193576     0.0239225     0.0392567     0.0227658    0.00541189       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 9263.11 MB
  Peak resident set size usage (VmHWM): 781.111 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0