Jobsub ID 240915.3@dunegpschedd01.fnal.gov

Jobsub ID	240915.3@dunegpschedd01.fnal.gov
Workflow ID	9468
Stage ID	1
User name	osbiston@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	GPU	No
	RSS bytes	2097152000 (2000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-10-30 12:23:40
Site	US_FNAL-FermiGrid
Entry	FNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat	2025-10-30 12:25:49
From worker node	Hostname	dunegli-7576962-0-fnpc9007.fnal.gov
	cpuinfo	Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	2097152000 (2000 MiB)
	Wall seconds limit	172800 (48 hours)
	GPU
	Inner Apptainer?	True
Job state	stalled
Started	2025-10-30 12:25:21
Input files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250210T063005Z_gen_002188_supernova_g4_halfActiveVol_detsim_reco_20250724T231813Z_reco2.root
Outputting started	2025-10-30 12:25:49
Output files
Finished	2025-10-30 12:31:30
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

cp: cannot stat '/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/7e4d8a7884851e80797d7d3c5855496e8acfe48e/*.xml': No such file or directory
Input PFN = root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/a5/04/prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250210T063005Z_gen_002188_supernova_g4_halfActiveVol_detsim_reco_20250724T231813Z_reco2.root
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/

MRB_PROJECT=larsoft
MRB_PROJECT_VERSION=v09_91_03
MRB_QUALS=e26:prof
MRB_TOP=/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/7e4d8a7884851e80797d7d3c5855496e8acfe48e
MRB_SOURCE=/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/7e4d8a7884851e80797d7d3c5855496e8acfe48e/srcs
MRB_BUILDDIR=/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/7e4d8a7884851e80797d7d3c5855496e8acfe48e/build_slf7.x86_64
MRB_INSTALL=/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/7e4d8a7884851e80797d7d3c5855496e8acfe48e/v09_91_03.prof/localProducts_larsoft_v09_91_03_e26_prof

PRODUCTS=/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/7e4d8a7884851e80797d7d3c5855496e8acfe48e/v09_91_03.prof/localProducts_larsoft_v09_91_03_e26_prof:/cvmfs/dune.opensciencegrid.org/products/dune:/cvmfs/larsoft.opensciencegrid.org/products:/cvmfs/larsoft.opensciencegrid.org/packages:/cvmfs/fermilab.opensciencegrid.org/products/common/db/
CETPKG_INSTALL=/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/7e4d8a7884851e80797d7d3c5855496e8acfe48e/v09_91_03.prof/localProducts_larsoft_v09_91_03_e26_prof

local product directory is /cvmfs/fifeuser1.opensciencegrid.org/sw/dune/7e4d8a7884851e80797d7d3c5855496e8acfe48e/v09_91_03.prof/localProducts_larsoft_v09_91_03_e26_prof
----------- this block should be empty ------------------
---------------------------------------------------------
/cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_5_5a/Linux64bit+3.10-2.17-e26-p3915-prof/lib/libXrdPosixPreload.so
=== Start last 100 lines of lar log file ===
hit initial : 64.6037 | hit correction : 1.12291
hit initial : 146.246 | hit correction : 1.14225
hit initial : 110.107 | hit correction : 1.14236
hit initial : 319.152 | hit correction : 1.1429
hit initial : 94.5458 | hit correction : 1.14243
hit initial : 126.015 | hit correction : 1.14227
hit initial : 61.9268 | hit correction : 1.14281
hit initial : 172.889 | hit correction : 1.1429
hit initial : 162.781 | hit correction : 1.14307
hit initial : 217.967 | hit correction : 1.14274
hit initial : 106.527 | hit correction : 1.14286
hit initial : 257.148 | hit correction : 1.14382
hit initial : 7.29951 | hit correction : 1.14422
hit initial : 43.1167 | hit correction : 1.14447
hit initial : 61.4748 | hit correction : 1.14457
hit initial : 122.08 | hit correction : 1.14492
hit initial : 239.977 | hit correction : 1.1453
hit initial : 14.1572 | hit correction : 1.14568
hit initial : 178.59 | hit correction : 1.14579
hit initial : 349.105 | hit correction : 1.14628
hit initial : 350.706 | hit correction : 1.1467
hit initial : 92.8783 | hit correction : 1.14678
hit initial : 119.164 | hit correction : 1.14771
hit initial : 535.449 | hit correction : 1.14787
hit initial : 109.539 | hit correction : 1.14812
hit initial : 682.575 | hit correction : 1.14826
 *** Total Charge : 5533.05
 Looking at view: 0 | Number of hits in view: 27
hit initial : 3.99589 | hit correction : 1.14871
hit initial : 112.45 | hit correction : 1.14232
hit initial : 99.971 | hit correction : 1.14293
hit initial : 273.098 | hit correction : 1.14233
hit initial : 231.158 | hit correction : 1.14288
hit initial : 324.282 | hit correction : 1.14289
hit initial : 197.158 | hit correction : 1.14234
hit initial : 299.249 | hit correction : 1.14303
hit initial : 274.832 | hit correction : 1.14376
hit initial : 334.025 | hit correction : 1.14437
hit initial : 130.333 | hit correction : 1.1449
hit initial : 187.468 | hit correction : 1.14497
hit initial : 60.5301 | hit correction : 1.14482
hit initial : 229.507 | hit correction : 1.14557
hit initial : 189.868 | hit correction : 1.14588
hit initial : 217.957 | hit correction : 1.14633
hit initial : 194.07 | hit correction : 1.14664
hit initial : 197.209 | hit correction : 1.1469
hit initial : 109.486 | hit correction : 1.14715
hit initial : 112.116 | hit correction : 1.14749
hit initial : 208.871 | hit correction : 1.14788
hit initial : 181.386 | hit correction : 1.14808
hit initial : 159.729 | hit correction : 1.14832
hit initial : 148.005 | hit correction : 1.1486
hit initial : 142.117 | hit correction : 1.14891
hit initial : 101.996 | hit correction : 1.14913
 *** Total Charge : 5406.29
 Looking at view: 1 | Number of hits in view: 26
hit initial : 51.1151 | hit correction : 1.14029
hit initial : 222.023 | hit correction : 1.14228
hit initial : 160.726 | hit correction : 1.14248
hit initial : 154.027 | hit correction : 1.14227
hit initial : 344.172 | hit correction : 1.14287
hit initial : 139.595 | hit correction : 1.14291
hit initial : 296.589 | hit correction : 1.14278
hit initial : 251.728 | hit correction : 1.14316
hit initial : 392.303 | hit correction : 1.14385
hit initial : 374.404 | hit correction : 1.14565
hit initial : 1319.9 | hit correction : 1.14624
hit initial : 37.6414 | hit correction : 1.14646
hit initial : 497.851 | hit correction : 1.14764
hit initial : 353.757 | hit correction : 1.14813
hit initial : 381.265 | hit correction : 1.14875
hit initial : 163.037 | hit correction : 1.14916
 *** Total Charge : 5887.65
 Looking at view: 2 | Number of hits in view: 16
Begin processing the 10th record. run: 5102 subRun: 0 event: 109360 at 30-Oct-2025 12:25:48 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.566
True Electron Energy: 2.08784
Completed MCParticle Loop
True Neutrino Energy: 5.37156
 Size of all hits: 0
30-Oct-2025 12:25:49 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/a5/04/prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250210T063005Z_gen_002188_supernova_g4_halfActiveVol_detsim_reco_20250724T231813Z_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0261835     0.0440213     0.082477      0.0349571     0.020496        10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.00151754    0.00308955     0.0129437     0.0020463    0.00329918       10     
end_path:ana:AnalyzeEvents         0.0239855     0.0407523     0.0797236     0.0288184     0.0206424       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 9265.26 MB
  Peak resident set size usage (VmHWM): 783.008 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0