Jobsub ID 272944.0@dunegpschedd01.fnal.gov

Jobsub ID	272944.0@dunegpschedd01.fnal.gov
Workflow ID	11594
Stage ID	1
User name	weishi@fnal.gov
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	24000 (6 hours)
Submitted time	2026-01-09 12:54:17
Site	US_UCSD
Entry	CMSHTPC_T2_US_UCSD_gw6
Last heartbeat	2026-01-09 15:19:51
From worker node	Hostname	mh-7763-11.t2.ucsd.edu
	cpuinfo	AMD EPYC 7763 64-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2026-01-09 12:56:08
Input files	fardet-vd:prodbackground_radiological_decay0_dunevd10kt_1x8x14_20250726T233127Z_gen_007735_supernova_g4stage1_g4stage2_detsim_reco_20250809T093541Z_reco2.root
Jobscript	Exit code	0
	Real time	2h (8597s)
	CPU time	2h (8014s = 93%)
	Max RSS bytes	2713825280 (2588 MiB)
Outputting started	2026-01-09 15:19:26
Output files	https://fndcadoor.fnal.gov:2880/dune/scratch/users/weishi/VDMarleyCCBkgAna/fnal/11594/1/001/reco2_QLMatch_ana_2026-01-09T_125827Z.root
Finished	2026-01-09 15:19:51
Saved logs	justin-logs:272944.0-dunegpschedd01.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

icles: 1607
  bkg 0 NParticles: 3
  bkg 0 NParticles: 0
  bkg 0 NParticles: 1
  bkg 0 NParticles: 18
  bkg 0 NParticles: 72
  bkg 0 NParticles: 56
  bkg 0 NParticles: 52
  bkg 0 NParticles: 41
  bkg 0 NParticles: 83870
  bkg 0 NParticles: 3420
  bkg 0 NParticles: 0
  bkg 0 NParticles: 0
  bkg 0 NParticles: 0

=========== BlipRecoAlg =========================
Event 77345 / run 79
Found 19892 hits from gaushit
Found 0 tracks from pandoraTrack
Found 132338 MC particles from largeant
Found 780978 SimEnergyDeposits from IonAndScint
Found 96768 SimChannels from tpcrawdecoder:simpleSC
Begin processing the 6th record. run: 79 subRun: 0 event: 77346 at 09-Jan-2026 06:10:43 PST
  bkg 0 NParticles: 30873
  bkg 0 NParticles: 3492
  bkg 0 NParticles: 2
  bkg 0 NParticles: 3
  bkg 0 NParticles: 30
  bkg 0 NParticles: 23
  bkg 0 NParticles: 45
  bkg 0 NParticles: 62
  bkg 0 NParticles: 11
  bkg 0 NParticles: 35
  bkg 0 NParticles: 232
  bkg 0 NParticles: 2989
  bkg 0 NParticles: 3602
  bkg 0 NParticles: 82
  bkg 0 NParticles: 1463
  bkg 0 NParticles: 1597
  bkg 0 NParticles: 3
  bkg 0 NParticles: 1
  bkg 0 NParticles: 1
  bkg 0 NParticles: 24
  bkg 0 NParticles: 35
  bkg 0 NParticles: 71
  bkg 0 NParticles: 36
  bkg 0 NParticles: 77
  bkg 0 NParticles: 83944
  bkg 0 NParticles: 3521
  bkg 0 NParticles: 1
  bkg 0 NParticles: 0
  bkg 0 NParticles: 1

=========== BlipRecoAlg =========================
Event 77346 / run 79
Found 19711 hits from gaushit
Found 0 tracks from pandoraTrack
Found 132280 MC particles from largeant
Found 770947 SimEnergyDeposits from IonAndScint
Found 96768 SimChannels from tpcrawdecoder:simpleSC
Begin processing the 7th record. run: 79 subRun: 0 event: 77347 at 09-Jan-2026 06:24:27 PST
  bkg 0 NParticles: 30977
  bkg 0 NParticles: 3562
  bkg 0 NParticles: 3
  bkg 0 NParticles: 0
  bkg 0 NParticles: 36
  bkg 0 NParticles: 29
  bkg 0 NParticles: 46
  bkg 0 NParticles: 55
  bkg 0 NParticles: 20
  bkg 0 NParticles: 21
  bkg 0 NParticles: 230
  bkg 0 NParticles: 3021
  bkg 0 NParticles: 3471
  bkg 0 NParticles: 107
  bkg 0 NParticles: 1310
  bkg 0 NParticles: 1682
  bkg 0 NParticles: 3
  bkg 0 NParticles: 3
  bkg 0 NParticles: 4
  bkg 0 NParticles: 10
  bkg 0 NParticles: 61
  bkg 0 NParticles: 65
  bkg 0 NParticles: 36
  bkg 0 NParticles: 65
  bkg 0 NParticles: 83597
  bkg 0 NParticles: 3465
  bkg 0 NParticles: 0
  bkg 0 NParticles: 0
  bkg 0 NParticles: 0

=========== BlipRecoAlg =========================
Event 77347 / run 79
Found 20089 hits from gaushit
Found 0 tracks from pandoraTrack
Found 131879 MC particles from largeant
Found 780777 SimEnergyDeposits from IonAndScint
Found 96768 SimChannels from tpcrawdecoder:simpleSC
Begin processing the 8th record. run: 79 subRun: 0 event: 77348 at 09-Jan-2026 06:38:16 PST
  bkg 0 NParticles: 31000
  bkg 0 NParticles: 3628
  bkg 0 NParticles: 0
  bkg 0 NParticles: 1
  bkg 0 NParticles: 29
  bkg 0 NParticles: 30
  bkg 0 NParticles: 52
  bkg 0 NParticles: 19
  bkg 0 NParticles: 30
  bkg 0 NParticles: 37
  bkg 0 NParticles: 211
  bkg 0 NParticles: 2923
  bkg 0 NParticles: 3598
  bkg 0 NParticles: 92
  bkg 0 NParticles: 1317
  bkg 0 NParticles: 1560
  bkg 0 NParticles: 3
  bkg 0 NParticles: 1
  bkg 0 NParticles: 3
  bkg 0 NParticles: 21
  bkg 0 NParticles: 32
  bkg 0 NParticles: 65
  bkg 0 NParticles: 30
  bkg 0 NParticles: 49
  bkg 0 NParticles: 83484
  bkg 0 NParticles: 3466
  bkg 0 NParticles: 0
  bkg 0 NParticles: 0
  bkg 0 NParticles: 0

=========== BlipRecoAlg =========================
Event 77348 / run 79
Found 19593 hits from gaushit
Found 0 tracks from pandoraTrack
Found 131681 MC particles from largeant
Found 770396 SimEnergyDeposits from IonAndScint
Found 96768 SimChannels from tpcrawdecoder:simpleSC
Begin processing the 9th record. run: 79 subRun: 0 event: 77349 at 09-Jan-2026 06:51:45 PST
  bkg 0 NParticles: 31089
  bkg 0 NParticles: 3620
  bkg 0 NParticles: 6
  bkg 0 NParticles: 0
  bkg 0 NParticles: 31
  bkg 0 NParticles: 30
  bkg 0 NParticles: 62
  bkg 0 NParticles: 36
  bkg 0 NParticles: 13
  bkg 0 NParticles: 47
  bkg 0 NParticles: 228
  bkg 0 NParticles: 3016
  bkg 0 NParticles: 3505
  bkg 0 NParticles: 92
  bkg 0 NParticles: 1321
  bkg 0 NParticles: 1497
  bkg 0 NParticles: 0
  bkg 0 NParticles: 2
  bkg 0 NParticles: 2
  bkg 0 NParticles: 20
  bkg 0 NParticles: 34
  bkg 0 NParticles: 43
  bkg 0 NParticles: 40
  bkg 0 NParticles: 59
  bkg 0 NParticles: 83844
  bkg 0 NParticles: 3566
  bkg 0 NParticles: 0
  bkg 0 NParticles: 0
  bkg 0 NParticles: 0

=========== BlipRecoAlg =========================
Event 77349 / run 79
Found 19708 hits from gaushit
Found 0 tracks from pandoraTrack
Found 132204 MC particles from largeant
Found 774178 SimEnergyDeposits from IonAndScint
Found 96768 SimChannels from tpcrawdecoder:simpleSC
Begin processing the 10th record. run: 79 subRun: 0 event: 77350 at 09-Jan-2026 07:05:26 PST
  bkg 0 NParticles: 31064
  bkg 0 NParticles: 3595
  bkg 0 NParticles: 2
  bkg 0 NParticles: 0
  bkg 0 NParticles: 32
  bkg 0 NParticles: 27
  bkg 0 NParticles: 56
  bkg 0 NParticles: 50
  bkg 0 NParticles: 18
  bkg 0 NParticles: 12
  bkg 0 NParticles: 245
  bkg 0 NParticles: 3113
  bkg 0 NParticles: 3591
  bkg 0 NParticles: 90
  bkg 0 NParticles: 1407
  bkg 0 NParticles: 1542
  bkg 0 NParticles: 0
  bkg 0 NParticles: 1
  bkg 0 NParticles: 1
  bkg 0 NParticles: 17
  bkg 0 NParticles: 51
  bkg 0 NParticles: 61
  bkg 0 NParticles: 36
  bkg 0 NParticles: 36
  bkg 0 NParticles: 84313
  bkg 0 NParticles: 3534
  bkg 0 NParticles: 1
  bkg 0 NParticles: 1
  bkg 0 NParticles: 0

=========== BlipRecoAlg =========================
Event 77350 / run 79
Found 20025 hits from gaushit
Found 0 tracks from pandoraTrack
Found 132918 MC particles from largeant
Found 782820 SimEnergyDeposits from IonAndScint
Found 96768 SimChannels from tpcrawdecoder:simpleSC
09-Jan-2026 07:19:22 PST  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/e0/2b/prodbackground_radiological_decay0_dunevd10kt_1x8x14_20250726T233127Z_gen_007735_supernova_g4stage1_g4stage2_detsim_reco_20250809T093541Z_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                          774.654       810.265       907.25        798.832       35.1446        10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.00113522     0.0412629     0.0849652     0.0394815     0.0330149       10     
end_path:QLMatchAna:QLMatchAna      774.642       810.211       907.177       798.777       35.1341        10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3574.99 MB
  Peak resident set size usage (VmHWM): 2713.83 MB
  Details saved in: 'mem.db'
====================================================================================================
terminate called after throwing an instance of 'cet::coded_exception<art::errors::ErrorCodes, &art::ExceptionDetail::translate[abi:cxx11]>'
  what():  ---- FatalRootError BEGIN
  Fatal Root Error: TClass::LoadClassInfo
  no interpreter information for class TF1 is available even though it has a TClass initialization routine.
  ROOT severity: 3000
---- FatalRootError END

../justin-jobscript: line 79:  1152 Aborted                 (core dumped) lar -c $ANA_FILE $events_option -T ${outAnaFile}.root $pfn
lar exit code 134
PYTHONHOME: 
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17
PYTHONPATH: 
/cvmfs/dune.opensciencegrid.org/products/dune/protoduneana/v09_91_04d01/slf7.x86_64.e26.prof/bin:/cvmfs/dune.opensciencegrid.org/products/dune/dunesim/v09_91_04d01/slf7.x86_64.e26.prof/bin:/cvmfs/larsoft.opensciencegrid.org/products/larsoft/v09_91_04_01/bin/python:/cvmfs/larsoft.opensciencegrid.org/products/torch_scatter/v2_1_2a/Linux64bit+3.10-2.17-e26/lib64:/cvmfs/larsoft.opensciencegrid.org/products/libtorch/v2_1_1b/Linux64bit+3.10-2.17-e26/lib:/cvmfs/larsoft.opensciencegrid.org/products/ifdhc/v2_7/Linux64bit+3.10-2.17-e26-p3915-prof/lib/python:/cvmfs/larsoft.opensciencegrid.org/products/pythia8/v8_3_10/Linux64bit+3.10-2.17-e26-p3915-prof/lib/python3.9/site-packages:/cvmfs/larsoft.opensciencegrid.org/products/hepmc3/v3_2_7/Linux64bit+3.10-2.17-e26-p3915-prof/lib/python3.9/site-packages:/cvmfs/larsoft.opensciencegrid.org/products/lhapdf/v6_5_4/Linux64bit+3.10-2.17-e26-p3915-prof/lib/python3.9/site-packages:/cvmfs/dune.opensciencegrid.org/products/dune/duneutil/v09_91_04d01/python:/cvmfs/fermilab.opensciencegrid.org/products/common/db//../prd/python_future_six_request/v1_3_1/Linux64bit-3-10-2-17-python3-9:/cvmfs/larsoft.opensciencegrid.org/products/larbatch/v01_59_05/python:/cvmfs/larsoft.opensciencegrid.org/products/larbatch/v01_59_05/bin:/cvmfs/fermilab.opensciencegrid.org/products/common/db//../prd/sam_web_client/v3_6/NULL/python:/cvmfs/larsoft.opensciencegrid.org/products/TRACE/v3_17_11/python:/cvmfs/larsoft.opensciencegrid.org/products/pygccxml/v2_2_1b/lib/python3.9/site-packages:/cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_5_5a/Linux64bit+3.10-2.17-e26-p3915-prof/lib:/cvmfs/larsoft.opensciencegrid.org/products/postgresql/v15_3/Linux64bit+3.10-2.17-p3915/lib/python3.9/site-packages:/cvmfs/larsoft.opensciencegrid.org/products/numpy/v1_24_3/Linux64bit+3.10-2.17-e26-p3915/lib/python3.9/site-packages:/cvmfs/larsoft.opensciencegrid.org/products/root/v6_28_12/Linux64bit+3.10-2.17-e26-p3915-prof/lib
job script exit code