Jobsub ID 267273.1@dunegpschedd02.fnal.gov

Jobsub ID	267273.1@dunegpschedd02.fnal.gov
Workflow ID	11594
Stage ID	1
User name	weishi@fnal.gov
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	24000 (6 hours)
Submitted time	2026-01-09 13:54:20
Site	US_UChicago
Entry	Engage_US_MWT2_iut2_condce_mcore
Last heartbeat	2026-01-09 18:15:11
From worker node	Hostname	iut2-c214.iu.edu
	cpuinfo	Intel(R) Xeon(R) CPU E5-2650 v3 @ 2.30GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	86400 (24 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2026-01-09 14:06:01
Input files	fardet-vd:prodbackground_radiological_decay0_dunevd10kt_1x8x14_20250728T090526Z_gen_001301_supernova_g4stage1_g4stage2_detsim_reco_20250809T075255Z_reco2.root
Jobscript	Exit code	0
	Real time	4h (14929s)
	CPU time	4h (14564s = 97%)
	Max RSS bytes	2751647744 (2624 MiB)
Outputting started	2026-01-09 18:14:51
Output files	https://fndcadoor.fnal.gov:2880/dune/scratch/users/weishi/VDMarleyCCBkgAna/fnal/11594/1/001/reco2_QLMatch_ana_2026-01-09T_140625Z.root
Finished	2026-01-09 18:15:11
Saved logs	justin-logs:267273.1-dunegpschedd02.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

s: 1598
  bkg 0 NParticles: 3
  bkg 0 NParticles: 1
  bkg 0 NParticles: 3
  bkg 0 NParticles: 26
  bkg 0 NParticles: 52
  bkg 0 NParticles: 54
  bkg 0 NParticles: 43
  bkg 0 NParticles: 31
  bkg 0 NParticles: 83870
  bkg 0 NParticles: 3420
  bkg 0 NParticles: 0
  bkg 0 NParticles: 0
  bkg 0 NParticles: 0

=========== BlipRecoAlg =========================
Event 13005 / run 87
Found 20278 hits from gaushit
Found 0 tracks from pandoraTrack
Found 131996 MC particles from largeant
Found 766938 SimEnergyDeposits from IonAndScint
Found 96768 SimChannels from tpcrawdecoder:simpleSC
Begin processing the 6th record. run: 87 subRun: 0 event: 13006 at 09-Jan-2026 11:05:51 EST
  bkg 0 NParticles: 30933
  bkg 0 NParticles: 3657
  bkg 0 NParticles: 4
  bkg 0 NParticles: 0
  bkg 0 NParticles: 21
  bkg 0 NParticles: 27
  bkg 0 NParticles: 47
  bkg 0 NParticles: 34
  bkg 0 NParticles: 27
  bkg 0 NParticles: 33
  bkg 0 NParticles: 230
  bkg 0 NParticles: 2960
  bkg 0 NParticles: 3705
  bkg 0 NParticles: 115
  bkg 0 NParticles: 1357
  bkg 0 NParticles: 1624
  bkg 0 NParticles: 0
  bkg 0 NParticles: 0
  bkg 0 NParticles: 2
  bkg 0 NParticles: 24
  bkg 0 NParticles: 31
  bkg 0 NParticles: 60
  bkg 0 NParticles: 27
  bkg 0 NParticles: 63
  bkg 0 NParticles: 83944
  bkg 0 NParticles: 3521
  bkg 0 NParticles: 1
  bkg 0 NParticles: 0
  bkg 0 NParticles: 1

=========== BlipRecoAlg =========================
Event 13006 / run 87
Found 20013 hits from gaushit
Found 0 tracks from pandoraTrack
Found 132464 MC particles from largeant
Found 775192 SimEnergyDeposits from IonAndScint
Found 96768 SimChannels from tpcrawdecoder:simpleSC
Begin processing the 7th record. run: 87 subRun: 0 event: 13007 at 09-Jan-2026 11:29:29 EST
  bkg 0 NParticles: 31332
  bkg 0 NParticles: 3469
  bkg 0 NParticles: 2
  bkg 0 NParticles: 0
  bkg 0 NParticles: 30
  bkg 0 NParticles: 31
  bkg 0 NParticles: 60
  bkg 0 NParticles: 50
  bkg 0 NParticles: 15
  bkg 0 NParticles: 30
  bkg 0 NParticles: 215
  bkg 0 NParticles: 2985
  bkg 0 NParticles: 3527
  bkg 0 NParticles: 102
  bkg 0 NParticles: 1399
  bkg 0 NParticles: 1491
  bkg 0 NParticles: 0
  bkg 0 NParticles: 1
  bkg 0 NParticles: 5
  bkg 0 NParticles: 12
  bkg 0 NParticles: 56
  bkg 0 NParticles: 75
  bkg 0 NParticles: 33
  bkg 0 NParticles: 29
  bkg 0 NParticles: 83597
  bkg 0 NParticles: 3465
  bkg 0 NParticles: 0
  bkg 0 NParticles: 0
  bkg 0 NParticles: 0

=========== BlipRecoAlg =========================
Event 13007 / run 87
Found 19687 hits from gaushit
Found 0 tracks from pandoraTrack
Found 132017 MC particles from largeant
Found 778498 SimEnergyDeposits from IonAndScint
Found 96768 SimChannels from tpcrawdecoder:simpleSC
Begin processing the 8th record. run: 87 subRun: 0 event: 13008 at 09-Jan-2026 11:58:32 EST
  bkg 0 NParticles: 31169
  bkg 0 NParticles: 3656
  bkg 0 NParticles: 4
  bkg 0 NParticles: 2
  bkg 0 NParticles: 41
  bkg 0 NParticles: 23
  bkg 0 NParticles: 58
  bkg 0 NParticles: 34
  bkg 0 NParticles: 17
  bkg 0 NParticles: 31
  bkg 0 NParticles: 244
  bkg 0 NParticles: 2838
  bkg 0 NParticles: 3591
  bkg 0 NParticles: 100
  bkg 0 NParticles: 1407
  bkg 0 NParticles: 1768
  bkg 0 NParticles: 1
  bkg 0 NParticles: 0
  bkg 0 NParticles: 1
  bkg 0 NParticles: 10
  bkg 0 NParticles: 28
  bkg 0 NParticles: 35
  bkg 0 NParticles: 36
  bkg 0 NParticles: 51
  bkg 0 NParticles: 83484
  bkg 0 NParticles: 3466
  bkg 0 NParticles: 0
  bkg 0 NParticles: 0
  bkg 0 NParticles: 0

=========== BlipRecoAlg =========================
Event 13008 / run 87
Found 20120 hits from gaushit
Found 0 tracks from pandoraTrack
Found 132098 MC particles from largeant
Found 774810 SimEnergyDeposits from IonAndScint
Found 96768 SimChannels from tpcrawdecoder:simpleSC
Begin processing the 9th record. run: 87 subRun: 0 event: 13009 at 09-Jan-2026 12:22:37 EST
  bkg 0 NParticles: 30965
  bkg 0 NParticles: 3536
  bkg 0 NParticles: 6
  bkg 0 NParticles: 1
  bkg 0 NParticles: 35
  bkg 0 NParticles: 28
  bkg 0 NParticles: 69
  bkg 0 NParticles: 57
  bkg 0 NParticles: 24
  bkg 0 NParticles: 37
  bkg 0 NParticles: 220
  bkg 0 NParticles: 3066
  bkg 0 NParticles: 3605
  bkg 0 NParticles: 102
  bkg 0 NParticles: 1374
  bkg 0 NParticles: 1558
  bkg 0 NParticles: 0
  bkg 0 NParticles: 1
  bkg 0 NParticles: 4
  bkg 0 NParticles: 16
  bkg 0 NParticles: 55
  bkg 0 NParticles: 22
  bkg 0 NParticles: 39
  bkg 0 NParticles: 52
  bkg 0 NParticles: 83844
  bkg 0 NParticles: 3566
  bkg 0 NParticles: 0
  bkg 0 NParticles: 0
  bkg 0 NParticles: 0

=========== BlipRecoAlg =========================
Event 13009 / run 87
Found 20045 hits from gaushit
Found 0 tracks from pandoraTrack
Found 132282 MC particles from largeant
Found 774414 SimEnergyDeposits from IonAndScint
Found 96768 SimChannels from tpcrawdecoder:simpleSC
Begin processing the 10th record. run: 87 subRun: 0 event: 13010 at 09-Jan-2026 12:47:51 EST
  bkg 0 NParticles: 31015
  bkg 0 NParticles: 3482
  bkg 0 NParticles: 2
  bkg 0 NParticles: 0
  bkg 0 NParticles: 31
  bkg 0 NParticles: 16
  bkg 0 NParticles: 53
  bkg 0 NParticles: 35
  bkg 0 NParticles: 26
  bkg 0 NParticles: 33
  bkg 0 NParticles: 247
  bkg 0 NParticles: 3018
  bkg 0 NParticles: 3588
  bkg 0 NParticles: 107
  bkg 0 NParticles: 1344
  bkg 0 NParticles: 1657
  bkg 0 NParticles: 1
  bkg 0 NParticles: 1
  bkg 0 NParticles: 2
  bkg 0 NParticles: 31
  bkg 0 NParticles: 36
  bkg 0 NParticles: 62
  bkg 0 NParticles: 43
  bkg 0 NParticles: 48
  bkg 0 NParticles: 84313
  bkg 0 NParticles: 3534
  bkg 0 NParticles: 1
  bkg 0 NParticles: 1
  bkg 0 NParticles: 0

=========== BlipRecoAlg =========================
Event 13010 / run 87
Found 20258 hits from gaushit
Found 0 tracks from pandoraTrack
Found 132738 MC particles from largeant
Found 786903 SimEnergyDeposits from IonAndScint
Found 96768 SimChannels from tpcrawdecoder:simpleSC
09-Jan-2026 13:14:44 EST  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/5b/59/prodbackground_radiological_decay0_dunevd10kt_1x8x14_20250728T090526Z_gen_001301_supernova_g4stage1_g4stage2_detsim_reco_20250809T075255Z_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                          1354.32       1477.91       1734.6        1439.86       113.587        10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000965943    0.0222906     0.0449685     0.0200129     0.0139102       10     
end_path:QLMatchAna:QLMatchAna      1354.29       1477.87       1734.55       1439.82       113.585        10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3572.49 MB
  Peak resident set size usage (VmHWM): 2751.65 MB
  Details saved in: 'mem.db'
====================================================================================================
terminate called after throwing an instance of 'cet::coded_exception<art::errors::ErrorCodes, &art::ExceptionDetail::translate[abi:cxx11]>'
  what():  ---- FatalRootError BEGIN
  Fatal Root Error: TClass::LoadClassInfo
  no interpreter information for class TF1 is available even though it has a TClass initialization routine.
  ROOT severity: 3000
---- FatalRootError END

../justin-jobscript: line 79:  1149 Aborted                 (core dumped) lar -c $ANA_FILE $events_option -T ${outAnaFile}.root $pfn
lar exit code 134
PYTHONHOME: 
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17
PYTHONPATH: 
/cvmfs/dune.opensciencegrid.org/products/dune/protoduneana/v09_91_04d01/slf7.x86_64.e26.prof/bin:/cvmfs/dune.opensciencegrid.org/products/dune/dunesim/v09_91_04d01/slf7.x86_64.e26.prof/bin:/cvmfs/larsoft.opensciencegrid.org/products/larsoft/v09_91_04_01/bin/python:/cvmfs/larsoft.opensciencegrid.org/products/torch_scatter/v2_1_2a/Linux64bit+3.10-2.17-e26/lib64:/cvmfs/larsoft.opensciencegrid.org/products/libtorch/v2_1_1b/Linux64bit+3.10-2.17-e26/lib:/cvmfs/larsoft.opensciencegrid.org/products/ifdhc/v2_7/Linux64bit+3.10-2.17-e26-p3915-prof/lib/python:/cvmfs/larsoft.opensciencegrid.org/products/pythia8/v8_3_10/Linux64bit+3.10-2.17-e26-p3915-prof/lib/python3.9/site-packages:/cvmfs/larsoft.opensciencegrid.org/products/hepmc3/v3_2_7/Linux64bit+3.10-2.17-e26-p3915-prof/lib/python3.9/site-packages:/cvmfs/larsoft.opensciencegrid.org/products/lhapdf/v6_5_4/Linux64bit+3.10-2.17-e26-p3915-prof/lib/python3.9/site-packages:/cvmfs/dune.opensciencegrid.org/products/dune/duneutil/v09_91_04d01/python:/cvmfs/fermilab.opensciencegrid.org/products/common/db//../prd/python_future_six_request/v1_3_1/Linux64bit-3-10-2-17-python3-9:/cvmfs/larsoft.opensciencegrid.org/products/larbatch/v01_59_05/python:/cvmfs/larsoft.opensciencegrid.org/products/larbatch/v01_59_05/bin:/cvmfs/fermilab.opensciencegrid.org/products/common/db//../prd/sam_web_client/v3_6/NULL/python:/cvmfs/larsoft.opensciencegrid.org/products/TRACE/v3_17_11/python:/cvmfs/larsoft.opensciencegrid.org/products/pygccxml/v2_2_1b/lib/python3.9/site-packages:/cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_5_5a/Linux64bit+3.10-2.17-e26-p3915-prof/lib:/cvmfs/larsoft.opensciencegrid.org/products/postgresql/v15_3/Linux64bit+3.10-2.17-p3915/lib/python3.9/site-packages:/cvmfs/larsoft.opensciencegrid.org/products/numpy/v1_24_3/Linux64bit+3.10-2.17-e26-p3915/lib/python3.9/site-packages:/cvmfs/larsoft.opensciencegrid.org/products/root/v6_28_12/Linux64bit+3.10-2.17-e26-p3915-prof/lib
job script exit code