Jobsub ID 241949.0@dunegpschedd02.fnal.gov

Jobsub ID	241949.0@dunegpschedd02.fnal.gov
Workflow ID	9965
Stage ID	1
User name	smanthey@fnal.gov
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-11-12 04:40:44
Site	US_FNAL-FermiGrid
Entry	FNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat	2025-11-13 07:46:12
From worker node	Hostname	dunegli-7572336-0-fnpc23037.fnal.gov
	cpuinfo	AMD EPYC 7543 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	172800 (48 hours)
	GPU
	Inner Apptainer?	True
Job state	outputting_failed
Started	2025-11-13 04:33:52
Input files	fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241024T014352Z_gen_004907_supernova_g4stage1_g4stage2_detsim_reco_20250611T113943Z_reco2.root
Jobscript	Exit code	0
	Real time	3h (11531s)
	CPU time	3h (11427s = 99%)
	Max RSS bytes	1915633664 (1826 MiB)
Outputting started	2025-11-13 07:46:03
Output files
Finished	2025-11-13 07:46:12
List job events (HTCondor job logs unavailable)
Jobscript log (last 10,000 characters)

6, 297.68)	132417	132290
marley	1000180400	0.00	(139.05, -450.76, 313.66)	132416	132290
marley	1000180400	0.00	(145.68, -449.45, 322.61)	132415	132290
marley	1000180400	0.00	(145.32, -449.94, 324.16)	132414	132290
marley	1000180360	0.00	(204.94, -589.97, 310.06)	132412	132290
marley	1000180360	0.00	(232.67, -583.36, 342.41)	132411	132290
marley	2212		0.00	(436.84, -613.12, 335.38)	132410	132290
marley	2212		0.00	(440.09, -613.33, 335.86)	132409	132290
marley	1000060120	0.00	(449.61, -616.71, 344.02)	132408	132290
marley	1000060120	0.00	(447.12, -618.39, 349.00)	132407	132290
marley	2212		0.00	(445.55, -612.35, 353.17)	132406	132290
marley	2212		0.00	(438.24, -611.06, 364.21)	132405	132290
marley	2212		0.01	(437.98, -610.79, 364.53)	132404	132290
marley	2212		0.09	(442.86, -581.71, 372.35)	132403	132290
marley	1000180400	0.00	(322.24, -441.12, 265.93)	132402	132290
marley	1000180400	0.00	(320.36, -428.60, 272.98)	132401	132290
marley	1000180400	0.00	(309.66, -428.59, 283.59)	132400	132290
marley	1000180400	0.01	(309.57, -428.91, 283.89)	132399	132290
marley	1000180400	0.01	(311.54, -377.55, 301.41)	132398	132290
marley	1000180400	0.00	(200.43, -406.39, 293.49)	132397	132290
marley	1000180400	0.00	(194.35, -402.56, 289.63)	132396	132290
marley	1000180400	0.01	(178.96, -377.90, 305.28)	132395	132290
marley	1000180400	0.00	(129.95, -360.41, 260.72)	132394	132290
marley	1000180400	0.01	(129.94, -354.07, 254.76)	132393	132290
marley	1000180360	0.02	(111.12, -393.70, 236.72)	132392	132290
marley	1000180400	0.00	(139.17, -367.52, 235.15)	132391	132290
marley	1000180400	0.02	(144.59, -366.85, 230.62)	132390	132290
marley	1000180400	0.01	(136.14, -372.36, 235.39)	132389	132290
marley	1000180400	0.02	(168.43, -365.12, 257.60)	132388	132290
marley	1000180400	0.01	(167.18, -365.38, 254.82)	132387	132290
marley	1000180400	0.00	(219.88, -329.30, 211.50)	132386	132290
marley	1000180400	0.01	(221.02, -325.54, 205.21)	132385	132290
marley	11		0.71	(172.96, -307.30, 209.02)	132339	132291
marley	-12		0.60	(172.96, -307.30, 209.02)	132338	132291
marley	1000200400	0.00	(172.96, -307.30, 209.02)	132337	132291
marley	22		0.03	(172.96, -307.30, 209.02)	132294	0
marley	22		1.61	(172.96, -307.30, 209.02)	132293	0
marley	22		0.65	(172.96, -307.30, 209.02)	132292	0
marley	1000190400	0.14	(172.96, -307.30, 209.02)	132291	0
marley	11		50.50	(172.96, -307.30, 209.02)	132290	0
Signal Track IDs: 132290; 132291; 132292; 132293; 132294; 132337; 132338; 132339; 132385; 132386; 132387; 132388; 132389; 132390; 132391; 132392; 132393; 132394; 132395; 132396; 132397; 132398; 132399; 132400; 132401; 132402; 132403; 132404; 132405; 132406; 132407; 132408; 132409; 132410; 132411; 132412; 132414; 132415; 132416; 132417; 132418; 132419; 132420; 132423; 132424; 132425; 132426; 132427; 132432; 132433; 132434; 132435; 132436; 132437; 132441; 132442; 132443; 132445; 132446; 132447; 132450; 132451; 132452; 132454; 132455; 132458; 132479; 132528; 132529; 132530; 132531; 132532; 132533; 132534; 132535; 132536; 132537; 132655; 132656; 132657; 132658; 132659; 132660; 132661; 132662; 132663; 132664; 132665; 132666; 132667; 132668; 132669; 132670; 132671; 132672; 132673; 132674; 132675; 132676; 132677; 132678; 132679; 132680; 132681; 132682; 132683; 132684; 132685; 132686; 132687; 132688; 132689; 132690; 132691; 132692; 132693; 132694; 132695; 132696; 132697; 132698; 132699; 132700; 132701; 132702; 132703; 132704; 132705; 132706; 132707; 132708; 132709; 132710; 132711; 132712; 132713; 132714; 132715; 132716; 132717; 132718; 132719; 132720; 132721; 132722; 132723; 132724; 132749; 132750; 132752; 132753; [0m
%MSG
%MSG-w ProducerUtils:  SolarNuAna:solarnuana@BeginModule  13-Nov-2025 07:46:00 UTC run: 3799 subRun: 0 event: 49069
[31m
Kinetic energy of signal particle is above threshold of 30.00 MeV. Skipping event.
[0m
%MSG
Begin processing the 10th record. run: 3799 subRun: 0 event: 49070 at 13-Nov-2025 07:46:00 UTC
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  13-Nov-2025 07:46:02 UTC run: 3799 subRun: 0 event: 49070
[35m
###################################################
Event:			49070
Flag:			1616310513
Geometry:		dunevd10kt_3view_30deg_v5_refactored_1x8x14ref (VD)
Signal Label:		marley
PDS Frequency in [MHz]:	62.50
PDS Tick in [us]:	0.016
TPC Frequency in [MHz]:	2.00
TPC Tick in [us]:	0.50
TPC Map:		112 TPCs
TPC DriftLength in [cm]:650.07
TPC DriftTime in [us]:	4048.69
Fiducial Volume in [cm]:850.32, 1507.84, 2306.12
###################################################[0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  13-Nov-2025 07:46:02 UTC run: 3799 subRun: 0 event: 49070
[91m
There are a total of 133155 Particles in the event

# of particles 8	from gen 1 marley
# of particles 31442	from gen 2 Ar39GenInLAr
# of particles 3584	from gen 3 Kr85GenInLAr
# of particles 4	from gen 4 Ar42GenInLAr
# of particles 1	from gen 5 K42From42ArGenInLAr
# of particles 21	from gen 6 Rn222ChainRn222GenInLAr
# of particles 28	from gen 7 Rn222ChainPo218GenInLAr
# of particles 71	from gen 8 Rn222ChainPb214GenInLAr
# of particles 61	from gen 9 Rn222ChainBi214GenInLAr
# of particles 28	from gen 10 Rn222ChainPb210GenInLAr
# of particles 33	from gen 11 Rn220ChainPb212GenInLAr
# of particles 222	from gen 12 K40GenInCathode
# of particles 3039	from gen 13 U238ChainGenInCathode
# of particles 3424	from gen 14 Th232ChainGenInCathode
# of particles 112	from gen 15 K40GenInAnode
# of particles 1464	from gen 16 U238ChainGenInAnode
# of particles 1500	from gen 17 Th232ChainGenInAnode
# of particles 0	from gen 18 Rn222ChainGenInPDS
# of particles 3	from gen 19 K42From42ArGenInUpperMesh1x8x14
# of particles 1	from gen 20 Rn222ChainFromPo218GenInUpperMesh1x8x14
# of particles 24	from gen 21 Rn222ChainFromPb214GenInUpperMesh1x8x14
# of particles 65	from gen 22 Rn222ChainFromBi214GenInUpperMesh1x8x14
# of particles 68	from gen 23 Rn222ChainFromPb210GenInUpperMesh1x8x14
# of particles 43	from gen 24 Rn222ChainFromBi210GenInUpperMesh1x8x14
# of particles 53	from gen 25 Rn220ChainFromPb212GenInUpperMesh1x8x14
# of particles 84313	from gen 26 CavernwallGammasAtLAr1x8x14
# of particles 3534	from gen 27 foamGammasAtLAr1x8x14
# of particles 8	from gen 28 CavernwallNeutronsAtLAr1x8x14
# of particles 1	from gen 29 CryostatNGammasAtLAr1x8x14
# of particles 0	from gen 30 CavernNGammasAtLAr1x8x14[0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  13-Nov-2025 07:46:03 UTC run: 3799 subRun: 0 event: 49070
[33m
Number of Signal Particles: 7
Neutrino Interaction: -1
Neutrino Energy: 32.87 MeV
Position (-265.06, 348.88, 1697.93) cm
Gen.	PdgCode		Energy		EndPosition		TrackID	MotherID
--------------------------------------------------------------------------------
marley	11		1.11	(-265.06, 348.88, 1697.93)	133174	133142
marley	-12		0.20	(-265.06, 348.88, 1697.93)	133173	133142
marley	1000200400	0.00	(-265.06, 348.88, 1697.93)	133172	133142
marley	22		0.03	(-265.06, 348.88, 1697.93)	133145	0
marley	22		1.61	(-265.06, 348.88, 1697.93)	133144	0
marley	22		1.09	(-265.06, 348.88, 1697.93)	133143	0
marley	1000190400	0.04	(-265.06, 348.88, 1697.93)	133142	0
marley	11		28.60	(-265.06, 348.88, 1697.93)	133141	0
Signal Track IDs: 133141; 133142; 133143; 133144; 133145; 133172; 133173; 133174; [0m
%MSG
%MSG-w ProducerUtils:  SolarNuAna:solarnuana@BeginModule  13-Nov-2025 07:46:03 UTC run: 3799 subRun: 0 event: 49070
[31m
Kinetic energy of signal particle is above threshold of 30.00 MeV. Skipping event.
[0m
%MSG
13-Nov-2025 07:46:03 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/2d/a9/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241024T014352Z_gen_004907_supernova_g4stage1_g4stage2_detsim_reco_20250611T113943Z_reco2.root"
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@EndJob 13-Nov-2025 07:46:03 UTC  ModuleEndJob
[35mFinished running the SolarNuAna module[0m
%MSG

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.558221       1147.54       3830.87       1.76458       1462.64        10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000491513   0.00643367     0.0168956    0.00445151    0.00607408       10     
end_path:solarnuana:SolarNuAna     0.557602       1147.53       3830.87       1.75992       1462.64        10     
====================================================================================================================
%MSG-i NuRandomService:  SolarNuAna:solarnuana@EndJob 13-Nov-2025 07:46:03 UTC  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1

%MSG

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 2537.69 MB
  Peak resident set size usage (VmHWM): 1915.63 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 10 passed = 10 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport         10         10          0 solarnuana

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 11404.985232 Real = 11503.668165

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 2537.69 VmHWM = 1915.63

Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0