Jobsub ID 294190.12@dunegpschedd01.fnal.gov

Jobsub ID	294190.12@dunegpschedd01.fnal.gov
Workflow ID	12013
Stage ID	1
User name	smanthey@fnal.gov
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2026-01-20 14:18:52
Site	US_FNAL-FermiGrid
Entry	FNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat	2026-01-20 18:19:06
From worker node	Hostname	dunegli-8384800-0-fnpc23044.fnal.gov
	cpuinfo	AMD EPYC 7543 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	172800 (48 hours)
	GPU
	Inner Apptainer?	True
Job state	outputting_failed
Started	2026-01-20 14:20:25
Input files	fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250122T040352Z_gen_000245_supernova_g4stage1_g4stage2_detsim_reco_20250730T033318Z_reco2.root
Jobscript	Exit code	0
	Real time	3h (14313s)
	CPU time	3h (14242s = 99%)
	Max RSS bytes	2563108864 (2444 MiB)
Outputting started	2026-01-20 18:18:59
Output files
Finished	2026-01-20 18:19:06
List job events Cached HTCondor job logs
Jobscript log (last 10,000 characters)

from 5468
 - MainTrackID 132608
 - Gen 1 -> marley TPC 41
 - Purity 1.00 Hits 5
 - Charge 1577.12 ( MaxHit 421.78 )
 - #AdjCl 4 ( 2 signal ):
    - Cluster 1946 at distance 85.91 with time 1611.63 and charge 57.74 in TPC 41 and hits 2
    - Cluster 1967 at distance 10.64 with time 1623.22 and charge 48.52 in TPC 41 and hits 2
    - Cluster 2008 at distance 37.34 with time 1658.44 and charge 403.98 in TPC 41 and hits 3
    - Cluster 2659 at distance 96.81 with time 2151.24 and charge 39.10 in TPC 41 and hits 1
 - RecoCol  Time,Y,Z ( 1660.63, -438.88, 800.53 )
 - RecoInd0 Time,Y,Z ( 1660.63, -438.87, 821.56 )
 - RecoInd1 Time,Y,Z ( 1660.82, -438.89, 821.56 )
*** Reconstructed Interaction Vertex: 
 - True X,Y,Z ( 59.53, -437.52, 800.55 )
 - Main X,Y,Z ( 57.86, -439.48, 801.28 )
 - EDep X,Y,Z ( 58.30, -439.03, 800.30 )
 - Reco X,Y,Z ( -1000000.00, -438.88, 800.53 )[0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  20-Jan-2026 18:18:54 UTC run: 4790 subRun: 0 event: 2449
[32m-----------------------------------------------------------------------------------------
[0m
%MSG
Begin processing the 10th record. run: 4790 subRun: 0 event: 2450 at 20-Jan-2026 18:18:54 UTC
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  20-Jan-2026 18:18:57 UTC run: 4790 subRun: 0 event: 2450
[35m
###################################################
Event:			2450
Flag:			864333667
Geometry:		dunevd10kt_3view_30deg_v5_refactored_1x8x14ref (VD)
Signal Label:		marley
PDS Frequency in [MHz]:	62.50
PDS Tick in [us]:	0.016
TPC Frequency in [MHz]:	2.00
TPC Tick in [us]:	0.50
TPC Map:		112 TPCs
TPC DriftLength in [cm]:650.07
TPC DriftTime in [us]:	4048.69
Fiducial Volume in [cm]:850.32, 1507.84, 2306.12
###################################################[0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  20-Jan-2026 18:18:57 UTC run: 4790 subRun: 0 event: 2450
[91m
There are a total of 132945 Particles in the event

# of particles 48	from gen 1 marley
# of particles 30981	from gen 2 Ar39GenInLAr
# of particles 3472	from gen 3 Kr85GenInLAr
# of particles 4	from gen 4 Ar42GenInLAr
# of particles 0	from gen 5 K42From42ArGenInLAr
# of particles 35	from gen 6 Rn222ChainRn222GenInLAr
# of particles 30	from gen 7 Rn222ChainPo218GenInLAr
# of particles 46	from gen 8 Rn222ChainPb214GenInLAr
# of particles 43	from gen 9 Rn222ChainBi214GenInLAr
# of particles 28	from gen 10 Rn222ChainPb210GenInLAr
# of particles 31	from gen 11 Rn220ChainPb212GenInLAr
# of particles 253	from gen 12 K40GenInCathode
# of particles 3058	from gen 13 U238ChainGenInCathode
# of particles 3764	from gen 14 Th232ChainGenInCathode
# of particles 106	from gen 15 K40GenInAnode
# of particles 1360	from gen 16 U238ChainGenInAnode
# of particles 1537	from gen 17 Th232ChainGenInAnode
# of particles 0	from gen 18 Rn222ChainGenInPDS
# of particles 2	from gen 19 K42From42ArGenInUpperMesh1x8x14
# of particles 2	from gen 20 Rn222ChainFromPo218GenInUpperMesh1x8x14
# of particles 15	from gen 21 Rn222ChainFromPb214GenInUpperMesh1x8x14
# of particles 66	from gen 22 Rn222ChainFromBi214GenInUpperMesh1x8x14
# of particles 73	from gen 23 Rn222ChainFromPb210GenInUpperMesh1x8x14
# of particles 39	from gen 24 Rn222ChainFromBi210GenInUpperMesh1x8x14
# of particles 69	from gen 25 Rn220ChainFromPb212GenInUpperMesh1x8x14
# of particles 84313	from gen 26 CavernwallGammasAtLAr1x8x14
# of particles 3534	from gen 27 foamGammasAtLAr1x8x14
# of particles 35	from gen 28 CavernwallNeutronsAtLAr1x8x14
# of particles 1	from gen 29 CryostatNGammasAtLAr1x8x14
# of particles 0	from gen 30 CavernNGammasAtLAr1x8x14[0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  20-Jan-2026 18:18:58 UTC run: 4790 subRun: 0 event: 2450
[33m
Number of Signal Particles: 5
Neutrino Interaction: -1
Neutrino Energy: 51.03 MeV
Position (-260.74, 664.03, 7.65) cm
Gen.	PdgCode		Energy		EndPosition		TrackID	MotherID
--------------------------------------------------------------------------------
marley	22		1.29	(-390.44, 491.27, 53.77)	132943	132939
marley	1000190410	0.00	(-390.44, 491.27, 53.77)	132942	132939
marley	11		0.39	(-390.44, 491.27, 53.77)	132941	132938
marley	-12		0.80	(-390.44, 491.27, 53.77)	132940	132938
marley	1000190419	0.00	(-390.44, 491.27, 53.77)	132939	132938
marley	1000180410	0.00	(-390.44, 491.27, 53.77)	132938	132866
marley	22		2.02	(-390.44, 491.27, 53.77)	132937	132866
marley	22		3.59	(-390.44, 491.27, 53.77)	132936	132866
marley	22		1.31	(-390.44, 491.27, 53.77)	132935	132866
marley	2212		0.00	(-427.70, 564.86, 114.03)	132925	132866
marley	1000180360	0.00	(-373.26, 602.57, 171.70)	132906	132866
marley	2212		0.00	(-328.28, 618.74, 137.33)	132900	132866
marley	1000060120	0.00	(-328.11, 618.66, 137.31)	132899	132866
marley	1000180360	0.00	(-331.30, 619.68, 138.56)	132898	132866
marley	1000180400	0.00	(-331.42, 609.05, 126.85)	132897	132866
marley	1000180400	0.00	(-331.73, 597.21, 149.15)	132896	132866
marley	2212		0.00	(-327.51, 592.56, 149.15)	132895	132866
marley	2212		0.00	(-327.28, 592.17, 148.92)	132894	132866
marley	2212		0.00	(-326.57, 588.48, 151.06)	132893	132866
marley	2212		0.00	(-326.48, 587.09, 151.17)	132892	132866
marley	1000180360	0.00	(-318.90, 572.49, 140.82)	132891	132866
marley	1000180360	0.00	(-321.10, 579.55, 141.86)	132890	132866
marley	1000180400	0.00	(-317.91, 596.13, 147.27)	132889	132866
marley	1000180400	0.00	(-291.59, 590.19, 163.33)	132888	132866
marley	1000180400	0.00	(-244.62, 519.89, 90.03)	132887	132866
marley	1000180400	0.00	(-211.27, 484.14, 103.66)	132886	132866
marley	1000180400	0.00	(-215.46, 482.50, 36.34)	132885	132866
marley	1000180400	0.00	(-231.43, 470.79, 29.20)	132884	132866
marley	2212		0.00	(-240.54, 558.39, -118.79)	132883	132866
marley	2212		0.00	(-250.16, 571.74, -131.97)	132882	132866
marley	2212		0.01	(-255.12, 582.17, -135.64)	132881	132866
marley	2212		0.04	(-262.85, 596.59, -128.79)	132880	132866
marley	2212		0.00	(-269.87, 596.65, -123.01)	132879	132866
marley	1000180400	0.00	(-333.19, 578.12, -48.82)	132878	132866
marley	1000180400	0.00	(-268.42, 659.74, -33.45)	132877	132866
marley	1000180400	0.01	(-268.55, 660.74, -35.37)	132876	132866
marley	1000180400	0.00	(-269.63, 660.66, -30.87)	132875	132866
marley	1000180400	0.00	(-269.27, 659.21, -30.37)	132874	132866
marley	1000180400	0.00	(-269.11, 657.53, -16.65)	132873	132866
marley	1000180400	0.00	(-269.42, 655.25, -12.38)	132872	132866
marley	1000130270	0.01	(-270.13, 662.89, -3.08)	132871	132866
marley	1000130270	0.00	(-269.91, 662.18, -2.61)	132870	132866
marley	1000180400	0.01	(-271.75, 652.83, 10.49)	132869	132866
marley	1000180400	0.01	(-253.97, 674.22, -24.88)	132868	132866
marley	1000130270	0.01	(-251.19, 658.07, -2.06)	132867	132866
marley	2112		0.12	(-260.74, 664.03, 7.65)	132866	0
marley	1000190390	0.12	(-260.74, 664.03, 7.65)	132865	0
marley	11		41.48	(-260.74, 664.03, 7.65)	132864	0
Signal Track IDs: 132864; 132865; 132866; 132867; 132868; 132869; 132870; 132871; 132872; 132873; 132874; 132875; 132876; 132877; 132878; 132879; 132880; 132881; 132882; 132883; 132884; 132885; 132886; 132887; 132888; 132889; 132890; 132891; 132892; 132893; 132894; 132895; 132896; 132897; 132898; 132899; 132900; 132906; 132925; 132935; 132936; 132937; 132938; 132939; 132940; 132941; 132942; 132943; [0m
%MSG
%MSG-w ProducerUtils:  SolarNuAna:solarnuana@BeginModule  20-Jan-2026 18:18:58 UTC run: 4790 subRun: 0 event: 2450
[31m
Kinetic energy of signal particle is above threshold of 30.00 MeV. Skipping event.
[0m
%MSG
20-Jan-2026 18:18:58 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/5c/89/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250122T040352Z_gen_000245_supernova_g4stage1_g4stage2_detsim_reco_20250730T033318Z_reco2.root"
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@EndJob 20-Jan-2026 18:18:58 UTC  ModuleEndJob
[35mFinished running the SolarNuAna module[0m
%MSG

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.621605       1427.6        2610.17       2207.01       1171.12        10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.00191493     0.0105053     0.0230937    0.00920539     0.0079062       10     
end_path:solarnuana:SolarNuAna     0.619543       1427.59       2610.16        2207         1171.11        10     
====================================================================================================================
%MSG-i NuRandomService:  SolarNuAna:solarnuana@EndJob 20-Jan-2026 18:18:58 UTC  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1

%MSG

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3190.57 MB
  Peak resident set size usage (VmHWM): 2563.11 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 10 passed = 10 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport         10         10          0 solarnuana

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 14231.465754 Real = 14301.490352

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 3190.57 VmHWM = 2563.11

Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===