Jobsub ID 275144.1@dunegpschedd01.fnal.gov

Jobsub ID	275144.1@dunegpschedd01.fnal.gov
Workflow ID	11695
Stage ID	1
User name	smanthey@fnal.gov
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2026-01-11 22:19:20
Site	US_BNL
Entry	DUNE_US_BNL_spoolce02
Last heartbeat	2026-01-12 13:01:54
From worker node	Hostname	spool0268.sdcc.bnl.gov
	cpuinfo	AMD EPYC 7351 16-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	GPU
	Inner Apptainer?	True
Job state	outputting_failed
Started	2026-01-12 10:56:35
Input files	fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250126T044017Z_gen_004839_supernova_g4stage1_g4stage2_detsim_reco_20250702T123307Z_reco2.root
Jobscript	Exit code	0
	Real time	2h (7498s)
	CPU time	2h (7312s = 97%)
	Max RSS bytes	2561474560 (2442 MiB)
Outputting started	2026-01-12 13:01:34
Output files
Finished	2026-01-12 13:01:54
List job events Cached HTCondor job logs
Jobscript log (last 10,000 characters)

, 620.22, 1325.39 )[0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  12-Jan-2026 07:00:28 EST run: 4909 subRun: 0 event: 48389
[32m*** Matched preselection cluster: 3968 from 5668
 - MainTrackID 132247
 - Gen 1 -> marley TPC 71
 - Purity 1.00 Hits 2
 - Charge 101.82 ( MaxHit 81.01 )
 - #AdjCl 3 ( 1 signal ):
    - Cluster 3739 at distance 75.14 with time 2821.87 and charge 34.80 in TPC 71 and hits 1
    - Cluster 4254 at distance 91.10 with time 3190.22 and charge 47.22 in TPC 67 and hits 1
    - Cluster 4676 at distance 98.85 with time 3535.41 and charge 49.43 in TPC 79 and hits 2
 - RecoCol  Time,Y,Z ( 2992.39, 83.35, 1310.97 )
 - RecoInd0 Time,Y,Z ( 2992.67, -475.20, 1427.74 )
 - RecoInd1 Time,Y,Z ( 2992.35, 641.91, 1270.32 )
*** Matched flash: 10014 from 18313
 - Purity 0.00 % Plane 0 #Hits 7
 - MainOpHitPE 8.80 (PE);  TotalPE 45.40 vs expected 53.78 (PE)
 - Time 390.09 (us) Fast 21.59 % Residual 15.22
 - Reco Time,Y,Z ( 390.09, 115.40, 1377.72 )
 - Found valid waveform: true
*** Reconstructed Interaction Vertex: 
 - True X,Y,Z ( -135.12, 621.08, 1325.60 )
 - Main X,Y,Z ( -158.50, 638.19, 1310.87 )
 - EDep X,Y,Z ( -155.40, 641.79, 1310.90 )
 - Reco X,Y,Z ( -92.80, 83.35, 1310.97 )[0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  12-Jan-2026 07:00:28 EST run: 4909 subRun: 0 event: 48389
[32m-----------------------------------------------------------------------------------------
[0m
%MSG
Begin processing the 10th record. run: 4909 subRun: 0 event: 48390 at 12-Jan-2026 07:00:29 EST
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  12-Jan-2026 07:00:36 EST run: 4909 subRun: 0 event: 48390
[35m
###################################################
Event:			48390
Flag:			220532547
Geometry:		dunevd10kt_3view_30deg_v5_refactored_1x8x14ref (VD)
Signal Label:		marley
PDS Frequency in [MHz]:	62.50
PDS Tick in [us]:	0.016
TPC Frequency in [MHz]:	2.00
TPC Tick in [us]:	0.50
TPC Map:		112 TPCs
TPC DriftLength in [cm]:650.07
TPC DriftTime in [us]:	4048.69
Fiducial Volume in [cm]:850.32, 1507.84, 2306.12
###################################################[0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  12-Jan-2026 07:00:38 EST run: 4909 subRun: 0 event: 48390
[91m
There are a total of 132955 Particles in the event

# of particles 13	from gen 1 marley
# of particles 31058	from gen 2 Ar39GenInLAr
# of particles 3605	from gen 3 Kr85GenInLAr
# of particles 4	from gen 4 Ar42GenInLAr
# of particles 0	from gen 5 K42From42ArGenInLAr
# of particles 25	from gen 6 Rn222ChainRn222GenInLAr
# of particles 24	from gen 7 Rn222ChainPo218GenInLAr
# of particles 44	from gen 8 Rn222ChainPb214GenInLAr
# of particles 41	from gen 9 Rn222ChainBi214GenInLAr
# of particles 28	from gen 10 Rn222ChainPb210GenInLAr
# of particles 30	from gen 11 Rn220ChainPb212GenInLAr
# of particles 271	from gen 12 K40GenInCathode
# of particles 3020	from gen 13 U238ChainGenInCathode
# of particles 3633	from gen 14 Th232ChainGenInCathode
# of particles 102	from gen 15 K40GenInAnode
# of particles 1348	from gen 16 U238ChainGenInAnode
# of particles 1632	from gen 17 Th232ChainGenInAnode
# of particles 1	from gen 18 Rn222ChainGenInPDS
# of particles 3	from gen 19 K42From42ArGenInUpperMesh1x8x14
# of particles 0	from gen 20 Rn222ChainFromPo218GenInUpperMesh1x8x14
# of particles 33	from gen 21 Rn222ChainFromPb214GenInUpperMesh1x8x14
# of particles 53	from gen 22 Rn222ChainFromBi214GenInUpperMesh1x8x14
# of particles 18	from gen 23 Rn222ChainFromPb210GenInUpperMesh1x8x14
# of particles 41	from gen 24 Rn222ChainFromBi210GenInUpperMesh1x8x14
# of particles 66	from gen 25 Rn220ChainFromPb212GenInUpperMesh1x8x14
# of particles 84313	from gen 26 CavernwallGammasAtLAr1x8x14
# of particles 3534	from gen 27 foamGammasAtLAr1x8x14
# of particles 14	from gen 28 CavernwallNeutronsAtLAr1x8x14
# of particles 1	from gen 29 CryostatNGammasAtLAr1x8x14
# of particles 0	from gen 30 CavernNGammasAtLAr1x8x14[0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  12-Jan-2026 07:00:38 EST run: 4909 subRun: 0 event: 48390
[33m
Number of Signal Particles: 8
Neutrino Interaction: -1
Neutrino Energy: 10.73 MeV
Position (251.09, 686.70, 1794.03) cm
Gen.	PdgCode		Energy		EndPosition		TrackID	MotherID
--------------------------------------------------------------------------------
marley	22		1.46	(251.09, 686.70, 1794.03)	132991	132929
marley	1000180400	0.00	(251.09, 686.70, 1794.03)	132990	132929
marley	12		0.04	(251.09, 686.70, 1794.03)	132989	132929
marley	11		0.00	(251.09, 686.70, 1794.03)	132987	132929
marley	11		0.00	(251.09, 686.70, 1794.03)	132986	132929
marley	11		0.00	(251.09, 686.70, 1794.03)	132985	132929
marley	11		0.00	(251.09, 686.70, 1794.03)	132984	132929
marley	22		0.03	(251.09, 686.70, 1794.03)	132933	0
marley	22		1.61	(251.09, 686.70, 1794.03)	132932	0
marley	22		0.65	(251.09, 686.70, 1794.03)	132931	0
marley	22		2.09	(251.09, 686.70, 1794.03)	132930	0
marley	1000190400	0.00	(251.09, 686.70, 1794.03)	132929	0
marley	11		4.84	(251.09, 686.70, 1794.03)	132928	0
Signal Track IDs: 132928; 132929; 132930; 132931; 132932; 132933; 132984; 132985; 132986; 132987; 132989; 132990; 132991; [0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  12-Jan-2026 07:00:38 EST run: 4909 subRun: 0 event: 48390
[33m
Kinetic energy of signal particle is below threshold of 30.00 MeV. Selecting event.
[0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  12-Jan-2026 07:02:16 EST run: 4909 subRun: 0 event: 48390
[34mOpFlash PE 14.35 with purity 0.28 time -0.96 plane 0
 - Vertex (-327.50, -8.48, 1161.15)
	*** 1st Sanity check: Ratio 0.58 <= 1.00
	*** 2nd Sanity check: #OpHits 2 >= 2
OpFlash PE 636.51 with purity 0.30 time 0.11 plane 1
 - Vertex (225.44, 743.30, 1759.28)
	*** 1st Sanity check: Ratio 0.09 <= 1.00
	*** 2nd Sanity check: #OpHits 20 >= 20
OpFlash PE 132.44 with purity 0.32 time 0.52 plane 0
 - Vertex (-327.50, 202.70, 1448.78)
	*** 1st Sanity check: Ratio 0.54 <= 1.00
	*** 2nd Sanity check: #OpHits 3 >= 3
OpFlash PE 125.91 with purity 0.09 time 1.63 plane 2
 - Vertex (125.60, -743.30, 474.41)
	*** 1st Sanity check: Ratio 0.16 <= 1.00
	*** 2nd Sanity check: #OpHits 11 >= 11
OpFlash PE 5.95 with purity 1.00 time 2.63 plane 0
 - Vertex (-327.50, 538.10, 1746.62)
	*** 1st Sanity check: Ratio 1.00 <= 1.00
	*** 2nd Sanity check: #OpHits 1 >= 1
OpFlash PE 17.49 with purity 0.47 time 3.02 plane 1
 - Vertex (234.19, 743.30, 1742.66)
	*** 1st Sanity check: Ratio 0.66 <= 1.00
	*** 2nd Sanity check: #OpHits 2 >= 2

# of OpHits (ophit10ppm): 52706 for times between -4254.23 us and 295147919198126080.00 us.
# of OpFlashes (solarflash): 18325 for times between -4254.23 us and 295147919198126080.00 us.
[0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  12-Jan-2026 07:02:20 EST run: 4909 subRun: 0 event: 48390
[36m
# OpHits (ophit10ppm) in full geometry: 52706
# OpFlashes (solarflash) in full geometry: 18325
# Hits (gaushit) in each view: 6821, 6749, 6647, 0
# Cluster from the hits: 2839, 2850, 5685, 0
# Tracks (pmtracktc) in full geometry: 451[0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  12-Jan-2026 08:01:32 EST run: 4909 subRun: 0 event: 48390
[32m
# ClusterReco: Looping over 5675 matched clusters[0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  12-Jan-2026 08:01:32 EST run: 4909 subRun: 0 event: 48390
[32m-----------------------------------------------------------------------------------------
[0m
%MSG
12-Jan-2026 08:01:33 EST  Closed input file "root://dcdndoor.sdcc.bnl.gov:1094//pnfs/sdcc.bnl.gov/data/dune/RSE/fardet-vd/15/08/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250126T044017Z_gen_004839_supernova_g4stage1_g4stage2_detsim_reco_20250702T123307Z_reco2.root"
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@EndJob 12-Jan-2026 08:01:33 EST  ModuleEndJob
[35mFinished running the SolarNuAna module[0m
%MSG

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                          1.31368       740.571       3735.64       1.58179       1477.86        10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000654003   0.00148065    0.00251031    0.00157783    0.000613895      10     
end_path:solarnuana:SolarNuAna      1.31162       740.569       3735.64       1.58083       1477.86        10     
====================================================================================================================
%MSG-i NuRandomService:  SolarNuAna:solarnuana@EndJob 12-Jan-2026 08:01:33 EST  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1

%MSG

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3185.72 MB
  Peak resident set size usage (VmHWM): 2561.47 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 10 passed = 10 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport         10         10          0 solarnuana

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 7288.183017 Real = 7468.431061

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 3185.72 VmHWM = 2561.47

Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===