Jobsub ID 242097.68@dunegpschedd02.fnal.gov

Jobsub ID	242097.68@dunegpschedd02.fnal.gov
Workflow ID	9960
Stage ID	1
User name	smanthey@fnal.gov
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-11-12 09:18:56
Site	US_FNAL-FermiGrid
Entry	FNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat	2025-11-13 10:20:59
From worker node	Hostname	dunegli-7572295-0-fnpc23118.fnal.gov
	cpuinfo	AMD EPYC 9354 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	172800 (48 hours)
	GPU
	Inner Apptainer?	True
Job state	outputting_failed
Started	2025-11-13 05:58:44
Input files	fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T044359Z_gen_003451_supernova_g4stage1_g4stage2_detsim_reco_20250731T010610Z_reco2.root
Jobscript	Exit code	0
	Real time	4h (15724s)
	CPU time	4h (15582s = 99%)
	Max RSS bytes	1912369152 (1823 MiB)
Outputting started	2025-11-13 10:20:48
Output files
Finished	2025-11-13 10:20:59
List job events (HTCondor job logs unavailable)
Jobscript log (last 10,000 characters)

32269; 132270; [0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  13-Nov-2025 09:44:33 UTC run: 5019 subRun: 0 event: 34509
[33m
Kinetic energy of signal particle is below threshold of 30.00 MeV. Selecting event.
[0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  13-Nov-2025 09:44:52 UTC run: 5019 subRun: 0 event: 34509
[34mOpFlash PE 44.44 with purity 0.11 time -0.04 plane 0
 - Vertex (-327.50, -377.98, 1846.30)
	*** 1st Sanity check: Ratio 0.22 <= 1.00
	*** 2nd Sanity check: #OpHits 7 >= 7
OpFlash PE 137.07 with purity 0.21 time -0.00 plane 0
 - Vertex (-327.50, 190.36, 1105.53)
	*** 1st Sanity check: Ratio 0.12 <= 1.00
	*** 2nd Sanity check: #OpHits 16 >= 16
OpFlash PE 243.22 with purity 0.29 time 0.02 plane 1
 - Vertex (232.43, 743.30, 1378.78)
	*** 1st Sanity check: Ratio 0.21 <= 1.00
	*** 2nd Sanity check: #OpHits 17 >= 17
OpFlash PE 429.82 with purity 0.01 time 0.01 plane 2
 - Vertex (135.07, -743.30, 1343.78)
	*** 1st Sanity check: Ratio 0.50 <= 1.00
	*** 2nd Sanity check: #OpHits 4 >= 4
OpFlash PE 51.14 with purity 0.09 time 0.02 plane 0
 - Vertex (-327.50, -495.57, 1902.03)
	*** 1st Sanity check: Ratio 0.33 <= 1.00
	*** 2nd Sanity check: #OpHits 6 >= 6
OpFlash PE 227.56 with purity 0.15 time 0.07 plane 1
 - Vertex (136.26, 743.30, 372.62)
	*** 1st Sanity check: Ratio 0.17 <= 1.00
	*** 2nd Sanity check: #OpHits 13 >= 13
OpFlash PE 40.54 with purity 0.31 time 0.08 plane 1
 - Vertex (137.19, 743.30, 1682.39)
	*** 1st Sanity check: Ratio 0.33 <= 1.00
	*** 2nd Sanity check: #OpHits 6 >= 6

# of OpFlashes (solarflash) in full geometry: 10615
[0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  13-Nov-2025 09:44:54 UTC run: 5019 subRun: 0 event: 34509
[36m
# OpHits (ophit10ppm) in full geometry: 52073
# OpFlashes (solarflash) in full geometry: 10615
# Hits (gaushit) in each view: 6711, 6673, 6652, 0
# Cluster from the hits: 2823, 2837, 5691, 0
# Tracks (pmtracktc) in full geometry: 459[0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  13-Nov-2025 10:20:44 UTC run: 5019 subRun: 0 event: 34509
[32m
# ClusterReco: Looping over 5678 matched clusters[0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  13-Nov-2025 10:20:44 UTC run: 5019 subRun: 0 event: 34509
[32m*** Matched preselection cluster: 872
 - MainTrackID 132225
 - Gen 1 -> marley TPC 78
 - Purity 1.00 Hits 6
 - Charge 1571.25 ( MaxHit 543.45 )
 - #AdjCl 7 ( 0 signal ):
    - Cluster 634 at distance 70.06 with time 538.87 and charge 28.31 in TPC 78 and hits 1
    - Cluster 1148 at distance 69.77 with time 904.81 and charge 47.99 in TPC 78 and hits 1
    - Cluster 1278 at distance 69.43 with time 993.25 and charge 55.32 in TPC 78 and hits 2
    - Cluster 1386 at distance 71.87 with time 1071.35 and charge 31.04 in TPC 68 and hits 1
    - Cluster 1409 at distance 96.54 with time 1093.30 and charge 104.49 in TPC 70 and hits 1
    - Cluster 1439 at distance 99.27 with time 1117.96 and charge 29.72 in TPC 68 and hits 1
    - Cluster 1480 at distance 93.97 with time 1151.76 and charge 52.26 in TPC 74 and hits 2
 - RecoCol  Time,Y,Z ( 722.02, 386.43, 1379.61 )
 - RecoInd0 Time,Y,Z ( 722.29, 386.40, 1405.12 )
 - RecoInd1 Time,Y,Z ( 722.08, 386.45, 1419.24 )
*** Matched flash: 
 - Purity 0.00 Plane 0 #Hits 19 PE 544.43 MaxPE 212.19
 - Time 51.06 Fast 0.79 Residual 5984.95
 - Reco Time,Y,Z ( 51.06, 373.40, 1377.78 )
*** Reconstructed Interaction Vertex: 
 - True X,Y,Z ( 208.91, 386.66, 1381.38 )
 - Main X,Y,Z ( 209.28, 387.61, 1378.98 )
 - EDep X,Y,Z ( 208.92, 386.07, 1380.40 )
 - Reco X,Y,Z ( 217.30, 386.43, 1379.61 )[0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  13-Nov-2025 10:20:44 UTC run: 5019 subRun: 0 event: 34509
[32m-----------------------------------------------------------------------------------------
[0m
%MSG
Begin processing the 10th record. run: 5019 subRun: 0 event: 34510 at 13-Nov-2025 10:20:44 UTC
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  13-Nov-2025 10:20:47 UTC run: 5019 subRun: 0 event: 34510
[35m
###################################################
Event:			34510
Flag:			2031344815
Geometry:		dunevd10kt_3view_30deg_v5_refactored_1x8x14ref (VD)
Signal Label:		marley
PDS Frequency in [MHz]:	62.50
PDS Tick in [us]:	0.016
TPC Frequency in [MHz]:	2.00
TPC Tick in [us]:	0.50
TPC Map:		112 TPCs
TPC DriftLength in [cm]:650.07
TPC DriftTime in [us]:	4048.69
Fiducial Volume in [cm]:850.32, 1507.84, 2306.12
###################################################[0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  13-Nov-2025 10:20:47 UTC run: 5019 subRun: 0 event: 34510
[91m
There are a total of 132392 Particles in the event

# of particles 8	from gen 1 marley
# of particles 31098	from gen 2 Ar39GenInLAr
# of particles 3571	from gen 3 Kr85GenInLAr
# of particles 5	from gen 4 Ar42GenInLAr
# of particles 1	from gen 5 K42From42ArGenInLAr
# of particles 37	from gen 6 Rn222ChainRn222GenInLAr
# of particles 30	from gen 7 Rn222ChainPo218GenInLAr
# of particles 59	from gen 8 Rn222ChainPb214GenInLAr
# of particles 41	from gen 9 Rn222ChainBi214GenInLAr
# of particles 26	from gen 10 Rn222ChainPb210GenInLAr
# of particles 22	from gen 11 Rn220ChainPb212GenInLAr
# of particles 232	from gen 12 K40GenInCathode
# of particles 2910	from gen 13 U238ChainGenInCathode
# of particles 3366	from gen 14 Th232ChainGenInCathode
# of particles 86	from gen 15 K40GenInAnode
# of particles 1226	from gen 16 U238ChainGenInAnode
# of particles 1607	from gen 17 Th232ChainGenInAnode
# of particles 0	from gen 18 Rn222ChainGenInPDS
# of particles 1	from gen 19 K42From42ArGenInUpperMesh1x8x14
# of particles 2	from gen 20 Rn222ChainFromPo218GenInUpperMesh1x8x14
# of particles 13	from gen 21 Rn222ChainFromPb214GenInUpperMesh1x8x14
# of particles 47	from gen 22 Rn222ChainFromBi214GenInUpperMesh1x8x14
# of particles 44	from gen 23 Rn222ChainFromPb210GenInUpperMesh1x8x14
# of particles 41	from gen 24 Rn222ChainFromBi210GenInUpperMesh1x8x14
# of particles 59	from gen 25 Rn220ChainFromPb212GenInUpperMesh1x8x14
# of particles 84313	from gen 26 CavernwallGammasAtLAr1x8x14
# of particles 3534	from gen 27 foamGammasAtLAr1x8x14
# of particles 12	from gen 28 CavernwallNeutronsAtLAr1x8x14
# of particles 1	from gen 29 CryostatNGammasAtLAr1x8x14
# of particles 0	from gen 30 CavernNGammasAtLAr1x8x14[0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  13-Nov-2025 10:20:48 UTC run: 5019 subRun: 0 event: 34510
[33m
Number of Signal Particles: 7
Neutrino Interaction: -1
Neutrino Energy: 47.25 MeV
Position (342.25, -729.06, 633.00) cm
Gen.	PdgCode		Energy		EndPosition		TrackID	MotherID
--------------------------------------------------------------------------------
marley	11		0.91	(342.25, -729.06, 632.99)	132440	132375
marley	-12		0.40	(342.25, -729.06, 632.99)	132439	132375
marley	1000200400	0.00	(342.25, -729.06, 632.99)	132438	132375
marley	22		0.03	(342.25, -729.06, 633.00)	132378	0
marley	22		2.71	(342.25, -729.06, 633.00)	132377	0
marley	22		1.64	(342.25, -729.06, 633.00)	132376	0
marley	1000190400	0.04	(342.25, -729.06, 633.00)	132375	0
marley	11		41.32	(342.25, -729.06, 633.00)	132374	0
Signal Track IDs: 132374; 132375; 132376; 132377; 132378; 132438; 132439; 132440; [0m
%MSG
%MSG-w ProducerUtils:  SolarNuAna:solarnuana@BeginModule  13-Nov-2025 10:20:48 UTC run: 5019 subRun: 0 event: 34510
[31m
Kinetic energy of signal particle is above threshold of 30.00 MeV. Skipping event.
[0m
%MSG
13-Nov-2025 10:20:48 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/9d/9d/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T044359Z_gen_003451_supernova_g4stage1_g4stage2_detsim_reco_20250731T010610Z_reco2.root"
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@EndJob 13-Nov-2025 10:20:48 UTC  ModuleEndJob
[35mFinished running the SolarNuAna module[0m
%MSG

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.786782       1567.05       3177.85       2070.58       1320.11        10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000626541    0.0121096     0.0368093    0.00696003     0.012554        10     
end_path:solarnuana:SolarNuAna     0.767311       1567.04       3177.85       2070.57       1320.11        10     
====================================================================================================================
%MSG-i NuRandomService:  SolarNuAna:solarnuana@EndJob 13-Nov-2025 10:20:48 UTC  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1

%MSG

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 2535.22 MB
  Peak resident set size usage (VmHWM): 1912.37 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 10 passed = 10 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport         10         10          0 solarnuana

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 15566.068948 Real = 15702.208592

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 2535.22 VmHWM = 1912.37

Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0