justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 241885.13@dunegpschedd02.fnal.gov

Jobsub ID241885.13@dunegpschedd02.fnal.gov
Workflow ID9965
Stage ID1
User namesmanthey@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-11-12 03:52:43
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-11-13 08:18:47
From worker nodeHostnamedunegli-7728514-0-fnpc23035.fnal.gov
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job stateoutputting_failed
Started2025-11-13 03:41:26
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250205T000306Z_gen_001268_supernova_g4stage1_g4stage2_detsim_reco_20250630T221640Z_reco2.root
JobscriptExit code0
Real time4h (16610s)
CPU time4h (16407s = 98%)
Max RSS bytes1911951360 (1823 MiB)
Outputting started2025-11-13 08:18:17
Output files
Finished2025-11-13 08:18:47
List job events     (HTCondor job logs unavailable)

Jobscript log (last 10,000 characters)

pperMesh1x8x14
# of particles 50	from gen 23 Rn222ChainFromPb210GenInUpperMesh1x8x14
# of particles 36	from gen 24 Rn222ChainFromBi210GenInUpperMesh1x8x14
# of particles 37	from gen 25 Rn220ChainFromPb212GenInUpperMesh1x8x14
# of particles 84313	from gen 26 CavernwallGammasAtLAr1x8x14
# of particles 3534	from gen 27 foamGammasAtLAr1x8x14
# of particles 18	from gen 28 CavernwallNeutronsAtLAr1x8x14
# of particles 1	from gen 29 CryostatNGammasAtLAr1x8x14
# of particles 0	from gen 30 CavernNGammasAtLAr1x8x14
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  13-Nov-2025 07:49:31 UTC run: 5055 subRun: 0 event: 12680

Number of Signal Particles: 5
Neutrino Interaction: -1
Neutrino Energy: 15.01 MeV
Position (-81.48, -252.12, 1959.92) cm
Gen.	PdgCode		Energy		EndPosition		TrackID	MotherID
--------------------------------------------------------------------------------
marley	1000240540	0.00	(296.14, -10.93, 2284.54)	133025	132971
marley	22		3.60	(296.14, -10.93, 2284.54)	133024	132971
marley	22		6.59	(296.14, -10.93, 2284.54)	133023	132971
marley	22		2.28	(296.14, -10.93, 2284.54)	133022	132971
marley	1000080160	0.00	(294.96, -15.73, 2283.73)	133014	132971
marley	1000070140	0.00	(295.03, -17.00, 2284.84)	133012	132971
marley	1000070140	0.00	(295.07, -16.97, 2284.77)	133011	132971
marley	1000070140	0.00	(294.89, -17.49, 2284.57)	133010	132971
marley	1000080160	0.00	(293.60, -16.85, 2283.66)	133008	132971
marley	1000070140	0.00	(293.52, -17.99, 2284.22)	133006	132971
marley	1000070140	0.00	(293.33, -18.52, 2284.42)	133005	132971
marley	1000070140	0.00	(293.48, -17.98, 2285.12)	133004	132971
marley	1000070140	0.00	(293.55, -17.85, 2285.09)	133003	132971
marley	1000070140	0.00	(292.92, -18.18, 2285.53)	133002	132971
marley	1000070140	0.00	(292.74, -18.71, 2285.07)	133001	132971
marley	1000070140	0.00	(292.74, -18.71, 2285.06)	133000	132971
marley	1000070140	0.00	(293.71, -18.51, 2284.69)	132999	132971
marley	1000070140	0.00	(293.78, -18.14, 2283.41)	132998	132971
marley	1000070140	0.00	(294.60, -18.57, 2282.41)	132997	132971
marley	1000260560	0.00	(294.50, -18.73, 2282.34)	132996	132971
marley	1000080160	0.00	(301.41, -17.16, 2273.45)	132995	132971
marley	2212		0.00	(304.79, -24.73, 2269.94)	132994	132971
marley	2212		0.00	(312.57, -33.43, 2273.13)	132993	132971
marley	2212		0.00	(315.82, -36.72, 2270.15)	132992	132971
marley	1000060120	0.00	(317.97, -44.23, 2264.51)	132991	132971
marley	2212		0.00	(330.04, -41.17, 2266.83)	132990	132971
marley	2212		0.00	(330.50, -41.25, 2267.18)	132989	132971
marley	2212		0.02	(337.06, -41.01, 2272.16)	132988	132971
marley	1000070140	0.01	(335.58, -40.72, 2279.45)	132987	132971
marley	2212		0.00	(326.82, -31.99, 2252.88)	132986	132971
marley	2212		0.01	(326.25, -30.82, 2245.83)	132985	132971
marley	2212		0.01	(299.51, -26.15, 2208.40)	132984	132971
marley	1000180400	0.01	(-321.61, -242.47, 1695.49)	132983	132971
marley	1000180400	0.00	(-251.97, -244.90, 1764.84)	132982	132971
marley	1000180400	0.00	(-255.35, -242.13, 1769.45)	132981	132971
marley	1000180400	0.00	(-255.38, -239.34, 1768.25)	132980	132971
marley	1000180400	0.01	(-257.06, -237.50, 1769.86)	132979	132971
marley	1000180400	0.01	(-257.53, -238.88, 1769.88)	132978	132971
marley	1000180360	0.01	(-254.36, -236.96, 1769.27)	132977	132971
marley	1000180400	0.01	(-259.22, -257.16, 1772.61)	132976	132971
marley	1000180400	0.01	(-137.49, -256.30, 1852.97)	132975	132971
marley	1000180400	0.01	(-141.57, -260.44, 1852.49)	132974	132971
marley	1000180400	0.01	(-142.56, -242.01, 1858.70)	132973	132971
marley	1000180400	0.01	(-144.64, -243.12, 1845.25)	132972	132971
marley	2112		0.14	(-81.48, -252.12, 1959.92)	132971	0
marley	1000190390	0.01	(-81.48, -252.12, 1959.92)	132970	0
marley	11		5.55	(-81.48, -252.12, 1959.92)	132969	0
Signal Track IDs: 132969; 132970; 132971; 132972; 132973; 132974; 132975; 132976; 132977; 132978; 132979; 132980; 132981; 132982; 132983; 132984; 132985; 132986; 132987; 132988; 132989; 132990; 132991; 132992; 132993; 132994; 132995; 132996; 132997; 132998; 132999; 133000; 133001; 133002; 133003; 133004; 133005; 133006; 133008; 133010; 133011; 133012; 133014; 133022; 133023; 133024; 133025; 
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  13-Nov-2025 07:49:31 UTC run: 5055 subRun: 0 event: 12680

Kinetic energy of signal particle is below threshold of 30.00 MeV. Selecting event.

%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  13-Nov-2025 07:49:53 UTC run: 5055 subRun: 0 event: 12680
OpFlash PE 112.20 with purity 0.29 time 0.02 plane 0
 - Vertex (-327.50, -210.98, 1842.01)
	*** 1st Sanity check: Ratio 0.12 <= 1.00
	*** 2nd Sanity check: #OpHits 14 >= 14

# of OpFlashes (solarflash) in full geometry: 10654

%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  13-Nov-2025 07:49:56 UTC run: 5055 subRun: 0 event: 12680

# OpHits (ophit10ppm) in full geometry: 52577
# OpFlashes (solarflash) in full geometry: 10654
# Hits (gaushit) in each view: 6594, 6526, 6539, 0
# Cluster from the hits: 2749, 2767, 5558, 0
# Tracks (pmtracktc) in full geometry: 460
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  13-Nov-2025 08:18:16 UTC run: 5055 subRun: 0 event: 12680

# ClusterReco: Looping over 5537 matched clusters
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  13-Nov-2025 08:18:17 UTC run: 5055 subRun: 0 event: 12680
*** Matched preselection cluster: 3269
 - MainTrackID 132969
 - Gen 1 -> marley TPC 106
 - Purity 1.00 Hits 3
 - Charge 602.06 ( MaxHit 325.05 )
 - #AdjCl 12 ( 0 signal ):
    - Cluster 2622 at distance 91.59 with time 2016.52 and charge 17.71 in TPC 98 and hits 1
    - Cluster 2706 at distance 94.15 with time 2081.26 and charge 67.56 in TPC 106 and hits 1
    - Cluster 2922 at distance 62.87 with time 2236.58 and charge 59.02 in TPC 102 and hits 2
    - Cluster 3055 at distance 80.82 with time 2348.69 and charge 183.00 in TPC 98 and hits 2
    - Cluster 3091 at distance 75.38 with time 2383.40 and charge 48.93 in TPC 98 and hits 1
    - Cluster 3128 at distance 48.73 with time 2412.51 and charge 170.14 in TPC 106 and hits 3
    - Cluster 3205 at distance 75.23 with time 2473.57 and charge 23.36 in TPC 96 and hits 1
    - Cluster 3296 at distance 94.53 with time 2548.86 and charge 31.68 in TPC 104 and hits 1
    - Cluster 3322 at distance 72.07 with time 2573.38 and charge 246.71 in TPC 106 and hits 2
    - Cluster 3400 at distance 83.36 with time 2630.32 and charge 65.36 in TPC 96 and hits 1
    - Cluster 3508 at distance 47.56 with time 2705.31 and charge 112.27 in TPC 106 and hits 1
    - Cluster 3710 at distance 75.35 with time 2867.73 and charge 50.70 in TPC 106 and hits 1
 - RecoCol  Time,Y,Z ( 2529.61, -251.59, 1960.08 )
 - RecoInd0 Time,Y,Z ( 2529.66, -251.59, 2016.92 )
 - RecoInd1 Time,Y,Z ( 2529.63, -251.59, 2016.92 )
*** Matched flash: 
 - Purity 0.00 Plane 0 #Hits 12 PE 631.79 MaxPE 169.80
 - Time 46.69 Fast 0.93 Residual 641.24
 - Reco Time,Y,Z ( 46.69, -297.40, 1973.46 )
*** Reconstructed Interaction Vertex: 
 - True X,Y,Z ( -81.48, -252.12, 1959.92 )
 - Main X,Y,Z ( -80.94, -251.33, 1960.54 )
 - EDep X,Y,Z ( -80.93, -251.35, 1960.30 )
 - Reco X,Y,Z ( -73.63, -251.59, 1960.08 )
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  13-Nov-2025 08:18:17 UTC run: 5055 subRun: 0 event: 12680
-----------------------------------------------------------------------------------------

%MSG
13-Nov-2025 08:18:17 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/c4/cb/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250205T000306Z_gen_001268_supernova_g4stage1_g4stage2_detsim_reco_20250630T221640Z_reco2.root"
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@EndJob 13-Nov-2025 08:18:17 UTC  ModuleEndJob
Finished running the SolarNuAna module
%MSG

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.600804       1654.4        2639.16       1895.6        867.803        10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000636949   0.00698159     0.0205464    0.00398595    0.00691145       10     
end_path:solarnuana:SolarNuAna     0.588889       1654.4        2639.15       1895.59       867.801        10     
====================================================================================================================
%MSG-i NuRandomService:  SolarNuAna:solarnuana@EndJob 13-Nov-2025 08:18:17 UTC  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1

%MSG

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 2535.45 MB
  Peak resident set size usage (VmHWM): 1911.95 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 10 passed = 10 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport         10         10          0 solarnuana

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 16392.799738 Real = 16572.204056

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 2535.45 VmHWM = 1911.95

Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-10 06:21:29 UTC       justIN version: 01.06.00