justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 274868.15@dunegpschedd01.fnal.gov

Jobsub ID274868.15@dunegpschedd01.fnal.gov
Workflow ID11707
Stage ID1
User namesmanthey@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-01-11 18:07:05
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2026-01-12 13:25:56
From worker nodeHostnamedunegli-8149077-0-fnpc19118.fnal.gov
cpuinfoAMD EPYC 7502 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job stateoutputting_failed
Started2026-01-12 13:13:11
Input filesfardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T180218Z_gen_003581_supernova_g4_detsim_reco_20250724T135128Z_reco2.root
JobscriptExit code0
Real time12m (730s)
CPU time11m (691s = 94%)
Max RSS bytes909717504 (867 MiB)
Outputting started2026-01-12 13:25:21
Output files
Finished2026-01-12 13:25:56
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

-18.05 (us) Fast 50.91 % Residual 1775.83
 - Reco Time,Y,Z ( -18.05, 342.49, 379.11 )
 - Found valid waveform: false
*** Reconstructed Interaction Vertex: 
 - True X,Y,Z ( -87.04, 309.54, 360.02 )
 - Main X,Y,Z ( -84.30, 310.25, 356.48 )
 - EDep X,Y,Z ( -84.21, 310.23, 356.58 )
 - Reco X,Y,Z ( -85.26, 309.70, 358.16 )
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  12-Jan-2026 13:25:20 UTC run: 5092 subRun: 0 event: 179049
-----------------------------------------------------------------------------------------

%MSG
Begin processing the 50th record. run: 5092 subRun: 0 event: 179050 at 12-Jan-2026 13:25:20 UTC
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  12-Jan-2026 13:25:20 UTC run: 5092 subRun: 0 event: 179050

###################################################
Event:			179050
Flag:			1745953731
Geometry:		dune10kt_v6_1x2x6 (HD)
Signal Label:		marley
PDS Frequency in [MHz]:	62.50
PDS Tick in [us]:	0.016
TPC Frequency in [MHz]:	2.00
TPC Tick in [us]:	0.50
TPC Map:		24 TPCs
TPC DriftLength in [cm]:360.37
TPC DriftTime in [us]:	2244.44
Fiducial Volume in [cm]:759.89, 1357.70, 1746.48
###################################################
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  12-Jan-2026 13:25:20 UTC run: 5092 subRun: 0 event: 179050

There are a total of 32941 Particles in the event

# of particles 59	from gen 1 marley
# of particles 10951	from gen 2 Ar39GenInLAr
# of particles 1284	from gen 3 Kr85GenInLAr
# of particles 1	from gen 4 Ar42GenInLAr
# of particles 0	from gen 5 K42From42ArGenInLAr
# of particles 6	from gen 6 Rn222ChainRn222GenInLAr
# of particles 4	from gen 7 Rn222ChainPo218GenInLAr
# of particles 13	from gen 8 Rn222ChainPb214GenInLAr
# of particles 7	from gen 9 Rn222ChainBi214GenInLAr
# of particles 23	from gen 10 Rn222ChainPb210GenInLAr
# of particles 5	from gen 11 Rn220ChainPb212GenInLAr
# of particles 14	from gen 12 K40GenInCPA
# of particles 210	from gen 13 U238ChainGenInCPA
# of particles 275	from gen 14 Th232ChainGenInCPA
# of particles 1	from gen 15 K42From42ArGenInCPA
# of particles 1	from gen 16 Rn222ChainPo218GenInCPA
# of particles 10	from gen 17 Rn222ChainPb214GenInCPA
# of particles 44	from gen 18 Rn222ChainBi214GenInCPA
# of particles 34	from gen 19 Rn222ChainPb210GenInCPA
# of particles 12	from gen 20 Rn222ChainFromBi210GenInCPA
# of particles 21	from gen 21 Rn220ChainFromPb212GenInCPA
# of particles 0	from gen 22 Co60GenInAPA
# of particles 1	from gen 23 U238ChainGenInAPA
# of particles 0	from gen 24 Th232ChainGenInAPA
# of particles 42	from gen 25 U238Th232K40GenInLArAPAboards1x2x6
# of particles 0	from gen 26 Rn222ChainGenInPDS
# of particles 19159	from gen 27 CavernwallGammasAtLAr1x2x6lowBgAPA
# of particles 764	from gen 28 foamGammasAtLAr1x2x6lowBgAPA
# of particles 0	from gen 29 CavernwallNeutronsAtLAr1x2x6lowBgAPA
# of particles 0	from gen 30 CryostatNGammasAtLAr1x2x6lowBgAPA
# of particles 0	from gen 31 CavernNGammasAtLAr1x2x6lowBgAPA
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  12-Jan-2026 13:25:21 UTC run: 5092 subRun: 0 event: 179050

Number of Signal Particles: 6
Neutrino Interaction: -1
Neutrino Energy: 55.69 MeV
Position (53.42, -374.72, 366.24) cm
Gen.	PdgCode		Energy		EndPosition		TrackID	MotherID
--------------------------------------------------------------------------------
marley	22		7.65	(379.93, -124.22, 506.23)	33002	32885
marley	1000260570	0.00	(379.93, -124.22, 506.23)	33001	32885
marley	2212		0.00	(407.44, -64.83, 468.73)	32971	32885
marley	2212		0.00	(409.68, -68.02, 466.86)	32970	32885
marley	2212		0.00	(407.96, -77.25, 474.05)	32969	32885
marley	2212		0.00	(408.51, -79.19, 473.21)	32968	32885
marley	2212		0.01	(410.76, -86.07, 476.21)	32967	32885
marley	2212		0.00	(406.04, -91.70, 480.95)	32966	32885
marley	1000060120	0.00	(404.34, -94.33, 483.97)	32965	32885
marley	2212		0.01	(398.92, -99.44, 483.03)	32964	32885
marley	2212		0.01	(394.52, -104.29, 485.37)	32963	32885
marley	2212		0.03	(388.55, -106.58, 491.75)	32962	32885
marley	1000180400	0.01	(248.51, -92.10, 526.11)	32961	32885
marley	1000180400	0.00	(249.13, -92.27, 526.77)	32960	32885
marley	1000180400	0.00	(249.21, -91.85, 527.20)	32959	32885
marley	1000180400	0.01	(249.17, -91.85, 527.22)	32958	32885
marley	1000180400	0.00	(249.06, -91.94, 526.75)	32957	32885
marley	1000180400	0.00	(250.27, -88.21, 524.07)	32956	32885
marley	1000180400	0.01	(246.82, -87.94, 520.82)	32955	32885
marley	1000180400	0.00	(247.01, -87.72, 521.11)	32954	32885
marley	1000180400	0.01	(254.30, -75.05, 527.77)	32953	32885
marley	1000180400	0.01	(208.19, -127.80, 541.34)	32952	32885
marley	1000180400	0.01	(207.61, -126.25, 539.51)	32951	32885
marley	1000180360	0.01	(203.64, -139.48, 547.45)	32950	32885
marley	1000180400	0.01	(217.05, -84.92, 567.73)	32949	32885
marley	1000180400	0.01	(89.63, -232.93, 399.87)	32948	32885
marley	1000180400	0.01	(150.11, -364.82, 460.03)	32947	32885
marley	1000180400	0.01	(149.54, -354.09, 463.89)	32946	32885
marley	1000180400	0.01	(149.06, -354.54, 464.19)	32945	32885
marley	1000180400	0.02	(152.10, -354.22, 468.88)	32944	32885
marley	1000180400	0.01	(144.81, -356.06, 461.24)	32943	32885
marley	1000180400	0.02	(144.90, -435.94, 444.12)	32942	32885
marley	1000180400	0.02	(187.35, -358.34, 436.32)	32941	32885
marley	1000180400	0.00	(186.16, -361.15, 438.78)	32940	32885
marley	1000180400	0.02	(187.57, -362.87, 440.81)	32939	32885
marley	1000180400	0.02	(192.47, -351.12, 435.62)	32938	32885
marley	1000180400	0.01	(117.90, -326.68, 433.61)	32937	32885
marley	1000180400	0.01	(119.58, -315.04, 430.66)	32936	32885
marley	1000180400	0.03	(124.15, -312.70, 421.98)	32935	32885
marley	1000180400	0.02	(113.30, -315.28, 428.85)	32934	32885
marley	1000180400	0.01	(113.63, -319.52, 427.84)	32933	32885
marley	1000180400	0.04	(97.75, -338.67, 415.22)	32932	32885
marley	1000180400	0.01	(156.73, -314.96, 486.73)	32931	32885
marley	1000180400	0.01	(192.99, -370.52, 514.44)	32930	32885
marley	1000180400	0.04	(126.05, -449.49, 550.99)	32929	32885
marley	1000180400	0.05	(135.93, -443.32, 554.78)	32928	32885
marley	1000180400	0.01	(132.20, -444.62, 551.26)	32927	32885
marley	1000180400	0.03	(69.44, -560.46, 529.20)	32926	32885
marley	1000180400	0.01	(56.32, -546.21, 521.20)	32925	32885
marley	1000180400	0.05	(-3.22, -539.56, 469.37)	32924	32885
marley	1000180400	0.04	(81.54, -495.52, 465.57)	32923	32885
marley	1000180400	0.04	(82.92, -541.26, 436.02)	32922	32885
marley	1000180400	0.00	(108.13, -522.79, 413.17)	32921	32885
marley	1000180400	0.03	(118.58, -518.26, 390.36)	32920	32885
marley	1000180400	0.08	(51.58, -373.08, 365.57)	32919	32885
marley	22		3.02	(53.42, -374.72, 366.24)	32886	0
marley	2112		0.83	(53.42, -374.72, 366.24)	32885	0
marley	1000190390	0.06	(53.42, -374.72, 366.24)	32884	0
marley	11		42.48	(53.42, -374.72, 366.24)	32883	0
Signal Track IDs: 32883; 32884; 32885; 32886; 32919; 32920; 32921; 32922; 32923; 32924; 32925; 32926; 32927; 32928; 32929; 32930; 32931; 32932; 32933; 32934; 32935; 32936; 32937; 32938; 32939; 32940; 32941; 32942; 32943; 32944; 32945; 32946; 32947; 32948; 32949; 32950; 32951; 32952; 32953; 32954; 32955; 32956; 32957; 32958; 32959; 32960; 32961; 32962; 32963; 32964; 32965; 32966; 32967; 32968; 32969; 32970; 32971; 33001; 33002; 
%MSG
%MSG-w ProducerUtils:  SolarNuAna:solarnuana@BeginModule  12-Jan-2026 13:25:21 UTC run: 5092 subRun: 0 event: 179050

Kinetic energy of signal particle is above threshold of 30.00 MeV. Skipping event.

%MSG
12-Jan-2026 13:25:21 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/a0/36/prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T180218Z_gen_003581_supernova_g4_detsim_reco_20250724T135128Z_reco2.root"
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@EndJob 12-Jan-2026 13:25:21 UTC  ModuleEndJob
Finished running the SolarNuAna module
%MSG

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.247956       12.604        43.6739      0.376195       17.2296        50     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.00161141    0.00735623     0.0238712     0.0023475    0.00717717       50     
end_path:solarnuana:SolarNuAna     0.245667       12.5965       43.6719      0.371574       17.2284        50     
====================================================================================================================
%MSG-i NuRandomService:  SolarNuAna:solarnuana@EndJob 12-Jan-2026 13:25:21 UTC  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1

%MSG

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1553.77 MB
  Peak resident set size usage (VmHWM): 908.222 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 50 passed = 50 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport         50         50          0 solarnuana

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 657.958068 Real = 667.854439

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 1553.77 VmHWM = 908.222

Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
justIN time: 2026-02-06 11:42:54 UTC       justIN version: 01.06.00