justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 275152.154@dunegpschedd01.fnal.gov

Jobsub ID275152.154@dunegpschedd01.fnal.gov
Workflow ID11706
Stage ID1
User namesmanthey@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-01-11 22:21:26
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2026-01-12 16:30:57
From worker nodeHostnamedunegli-8149243-0-fnpc18007.fnal.gov
cpuinfoAMD EPYC 7551 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job stateoutputting_failed
Started2026-01-12 16:06:45
Input filesfardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T161306Z_gen_000809_supernova_g4_detsim_reco_20250724T105809Z_reco2.root
JobscriptExit code0
Real time23m (1431s)
CPU time23m (1398s = 97%)
Max RSS bytes905797632 (863 MiB)
Outputting started2026-01-12 16:30:36
Output files
Finished2026-01-12 16:30:57
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

PA
# of particles 19	from gen 19 Rn222ChainPb210GenInCPA
# of particles 31	from gen 20 Rn222ChainFromBi210GenInCPA
# of particles 46	from gen 21 Rn220ChainFromPb212GenInCPA
# of particles 0	from gen 22 Co60GenInAPA
# of particles 2	from gen 23 U238ChainGenInAPA
# of particles 9	from gen 24 Th232ChainGenInAPA
# of particles 55	from gen 25 U238Th232K40GenInLArAPAboards1x2x6
# of particles 1	from gen 26 Rn222ChainGenInPDS
# of particles 19144	from gen 27 CavernwallGammasAtLAr1x2x6lowBgAPA
# of particles 814	from gen 28 foamGammasAtLAr1x2x6lowBgAPA
# of particles 0	from gen 29 CavernwallNeutronsAtLAr1x2x6lowBgAPA
# of particles 0	from gen 30 CryostatNGammasAtLAr1x2x6lowBgAPA
# of particles 0	from gen 31 CavernNGammasAtLAr1x2x6lowBgAPA
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  12-Jan-2026 16:30:33 UTC run: 5092 subRun: 0 event: 40449

Number of Signal Particles: 8
Neutrino Interaction: -1
Neutrino Energy: 44.79 MeV
Position (-65.24, -518.05, 1115.43) cm
Gen.	PdgCode		Energy		EndPosition		TrackID	MotherID
--------------------------------------------------------------------------------
marley	11		0.37	(-65.24, -518.05, 1115.43)	33014	32943
marley	-12		0.94	(-65.24, -518.05, 1115.43)	33013	32943
marley	1000200400	0.00	(-65.24, -518.05, 1115.43)	33012	32943
marley	22		0.03	(-65.24, -518.05, 1115.43)	32947	0
marley	22		1.61	(-65.24, -518.05, 1115.43)	32946	0
marley	22		0.65	(-65.24, -518.05, 1115.43)	32945	0
marley	22		2.09	(-65.24, -518.05, 1115.43)	32944	0
marley	1000190400	0.01	(-65.24, -518.05, 1115.43)	32943	0
marley	11		38.90	(-65.24, -518.05, 1115.43)	32942	0
Signal Track IDs: 32942; 32943; 32944; 32945; 32946; 32947; 33012; 33013; 33014; 
%MSG
%MSG-w ProducerUtils:  SolarNuAna:solarnuana@BeginModule  12-Jan-2026 16:30:33 UTC run: 5092 subRun: 0 event: 40449

Kinetic energy of signal particle is above threshold of 30.00 MeV. Skipping event.

%MSG
Begin processing the 50th record. run: 5092 subRun: 0 event: 40450 at 12-Jan-2026 16:30:33 UTC
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  12-Jan-2026 16:30:34 UTC run: 5092 subRun: 0 event: 40450

###################################################
Event:			40450
Flag:			1659622766
Geometry:		dune10kt_v6_1x2x6 (HD)
Signal Label:		marley
PDS Frequency in [MHz]:	62.50
PDS Tick in [us]:	0.016
TPC Frequency in [MHz]:	2.00
TPC Tick in [us]:	0.50
TPC Map:		24 TPCs
TPC DriftLength in [cm]:360.37
TPC DriftTime in [us]:	2244.44
Fiducial Volume in [cm]:759.89, 1357.70, 1746.48
###################################################
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  12-Jan-2026 16:30:34 UTC run: 5092 subRun: 0 event: 40450

There are a total of 32963 Particles in the event

# of particles 39	from gen 1 marley
# of particles 10953	from gen 2 Ar39GenInLAr
# of particles 1242	from gen 3 Kr85GenInLAr
# of particles 0	from gen 4 Ar42GenInLAr
# of particles 0	from gen 5 K42From42ArGenInLAr
# of particles 18	from gen 6 Rn222ChainRn222GenInLAr
# of particles 8	from gen 7 Rn222ChainPo218GenInLAr
# of particles 13	from gen 8 Rn222ChainPb214GenInLAr
# of particles 16	from gen 9 Rn222ChainBi214GenInLAr
# of particles 0	from gen 10 Rn222ChainPb210GenInLAr
# of particles 9	from gen 11 Rn220ChainPb212GenInLAr
# of particles 20	from gen 12 K40GenInCPA
# of particles 263	from gen 13 U238ChainGenInCPA
# of particles 268	from gen 14 Th232ChainGenInCPA
# of particles 3	from gen 15 K42From42ArGenInCPA
# of particles 2	from gen 16 Rn222ChainPo218GenInCPA
# of particles 10	from gen 17 Rn222ChainPb214GenInCPA
# of particles 25	from gen 18 Rn222ChainBi214GenInCPA
# of particles 34	from gen 19 Rn222ChainPb210GenInCPA
# of particles 37	from gen 20 Rn222ChainFromBi210GenInCPA
# of particles 38	from gen 21 Rn220ChainFromPb212GenInCPA
# of particles 0	from gen 22 Co60GenInAPA
# of particles 0	from gen 23 U238ChainGenInAPA
# of particles 0	from gen 24 Th232ChainGenInAPA
# of particles 42	from gen 25 U238Th232K40GenInLArAPAboards1x2x6
# of particles 0	from gen 26 Rn222ChainGenInPDS
# of particles 19159	from gen 27 CavernwallGammasAtLAr1x2x6lowBgAPA
# of particles 764	from gen 28 foamGammasAtLAr1x2x6lowBgAPA
# of particles 0	from gen 29 CavernwallNeutronsAtLAr1x2x6lowBgAPA
# of particles 0	from gen 30 CryostatNGammasAtLAr1x2x6lowBgAPA
# of particles 0	from gen 31 CavernNGammasAtLAr1x2x6lowBgAPA
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  12-Jan-2026 16:30:35 UTC run: 5092 subRun: 0 event: 40450

Number of Signal Particles: 6
Neutrino Interaction: -1
Neutrino Energy: 50.74 MeV
Position (228.83, 587.22, 924.89) cm
Gen.	PdgCode		Energy		EndPosition		TrackID	MotherID
--------------------------------------------------------------------------------
marley	22		2.22	(447.40, 339.35, 1049.47)	33033	32983
marley	1000010020	0.00	(447.40, 339.35, 1049.47)	33032	32983
marley	2212		0.00	(380.35, 381.35, 1043.61)	33026	32983
marley	2212		0.00	(388.15, 393.23, 1040.04)	33025	32983
marley	2212		0.00	(389.01, 395.07, 1037.44)	33024	32983
marley	1000060120	0.00	(392.39, 404.88, 1041.32)	33023	32983
marley	2212		0.00	(395.31, 407.37, 1043.15)	33022	32983
marley	2212		0.02	(395.84, 412.78, 1043.91)	33021	32983
marley	2212		0.02	(392.65, 414.65, 1043.91)	33020	32983
marley	1000060120	0.01	(389.87, 423.78, 1041.59)	33019	32983
marley	2212		0.12	(399.59, 421.77, 1051.42)	33018	32983
marley	2212		0.11	(404.18, 409.57, 1055.96)	33017	32983
marley	1000060120	0.08	(398.13, 380.09, 1055.78)	33016	32983
marley	1000180400	0.01	(367.27, 405.50, 1053.32)	33015	32983
marley	1000180400	0.04	(213.76, 345.58, 1083.01)	33014	32983
marley	1000180360	0.01	(226.09, 346.00, 1080.28)	33013	32983
marley	1000180400	0.03	(257.49, 351.02, 1100.68)	33012	32983
marley	1000180400	0.03	(250.69, 355.82, 1104.42)	33011	32983
marley	1000180400	0.02	(253.14, 355.68, 1106.82)	33010	32983
marley	1000180400	0.03	(242.92, 385.37, 1121.71)	33009	32983
marley	1000180400	0.00	(246.47, 388.66, 1117.21)	33008	32983
marley	1000180400	0.01	(326.53, 457.98, 1033.80)	33007	32983
marley	1000180400	0.02	(327.60, 516.02, 1029.07)	33006	32983
marley	1000180400	0.04	(244.39, 545.40, 996.00)	33005	32983
marley	22		1.46	(273.29, 578.37, 984.30)	32985	32941
marley	1000180400	0.07	(273.29, 578.37, 984.30)	32984	32941
marley	2112		0.62	(273.29, 578.37, 984.30)	32983	32941
marley	1000180360	0.02	(272.46, 579.00, 984.04)	32982	32941
marley	1000180400	0.02	(253.14, 583.85, 963.65)	32981	32941
marley	1000180360	0.02	(252.96, 583.66, 963.05)	32980	32941
marley	22		1.46	(245.30, 585.03, 954.07)	32943	32927
marley	1000180400	0.01	(245.30, 585.03, 954.07)	32942	32927
marley	2112		2.21	(245.30, 585.03, 954.07)	32941	32927
marley	1000180400	0.03	(238.64, 590.76, 941.28)	32940	32927
marley	1000180400	0.01	(236.10, 589.06, 934.12)	32939	32927
marley	22		3.02	(228.83, 587.22, 924.89)	32928	0
marley	2112		3.73	(228.83, 587.22, 924.89)	32927	0
marley	1000190390	0.02	(228.83, 587.22, 924.89)	32926	0
marley	11		34.67	(228.83, 587.22, 924.89)	32925	0
Signal Track IDs: 32925; 32926; 32927; 32928; 32939; 32940; 32941; 32942; 32943; 32980; 32981; 32982; 32983; 32984; 32985; 33005; 33006; 33007; 33008; 33009; 33010; 33011; 33012; 33013; 33014; 33015; 33016; 33017; 33018; 33019; 33020; 33021; 33022; 33023; 33024; 33025; 33026; 33032; 33033; 
%MSG
%MSG-w ProducerUtils:  SolarNuAna:solarnuana@BeginModule  12-Jan-2026 16:30:35 UTC run: 5092 subRun: 0 event: 40450

Kinetic energy of signal particle is above threshold of 30.00 MeV. Skipping event.

%MSG
12-Jan-2026 16:30:35 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/4d/1b/prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T161306Z_gen_000809_supernova_g4_detsim_reco_20250724T105809Z_reco2.root"
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@EndJob 12-Jan-2026 16:30:35 UTC  ModuleEndJob
Finished running the SolarNuAna module
%MSG

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.484616       26.3704       77.9644       0.95988       32.9339        50     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)             0.0019295    0.00668069     0.0263109    0.00307333    0.00640275       50     
end_path:solarnuana:SolarNuAna     0.481172       26.3635       77.9611      0.956144       32.9333        50     
====================================================================================================================
%MSG-i NuRandomService:  SolarNuAna:solarnuana@EndJob 12-Jan-2026 16:30:35 UTC  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1

%MSG

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1552.22 MB
  Peak resident set size usage (VmHWM): 906.822 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 50 passed = 50 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport         50         50          0 solarnuana

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 1354.358641 Real = 1379.442977

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 1552.22 VmHWM = 906.822

Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
justIN time: 2026-02-06 11:42:56 UTC       justIN version: 01.06.00