justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 288544.40@dunegpschedd02.fnal.gov

Jobsub ID288544.40@dunegpschedd02.fnal.gov
Workflow ID12007
Stage ID1
User namesmanthey@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-01-20 14:48:38
SiteUS_FNAL-T1
EntryCMSHTPC_T1_US_FNAL_condce_opp1_whole
Last heartbeat2026-01-20 18:57:53
From worker nodeHostnamedunegli-50386-0-cmswn2355.fnal.gov
cpuinfoIntel(R) Xeon(R) CPU E5-2670 v3 @ 2.30GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job stateoutputting_failed
Started2026-01-20 14:57:40
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250117T062939Z_gen_000250_supernova_g4stage1_g4stage2_detsim_reco_20250619T230121Z_reco2.root
JobscriptExit code0
Real time3h (14392s)
CPU time3h (14101s = 97%)
Max RSS bytes2561867776 (2443 MiB)
Outputting started2026-01-20 18:57:33
Output files
Finished2026-01-20 18:57:53
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

1x8x14
# of particles 15	from gen 21 Rn222ChainFromPb214GenInUpperMesh1x8x14
# of particles 68	from gen 22 Rn222ChainFromBi214GenInUpperMesh1x8x14
# of particles 55	from gen 23 Rn222ChainFromPb210GenInUpperMesh1x8x14
# of particles 42	from gen 24 Rn222ChainFromBi210GenInUpperMesh1x8x14
# of particles 43	from gen 25 Rn220ChainFromPb212GenInUpperMesh1x8x14
# of particles 83844	from gen 26 CavernwallGammasAtLAr1x8x14
# of particles 3566	from gen 27 foamGammasAtLAr1x8x14
# of particles 0	from gen 28 CavernwallNeutronsAtLAr1x8x14
# of particles 0	from gen 29 CryostatNGammasAtLAr1x8x14
# of particles 0	from gen 30 CavernNGammasAtLAr1x8x14
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  20-Jan-2026 18:57:22 UTC run: 4705 subRun: 0 event: 2499

Number of Signal Particles: 5
Neutrino Interaction: -1
Neutrino Energy: 43.64 MeV
Position (-276.82, -223.22, 1181.58) cm
Gen.	PdgCode		Energy		EndPosition		TrackID	MotherID
--------------------------------------------------------------------------------
marley	22		2.22	(-470.08, -377.04, 1277.41)	132060	132021
marley	1000010020	0.00	(-470.08, -377.04, 1277.41)	132059	132021
marley	2212		0.00	(-477.93, -351.51, 1250.15)	132050	132021
marley	2212		0.00	(-477.23, -351.29, 1249.70)	132049	132021
marley	1000060120	0.00	(-467.82, -345.82, 1249.22)	132048	132021
marley	2212		0.00	(-459.85, -357.35, 1248.75)	132047	132021
marley	2212		0.00	(-440.06, -358.92, 1247.93)	132046	132021
marley	2212		0.00	(-428.42, -350.11, 1256.84)	132045	132021
marley	1000060120	0.00	(-426.70, -342.48, 1257.89)	132044	132021
marley	1000060120	0.00	(-425.43, -342.10, 1257.58)	132043	132021
marley	1000260560	0.00	(-425.09, -353.26, 1240.87)	132042	132021
marley	2212		0.02	(-431.47, -354.88, 1236.80)	132041	132021
marley	2212		0.02	(-428.30, -358.24, 1229.60)	132040	132021
marley	2212		0.02	(-425.75, -358.97, 1228.35)	132039	132021
marley	1000180400	0.00	(-410.19, -353.00, 1222.78)	132038	132021
marley	1000180400	0.01	(-386.87, -328.68, 1150.81)	132037	132021
marley	1000180400	0.01	(-375.35, -333.57, 1171.85)	132036	132021
marley	1000180400	0.00	(-377.80, -341.06, 1166.42)	132035	132021
marley	1000180400	0.00	(-379.49, -342.12, 1168.49)	132034	132021
marley	1000180400	0.01	(-389.50, -344.31, 1164.68)	132033	132021
marley	1000180400	0.00	(-387.42, -343.25, 1163.56)	132032	132021
marley	1000180400	0.01	(-383.30, -334.31, 1174.04)	132031	132021
marley	1000180400	0.01	(-394.84, -323.57, 1167.04)	132030	132021
marley	1000180400	0.01	(-341.42, -311.35, 1142.53)	132029	132021
marley	1000180400	0.00	(-342.64, -302.09, 1145.36)	132028	132021
marley	1000180400	0.00	(-343.70, -281.61, 1138.16)	132027	132021
marley	1000180360	0.01	(-334.47, -266.62, 1145.69)	132026	132021
marley	1000180400	0.01	(-329.44, -268.25, 1136.31)	132025	132021
marley	1000180400	0.01	(-295.47, -223.52, 1205.90)	132024	132021
marley	1000180400	0.00	(-297.98, -236.39, 1203.91)	132023	132021
marley	1000180400	0.01	(-315.50, -242.95, 1187.55)	132022	132021
marley	2112		0.16	(-276.82, -223.22, 1181.58)	132021	0
marley	1000190390	0.05	(-276.82, -223.22, 1181.58)	132020	0
marley	11		34.13	(-276.82, -223.22, 1181.58)	132019	0
Signal Track IDs: 132019; 132020; 132021; 132022; 132023; 132024; 132025; 132026; 132027; 132028; 132029; 132030; 132031; 132032; 132033; 132034; 132035; 132036; 132037; 132038; 132039; 132040; 132041; 132042; 132043; 132044; 132045; 132046; 132047; 132048; 132049; 132050; 132059; 132060; 
%MSG
%MSG-w ProducerUtils:  SolarNuAna:solarnuana@BeginModule  20-Jan-2026 18:57:22 UTC run: 4705 subRun: 0 event: 2499

Kinetic energy of signal particle is above threshold of 30.00 MeV. Skipping event.

%MSG
Begin processing the 10th record. run: 4705 subRun: 0 event: 2500 at 20-Jan-2026 18:57:23 UTC
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  20-Jan-2026 18:57:29 UTC run: 4705 subRun: 0 event: 2500

###################################################
Event:			2500
Flag:			1688347139
Geometry:		dunevd10kt_3view_30deg_v5_refactored_1x8x14ref (VD)
Signal Label:		marley
PDS Frequency in [MHz]:	62.50
PDS Tick in [us]:	0.016
TPC Frequency in [MHz]:	2.00
TPC Tick in [us]:	0.50
TPC Map:		112 TPCs
TPC DriftLength in [cm]:650.07
TPC DriftTime in [us]:	4048.69
Fiducial Volume in [cm]:850.32, 1507.84, 2306.12
###################################################
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  20-Jan-2026 18:57:31 UTC run: 4705 subRun: 0 event: 2500

There are a total of 133334 Particles in the event

# of particles 11	from gen 1 marley
# of particles 31342	from gen 2 Ar39GenInLAr
# of particles 3499	from gen 3 Kr85GenInLAr
# of particles 2	from gen 4 Ar42GenInLAr
# of particles 0	from gen 5 K42From42ArGenInLAr
# of particles 30	from gen 6 Rn222ChainRn222GenInLAr
# of particles 24	from gen 7 Rn222ChainPo218GenInLAr
# of particles 59	from gen 8 Rn222ChainPb214GenInLAr
# of particles 57	from gen 9 Rn222ChainBi214GenInLAr
# of particles 23	from gen 10 Rn222ChainPb210GenInLAr
# of particles 43	from gen 11 Rn220ChainPb212GenInLAr
# of particles 257	from gen 12 K40GenInCathode
# of particles 3026	from gen 13 U238ChainGenInCathode
# of particles 3556	from gen 14 Th232ChainGenInCathode
# of particles 98	from gen 15 K40GenInAnode
# of particles 1434	from gen 16 U238ChainGenInAnode
# of particles 1794	from gen 17 Th232ChainGenInAnode
# of particles 0	from gen 18 Rn222ChainGenInPDS
# of particles 5	from gen 19 K42From42ArGenInUpperMesh1x8x14
# of particles 3	from gen 20 Rn222ChainFromPo218GenInUpperMesh1x8x14
# of particles 20	from gen 21 Rn222ChainFromPb214GenInUpperMesh1x8x14
# of particles 73	from gen 22 Rn222ChainFromBi214GenInUpperMesh1x8x14
# of particles 54	from gen 23 Rn222ChainFromPb210GenInUpperMesh1x8x14
# of particles 33	from gen 24 Rn222ChainFromBi210GenInUpperMesh1x8x14
# of particles 31	from gen 25 Rn220ChainFromPb212GenInUpperMesh1x8x14
# of particles 84313	from gen 26 CavernwallGammasAtLAr1x8x14
# of particles 3534	from gen 27 foamGammasAtLAr1x8x14
# of particles 12	from gen 28 CavernwallNeutronsAtLAr1x8x14
# of particles 1	from gen 29 CryostatNGammasAtLAr1x8x14
# of particles 0	from gen 30 CavernNGammasAtLAr1x8x14
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  20-Jan-2026 18:57:31 UTC run: 4705 subRun: 0 event: 2500

Number of Signal Particles: 7
Neutrino Interaction: -1
Neutrino Energy: 46.48 MeV
Position (-249.28, 455.83, 755.52) cm
Gen.	PdgCode		Energy		EndPosition		TrackID	MotherID
--------------------------------------------------------------------------------
marley	22		1.46	(-249.28, 455.83, 755.52)	133371	133313
marley	1000180400	0.00	(-249.28, 455.83, 755.52)	133370	133313
marley	12		0.04	(-249.28, 455.83, 755.52)	133369	133313
marley	11		0.00	(-249.28, 455.83, 755.52)	133367	133313
marley	11		0.00	(-249.28, 455.83, 755.52)	133366	133313
marley	11		0.00	(-249.28, 455.83, 755.52)	133365	133313
marley	22		0.03	(-249.28, 455.83, 755.52)	133316	0
marley	22		0.77	(-249.28, 455.83, 755.52)	133315	0
marley	22		4.40	(-249.28, 455.83, 755.52)	133314	0
marley	1000190400	0.03	(-249.28, 455.83, 755.52)	133313	0
marley	11		39.75	(-249.28, 455.83, 755.52)	133312	0
Signal Track IDs: 133312; 133313; 133314; 133315; 133316; 133365; 133366; 133367; 133369; 133370; 133371; 
%MSG
%MSG-w ProducerUtils:  SolarNuAna:solarnuana@BeginModule  20-Jan-2026 18:57:31 UTC run: 4705 subRun: 0 event: 2500

Kinetic energy of signal particle is above threshold of 30.00 MeV. Skipping event.

%MSG
20-Jan-2026 18:57:32 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/e1/b1/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250117T062939Z_gen_000250_supernova_g4stage1_g4stage2_detsim_reco_20250619T230121Z_reco2.root"
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@EndJob 20-Jan-2026 18:57:32 UTC  ModuleEndJob
Finished running the SolarNuAna module
%MSG

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                          2.27289       1416.18       4944.77       4.28204       2146.14        10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)             0.0012865      6.82815       68.173      0.00942264      20.4483        10     
end_path:solarnuana:SolarNuAna      2.25473       1409.35       4944.76       3.11137       2150.5         10     
====================================================================================================================
%MSG-i NuRandomService:  SolarNuAna:solarnuana@EndJob 20-Jan-2026 18:57:32 UTC  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1

%MSG

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3186.79 MB
  Peak resident set size usage (VmHWM): 2561.87 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 10 passed = 10 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport         10         10          0 solarnuana

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 14066.889363 Real = 14356.968484

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 3186.79 VmHWM = 2561.87

Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
justIN time: 2026-02-04 07:36:57 UTC       justIN version: 01.06.00