justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 296373.0@dunegpschedd02.fnal.gov

Jobsub ID296373.0@dunegpschedd02.fnal.gov
Workflow ID12416
Stage ID1
User namesmanthey@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-01-28 17:18:54
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2026-01-28 20:59:41
From worker nodeHostnamedunegli-8463070-0-fnpc23031.fnal.gov
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job stateoutputting_failed
Started2026-01-28 17:19:59
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T130214Z_gen_000781_supernova_g4stage1_g4stage2_detsim_reco_20250611T004747Z_reco2.root
JobscriptExit code0
Real time3h (13173s)
CPU time3h (13065s = 99%)
Max RSS bytes2565824512 (2446 MiB)
Outputting started2026-01-28 20:59:32
Output files
Finished2026-01-28 20:59:41
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

icles 30945	from gen 2 Ar39GenInLAr
# of particles 3650	from gen 3 Kr85GenInLAr
# of particles 7	from gen 4 Ar42GenInLAr
# of particles 0	from gen 5 K42From42ArGenInLAr
# of particles 27	from gen 6 Rn222ChainRn222GenInLAr
# of particles 33	from gen 7 Rn222ChainPo218GenInLAr
# of particles 42	from gen 8 Rn222ChainPb214GenInLAr
# of particles 46	from gen 9 Rn222ChainBi214GenInLAr
# of particles 19	from gen 10 Rn222ChainPb210GenInLAr
# of particles 28	from gen 11 Rn220ChainPb212GenInLAr
# of particles 178	from gen 12 K40GenInCathode
# of particles 3003	from gen 13 U238ChainGenInCathode
# of particles 3707	from gen 14 Th232ChainGenInCathode
# of particles 113	from gen 15 K40GenInAnode
# of particles 1339	from gen 16 U238ChainGenInAnode
# of particles 1628	from gen 17 Th232ChainGenInAnode
# of particles 0	from gen 18 Rn222ChainGenInPDS
# of particles 1	from gen 19 K42From42ArGenInUpperMesh1x8x14
# of particles 0	from gen 20 Rn222ChainFromPo218GenInUpperMesh1x8x14
# of particles 31	from gen 21 Rn222ChainFromPb214GenInUpperMesh1x8x14
# of particles 63	from gen 22 Rn222ChainFromBi214GenInUpperMesh1x8x14
# of particles 37	from gen 23 Rn222ChainFromPb210GenInUpperMesh1x8x14
# of particles 32	from gen 24 Rn222ChainFromBi210GenInUpperMesh1x8x14
# of particles 58	from gen 25 Rn220ChainFromPb212GenInUpperMesh1x8x14
# of particles 84313	from gen 26 CavernwallGammasAtLAr1x8x14
# of particles 3534	from gen 27 foamGammasAtLAr1x8x14
# of particles 17	from gen 28 CavernwallNeutronsAtLAr1x8x14
# of particles 1	from gen 29 CryostatNGammasAtLAr1x8x14
# of particles 0	from gen 30 CavernNGammasAtLAr1x8x14
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  28-Jan-2026 20:59:32 UTC run: 4966 subRun: 0 event: 7810

Number of Signal Particles: 6
Neutrino Interaction: -1
Neutrino Energy: 54.61 MeV
Position (-240.76, 353.47, 1880.79) cm
Gen.	PdgCode		Energy		EndPosition		TrackID	MotherID
--------------------------------------------------------------------------------
marley	22		1.29	(-298.02, -217.51, 2157.12)	133009	133000
marley	1000190410	0.00	(-298.02, -217.51, 2157.12)	133008	133000
marley	11		1.07	(-298.02, -217.51, 2157.12)	133002	132999
marley	-12		0.13	(-298.02, -217.51, 2157.12)	133001	132999
marley	1000190419	0.00	(-298.02, -217.51, 2157.12)	133000	132999
marley	1000180410	0.00	(-298.02, -217.51, 2157.12)	132999	132838
marley	22		2.43	(-298.02, -217.51, 2157.12)	132998	132838
marley	22		3.95	(-298.02, -217.51, 2157.12)	132997	132838
marley	22		3.12	(-298.02, -217.51, 2157.12)	132996	132838
marley	22		1.02	(-298.02, -217.51, 2157.12)	132995	132838
marley	22		1.89	(-298.02, -217.51, 2157.12)	132994	132838
marley	22		3.35	(-298.02, -217.51, 2157.12)	132993	132838
marley	22		0.99	(-298.02, -217.51, 2157.12)	132992	132838
marley	2212		0.00	(-296.93, -249.12, 2223.17)	132919	132838
marley	2212		0.00	(-298.78, -249.67, 2222.80)	132918	132838
marley	2212		0.00	(-299.33, -249.51, 2222.98)	132917	132838
marley	1000060120	0.00	(-300.01, -248.43, 2224.08)	132916	132838
marley	2212		0.00	(-297.92, -249.16, 2224.48)	132915	132838
marley	2212		0.05	(-294.13, -252.65, 2225.00)	132914	132838
marley	1000180380	0.00	(-96.13, -230.97, 1790.72)	132913	132838
marley	1000180360	0.01	(-82.66, -241.59, 1794.12)	132912	132838
marley	1000180360	0.00	(-88.01, 85.32, 1876.50)	132911	132838
marley	1000180400	0.00	(-107.00, 105.42, 1867.65)	132910	132838
marley	1000180400	0.01	(-8.25, 177.04, 1915.28)	132909	132838
marley	1000180400	0.00	(-8.74, 176.20, 1914.79)	132908	132838
marley	1000180400	0.00	(-9.97, 175.37, 1914.64)	132907	132838
marley	1000180400	0.00	(-11.93, 177.05, 1915.57)	132906	132838
marley	1000180400	0.00	(-13.35, 178.41, 1914.07)	132905	132838
marley	1000180400	0.01	(-12.54, 177.94, 1912.83)	132904	132838
marley	1000180400	0.01	(-17.38, 179.65, 1914.18)	132903	132838
marley	1000180400	0.00	(-23.03, 165.50, 1908.90)	132902	132838
marley	1000180400	0.01	(-27.39, 146.33, 1913.23)	132901	132838
marley	1000180400	0.01	(-32.47, 148.55, 1915.13)	132900	132838
marley	1000180400	0.01	(-28.95, 149.04, 1913.55)	132899	132838
marley	1000180400	0.01	(-44.01, 164.39, 1891.50)	132898	132838
marley	1000180400	0.00	(-90.71, 159.70, 1948.80)	132897	132838
marley	1000180400	0.00	(-157.64, 257.23, 1957.13)	132896	132838
marley	1000180400	0.00	(-156.33, 268.58, 1957.96)	132895	132838
marley	1000180400	0.01	(-143.14, 277.35, 1978.66)	132894	132838
marley	1000180400	0.00	(-226.61, 302.44, 1973.24)	132893	132838
marley	1000180400	0.01	(-230.81, 311.25, 1968.09)	132892	132838
marley	1000180400	0.01	(-222.08, 274.63, 1917.31)	132891	132838
marley	1000180400	0.02	(-221.84, 278.86, 1915.57)	132890	132838
marley	1000180400	0.01	(-224.00, 276.71, 1916.63)	132889	132838
marley	1000180360	0.01	(-228.32, 284.30, 1913.20)	132888	132838
marley	1000180400	0.02	(-245.84, 260.38, 1902.36)	132887	132838
marley	1000180400	0.01	(-215.88, 249.10, 1924.67)	132886	132838
marley	1000180400	0.02	(-197.66, 247.77, 1910.01)	132885	132838
marley	1000180400	0.01	(-195.65, 248.99, 1932.44)	132884	132838
marley	1000180400	0.03	(-202.73, 240.34, 1935.92)	132883	132838
marley	1000180400	0.01	(-194.68, 263.15, 1938.41)	132882	132838
marley	1000180400	0.00	(-187.39, 263.68, 1931.08)	132881	132838
marley	1000180400	0.02	(-165.13, 268.15, 1856.46)	132880	132838
marley	1000180400	0.01	(-158.76, 274.47, 1859.83)	132879	132838
marley	1000180400	0.04	(-162.55, 278.89, 1869.98)	132878	132838
marley	1000180400	0.02	(-162.71, 275.47, 1858.16)	132877	132838
marley	1000180400	0.04	(-164.12, 270.74, 1858.45)	132876	132838
marley	1000180360	0.01	(-163.16, 271.42, 1858.44)	132875	132838
marley	1000180400	0.03	(-159.84, 276.79, 1863.49)	132874	132838
marley	1000180400	0.04	(-190.68, 302.80, 1841.07)	132873	132838
marley	1000180400	0.05	(-155.73, 305.67, 1829.48)	132872	132838
marley	1000180400	0.01	(-163.29, 308.24, 1838.24)	132871	132838
marley	1000180400	0.05	(-164.75, 321.35, 1845.67)	132870	132838
marley	1000180400	0.06	(-165.29, 319.62, 1846.08)	132869	132838
marley	1000180400	0.00	(-161.03, 326.67, 1845.63)	132868	132838
marley	1000180400	0.01	(-160.74, 341.22, 1845.81)	132867	132838
marley	1000180400	0.02	(-161.38, 342.43, 1846.31)	132866	132838
marley	1000180400	0.02	(-234.08, 346.95, 1824.52)	132865	132838
marley	1000180400	0.04	(-240.78, 338.53, 1826.16)	132864	132838
marley	1000180400	0.00	(-244.49, 342.85, 1831.00)	132863	132838
marley	1000180400	0.01	(-246.88, 346.60, 1836.37)	132862	132838
marley	22		5.81	(-240.76, 353.47, 1880.79)	132839	0
marley	2112		0.77	(-240.76, 353.47, 1880.79)	132838	0
marley	1000190390	0.13	(-240.76, 353.47, 1880.79)	132837	0
marley	11		38.59	(-240.76, 353.47, 1880.79)	132836	0
Signal Track IDs: 132836; 132837; 132838; 132839; 132862; 132863; 132864; 132865; 132866; 132867; 132868; 132869; 132870; 132871; 132872; 132873; 132874; 132875; 132876; 132877; 132878; 132879; 132880; 132881; 132882; 132883; 132884; 132885; 132886; 132887; 132888; 132889; 132890; 132891; 132892; 132893; 132894; 132895; 132896; 132897; 132898; 132899; 132900; 132901; 132902; 132903; 132904; 132905; 132906; 132907; 132908; 132909; 132910; 132911; 132912; 132913; 132914; 132915; 132916; 132917; 132918; 132919; 132992; 132993; 132994; 132995; 132996; 132997; 132998; 132999; 133000; 133001; 133002; 133008; 133009; 
%MSG
%MSG-w ProducerUtils:  SolarNuAna:solarnuana@BeginModule  28-Jan-2026 20:59:32 UTC run: 4966 subRun: 0 event: 7810

Kinetic energy of signal particle is above threshold of 30.00 MeV. Skipping event.

%MSG
28-Jan-2026 20:59:32 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/ba/97/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T130214Z_gen_000781_supernova_g4stage1_g4stage2_detsim_reco_20250611T004747Z_reco2.root"
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@EndJob 28-Jan-2026 20:59:32 UTC  ModuleEndJob
Finished running the SolarNuAna module
%MSG

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                          0.61253       1313.6        1943.56       1849.08       859.987        10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.00135009     0.010146      0.0270791    0.00651121    0.00901916       10     
end_path:solarnuana:SolarNuAna      0.58529       1313.59       1943.56       1849.07       859.99         10     
====================================================================================================================
%MSG-i NuRandomService:  SolarNuAna:solarnuana@EndJob 28-Jan-2026 20:59:32 UTC  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1

%MSG

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3186.59 MB
  Peak resident set size usage (VmHWM): 2565.82 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 10 passed = 10 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport         10         10          0 solarnuana

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 13052.866062 Real = 13158.698463

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 3186.59 VmHWM = 2565.82

Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
justIN time: 2026-02-04 04:29:24 UTC       justIN version: 01.06.00