justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 288544.29@dunegpschedd02.fnal.gov

Jobsub ID288544.29@dunegpschedd02.fnal.gov
Workflow ID12007
Stage ID1
User namesmanthey@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-01-20 14:48:38
SiteUS_FNAL-T1
EntryCMSHTPC_T1_US_FNAL_condce_opp1_whole
Last heartbeat2026-01-20 20:00:13
From worker nodeHostnamedunegli-50386-0-cmswn2355.fnal.gov
cpuinfoIntel(R) Xeon(R) CPU E5-2670 v3 @ 2.30GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job stateoutputting_failed
Started2026-01-20 14:57:25
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250127T055504Z_gen_000496_supernova_g4stage1_g4stage2_detsim_reco_20250701T190102Z_reco2.root
JobscriptExit code0
Real time5h (18149s)
CPU time4h (17757s = 97%)
Max RSS bytes2564792320 (2445 MiB)
Outputting started2026-01-20 19:59:54
Output files
Finished2026-01-20 20:00:13
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

08.51)	132591	132575
marley	2212		0.00	(463.66, 641.35, 502.52)	132590	132575
marley	2212		0.00	(481.89, 645.39, 484.36)	132589	132575
marley	1000070140	0.00	(494.39, 644.34, 487.43)	132588	132575
marley	2212		0.02	(495.41, 645.15, 505.97)	132587	132575
marley	2212		0.00	(479.87, 655.47, 513.15)	132586	132575
marley	2212		0.01	(477.04, 656.09, 514.41)	132585	132575
marley	1000060120	0.01	(473.31, 657.44, 520.40)	132584	132575
marley	2212		0.01	(464.68, 647.79, 516.02)	132583	132575
marley	2212		0.03	(459.13, 636.48, 509.08)	132582	132575
marley	1000060120	0.01	(458.69, 636.23, 508.30)	132581	132575
marley	1000060120	0.02	(451.57, 646.51, 507.19)	132580	132575
marley	1000180400	0.01	(304.97, 536.78, 710.48)	132579	132575
marley	1000180360	0.00	(304.86, 536.82, 709.40)	132578	132575
marley	1000180400	0.01	(312.70, 579.29, 673.88)	132577	132575
marley	1000180400	0.01	(301.71, 598.17, 691.16)	132576	132575
marley	2112		0.16	(304.24, 641.32, 653.49)	132575	0
marley	1000190390	0.00	(304.24, 641.32, 653.49)	132574	0
marley	11		9.31	(304.24, 641.32, 653.49)	132573	0
Signal Track IDs: 132573; 132574; 132575; 132576; 132577; 132578; 132579; 132580; 132581; 132582; 132583; 132584; 132585; 132586; 132587; 132588; 132589; 132590; 132591; 132592; 132593; 132594; 132652; 132653; 132655; 132656; 
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  20-Jan-2026 18:47:52 UTC run: 4933 subRun: 0 event: 4959

Kinetic energy of signal particle is below threshold of 30.00 MeV. Selecting event.

%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  20-Jan-2026 18:51:41 UTC run: 4933 subRun: 0 event: 4959
OpFlash PE 43.71 with purity 0.08 time -2.64 plane 1
 - Vertex (60.07, 743.30, 1105.86)
	*** 1st Sanity check: Ratio 0.46 <= 1.00
	*** 2nd Sanity check: #OpHits 5 >= 5
OpFlash PE 9.35 with purity 0.20 time -1.40 plane 0
 - Vertex (-327.50, -206.70, 1231.78)
	*** 1st Sanity check: Ratio 1.00 <= 1.00
	*** 2nd Sanity check: #OpHits 1 >= 1
OpFlash PE 235.68 with purity 0.32 time 0.04 plane 1
 - Vertex (231.43, 743.30, 730.24)
	*** 1st Sanity check: Ratio 0.17 <= 1.00
	*** 2nd Sanity check: #OpHits 13 >= 13
OpFlash PE 47.90 with purity 0.23 time 3.79 plane 1
 - Vertex (139.96, 743.30, 1054.12)
	*** 1st Sanity check: Ratio 0.26 <= 1.00
	*** 2nd Sanity check: #OpHits 6 >= 6

# of OpHits (ophit10ppm): 52010 for times between -4254.34 us and 295147919198126080.00 us.
# of OpFlashes (solarflash): 18296 for times between -4254.22 us and 295147919198126080.00 us.

%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  20-Jan-2026 18:51:45 UTC run: 4933 subRun: 0 event: 4959

# OpHits (ophit10ppm) in full geometry: 52010
# OpFlashes (solarflash) in full geometry: 18296
# Hits (gaushit) in each view: 6675, 6677, 6751, 0
# Cluster from the hits: 2741, 2807, 5751, 0
# Tracks (pmtracktc) in full geometry: 476
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  20-Jan-2026 19:58:33 UTC run: 4933 subRun: 0 event: 4959

# ClusterReco: Looping over 5739 matched clusters
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  20-Jan-2026 19:58:33 UTC run: 4933 subRun: 0 event: 4959
*** Matched preselection cluster: 105 from 5739
 - MainTrackID 132573
 - Gen 1 -> marley TPC 39
 - Purity 1.00 Hits 4
 - Charge 1157.29 ( MaxHit 520.05 )
 - #AdjCl 0 ( 0 signal ):
 - RecoCol  Time,Y,Z ( 127.85, 642.96, 653.91 )
 - RecoInd0 Time,Y,Z ( 127.83, 643.05, 675.11 )
 - RecoInd1 Time,Y,Z ( 128.61, 642.86, 651.60 )
*** Reconstructed Interaction Vertex: 
 - True X,Y,Z ( 304.24, 641.32, 653.49 )
 - Main X,Y,Z ( 305.34, 644.31, 654.08 )
 - EDep X,Y,Z ( 304.31, 642.90, 653.71 )
 - Reco X,Y,Z ( -1000000.00, 642.96, 653.91 )
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  20-Jan-2026 19:58:33 UTC run: 4933 subRun: 0 event: 4959
-----------------------------------------------------------------------------------------

%MSG
Begin processing the 10th record. run: 4933 subRun: 0 event: 4960 at 20-Jan-2026 19:58:33 UTC
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  20-Jan-2026 19:59:50 UTC run: 4933 subRun: 0 event: 4960

###################################################
Event:			4960
Flag:			1874384567
Geometry:		dunevd10kt_3view_30deg_v5_refactored_1x8x14ref (VD)
Signal Label:		marley
PDS Frequency in [MHz]:	62.50
PDS Tick in [us]:	0.016
TPC Frequency in [MHz]:	2.00
TPC Tick in [us]:	0.50
TPC Map:		112 TPCs
TPC DriftLength in [cm]:650.07
TPC DriftTime in [us]:	4048.69
Fiducial Volume in [cm]:850.32, 1507.84, 2306.12
###################################################
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  20-Jan-2026 19:59:52 UTC run: 4933 subRun: 0 event: 4960

There are a total of 133148 Particles in the event

# of particles 7	from gen 1 marley
# of particles 31448	from gen 2 Ar39GenInLAr
# of particles 3530	from gen 3 Kr85GenInLAr
# of particles 4	from gen 4 Ar42GenInLAr
# of particles 0	from gen 5 K42From42ArGenInLAr
# of particles 36	from gen 6 Rn222ChainRn222GenInLAr
# of particles 23	from gen 7 Rn222ChainPo218GenInLAr
# of particles 46	from gen 8 Rn222ChainPb214GenInLAr
# of particles 33	from gen 9 Rn222ChainBi214GenInLAr
# of particles 25	from gen 10 Rn222ChainPb210GenInLAr
# of particles 13	from gen 11 Rn220ChainPb212GenInLAr
# of particles 242	from gen 12 K40GenInCathode
# of particles 3103	from gen 13 U238ChainGenInCathode
# of particles 3653	from gen 14 Th232ChainGenInCathode
# of particles 109	from gen 15 K40GenInAnode
# of particles 1291	from gen 16 U238ChainGenInAnode
# of particles 1482	from gen 17 Th232ChainGenInAnode
# of particles 0	from gen 18 Rn222ChainGenInPDS
# of particles 0	from gen 19 K42From42ArGenInUpperMesh1x8x14
# of particles 1	from gen 20 Rn222ChainFromPo218GenInUpperMesh1x8x14
# of particles 17	from gen 21 Rn222ChainFromPb214GenInUpperMesh1x8x14
# of particles 46	from gen 22 Rn222ChainFromBi214GenInUpperMesh1x8x14
# of particles 70	from gen 23 Rn222ChainFromPb210GenInUpperMesh1x8x14
# of particles 44	from gen 24 Rn222ChainFromBi210GenInUpperMesh1x8x14
# of particles 62	from gen 25 Rn220ChainFromPb212GenInUpperMesh1x8x14
# of particles 84313	from gen 26 CavernwallGammasAtLAr1x8x14
# of particles 3534	from gen 27 foamGammasAtLAr1x8x14
# of particles 15	from gen 28 CavernwallNeutronsAtLAr1x8x14
# of particles 1	from gen 29 CryostatNGammasAtLAr1x8x14
# of particles 0	from gen 30 CavernNGammasAtLAr1x8x14
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  20-Jan-2026 19:59:53 UTC run: 4933 subRun: 0 event: 4960

Number of Signal Particles: 7
Neutrino Interaction: -1
Neutrino Energy: 31.31 MeV
Position (27.74, 621.38, 1423.90) cm
Gen.	PdgCode		Energy		EndPosition		TrackID	MotherID
--------------------------------------------------------------------------------
marley	11		0.28	(27.74, 621.38, 1423.90)	133175	133128
marley	-12		0.29	(27.74, 621.38, 1423.90)	133174	133128
marley	22		2.09	(27.74, 621.38, 1423.90)	133131	0
marley	22		2.62	(27.74, 621.38, 1423.90)	133130	0
marley	2212		2.10	(27.74, 621.38, 1423.90)	133129	0
marley	1000180390	0.11	(27.74, 621.38, 1423.90)	133128	0
marley	11		15.30	(27.74, 621.38, 1423.90)	133127	0
Signal Track IDs: 133127; 133128; 133129; 133130; 133131; 133174; 133175; 
%MSG
%MSG-w ProducerUtils:  SolarNuAna:solarnuana@BeginModule  20-Jan-2026 19:59:53 UTC run: 4933 subRun: 0 event: 4960

Kinetic energy of signal particle is above threshold of 30.00 MeV. Skipping event.

%MSG
20-Jan-2026 19:59:53 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/a0/de/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250127T055504Z_gen_000496_supernova_g4stage1_g4stage2_detsim_reco_20250701T190102Z_reco2.root"
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@EndJob 20-Jan-2026 19:59:53 UTC  ModuleEndJob
Finished running the SolarNuAna module
%MSG

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                          2.23602       1792.2        5322.32       70.578        2188.19        10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.00221422      13.6438       74.1984     0.00985995      27.4033        10     
end_path:solarnuana:SolarNuAna      2.21864       1778.55       5322.3        3.04559       2199.06        10     
====================================================================================================================
%MSG-i NuRandomService:  SolarNuAna:solarnuana@EndJob 20-Jan-2026 19:59:53 UTC  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1

%MSG

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3189.98 MB
  Peak resident set size usage (VmHWM): 2564.79 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 10 passed = 10 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport         10         10          0 solarnuana

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 17736.491423 Real = 18126.372920

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 3189.98 VmHWM = 2564.79

Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
justIN time: 2026-02-04 04:29:09 UTC       justIN version: 01.06.00