justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 266788.25@dunegpschedd02.fnal.gov

Jobsub ID266788.25@dunegpschedd02.fnal.gov
Workflow ID11592
Stage ID1
User namesmanthey@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-01-08 07:04:42
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2026-01-08 07:15:53
From worker nodeHostnamedunegli-8270939-0-fnpc22008.fnal.gov
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-08 07:05:12
Input filesfardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250208T110619Z_gen_000753_supernova_g4_detsim_reco_20250724T105138Z_reco2.root
JobscriptExit code0
Real time10m (629s)
CPU time10m (614s = 97%)
Max RSS bytes909860864 (867 MiB)
Outputting started2026-01-08 07:15:42
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/smanthey/ana/solar_ana_marley_flash_radiological_decay0_dune10kt_1x2x6_centralAPA/fnal/11592/1/001/fdhd_ana_2026-01-08T_070519Z_hist.root
Finished2026-01-08 07:15:53
Saved logsjustin-logs:266788.25-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

00180400	0.01	(-226.87, 287.84, 771.23)	33018	32952
marley	1000180400	0.01	(-240.87, 291.62, 755.89)	33017	32952
marley	1000180360	0.00	(-159.54, 281.99, 785.96)	33016	32952
marley	1000180360	0.00	(-125.96, 290.34, 846.93)	33015	32952
marley	1000180400	0.00	(-125.05, 249.12, 886.55)	33014	32952
marley	1000180360	0.01	(-123.38, 250.11, 889.80)	33013	32952
marley	1000180360	0.02	(-178.41, 230.80, 898.62)	33012	32952
marley	1000180400	0.01	(-114.79, 291.07, 899.07)	33011	32952
marley	1000180400	0.02	(-116.26, 292.27, 898.70)	33010	32952
marley	1000180400	0.01	(-104.75, 266.29, 905.00)	33009	32952
marley	1000180400	0.01	(-107.49, 265.56, 907.94)	33008	32952
marley	1000180400	0.01	(-107.13, 264.53, 908.24)	33007	32952
marley	1000180400	0.02	(-106.56, 266.00, 910.37)	33006	32952
marley	1000180400	0.02	(-106.66, 265.85, 910.10)	33005	32952
marley	1000180400	0.01	(-106.96, 265.70, 912.04)	33004	32952
marley	1000180400	0.02	(-105.86, 266.98, 912.50)	33003	32952
marley	1000180360	0.02	(-102.21, 265.48, 906.48)	33002	32952
marley	1000180360	0.04	(-126.61, 282.75, 889.66)	33001	32952
marley	1000180400	0.01	(-82.84, 260.25, 921.53)	33000	32952
marley	1000180400	0.04	(-60.01, 272.24, 912.47)	32999	32952
marley	1000180400	0.00	(-73.31, 268.09, 924.85)	32998	32952
marley	1000180400	0.01	(-88.71, 269.36, 931.32)	32997	32952
marley	1000180360	0.04	(-99.00, 285.31, 936.97)	32996	32952
marley	1000180400	0.01	(-105.07, 270.16, 919.72)	32995	32952
marley	1000180400	0.01	(-93.47, 262.89, 905.96)	32994	32952
marley	1000180400	0.02	(-51.44, 180.52, 898.67)	32993	32952
marley	1000180400	0.04	(-51.30, 180.45, 899.16)	32992	32952
marley	1000180400	0.04	(-42.00, 153.07, 870.10)	32991	32952
marley	1000180400	0.01	(-41.28, 153.18, 870.74)	32990	32952
marley	1000180400	0.00	(-41.82, 140.99, 918.79)	32989	32952
marley	1000180400	0.03	(-40.00, 139.80, 922.14)	32988	32952
marley	1000180400	0.01	(-44.58, 142.44, 924.98)	32987	32952
marley	22		6.04	(-198.82, 220.44, 856.72)	32953	0
marley	2112		0.62	(-198.82, 220.44, 856.72)	32952	0
marley	1000190390	0.02	(-198.82, 220.44, 856.72)	32951	0
marley	11		10.85	(-198.82, 220.44, 856.72)	32950	0
Signal Track IDs: 32950; 32951; 32952; 32953; 32987; 32988; 32989; 32990; 32991; 32992; 32993; 32994; 32995; 32996; 32997; 32998; 32999; 33000; 33001; 33002; 33003; 33004; 33005; 33006; 33007; 33008; 33009; 33010; 33011; 33012; 33013; 33014; 33015; 33016; 33017; 33018; 33019; 33020; 33021; 33022; 33023; 33024; 33025; 33026; 33027; 33028; 33029; 33030; 33031; 33032; 33033; 33034; 33035; 33036; 33037; 33038; 33039; 33040; 33041; 33042; 33043; 33045; 33107; 33108; 33109; 33110; 33111; 33112; 33114; 33115; 33116; 33117; 
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  08-Jan-2026 07:15:22 UTC run: 5097 subRun: 0 event: 37650

Kinetic energy of signal particle is below threshold of 30.00 MeV. Selecting event.

%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  08-Jan-2026 07:15:25 UTC run: 5097 subRun: 0 event: 37650
OpFlash PE 45.81 with purity 0.33 time -20.00 plane 0
 - Vertex (0.05, 529.30, 964.68)
	*** 1st Sanity check: Ratio 1.00 <= 1.00
	*** 2nd Sanity check: #OpHits 1 >= 1
OpFlash PE 163.59 with purity 0.86 time -17.96 plane 0
 - Vertex (0.05, 207.18, 964.38)
	*** 1st Sanity check: Ratio 0.14 <= 1.00
	*** 2nd Sanity check: #OpHits 26 >= 26
OpFlash PE 54.37 with purity 0.96 time -17.36 plane 0
 - Vertex (0.02, 136.60, 905.32)
	*** 1st Sanity check: Ratio 0.13 <= 1.00
	*** 2nd Sanity check: #OpHits 16 >= 16
OpFlash PE 105.06 with purity 0.28 time -16.97 plane 0
 - Vertex (0.05, 155.68, 1357.47)
	*** 1st Sanity check: Ratio 0.62 <= 1.00
	*** 2nd Sanity check: #OpHits 15 >= 15
OpFlash PE 22.08 with purity 0.86 time -15.61 plane 0
 - Vertex (0.05, 315.07, 843.89)
	*** 1st Sanity check: Ratio 0.36 <= 1.00
	*** 2nd Sanity check: #OpHits 8 >= 8
OpFlash PE 17.79 with purity 0.94 time -14.71 plane 0
 - Vertex (0.05, 246.62, 821.23)
	*** 1st Sanity check: Ratio 0.23 <= 1.00
	*** 2nd Sanity check: #OpHits 8 >= 8
OpFlash PE 3.54 with purity 0.50 time -14.10 plane 0
 - Vertex (0.05, 186.81, 1136.67)
	*** 1st Sanity check: Ratio 0.50 <= 1.00
	*** 2nd Sanity check: #OpHits 2 >= 2
OpFlash PE 1.94 with purity 1.00 time -13.62 plane 0
 - Vertex (-0.05, -31.14, 781.09)
	*** 1st Sanity check: Ratio 1.00 <= 1.00
	*** 2nd Sanity check: #OpHits 1 >= 1
OpFlash PE 8.63 with purity 0.66 time -12.77 plane 0
 - Vertex (-0.05, -135.26, 954.38)
	*** 1st Sanity check: Ratio 0.66 <= 1.00
	*** 2nd Sanity check: #OpHits 2 >= 2
OpFlash PE 1.90 with purity 1.00 time -11.79 plane 0
 - Vertex (0.05, 529.30, 427.91)
	*** 1st Sanity check: Ratio 1.00 <= 1.00
	*** 2nd Sanity check: #OpHits 1 >= 1
OpFlash PE 3.55 with purity 0.48 time -11.57 plane 0
 - Vertex (-0.05, -248.12, 844.88)
	*** 1st Sanity check: Ratio 0.52 <= 1.00
	*** 2nd Sanity check: #OpHits 2 >= 2
OpFlash PE 1.91 with purity 1.00 time -10.99 plane 0
 - Vertex (0.05, 93.41, 379.11)
	*** 1st Sanity check: Ratio 1.00 <= 1.00
	*** 2nd Sanity check: #OpHits 1 >= 1
OpFlash PE 54.42 with purity 0.27 time 0.67 plane 0
 - Vertex (0.05, 529.30, 781.09)
	*** 1st Sanity check: Ratio 0.82 <= 1.00
	*** 2nd Sanity check: #OpHits 3 >= 3

# of OpHits (ophitspe): 71582 for times between -2262.27 us and 2227.41 us.
# of OpFlashes (solarflash): 9540 for times between -2262.27 us and 2227.41 us.

%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  08-Jan-2026 07:15:25 UTC run: 5097 subRun: 0 event: 37650

# OpHits (ophitspe) in full geometry: 71582
# OpFlashes (solarflash) in full geometry: 9540
# Hits (hitfd) in each view: 174, 181, 429, 0
# Cluster from the hits: 111, 111, 388, 0
# Tracks (pmtracktc) in full geometry: 1
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  08-Jan-2026 07:15:41 UTC run: 5097 subRun: 0 event: 37650

# ClusterReco: Looping over 229 matched clusters
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  08-Jan-2026 07:15:41 UTC run: 5097 subRun: 0 event: 37650
*** Matched preselection cluster: 93 from 229
 - MainTrackID 32950
 - Gen 1 -> marley TPC 14
 - Purity 1.00 Hits 1
 - Charge 45.61 ( MaxHit 45.61 )
 - #AdjCl 0 ( 0 signal ):
 - RecoCol  Time,Y,Z ( 1119.68, 443.50, 859.19 )
 - RecoInd0 Time,Y,Z ( 1128.44, 96.62, 1044.88 )
 - RecoInd1 Time,Y,Z ( 1128.27, 790.38, 1077.95 )
*** Reconstructed Interaction Vertex: 
 - True X,Y,Z ( -198.82, 220.44, 856.72 )
 - Main X,Y,Z ( -196.22, 218.82, 857.26 )
 - EDep X,Y,Z ( -196.15, 219.15, 857.14 )
 - Reco X,Y,Z ( -1000000.00, 443.50, 859.19 )
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  08-Jan-2026 07:15:41 UTC run: 5097 subRun: 0 event: 37650
*** Matched preselection cluster: 101 from 229
 - MainTrackID 32950
 - Gen 1 -> marley TPC 14
 - Purity 1.00 Hits 4
 - Charge 1174.27 ( MaxHit 325.96 )
 - #AdjCl 1 ( 1 signal ):
    - Cluster 105 at distance 15.19 with time 1281.73 and charge 591.80 in TPC 14 and hits 3
 - RecoCol  Time,Y,Z ( 1208.67, 219.89, 856.92 )
 - RecoInd0 Time,Y,Z ( 1209.18, 219.93, 812.49 )
 - RecoInd1 Time,Y,Z ( 1208.58, 219.85, 813.66 )
*** Reconstructed Interaction Vertex: 
 - True X,Y,Z ( -198.82, 220.44, 856.72 )
 - Main X,Y,Z ( -196.22, 218.82, 857.26 )
 - EDep X,Y,Z ( -196.15, 219.15, 857.14 )
 - Reco X,Y,Z ( -1000000.00, 219.89, 856.92 )
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  08-Jan-2026 07:15:41 UTC run: 5097 subRun: 0 event: 37650
-----------------------------------------------------------------------------------------

%MSG
08-Jan-2026 07:15:41 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/4f/b5/prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250208T110619Z_gen_000753_supernova_g4_detsim_reco_20250724T105138Z_reco2.root"
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@EndJob 08-Jan-2026 07:15:41 UTC  ModuleEndJob
Finished running the SolarNuAna module
%MSG

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.158554       11.6416       28.5242       0.80748       11.9054        50     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.00130122    0.00626868     0.0426762    0.00243588    0.00736057       50     
end_path:solarnuana:SolarNuAna     0.156438       11.6352       28.522       0.805173       11.9058        50     
====================================================================================================================
%MSG-i NuRandomService:  SolarNuAna:solarnuana@EndJob 08-Jan-2026 07:15:41 UTC  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1

%MSG

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1553.17 MB
  Peak resident set size usage (VmHWM): 907.37 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 50 passed = 50 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport         50         50          0 solarnuana

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 599.882456 Real = 611.113963

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 1553.17 VmHWM = 907.37

Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
justIN time: 2026-02-06 15:47:17 UTC       justIN version: 01.06.00