justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 270723.5@dunegpschedd02.fnal.gov

Jobsub ID270723.5@dunegpschedd02.fnal.gov
Workflow ID11754
Stage ID1
User namesmanthey@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-01-12 17:48:25
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2026-01-12 21:44:46
From worker nodeHostnamedunegli-8147871-0-fnpc23047.fnal.gov
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-12 21:37:35
Input filesfardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T164113Z_gen_001720_supernova_g4_detsim_reco_20250724T110027Z_reco2.root
JobscriptExit code0
Real time6m (398s)
CPU time6m (377s = 94%)
Max RSS bytes907747328 (865 MiB)
Outputting started2026-01-12 21:44:14
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/smanthey/ana/solar_ana_marley_flash_radiological_decay0_dune10kt_1x2x6_centralAPA/fnal/11754/1/001/fdhd_ana_2026-01-12T_213742Z_hist.root
Finished2026-01-12 21:44:46
Saved logsjustin-logs:270723.5-dunegpschedd02.fnal.gov.logs.tgz
List job events     (HTCondor job logs unavailable)

Jobscript log (last 10,000 characters)

665.29)	33053	33036
marley	2212		0.01	(-451.62, 527.73, 666.06)	33052	33036
marley	1000260560	0.00	(-466.11, 534.16, 656.99)	33051	33036
marley	2212		0.00	(-458.24, 528.81, 669.96)	33050	33036
marley	2212		0.00	(-456.82, 528.42, 671.23)	33049	33036
marley	2212		0.01	(-456.00, 528.60, 672.26)	33048	33036
marley	2212		0.03	(-454.09, 529.66, 673.40)	33047	33036
marley	1000280620	0.00	(-379.76, 503.18, 692.94)	33046	33036
marley	1000180400	0.00	(-333.04, 330.89, 876.46)	33045	33036
marley	1000060120	0.02	(-0.36, -25.89, 1030.13)	33044	33036
marley	2212		0.02	(-0.65, -25.38, 1029.71)	33043	33036
marley	1000180400	0.00	(-0.29, -17.99, 1024.99)	33042	33036
marley	1000180400	0.00	(2.61, -17.55, 1012.09)	33041	33036
marley	1000180400	0.01	(-22.79, -0.41, 1004.23)	33040	33036
marley	1000180400	0.00	(-8.69, 7.58, 1004.70)	33039	33036
marley	1000260540	0.01	(-2.44, 5.40, 988.33)	33038	33036
marley	1000240520	0.01	(-2.49, 5.22, 988.50)	33037	33036
marley	2112		0.14	(-22.81, 135.97, 911.69)	33036	0
marley	1000190390	0.03	(-22.81, 135.97, 911.69)	33035	0
marley	11		14.69	(-22.81, 135.97, 911.69)	33034	0
Signal Track IDs: 33034; 33035; 33036; 33037; 33038; 33039; 33040; 33041; 33042; 33043; 33044; 33045; 33046; 33047; 33048; 33049; 33050; 33051; 33052; 33053; 33054; 33055; 33056; 33058; 33059; 33077; 33078; 33079; 33080; 
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  12-Jan-2026 21:43:58 UTC run: 5092 subRun: 0 event: 86000

Kinetic energy of signal particle is below threshold of 30.00 MeV. Selecting event.

%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  12-Jan-2026 21:44:00 UTC run: 5092 subRun: 0 event: 86000
OpFlash PE 18.11 with purity 0.20 time -20.18 plane 0
 - Vertex (0.05, 31.14, 267.51)
	*** 1st Sanity check: Ratio 1.00 <= 1.00
	*** 2nd Sanity check: #OpHits 1 >= 1
OpFlash PE 9.69 with purity 0.37 time -18.54 plane 0
 - Vertex (-0.03, -115.76, 928.04)
	*** 1st Sanity check: Ratio 0.39 <= 1.00
	*** 2nd Sanity check: #OpHits 4 >= 4
OpFlash PE 718.31 with purity 0.92 time -18.07 plane 0
 - Vertex (0.05, 155.68, 913.32)
	*** 1st Sanity check: Ratio 0.33 <= 1.00
	*** 2nd Sanity check: #OpHits 49 >= 49
OpFlash PE 62.93 with purity 0.91 time -17.42 plane 0
 - Vertex (0.05, 211.42, 799.45)
	*** 1st Sanity check: Ratio 0.26 <= 1.00
	*** 2nd Sanity check: #OpHits 17 >= 17
OpFlash PE 40.96 with purity 0.88 time -16.99 plane 0
 - Vertex (0.05, 258.58, 1002.67)
	*** 1st Sanity check: Ratio 0.22 <= 1.00
	*** 2nd Sanity check: #OpHits 13 >= 13
OpFlash PE 49.12 with purity 0.75 time -16.07 plane 0
 - Vertex (0.05, 174.03, 924.97)
	*** 1st Sanity check: Ratio 0.23 <= 1.00
	*** 2nd Sanity check: #OpHits 15 >= 15
OpFlash PE 27.56 with purity 0.70 time -15.57 plane 0
 - Vertex (0.05, 149.12, 1000.51)
	*** 1st Sanity check: Ratio 0.19 <= 1.00
	*** 2nd Sanity check: #OpHits 10 >= 10
OpFlash PE 24.99 with purity 0.68 time -15.11 plane 0
 - Vertex (0.03, 104.67, 921.40)
	*** 1st Sanity check: Ratio 0.18 <= 1.00
	*** 2nd Sanity check: #OpHits 10 >= 10
OpFlash PE 5.86 with purity 0.66 time -14.26 plane 0
 - Vertex (0.05, 196.48, 959.61)
	*** 1st Sanity check: Ratio 0.67 <= 1.00
	*** 2nd Sanity check: #OpHits 2 >= 2
OpFlash PE 5.74 with purity 0.64 time -13.81 plane 0
 - Vertex (0.05, 328.60, 1104.28)
	*** 1st Sanity check: Ratio 0.64 <= 1.00
	*** 2nd Sanity check: #OpHits 2 >= 2
OpFlash PE 5.50 with purity 1.00 time -13.30 plane 0
 - Vertex (0.05, 249.08, 964.68)
	*** 1st Sanity check: Ratio 0.36 <= 1.00
	*** 2nd Sanity check: #OpHits 3 >= 3
OpFlash PE 12.88 with purity 0.83 time -12.80 plane 0
 - Vertex (0.04, 93.45, 863.92)
	*** 1st Sanity check: Ratio 0.33 <= 1.00
	*** 2nd Sanity check: #OpHits 5 >= 5
OpFlash PE 5.85 with purity 1.00 time -11.81 plane 0
 - Vertex (0.05, 215.39, 931.30)
	*** 1st Sanity check: Ratio 0.68 <= 1.00
	*** 2nd Sanity check: #OpHits 2 >= 2
OpFlash PE 7.28 with purity 0.45 time -11.33 plane 0
 - Vertex (0.02, 115.57, 853.56)
	*** 1st Sanity check: Ratio 0.45 <= 1.00
	*** 2nd Sanity check: #OpHits 3 >= 3
OpFlash PE 4.13 with purity 0.53 time -11.09 plane 0
 - Vertex (0.00, -86.47, 1002.41)
	*** 1st Sanity check: Ratio 0.53 <= 1.00
	*** 2nd Sanity check: #OpHits 2 >= 2
OpFlash PE 5.88 with purity 0.35 time -10.48 plane 0
 - Vertex (0.05, 174.10, 914.18)
	*** 1st Sanity check: Ratio 0.35 <= 1.00
	*** 2nd Sanity check: #OpHits 3 >= 3
OpFlash PE 10.60 with purity 0.30 time -9.74 plane 0
 - Vertex (0.05, 242.74, 533.67)
	*** 1st Sanity check: Ratio 0.70 <= 1.00
	*** 2nd Sanity check: #OpHits 2 >= 2
OpFlash PE 72.25 with purity 0.97 time -0.03 plane 0
 - Vertex (0.05, 217.95, 1013.48)
	*** 1st Sanity check: Ratio 0.97 <= 1.00
	*** 2nd Sanity check: #OpHits 2 >= 2

# of OpHits (ophitspe): 72277 for times between -2262.27 us and 2227.79 us.
# of OpFlashes (solarflash): 9749 for times between -2262.27 us and 2227.70 us.

%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  12-Jan-2026 21:44:00 UTC run: 5092 subRun: 0 event: 86000

# OpHits (ophitspe) in full geometry: 72277
# OpFlashes (solarflash) in full geometry: 9749
# Hits (hitfd) in each view: 182, 182, 476, 0
# Cluster from the hits: 112, 127, 445, 0
# Tracks (pmtracktc) in full geometry: 1
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  12-Jan-2026 21:44:14 UTC run: 5092 subRun: 0 event: 86000

# ClusterReco: Looping over 292 matched clusters
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  12-Jan-2026 21:44:14 UTC run: 5092 subRun: 0 event: 86000
*** Matched preselection cluster: 13 from 292
 - MainTrackID 33034
 - Gen 1 -> marley TPC 14
 - Purity 1.00 Hits 6
 - Charge 1231.09 ( MaxHit 308.51 )
 - #AdjCl 1 ( 1 signal ):
    - Cluster 14 at distance 12.18 with time 140.50 and charge 334.39 in TPC 14 and hits 1
 - RecoCol  Time,Y,Z ( 126.51, 137.22, 913.12 )
 - RecoInd0 Time,Y,Z ( 126.57, 137.23, 812.49 )
 - RecoInd1 Time,Y,Z ( 126.64, 137.21, 871.46 )
*** Matched flash: 4801 from 9749
 - Purity 92.47 % Plane 0 #Hits 49
 - MainOpHitPE 238.62 (PE);  TotalPE 718.31 vs expected 735.72 (PE)
 - Time -18.07 (us) Fast 74.17 % Residual 155.82
 - Reco Time,Y,Z ( -18.07, 155.68, 913.32 )
 - Found valid waveform: false
*** Reconstructed Interaction Vertex: 
 - True X,Y,Z ( -22.81, 135.97, 911.69 )
 - Main X,Y,Z ( -23.20, 137.76, 914.11 )
 - EDep X,Y,Z ( -23.20, 137.88, 914.02 )
 - Reco X,Y,Z ( -23.21, 137.22, 913.12 )
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  12-Jan-2026 21:44:14 UTC run: 5092 subRun: 0 event: 86000
*** Matched preselection cluster: 22 from 292
 - MainTrackID 33034
 - Gen 1 -> marley TPC 18
 - Purity 1.00 Hits 1
 - Charge 54.14 ( MaxHit 54.14 )
 - #AdjCl 0 ( 0 signal ):
 - RecoCol  Time,Y,Z ( 260.47, 408.19, 981.41 )
 - RecoInd0 Time,Y,Z ( 260.83, 843.02, 1158.14 )
 - RecoInd1 Time,Y,Z ( 267.30, -26.64, 1277.27 )
*** Matched flash: 2649 from 9749
 - Purity 0.00 % Plane 0 #Hits 6
 - MainOpHitPE 8.15 (PE);  TotalPE 19.67 vs expected 46.33 (PE)
 - Time -1050.41 (us) Fast 56.64 % Residual 26.50
 - Reco Time,Y,Z ( -1050.41, 446.92, 1050.04 )
 - Found valid waveform: false
*** Reconstructed Interaction Vertex: 
 - True X,Y,Z ( -22.81, 135.97, 911.69 )
 - Main X,Y,Z ( -23.20, 137.76, 914.11 )
 - EDep X,Y,Z ( -23.20, 137.88, 914.02 )
 - Reco X,Y,Z ( -210.48, 408.19, 981.41 )
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  12-Jan-2026 21:44:14 UTC run: 5092 subRun: 0 event: 86000
-----------------------------------------------------------------------------------------

%MSG
12-Jan-2026 21:44:14 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/fa/46/prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T164113Z_gen_001720_supernova_g4_detsim_reco_20250724T110027Z_reco2.root"
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@EndJob 12-Jan-2026 21:44:14 UTC  ModuleEndJob
Finished running the SolarNuAna module
%MSG

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.146042       7.13181       19.0689      0.592115       8.07091        50     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000977987    0.0065823     0.0260067    0.00199908    0.00760213       50     
end_path:solarnuana:SolarNuAna     0.144231       7.12505       19.0462      0.582411       8.07008        50     
====================================================================================================================
%MSG-i NuRandomService:  SolarNuAna:solarnuana@EndJob 12-Jan-2026 21:44:14 UTC  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1

%MSG

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1552.2 MB
  Peak resident set size usage (VmHWM): 904.741 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 50 passed = 50 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport         50         50          0 solarnuana

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 363.365699 Real = 381.576289

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 1552.2 VmHWM = 904.741

Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
justIN time: 2026-02-06 09:01:35 UTC       justIN version: 01.06.00