justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 265934.6@dunegpschedd02.fnal.gov

Jobsub ID265934.6@dunegpschedd02.fnal.gov
Workflow ID11516
Stage ID1
User namesmanthey@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-01-04 21:31:33
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2026-01-04 21:49:46
From worker nodeHostnamedunegli-8084275-0-fnpc8001.fnal.gov
cpuinfoIntel(R) Xeon(R) CPU E5-2670 v3 @ 2.30GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-04 21:33:27
Input filesfardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T172802Z_gen_002903_supernova_g4_detsim_reco_20250724T084325Z_reco2.root
JobscriptExit code0
Real time16m (965s)
CPU time15m (950s = 98%)
Max RSS bytes907300864 (865 MiB)
Outputting started2026-01-04 21:49:33
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/smanthey/ana/solar_ana_marley_flash_radiological_decay0_dune10kt_1x2x6_centralAPA/fnal/11516/1/001/fdhd_ana_2026-01-04T_213337Z_hist.root
Finished2026-01-04 21:49:46
Saved logsjustin-logs:265934.6-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

marley	1000180400	0.00	(-310.23, 298.46, 989.20)	32747	32705
marley	1000180400	0.02	(-305.07, 294.02, 983.21)	32746	32705
marley	1000180400	0.03	(-304.17, 296.77, 984.53)	32745	32705
marley	1000180400	0.02	(-336.52, 275.07, 966.08)	32744	32705
marley	1000180400	0.02	(-337.64, 274.97, 967.98)	32743	32705
marley	1000180400	0.01	(-343.94, 293.08, 968.30)	32742	32705
marley	1000180360	0.00	(-344.83, 293.28, 968.25)	32741	32705
marley	1000180400	0.00	(-345.78, 293.15, 967.62)	32740	32705
marley	1000180400	0.03	(-346.22, 293.30, 967.50)	32739	32705
marley	1000180400	0.04	(-332.22, 253.77, 938.38)	32738	32705
marley	1000180400	0.04	(-332.60, 264.04, 939.25)	32737	32705
marley	1000180360	0.01	(-332.04, 260.76, 940.93)	32736	32705
marley	1000180400	0.05	(-293.89, 235.45, 933.70)	32735	32705
marley	1000180400	0.01	(-352.02, 285.80, 977.10)	32734	32705
marley	1000180380	0.01	(-362.30, 282.38, 982.13)	32733	32705
marley	1000180400	0.01	(-364.10, 283.46, 984.34)	32732	32705
marley	1000180400	0.06	(-367.57, 290.08, 984.26)	32731	32705
marley	1000180400	0.01	(-364.88, 278.77, 982.61)	32730	32705
marley	1000180400	0.00	(-357.54, 272.87, 978.86)	32729	32705
marley	1000180360	0.01	(-346.78, 270.46, 970.89)	32728	32705
marley	1000180400	0.02	(-338.73, 270.50, 971.77)	32727	32705
marley	1000180400	0.03	(-329.85, 293.30, 1004.34)	32726	32705
marley	1000180400	0.00	(-326.35, 299.77, 999.78)	32725	32705
marley	1000180400	0.04	(-318.87, 320.08, 984.68)	32724	32705
marley	22		3.02	(-303.02, 298.01, 965.60)	32706	0
marley	2112		0.75	(-303.02, 298.01, 965.60)	32705	0
marley	1000190390	0.03	(-303.02, 298.01, 965.60)	32704	0
marley	11		7.55	(-303.02, 298.01, 965.60)	32703	0
Signal Track IDs: 32703; 32704; 32705; 32706; 32724; 32725; 32726; 32727; 32728; 32729; 32730; 32731; 32732; 32733; 32734; 32735; 32736; 32737; 32738; 32739; 32740; 32741; 32742; 32743; 32744; 32745; 32746; 32747; 32748; 32749; 32750; 32751; 32752; 32753; 32754; 32755; 32756; 32757; 32758; 32759; 32760; 32761; 32762; 32766; 32767; 32768; 32769; 
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  04-Jan-2026 21:48:59 UTC run: 5092 subRun: 0 event: 145150

Kinetic energy of signal particle is below threshold of 30.00 MeV. Selecting event.

%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  04-Jan-2026 21:49:11 UTC run: 5092 subRun: 0 event: 145150
OpFlash PE 12.87 with purity 0.33 time -23.62 plane 0
 - Vertex (0.05, 155.68, 499.90)
	*** 1st Sanity check: Ratio 1.00 <= 1.00
	*** 2nd Sanity check: #OpHits 1 >= 1
OpFlash PE 44.19 with purity 0.92 time -17.94 plane 0
 - Vertex (0.04, 284.71, 987.07)
	*** 1st Sanity check: Ratio 0.08 <= 1.00
	*** 2nd Sanity check: #OpHits 20 >= 20
OpFlash PE 21.61 with purity 0.70 time -17.12 plane 0
 - Vertex (-0.05, -31.14, 892.69)
	*** 1st Sanity check: Ratio 0.48 <= 1.00
	*** 2nd Sanity check: #OpHits 6 >= 6
OpFlash PE 11.05 with purity 0.41 time -16.42 plane 0
 - Vertex (0.05, 422.68, 958.70)
	*** 1st Sanity check: Ratio 0.27 <= 1.00
	*** 2nd Sanity check: #OpHits 5 >= 5
OpFlash PE 7.95 with purity 0.93 time -15.90 plane 0
 - Vertex (0.02, 183.16, 865.16)
	*** 1st Sanity check: Ratio 0.29 <= 1.00
	*** 2nd Sanity check: #OpHits 4 >= 4
OpFlash PE 6.86 with purity 0.51 time -14.95 plane 0
 - Vertex (0.05, 406.48, 862.56)
	*** 1st Sanity check: Ratio 0.44 <= 1.00
	*** 2nd Sanity check: #OpHits 3 >= 3
OpFlash PE 3.84 with purity 0.49 time -14.57 plane 0
 - Vertex (0.05, 217.95, 1162.61)
	*** 1st Sanity check: Ratio 0.51 <= 1.00
	*** 2nd Sanity check: #OpHits 2 >= 2
OpFlash PE 1.95 with purity 1.00 time -10.13 plane 0
 - Vertex (0.05, 529.30, 1357.47)
	*** 1st Sanity check: Ratio 1.00 <= 1.00
	*** 2nd Sanity check: #OpHits 1 >= 1
OpFlash PE 4.65 with purity 0.30 time -10.13 plane 0
 - Vertex (0.05, 219.87, 781.09)
	*** 1st Sanity check: Ratio 0.61 <= 1.00
	*** 2nd Sanity check: #OpHits 2 >= 2

# of OpHits (ophitspe): 70660 for times between -2260.72 us and 2227.74 us.
# of OpFlashes (solarflash): 9502 for times between -2260.70 us and 2227.70 us.

%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  04-Jan-2026 21:49:11 UTC run: 5092 subRun: 0 event: 145150

# OpHits (ophitspe) in full geometry: 70660
# OpFlashes (solarflash) in full geometry: 9502
# Hits (hitfd) in each view: 205, 216, 491, 0
# Cluster from the hits: 131, 132, 446, 0
# Tracks (pmtracktc) in full geometry: 1
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  04-Jan-2026 21:49:32 UTC run: 5092 subRun: 0 event: 145150

# ClusterReco: Looping over 291 matched clusters
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  04-Jan-2026 21:49:32 UTC run: 5092 subRun: 0 event: 145150
*** Matched preselection cluster: 203 from 291
 - MainTrackID 32703
 - Gen 1 -> marley TPC 18
 - Purity 1.00 Hits 3
 - Charge 763.19 ( MaxHit 541.44 )
 - #AdjCl 2 ( 2 signal ):
    - Cluster 208 at distance 5.11 with time 1895.29 and charge 86.18 in TPC 18 and hits 1
    - Cluster 211 at distance 11.45 with time 1910.51 and charge 177.51 in TPC 18 and hits 2
 - RecoCol  Time,Y,Z ( 1876.43, 298.77, 965.26 )
 - RecoInd0 Time,Y,Z ( 1876.50, 298.74, 1044.88 )
 - RecoInd1 Time,Y,Z ( 1876.17, 298.79, 1044.88 )
*** Matched flash: 4696 from 9502
 - Purity 92.17 % Plane 0 #Hits 20 PE 44.19 MainOpHitPE 3.34
 - Time -17.94 (us) Fast 13.28 % Residual 1.98
 - Reco Time,Y,Z ( -17.94, 284.71, 987.07 )
 - Found valid waveform: false
*** Reconstructed Interaction Vertex: 
 - True X,Y,Z ( -303.02, 298.01, 965.60 )
 - Main X,Y,Z ( -305.09, 298.85, 965.32 )
 - EDep X,Y,Z ( -305.01, 298.79, 965.21 )
 - Reco X,Y,Z ( -304.17, 298.77, 965.26 )
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  04-Jan-2026 21:49:32 UTC run: 5092 subRun: 0 event: 145150
*** Matched preselection cluster: 226 from 291
 - MainTrackID 32767
 - Gen 1 -> marley TPC 22
 - Purity 1.00 Hits 2
 - Charge 153.50 ( MaxHit 94.40 )
 - #AdjCl 1 ( 0 signal ):
    - Cluster 182 at distance 79.84 with time 1665.18 and charge 60.94 in TPC 20 and hits 1
 - RecoCol  Time,Y,Z ( 2159.30, 369.95, 1238.89 )
 - RecoInd0 Time,Y,Z ( 2159.70, 369.95, 1277.27 )
 - RecoInd1 Time,Y,Z ( -1000000.00, -1000000.00, -1000000.00 )
*** Matched flash: 5360 from 9502
 - Purity 0.00 % Plane 0 #Hits 3 PE 23.73 MainOpHitPE 20.33
 - Time 294.25 (us) Fast 89.89 % Residual 159.45
 - Reco Time,Y,Z ( 294.25, 404.76, 1197.07 )
 - Found valid waveform: false
*** Reconstructed Interaction Vertex: 
 - True X,Y,Z ( -303.02, 298.01, 965.60 )
 - Main X,Y,Z ( -379.52, 352.18, 1196.41 )
 - EDep X,Y,Z ( -352.07, 370.85, 1235.11 )
 - Reco X,Y,Z ( -299.46, 369.95, 1238.89 )
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  04-Jan-2026 21:49:32 UTC run: 5092 subRun: 0 event: 145150
*** Matched preselection cluster: 232 from 291
 - MainTrackID 32767
 - Gen 1 -> marley TPC 22
 - Purity 1.00 Hits 2
 - Charge 189.46 ( MaxHit 101.24 )
 - #AdjCl 0 ( 0 signal ):
 - RecoCol  Time,Y,Z ( 2181.97, 1006.31, 1235.10 )
 - RecoInd0 Time,Y,Z ( 2181.96, 370.82, 1277.27 )
 - RecoInd1 Time,Y,Z ( 2182.99, 1641.81, 347.71 )
*** Reconstructed Interaction Vertex: 
 - True X,Y,Z ( -303.02, 298.01, 965.60 )
 - Main X,Y,Z ( -379.52, 352.18, 1196.41 )
 - EDep X,Y,Z ( -352.07, 370.85, 1235.11 )
 - Reco X,Y,Z ( -1000000.00, 1006.31, 1235.10 )
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  04-Jan-2026 21:49:32 UTC run: 5092 subRun: 0 event: 145150
-----------------------------------------------------------------------------------------

%MSG
04-Jan-2026 21:49:32 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/c6/a2/prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T172802Z_gen_002903_supernova_g4_detsim_reco_20250724T084325Z_reco2.root"
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@EndJob 04-Jan-2026 21:49:32 UTC  ModuleEndJob
Finished running the SolarNuAna module
%MSG

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.315424       17.4068       45.2569      0.858884       18.5488        50     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.00136636    0.00928055     0.0560385    0.00356374     0.0100399       50     
end_path:solarnuana:SolarNuAna     0.293565       17.3973       45.2548      0.849604       18.5472        50     
====================================================================================================================
%MSG-i NuRandomService:  SolarNuAna:solarnuana@EndJob 04-Jan-2026 21:49:32 UTC  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1

%MSG

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1552.48 MB
  Peak resident set size usage (VmHWM): 906.879 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 50 passed = 50 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport         50         50          0 solarnuana

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 911.059887 Real = 924.112084

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 1552.48 VmHWM = 906.879

Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
justIN time: 2026-02-06 21:01:21 UTC       justIN version: 01.06.00