justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 267146.2@dunegpschedd02.fnal.gov

Jobsub ID267146.2@dunegpschedd02.fnal.gov
Workflow ID11625
Stage ID1
User namesmanthey@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-01-08 19:39:25
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2026-01-08 19:49:29
From worker nodeHostnamedunegli-8121712-0-fnpc19118.fnal.gov
cpuinfoAMD EPYC 7502 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-08 19:40:05
Input filesfardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T105359Z_gen_002917_supernova_g4_detsim_reco_20250723T183847Z_reco2.root
JobscriptExit code0
Real time9m (554s)
CPU time8m (538s = 97%)
Max RSS bytes907841536 (865 MiB)
Outputting started2026-01-08 19:49:20
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/smanthey/ana/solar_ana_marley_flash_radiological_decay0_dune10kt_1x2x6_centralAPA/fnal/11625/1/001/fdhd_ana_2026-01-08T_194013Z_hist.root
Finished2026-01-08 19:49:29
Saved logsjustin-logs:267146.2-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

2868
marley	22		0.03	(2.90, -293.02, 220.89)	32871	0
marley	22		1.61	(2.90, -293.02, 220.89)	32870	0
marley	22		1.09	(2.90, -293.02, 220.89)	32869	0
marley	1000190400	0.01	(2.90, -293.02, 220.89)	32868	0
marley	11		18.06	(2.90, -293.02, 220.89)	32867	0
Signal Track IDs: 32867; 32868; 32869; 32870; 32871; 32906; 32907; 32908; 
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  08-Jan-2026 19:48:58 UTC run: 5089 subRun: 0 event: 145850

Kinetic energy of signal particle is below threshold of 30.00 MeV. Selecting event.

%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  08-Jan-2026 19:49:04 UTC run: 5089 subRun: 0 event: 145850
OpFlash PE 24.25 with purity 0.50 time -26.82 plane 0
 - Vertex (-0.05, -342.49, 427.91)
	*** 1st Sanity check: Ratio 1.00 <= 1.00
	*** 2nd Sanity check: #OpHits 1 >= 1
OpFlash PE 1198.67 with purity 0.99 time -18.07 plane 0
 - Vertex (-0.05, -280.22, 195.52)
	*** 1st Sanity check: Ratio 0.56 <= 1.00
	*** 2nd Sanity check: #OpHits 49 >= 49
OpFlash PE 96.13 with purity 0.75 time -17.69 plane 0
 - Vertex (-0.05, -123.25, 465.95)
	*** 1st Sanity check: Ratio 0.36 <= 1.00
	*** 2nd Sanity check: #OpHits 11 >= 11
OpFlash PE 23.24 with purity 0.05 time -17.70 plane 0
 - Vertex (0.05, 546.60, 95.21)
	*** 1st Sanity check: Ratio 0.26 <= 1.00
	*** 2nd Sanity check: #OpHits 7 >= 7
OpFlash PE 92.44 with purity 0.97 time -17.39 plane 0
 - Vertex (-0.05, -280.22, 146.72)
	*** 1st Sanity check: Ratio 0.43 <= 1.00
	*** 2nd Sanity check: #OpHits 16 >= 16
OpFlash PE 17.38 with purity 1.00 time -17.11 plane 0
 - Vertex (-0.05, -354.74, 419.17)
	*** 1st Sanity check: Ratio 0.21 <= 1.00
	*** 2nd Sanity check: #OpHits 7 >= 7
OpFlash PE 45.92 with purity 1.00 time -16.32 plane 0
 - Vertex (-0.05, -308.83, 235.13)
	*** 1st Sanity check: Ratio 0.24 <= 1.00
	*** 2nd Sanity check: #OpHits 12 >= 12
OpFlash PE 7.63 with purity 0.62 time -16.17 plane 0
 - Vertex (0.01, 216.23, 130.50)
	*** 1st Sanity check: Ratio 0.38 <= 1.00
	*** 2nd Sanity check: #OpHits 3 >= 3
OpFlash PE 45.03 with purity 1.00 time -15.95 plane 0
 - Vertex (-0.05, -295.37, 259.26)
	*** 1st Sanity check: Ratio 0.25 <= 1.00
	*** 2nd Sanity check: #OpHits 11 >= 11
OpFlash PE 32.14 with purity 0.89 time -15.59 plane 0
 - Vertex (-0.05, -284.52, 235.33)
	*** 1st Sanity check: Ratio 0.20 <= 1.00
	*** 2nd Sanity check: #OpHits 9 >= 9
OpFlash PE 26.27 with purity 0.81 time -14.81 plane 0
 - Vertex (-0.05, -214.27, 248.45)
	*** 1st Sanity check: Ratio 0.31 <= 1.00
	*** 2nd Sanity check: #OpHits 10 >= 10
OpFlash PE 27.71 with purity 0.49 time -14.23 plane 0
 - Vertex (-0.05, -162.75, 463.82)
	*** 1st Sanity check: Ratio 0.40 <= 1.00
	*** 2nd Sanity check: #OpHits 6 >= 6
OpFlash PE 10.98 with purity 1.00 time -13.76 plane 0
 - Vertex (-0.05, -342.49, 267.51)
	*** 1st Sanity check: Ratio 0.83 <= 1.00
	*** 2nd Sanity check: #OpHits 2 >= 2
OpFlash PE 12.05 with purity 0.69 time -13.42 plane 0
 - Vertex (-0.03, -201.15, 449.25)
	*** 1st Sanity check: Ratio 0.21 <= 1.00
	*** 2nd Sanity check: #OpHits 6 >= 6
OpFlash PE 5.74 with purity 0.72 time -12.89 plane 0
 - Vertex (-0.05, -422.72, 245.27)
	*** 1st Sanity check: Ratio 0.36 <= 1.00
	*** 2nd Sanity check: #OpHits 3 >= 3
OpFlash PE 23.55 with purity 0.79 time -11.93 plane 0
 - Vertex (-0.05, -273.25, 356.10)
	*** 1st Sanity check: Ratio 0.43 <= 1.00
	*** 2nd Sanity check: #OpHits 4 >= 4
OpFlash PE 3.20 with purity 1.00 time -10.00 plane 0
 - Vertex (-0.05, -280.22, 195.52)
	*** 1st Sanity check: Ratio 1.00 <= 1.00
	*** 2nd Sanity check: #OpHits 1 >= 1
OpFlash PE 3.44 with purity 0.54 time -7.74 plane 0
 - Vertex (-0.05, -353.48, 289.76)
	*** 1st Sanity check: Ratio 0.54 <= 1.00
	*** 2nd Sanity check: #OpHits 2 >= 2
OpFlash PE 2.02 with purity 1.00 time -5.04 plane 0
 - Vertex (0.05, 217.95, 35.12)
	*** 1st Sanity check: Ratio 1.00 <= 1.00
	*** 2nd Sanity check: #OpHits 1 >= 1

# of OpHits (ophitspe): 71554 for times between -2261.73 us and 2233.39 us.
# of OpFlashes (solarflash): 9670 for times between -2261.68 us and 2233.39 us.

%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  08-Jan-2026 19:49:04 UTC run: 5089 subRun: 0 event: 145850

# OpHits (ophitspe) in full geometry: 71554
# OpFlashes (solarflash) in full geometry: 9670
# Hits (hitfd) in each view: 173, 165, 457, 0
# Cluster from the hits: 113, 103, 422, 0
# Tracks (pmtracktc) in full geometry: 3
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  08-Jan-2026 19:49:19 UTC run: 5089 subRun: 0 event: 145850

# ClusterReco: Looping over 231 matched clusters
%MSG
Selecting flash with PE 9.11 closer to predicted PE 76.222 than reference flash PE 7.69667 predicted PE 73.6858
The differences are 7.36685 and 8.57372
Selecting flash with PE 21.47 closer to predicted PE 80.336 than reference flash PE 9.11 predicted PE 76.222
The differences are 2.74178 and 7.36685
Selecting flash with PE 1198.67 closer to predicted PE 1095.86 than reference flash PE 21.47 predicted PE 80.336
The differences are 0.0857668 and 2.74178
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  08-Jan-2026 19:49:19 UTC run: 5089 subRun: 0 event: 145850
*** Matched preselection cluster: 3 from 231
 - MainTrackID 32867
 - Gen 1 -> marley TPC 1
 - Purity 1.00 Hits 7
 - Charge 1607.11 ( MaxHit 415.44 )
 - #AdjCl 5 ( 4 signal ):
    - Cluster 5 at distance 12.62 with time 29.19 and charge 60.13 in TPC 1 and hits 1
    - Cluster 7 at distance 5.19 with time 32.25 and charge 88.68 in TPC 1 and hits 1
    - Cluster 12 at distance 50.62 with time 65.31 and charge 43.33 in TPC 5 and hits 1
    - Cluster 15 at distance 13.62 with time 69.22 and charge 112.32 in TPC 1 and hits 1
    - Cluster 20 at distance 29.37 with time 198.65 and charge 68.09 in TPC 1 and hits 1
 - RecoCol  Time,Y,Z ( 17.02, -291.91, 217.35 )
 - RecoInd0 Time,Y,Z ( 17.29, -291.86, 115.32 )
 - RecoInd1 Time,Y,Z ( 17.04, -291.97, 119.37 )
*** Matched flash: 4845 from 9670
 - Purity 99.18 % Plane 0 #Hits 49
 - MainOpHitPE 672.90 (PE);  TotalPE 1198.67 vs expected 1095.86 (PE)
 - Time -18.07 (us) Fast 82.84 % Residual 455.95
 - Reco Time,Y,Z ( -18.07, -280.22, 195.52 )
 - Found valid waveform: false
*** Reconstructed Interaction Vertex: 
 - True X,Y,Z ( 2.90, -293.02, 220.89 )
 - Main X,Y,Z ( 6.68, -291.32, 216.22 )
 - EDep X,Y,Z ( 14.13, -297.86, 208.55 )
 - Reco X,Y,Z ( 5.63, -291.91, 217.35 )
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  08-Jan-2026 19:49:19 UTC run: 5089 subRun: 0 event: 145850
*** Matched preselection cluster: 10 from 231
 - MainTrackID 32870
 - Gen 1 -> marley TPC 1
 - Purity 1.00 Hits 1
 - Charge 71.44 ( MaxHit 71.44 )
 - #AdjCl 0 ( 0 signal ):
 - RecoCol  Time,Y,Z ( 54.77, 676.09, 218.06 )
 - RecoInd0 Time,Y,Z ( 55.23, -287.97, 115.32 )
 - RecoInd1 Time,Y,Z ( 51.17, 1640.15, 1277.27 )
*** Matched flash: 1052 from 9670
 - Purity 0.00 % Plane 0 #Hits 4
 - MainOpHitPE 8.92 (PE);  TotalPE 15.08 vs expected 25.25 (PE)
 - Time -1769.91 (us) Fast 65.78 % Residual 17.78
 - Reco Time,Y,Z ( -1769.91, 591.57, 195.52 )
 - Found valid waveform: false
*** Reconstructed Interaction Vertex: 
 - True X,Y,Z ( 2.90, -293.02, 220.89 )
 - Main X,Y,Z ( 32.33, -293.62, 207.22 )
 - EDep X,Y,Z ( 11.82, -287.76, 218.10 )
 - Reco X,Y,Z ( 292.98, 676.09, 218.06 )
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  08-Jan-2026 19:49:19 UTC run: 5089 subRun: 0 event: 145850
-----------------------------------------------------------------------------------------

%MSG
08-Jan-2026 19:49:19 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/cd/af/prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T105359Z_gen_002917_supernova_g4_detsim_reco_20250723T183847Z_reco2.root"
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@EndJob 08-Jan-2026 19:49:19 UTC  ModuleEndJob
Finished running the SolarNuAna module
%MSG

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.212428       9.9405        27.7885      0.576837       11.3472        50     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.00136272    0.00949037     0.132321      0.0028412     0.0195424       50     
end_path:solarnuana:SolarNuAna     0.206985       9.93081       27.7638       0.57472       11.3488        50     
====================================================================================================================
%MSG-i NuRandomService:  SolarNuAna:solarnuana@EndJob 08-Jan-2026 19:49:19 UTC  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1

%MSG

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1552.04 MB
  Peak resident set size usage (VmHWM): 906.301 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 50 passed = 50 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport         50         50          0 solarnuana

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 515.036685 Real = 528.886563

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 1552.04 VmHWM = 906.301

Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
justIN time: 2026-02-06 14:26:16 UTC       justIN version: 01.06.00