Jobsub ID 250056.0@dunegpschedd02.fnal.gov

Jobsub ID	250056.0@dunegpschedd02.fnal.gov
Workflow ID	10534
Stage ID	1
User name	smanthey@fnal.gov
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-11-23 14:26:56
Site	US_FNAL-FermiGrid
Entry	FNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat	2025-11-23 14:56:01
From worker node	Hostname	dunegli-7706434-0-fnpc18009.fnal.gov
	cpuinfo	AMD EPYC 7551 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	172800 (48 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-11-23 14:27:51
Input files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T131710Z_gen_004909_supernova_g4_detsim_reco_20250724T124544Z_reco2.root
Jobscript	Exit code	0
	Real time	27m (1669s)
	CPU time	27m (1640s = 98%)
	Max RSS bytes	914554880 (872 MiB)
Outputting started	2025-11-23 14:55:40
Output files	https://fndcadoor.fnal.gov:2880/dune/scratch/users/smanthey/ana/solar_ana_marley_flash_radiological_decay0_dune10kt_1x2x6_centralAPA/fnal/10534/1/001/fdhd_ana_2025-11-23T_142804Z_hist.root
Finished	2025-11-23 14:56:01
Saved logs	justin-logs:250056.0-dunegpschedd02.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

olarnuana@BeginModule  23-Nov-2025 14:55:17 UTC run: 5089 subRun: 0 event: 245450
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  23-Nov-2025 14:55:17 UTC run: 5089 subRun: 0 event: 245450
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  23-Nov-2025 14:55:17 UTC run: 5089 subRun: 0 event: 245450
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  23-Nov-2025 14:55:17 UTC run: 5089 subRun: 0 event: 245450
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  23-Nov-2025 14:55:17 UTC run: 5089 subRun: 0 event: 245450
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  23-Nov-2025 14:55:17 UTC run: 5089 subRun: 0 event: 245450
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  23-Nov-2025 14:55:17 UTC run: 5089 subRun: 0 event: 245450
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  23-Nov-2025 14:55:17 UTC run: 5089 subRun: 0 event: 245450
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  23-Nov-2025 14:55:17 UTC run: 5089 subRun: 0 event: 245450
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  23-Nov-2025 14:55:17 UTC run: 5089 subRun: 0 event: 245450
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  23-Nov-2025 14:55:17 UTC run: 5089 subRun: 0 event: 245450
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  23-Nov-2025 14:55:17 UTC run: 5089 subRun: 0 event: 245450
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  23-Nov-2025 14:55:17 UTC run: 5089 subRun: 0 event: 245450
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  23-Nov-2025 14:55:17 UTC run: 5089 subRun: 0 event: 245450
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  23-Nov-2025 14:55:17 UTC run: 5089 subRun: 0 event: 245450
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  23-Nov-2025 14:55:17 UTC run: 5089 subRun: 0 event: 245450
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  23-Nov-2025 14:55:17 UTC run: 5089 subRun: 0 event: 245450
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  23-Nov-2025 14:55:17 UTC run: 5089 subRun: 0 event: 245450
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  23-Nov-2025 14:55:17 UTC run: 5089 subRun: 0 event: 245450
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  23-Nov-2025 14:55:17 UTC run: 5089 subRun: 0 event: 245450
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  23-Nov-2025 14:55:17 UTC run: 5089 subRun: 0 event: 245450
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  23-Nov-2025 14:55:17 UTC run: 5089 subRun: 0 event: 245450
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  23-Nov-2025 14:55:17 UTC run: 5089 subRun: 0 event: 245450
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  23-Nov-2025 14:55:17 UTC run: 5089 subRun: 0 event: 245450
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  23-Nov-2025 14:55:17 UTC run: 5089 subRun: 0 event: 245450
[34mOpFlash PE 246.77 with purity 0.87 time -17.99 plane 0
 - Vertex (-0.05, -93.41, 83.92)
	*** 1st Sanity check: Ratio 0.42 <= 1.00
	*** 2nd Sanity check: #OpHits 30 >= 30
OpFlash PE 25.14 with purity 0.08 time -16.38 plane 0
 - Vertex (-0.05, -591.57, 843.89)
	*** 1st Sanity check: Ratio 0.92 <= 1.00
	*** 2nd Sanity check: #OpHits 2 >= 2
OpFlash PE 21.57 with purity 0.91 time -16.18 plane 0
 - Vertex (-0.04, -138.95, 160.69)
	*** 1st Sanity check: Ratio 0.23 <= 1.00
	*** 2nd Sanity check: #OpHits 9 >= 9
OpFlash PE 21.13 with purity 0.51 time -14.83 plane 0
 - Vertex (-0.05, -144.73, 209.16)
	*** 1st Sanity check: Ratio 0.40 <= 1.00
	*** 2nd Sanity check: #OpHits 6 >= 6
OpFlash PE 17.98 with purity 0.46 time -13.15 plane 0
 - Vertex (-0.04, -217.47, 208.75)
	*** 1st Sanity check: Ratio 0.35 <= 1.00
	*** 2nd Sanity check: #OpHits 6 >= 6
OpFlash PE 5.93 with purity 0.33 time -12.42 plane 0
 - Vertex (0.02, -4.25, 224.21)
	*** 1st Sanity check: Ratio 0.35 <= 1.00
	*** 2nd Sanity check: #OpHits 3 >= 3

# of OpHits (ophitspe): 69477 for times between -2262.53 us and 2227.76 us.
# of OpFlashes (solarflash): 5074 for times between -2262.53 us and 2227.76 us.
[0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  23-Nov-2025 14:55:17 UTC run: 5089 subRun: 0 event: 245450
[36m
# OpHits (ophitspe) in full geometry: 69477
# OpFlashes (solarflash) in full geometry: 5074
# Hits (hitfd) in each view: 166, 155, 434, 0
# Cluster from the hits: 109, 110, 406, 0
# Tracks (pmtracktc) in full geometry: 1[0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  23-Nov-2025 14:55:39 UTC run: 5089 subRun: 0 event: 245450
[32m
# ClusterReco: Looping over 256 matched clusters[0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  23-Nov-2025 14:55:39 UTC run: 5089 subRun: 0 event: 245450
[32m*** Matched preselection cluster: 0 from 256
 - MainTrackID 32867
 - Gen 1 -> marley TPC 1
 - Purity 1.00 Hits 2
 - Charge 234.37 ( MaxHit 181.02 )
 - #AdjCl 0 ( 0 signal ):
 - RecoCol  Time,Y,Z ( 8.45, -118.45, 97.25 )
 - RecoInd0 Time,Y,Z ( 7.92, -118.22, 90.76 )
 - RecoInd1 Time,Y,Z ( 7.88, -118.67, 121.60 )
*** Matched flash: 2565 from 5074
 - Purity 87.09 % Plane 0 #Hits 30 PE 246.77 MainOpHitPE 102.79
 - Time -17.99 (us) Fast 74.90 % Residual 69.48
 - Reco Time,Y,Z ( -17.99, -93.41, 83.92 )
 - Found valid waveform: false
*** Reconstructed Interaction Vertex: 
 - True X,Y,Z ( 4.60, -118.48, 97.32 )
 - Main X,Y,Z ( 3.78, -118.32, 97.11 )
 - EDep X,Y,Z ( 4.01, -118.44, 97.15 )
 - Reco X,Y,Z ( 4.25, -118.45, 97.25 )[0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  23-Nov-2025 14:55:39 UTC run: 5089 subRun: 0 event: 245450
[32m*** Matched preselection cluster: 2 from 256
 - MainTrackID 32869
 - Gen 1 -> marley TPC 1
 - Purity 1.00 Hits 1
 - Charge 77.32 ( MaxHit 77.32 )
 - #AdjCl 0 ( 0 signal ):
 - RecoCol  Time,Y,Z ( 22.72, -325.54, 92.09 )
 - RecoInd0 Time,Y,Z ( 22.65, -105.01, 83.43 )
 - RecoInd1 Time,Y,Z ( 30.40, -546.08, 806.72 )
*** Matched flash: 2395 from 5074
 - Purity 0.00 % Plane 0 #Hits 3 PE 41.33 MainOpHitPE 37.48
 - Time -163.48 (us) Fast 93.10 % Residual 19.04
 - Reco Time,Y,Z ( -163.48, -280.22, 83.92 )
 - Found valid waveform: false
*** Reconstructed Interaction Vertex: 
 - True X,Y,Z ( 4.60, -118.48, 97.32 )
 - Main X,Y,Z ( 8.21, -75.60, 112.66 )
 - EDep X,Y,Z ( 6.63, -104.75, 92.21 )
 - Reco X,Y,Z ( 29.90, -325.54, 92.09 )[0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  23-Nov-2025 14:55:39 UTC run: 5089 subRun: 0 event: 245450
[32m-----------------------------------------------------------------------------------------
[0m
%MSG
23-Nov-2025 14:55:39 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/e2/08/prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T131710Z_gen_004909_supernova_g4_detsim_reco_20250724T124544Z_reco2.root"
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@EndJob 23-Nov-2025 14:55:39 UTC  ModuleEndJob
[35mFinished running the SolarNuAna module[0m
%MSG

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.332588       31.2707       99.9279      0.972732       38.9904        50     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.00186372    0.00983071     0.0292287    0.00873123    0.00702233       50     
end_path:solarnuana:SolarNuAna     0.327762       31.2606       99.9059      0.969396       38.9878        50     
====================================================================================================================
%MSG-i NuRandomService:  SolarNuAna:solarnuana@EndJob 23-Nov-2025 14:55:40 UTC  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1

%MSG

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1551.56 MB
  Peak resident set size usage (VmHWM): 904.675 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 50 passed = 50 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport         50         50          0 solarnuana

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 1595.701088 Real = 1618.838837

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 1551.56 VmHWM = 904.675

Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0