Jobsub ID 257958.1@dunegpschedd01.fnal.gov

Jobsub ID	257958.1@dunegpschedd01.fnal.gov
Workflow ID	10700
Stage ID	1
User name	smanthey@fnal.gov
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-11-26 00:19:55
Site	US_FNAL-FermiGrid
Entry	FNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat	2025-11-26 00:29:34
From worker node	Hostname	dunegli-7738274-0-fnpc23106.fnal.gov
	cpuinfo	AMD EPYC 9354 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	172800 (48 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-11-26 00:21:43
Input files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T100738Z_gen_002308_supernova_g4_detsim_reco_20250724T084544Z_reco2.root
Jobscript	Exit code	0
	Real time	7m (451s)
	CPU time	7m (434s = 96%)
	Max RSS bytes	908677120 (866 MiB)
Outputting started	2025-11-26 00:29:15
Output files	https://fndcadoor.fnal.gov:2880/dune/scratch/users/smanthey/ana/solar_ana_marley_flash_radiological_decay0_dune10kt_1x2x6_centralAPA/fnal/10700/1/001/fdhd_ana_2025-11-26T_002151Z_hist.root
Finished	2025-11-26 00:29:34
Saved logs	justin-logs:257958.1-dunegpschedd01.fnal.gov.logs.tgz
List job events (HTCondor job logs unavailable)

Jobscript log (last 10,000 characters)

[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Nov-2025 00:29:01 UTC run: 5089 subRun: 0 event: 115400
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Nov-2025 00:29:01 UTC run: 5089 subRun: 0 event: 115400
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Nov-2025 00:29:01 UTC run: 5089 subRun: 0 event: 115400
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Nov-2025 00:29:01 UTC run: 5089 subRun: 0 event: 115400
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Nov-2025 00:29:01 UTC run: 5089 subRun: 0 event: 115400
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Nov-2025 00:29:01 UTC run: 5089 subRun: 0 event: 115400
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Nov-2025 00:29:01 UTC run: 5089 subRun: 0 event: 115400
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Nov-2025 00:29:01 UTC run: 5089 subRun: 0 event: 115400
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Nov-2025 00:29:01 UTC run: 5089 subRun: 0 event: 115400
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Nov-2025 00:29:01 UTC run: 5089 subRun: 0 event: 115400
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Nov-2025 00:29:01 UTC run: 5089 subRun: 0 event: 115400
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Nov-2025 00:29:01 UTC run: 5089 subRun: 0 event: 115400
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Nov-2025 00:29:01 UTC run: 5089 subRun: 0 event: 115400
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Nov-2025 00:29:01 UTC run: 5089 subRun: 0 event: 115400
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Nov-2025 00:29:01 UTC run: 5089 subRun: 0 event: 115400
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Nov-2025 00:29:01 UTC run: 5089 subRun: 0 event: 115400
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Nov-2025 00:29:01 UTC run: 5089 subRun: 0 event: 115400
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Nov-2025 00:29:01 UTC run: 5089 subRun: 0 event: 115400
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Nov-2025 00:29:01 UTC run: 5089 subRun: 0 event: 115400
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Nov-2025 00:29:01 UTC run: 5089 subRun: 0 event: 115400
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Nov-2025 00:29:01 UTC run: 5089 subRun: 0 event: 115400
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Nov-2025 00:29:01 UTC run: 5089 subRun: 0 event: 115400
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Nov-2025 00:29:01 UTC run: 5089 subRun: 0 event: 115400
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Nov-2025 00:29:01 UTC run: 5089 subRun: 0 event: 115400
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Nov-2025 00:29:01 UTC run: 5089 subRun: 0 event: 115400
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Nov-2025 00:29:01 UTC run: 5089 subRun: 0 event: 115400
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Nov-2025 00:29:01 UTC run: 5089 subRun: 0 event: 115400
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Nov-2025 00:29:01 UTC run: 5089 subRun: 0 event: 115400
[31mInvalid OpWaveform handle[0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Nov-2025 00:29:01 UTC run: 5089 subRun: 0 event: 115400
[34mOpFlash PE 195.15 with purity 0.91 time -18.03 plane 0
 - Vertex (0.05, 549.18, 35.12)
	*** 1st Sanity check: Ratio 0.38 <= 1.00
	*** 2nd Sanity check: #OpHits 22 >= 22
OpFlash PE 11.21 with purity 0.95 time -15.53 plane 0
 - Vertex (0.05, 576.18, 110.60)
	*** 1st Sanity check: Ratio 0.25 <= 1.00
	*** 2nd Sanity check: #OpHits 5 >= 5
OpFlash PE 4.74 with purity 0.71 time -13.12 plane 0
 - Vertex (0.05, 482.00, 180.01)
	*** 1st Sanity check: Ratio 0.59 <= 1.00
	*** 2nd Sanity check: #OpHits 2 >= 2
OpFlash PE 46.84 with purity 0.55 time -8.94 plane 0
 - Vertex (0.05, 467.03, 83.92)
	*** 1st Sanity check: Ratio 0.96 <= 1.00
	*** 2nd Sanity check: #OpHits 2 >= 2
OpFlash PE 18.82 with purity 0.27 time -6.17 plane 0
 - Vertex (0.05, 591.57, 146.72)
	*** 1st Sanity check: Ratio 0.81 <= 1.00
	*** 2nd Sanity check: #OpHits 3 >= 3
OpFlash PE 144.57 with purity 0.12 time 0.38 plane 0
 - Vertex (0.05, 467.03, 499.90)
	*** 1st Sanity check: Ratio 0.83 <= 1.00
	*** 2nd Sanity check: #OpHits 12 >= 12

# of OpHits (ophitspe): 72088 for times between -2261.44 us and 2227.38 us.
# of OpFlashes (solarflash): 5152 for times between -2261.02 us and 2227.38 us.
[0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Nov-2025 00:29:01 UTC run: 5089 subRun: 0 event: 115400
[36m
# OpHits (ophitspe) in full geometry: 72088
# OpFlashes (solarflash) in full geometry: 5152
# Hits (hitfd) in each view: 178, 173, 448, 0
# Cluster from the hits: 109, 105, 410, 0
# Tracks (pmtracktc) in full geometry: 2[0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Nov-2025 00:29:14 UTC run: 5089 subRun: 0 event: 115400
[32m
# ClusterReco: Looping over 232 matched clusters[0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Nov-2025 00:29:14 UTC run: 5089 subRun: 0 event: 115400
[32m*** Matched preselection cluster: 17 from 232
 - MainTrackID 32920
 - Gen 1 -> marley TPC 3
 - Purity 1.00 Hits 4
 - Charge 889.87 ( MaxHit 623.60 )
 - #AdjCl 2 ( 1 signal ):
    - Cluster 27 at distance 49.43 with time 298.49 and charge 153.48 in TPC 3 and hits 2
    - Cluster 33 at distance 21.78 with time 369.33 and charge 648.49 in TPC 3 and hits 4
 - RecoCol  Time,Y,Z ( 242.25, 583.09, 20.21 )
 - RecoInd0 Time,Y,Z ( 241.90, 583.14, 16.17 )
 - RecoInd1 Time,Y,Z ( 242.51, 583.03, 119.16 )
*** Matched flash: 2619 from 5152
 - Purity 90.90 % Plane 0 #Hits 22 PE 195.15 MainOpHitPE 74.35
 - Time -18.03 (us) Fast 72.79 % Residual 1142.74
 - Reco Time,Y,Z ( -18.03, 549.18, 35.12 )
 - Found valid waveform: false
*** Reconstructed Interaction Vertex: 
 - True X,Y,Z ( 40.91, 583.60, 20.09 )
 - Main X,Y,Z ( 36.64, 587.13, 25.52 )
 - EDep X,Y,Z ( 40.89, 582.82, 19.85 )
 - Reco X,Y,Z ( 41.79, 583.09, 20.21 )[0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Nov-2025 00:29:14 UTC run: 5089 subRun: 0 event: 115400
[32m-----------------------------------------------------------------------------------------
[0m
%MSG
26-Nov-2025 00:29:14 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/2a/45/prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T100738Z_gen_002308_supernova_g4_detsim_reco_20250724T084544Z_reco2.root"
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@EndJob 26-Nov-2025 00:29:14 UTC  ModuleEndJob
[35mFinished running the SolarNuAna module[0m
%MSG

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.126188       8.16635       21.955        7.05932       8.03203        50     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000876697   0.00496191     0.0401405    0.00163894    0.00856658       50     
end_path:solarnuana:SolarNuAna      0.12432       8.16126       21.9535       7.05298       8.03365        50     
====================================================================================================================
%MSG-i NuRandomService:  SolarNuAna:solarnuana@EndJob 26-Nov-2025 00:29:14 UTC  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1

%MSG

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1549.08 MB
  Peak resident set size usage (VmHWM): 902.951 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 50 passed = 50 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport         50         50          0 solarnuana

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 420.315733 Real = 432.990449

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 1549.08 VmHWM = 902.951

Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===