justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 255450.0@dunegpschedd01.fnal.gov

Jobsub ID255450.0@dunegpschedd01.fnal.gov
Workflow ID10516
Stage ID1
User namesmanthey@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-11-22 19:55:18
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-11-22 20:42:41
From worker nodeHostnamedunegli-7693576-0-fnpc18006.fnal.gov
cpuinfoAMD EPYC 7551 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-22 20:14:48
Input filesfardet-hd:fardet-hd_mc_le_mc_2024a_list-4367_input.list_out1_reco1_supernova_dune10kt_1x2x6_hit-reconstructed_merged_skip000125_lim000025_mergeLar_20241218T154015_20250723T161022Z_reco2.root
JobscriptExit code0
Real time2m (146s)
CPU time1m (117s = 80%)
Max RSS bytes639488000 (609 MiB)
Outputting started2025-11-22 20:17:15
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/smanthey/ana/solar_ana_marley_flash_dune10kt_1x2x6/fnal/10516/1/001/fdhd_ana_2025-11-22T_201501Z_hist.root
Finished2025-11-22 20:42:41
Saved logsjustin-logs:255450.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

: 0 event: 15000
Invalid OpWaveform handle
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  22-Nov-2025 20:17:14 UTC run: 4367 subRun: 0 event: 15000
Invalid OpWaveform handle
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  22-Nov-2025 20:17:14 UTC run: 4367 subRun: 0 event: 15000
Invalid OpWaveform handle
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  22-Nov-2025 20:17:14 UTC run: 4367 subRun: 0 event: 15000
Invalid OpWaveform handle
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  22-Nov-2025 20:17:14 UTC run: 4367 subRun: 0 event: 15000
Invalid OpWaveform handle
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  22-Nov-2025 20:17:14 UTC run: 4367 subRun: 0 event: 15000
Invalid OpWaveform handle
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  22-Nov-2025 20:17:14 UTC run: 4367 subRun: 0 event: 15000
Invalid OpWaveform handle
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  22-Nov-2025 20:17:14 UTC run: 4367 subRun: 0 event: 15000
Invalid OpWaveform handle
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  22-Nov-2025 20:17:14 UTC run: 4367 subRun: 0 event: 15000
Invalid OpWaveform handle
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  22-Nov-2025 20:17:14 UTC run: 4367 subRun: 0 event: 15000
Invalid OpWaveform handle
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  22-Nov-2025 20:17:14 UTC run: 4367 subRun: 0 event: 15000
Invalid OpWaveform handle
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  22-Nov-2025 20:17:14 UTC run: 4367 subRun: 0 event: 15000
Invalid OpWaveform handle
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  22-Nov-2025 20:17:14 UTC run: 4367 subRun: 0 event: 15000
Invalid OpWaveform handle
%MSG
%MSG-e ProducerUtils:  SolarNuAna:solarnuana@BeginModule  22-Nov-2025 20:17:14 UTC run: 4367 subRun: 0 event: 15000
Invalid OpWaveform handle
%MSG
%MSG-w PhotonBackTracker:  SolarNuAna:solarnuana@BeginModule  22-Nov-2025 20:17:14 UTC run: 4367 subRun: 0 event: 15000
Exception caught
---- PhotonBackTracker2 BEGIN
  No sim:: OpDetBacktrackerRecord corresponding to opDetNum: 264
---- PhotonBackTracker2 END

%MSG
%MSG-w PhotonBackTracker:  SolarNuAna:solarnuana@BeginModule  22-Nov-2025 20:17:14 UTC run: 4367 subRun: 0 event: 15000
Exception caught
---- PhotonBackTracker2 BEGIN
  No sim:: OpDetBacktrackerRecord corresponding to opDetNum: 307
---- PhotonBackTracker2 END

%MSG
%MSG-w PhotonBackTracker:  SolarNuAna:solarnuana@BeginModule  22-Nov-2025 20:17:14 UTC run: 4367 subRun: 0 event: 15000
Exception caught
---- PhotonBackTracker2 BEGIN
  No sim:: OpDetBacktrackerRecord corresponding to opDetNum: 152
---- PhotonBackTracker2 END

%MSG
%MSG-w PhotonBackTracker:  SolarNuAna:solarnuana@BeginModule  22-Nov-2025 20:17:14 UTC run: 4367 subRun: 0 event: 15000
Exception caught
---- PhotonBackTracker2 BEGIN
  No sim:: OpDetBacktrackerRecord corresponding to opDetNum: 54
---- PhotonBackTracker2 END

%MSG
%MSG-w PhotonBackTracker:  SolarNuAna:solarnuana@BeginModule  22-Nov-2025 20:17:14 UTC run: 4367 subRun: 0 event: 15000
Exception caught
---- PhotonBackTracker2 BEGIN
  No sim:: OpDetBacktrackerRecord corresponding to opDetNum: 293
---- PhotonBackTracker2 END

%MSG
%MSG-w PhotonBackTracker:  SolarNuAna:solarnuana@BeginModule  22-Nov-2025 20:17:14 UTC run: 4367 subRun: 0 event: 15000
Exception caught
---- PhotonBackTracker2 BEGIN
  No sim:: OpDetBacktrackerRecord corresponding to opDetNum: 18
---- PhotonBackTracker2 END

%MSG
%MSG-w PhotonBackTracker:  SolarNuAna:solarnuana@BeginModule  22-Nov-2025 20:17:14 UTC run: 4367 subRun: 0 event: 15000
Exception caught
---- PhotonBackTracker2 BEGIN
  No sim:: OpDetBacktrackerRecord corresponding to opDetNum: 118
---- PhotonBackTracker2 END

%MSG
%MSG-w PhotonBackTracker:  SolarNuAna:solarnuana@BeginModule  22-Nov-2025 20:17:14 UTC run: 4367 subRun: 0 event: 15000
Exception caught
---- PhotonBackTracker2 BEGIN
  No sim:: OpDetBacktrackerRecord corresponding to opDetNum: 330
---- PhotonBackTracker2 END

%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  22-Nov-2025 20:17:14 UTC run: 4367 subRun: 0 event: 15000
OpFlash PE 491.52 with purity 1.00 time -17.93 plane 0
 - Vertex (-0.05, -194.73, 326.92)
	*** 1st Sanity check: Ratio 0.05 <= 1.00
	*** 2nd Sanity check: #OpHits 128 >= 128
OpFlash PE 181.30 with purity 0.99 time -16.49 plane 0
 - Vertex (-0.04, -150.85, 293.71)
	*** 1st Sanity check: Ratio 0.05 <= 1.00
	*** 2nd Sanity check: #OpHits 86 >= 86
OpFlash PE 77.60 with purity 0.97 time -15.29 plane 0
 - Vertex (-0.05, -194.81, 281.35)
	*** 1st Sanity check: Ratio 0.06 <= 1.00
	*** 2nd Sanity check: #OpHits 52 >= 52
OpFlash PE 54.41 with purity 0.94 time -13.85 plane 0
 - Vertex (-0.04, -177.29, 259.12)
	*** 1st Sanity check: Ratio 0.06 <= 1.00
	*** 2nd Sanity check: #OpHits 47 >= 47
OpFlash PE 31.52 with purity 0.94 time -12.88 plane 0
 - Vertex (-0.03, -154.20, 213.97)
	*** 1st Sanity check: Ratio 0.10 <= 1.00
	*** 2nd Sanity check: #OpHits 27 >= 27
OpFlash PE 12.57 with purity 0.79 time -12.33 plane 0
 - Vertex (-0.01, -70.20, 458.45)
	*** 1st Sanity check: Ratio 0.17 <= 1.00
	*** 2nd Sanity check: #OpHits 11 >= 11
OpFlash PE 5.24 with purity 0.88 time -9.97 plane 0
 - Vertex (-0.00, -97.88, 295.39)
	*** 1st Sanity check: Ratio 0.40 <= 1.00
	*** 2nd Sanity check: #OpHits 4 >= 4

# of OpHits (ophitspe): 408 for times between -2212.75 us and 1855.06 us.
# of OpFlashes (solarflash): 34 for times between -2212.75 us and 1855.06 us.

%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  22-Nov-2025 20:17:14 UTC run: 4367 subRun: 0 event: 15000

# OpHits (ophitspe) in full geometry: 408
# OpFlashes (solarflash) in full geometry: 34
# Hits (hitfd) in each view: 8, 7, 5, 0
# Cluster from the hits: 3, 3, 3, 0
# Tracks (pmtracktc) in full geometry: 0
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  22-Nov-2025 20:17:14 UTC run: 4367 subRun: 0 event: 15000

# ClusterReco: Looping over 3 matched clusters
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  22-Nov-2025 20:17:14 UTC run: 4367 subRun: 0 event: 15000
*** Matched preselection cluster: 2 from 3
 - MainTrackID 1
 - Gen 1 -> marley TPC 4
 - Purity 1.00 Hits 2
 - Charge 668.42 ( MaxHit 543.62 )
 - #AdjCl 2 ( 2 signal ):
    - Cluster 0 at distance 13.05 with time 780.70 and charge 145.66 in TPC 4 and hits 2
    - Cluster 1 at distance 7.19 with time 816.24 and charge 138.92 in TPC 4 and hits 1
 - RecoCol  Time,Y,Z ( 826.62, -183.34, 278.88 )
 - RecoInd0 Time,Y,Z ( 826.54, -183.41, 347.71 )
 - RecoInd1 Time,Y,Z ( 826.57, -183.28, 321.20 )
*** Matched flash: 12 from 34
 - Purity 99.71 % Plane 0 #Hits 128 PE 491.52 MainOpHitPE 26.24
 - Time -17.93 (us) Fast 44.80 % Residual 33.85
 - Reco Time,Y,Z ( -17.93, -194.73, 326.92 )
 - Found valid waveform: false
*** Reconstructed Interaction Vertex: 
 - True X,Y,Z ( -134.93, -183.27, 279.11 )
 - Main X,Y,Z ( -136.44, -183.21, 278.88 )
 - EDep X,Y,Z ( -135.69, -183.39, 278.83 )
 - Reco X,Y,Z ( -135.60, -183.34, 278.88 )
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  22-Nov-2025 20:17:14 UTC run: 4367 subRun: 0 event: 15000
-----------------------------------------------------------------------------------------

%MSG
22-Nov-2025 20:17:14 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/bc/56/fardet-hd_mc_le_mc_2024a_list-4367_input.list_out1_reco1_supernova_dune10kt_1x2x6_hit-reconstructed_merged_skip000125_lim000025_mergeLar_20241218T154015_20250723T161022Z_reco2.root"
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@EndJob 22-Nov-2025 20:17:14 UTC  ModuleEndJob
Finished running the SolarNuAna module
%MSG

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                        0.00388974     0.0210062     0.292548      0.0180625     0.017057       2500    
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.00082132    0.00178181     0.198894      0.0014803     0.004325       2500    
end_path:solarnuana:SolarNuAna    0.00257359     0.0190373     0.290544      0.0162988     0.0164785      2500    
====================================================================================================================
%MSG-i NuRandomService:  SolarNuAna:solarnuana@EndJob 22-Nov-2025 20:17:14 UTC  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1

%MSG

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1328.69 MB
  Peak resident set size usage (VmHWM): 639.488 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 2500 passed = 2500 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport       2500       2500          0 solarnuana

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 79.617157 Real = 102.399357

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 1328.69 VmHWM = 639.488

Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-08 21:41:24 UTC       justIN version: 01.06.00