justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 273642.0@dunegpschedd01.fnal.gov

Jobsub ID273642.0@dunegpschedd01.fnal.gov
Workflow ID11681
Stage ID1
User namesmanthey@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-01-10 18:19:49
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2026-01-10 18:41:06
From worker nodeHostnamedunegli-8297197-0-fnpc18010.fnal.gov
cpuinfoAMD EPYC 7551 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-10 18:20:13
Input filesfardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250205T171500Z_gen_000710_supernova_g4_detsim_reco_20250724T083615Z_reco2.root
JobscriptExit code0
Real time20m (1230s)
CPU time20m (1207s = 98%)
Max RSS bytes899268608 (857 MiB)
Outputting started2026-01-10 18:40:43
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/smanthey/ana/solar_ana_marley_flash_radiological_decay0_dune10kt_1x2x6_centralAPA/fnal/11681/1/001/fdhd_ana_2026-01-10T_182027Z_hist.root
Finished2026-01-10 18:41:06
Saved logsjustin-logs:273642.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

0.00	(-220.25, -159.36, -304.71)	32792	32759
marley	2212		0.00	(-219.06, -158.20, -303.71)	32790	32759
marley	2212		0.00	(-219.74, -153.06, -303.04)	32789	32759
marley	1000080160	0.00	(-217.82, -141.51, -309.56)	32788	32759
marley	2212		0.00	(-215.72, -139.62, -309.51)	32787	32759
marley	1000080160	0.00	(-214.50, -137.83, -309.83)	32786	32759
marley	2212		0.00	(-212.56, -141.56, -309.15)	32785	32759
marley	2212		0.00	(-209.42, -142.71, -311.52)	32784	32759
marley	2212		0.00	(-197.78, -151.77, -315.60)	32783	32759
marley	2212		0.04	(-180.86, -171.57, -312.80)	32782	32759
marley	2212		0.02	(-187.48, -181.31, -303.18)	32781	32759
marley	1000180400	0.00	(-181.65, -314.09, -94.62)	32780	32759
marley	1000180400	0.00	(-140.77, 115.08, 164.00)	32779	32759
marley	1000180400	0.00	(-93.40, 87.00, 163.04)	32778	32759
marley	1000180400	0.00	(-99.78, 79.89, 163.07)	32777	32759
marley	1000180400	0.00	(-99.61, 78.61, 164.01)	32776	32759
marley	1000180400	0.01	(-102.72, 73.54, 165.59)	32775	32759
marley	1000180400	0.01	(-99.48, 77.58, 165.14)	32774	32759
marley	1000180400	0.01	(-105.81, 80.74, 169.47)	32773	32759
marley	1000180400	0.00	(-77.56, 96.50, 168.35)	32772	32759
marley	1000180400	0.01	(-76.90, 105.97, 167.67)	32771	32759
marley	1000180400	0.00	(-71.69, 105.38, 169.62)	32770	32759
marley	1000180400	0.00	(-57.36, 121.00, 162.43)	32769	32759
marley	1000180400	0.01	(-57.54, 121.26, 161.85)	32768	32759
marley	1000180400	0.00	(-61.11, 106.91, 167.76)	32767	32759
marley	1000180400	0.00	(-70.20, 86.97, 172.31)	32766	32759
marley	1000180400	0.01	(-71.94, 76.09, 193.67)	32765	32759
marley	1000180400	0.00	(25.65, 154.90, 224.60)	32764	32759
marley	1000180400	0.00	(26.27, 185.89, 253.48)	32763	32759
marley	1000180400	0.00	(21.82, 188.58, 259.45)	32762	32759
marley	1000180400	0.01	(121.76, 45.94, 509.36)	32761	32759
marley	1000180360	0.01	(75.13, -75.35, 500.10)	32760	32759
marley	2112		0.15	(89.21, -81.39, 502.65)	32759	0
marley	1000190390	0.02	(89.21, -81.39, 502.65)	32758	0
marley	11		18.26	(89.21, -81.39, 502.65)	32757	0
Signal Track IDs: 32757; 32758; 32759; 32760; 32761; 32762; 32763; 32764; 32765; 32766; 32767; 32768; 32769; 32770; 32771; 32772; 32773; 32774; 32775; 32776; 32777; 32778; 32779; 32780; 32781; 32782; 32783; 32784; 32785; 32786; 32787; 32788; 32789; 32790; 32792; 32823; 32824; 
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  10-Jan-2026 18:40:05 UTC run: 5064 subRun: 0 event: 35500

Kinetic energy of signal particle is below threshold of 30.00 MeV. Selecting event.

%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  10-Jan-2026 18:40:14 UTC run: 5064 subRun: 0 event: 35500
OpFlash PE 87.96 with purity 0.31 time -31.95 plane 0
 - Vertex (-0.05, -217.95, 267.51)
	*** 1st Sanity check: Ratio 0.94 <= 1.00
	*** 2nd Sanity check: #OpHits 4 >= 4
OpFlash PE 24.35 with purity 0.20 time -30.77 plane 0
 - Vertex (-0.05, -404.76, 1076.28)
	*** 1st Sanity check: Ratio 1.00 <= 1.00
	*** 2nd Sanity check: #OpHits 1 >= 1
OpFlash PE 58.95 with purity 0.48 time -28.61 plane 0
 - Vertex (-0.05, -342.49, 427.91)
	*** 1st Sanity check: Ratio 0.96 <= 1.00
	*** 2nd Sanity check: #OpHits 2 >= 2
OpFlash PE 16.24 with purity 0.07 time -18.56 plane 0
 - Vertex (-0.04, -398.35, 239.00)
	*** 1st Sanity check: Ratio 0.24 <= 1.00
	*** 2nd Sanity check: #OpHits 7 >= 7
OpFlash PE 453.06 with purity 0.38 time -17.68 plane 0
 - Vertex (-0.05, -31.14, 548.70)
	*** 1st Sanity check: Ratio 0.82 <= 1.00
	*** 2nd Sanity check: #OpHits 21 >= 21
OpFlash PE 1.94 with purity 1.00 time -17.30 plane 0
 - Vertex (-0.05, -342.49, 1357.47)
	*** 1st Sanity check: Ratio 1.00 <= 1.00
	*** 2nd Sanity check: #OpHits 1 >= 1
OpFlash PE 97.33 with purity 0.81 time -17.19 plane 0
 - Vertex (-0.03, -104.08, 541.89)
	*** 1st Sanity check: Ratio 0.26 <= 1.00
	*** 2nd Sanity check: #OpHits 17 >= 17
OpFlash PE 57.41 with purity 0.71 time -17.14 plane 0
 - Vertex (0.05, 367.25, 532.67)
	*** 1st Sanity check: Ratio 0.39 <= 1.00
	*** 2nd Sanity check: #OpHits 10 >= 10
OpFlash PE 24.13 with purity 0.96 time -16.53 plane 0
 - Vertex (-0.05, -65.09, 452.42)
	*** 1st Sanity check: Ratio 0.38 <= 1.00
	*** 2nd Sanity check: #OpHits 8 >= 8
OpFlash PE 28.24 with purity 0.91 time -16.01 plane 0
 - Vertex (-0.02, -47.88, 566.56)
	*** 1st Sanity check: Ratio 0.18 <= 1.00
	*** 2nd Sanity check: #OpHits 12 >= 12
OpFlash PE 32.24 with purity 0.80 time -15.16 plane 0
 - Vertex (-0.03, -52.94, 518.88)
	*** 1st Sanity check: Ratio 0.37 <= 1.00
	*** 2nd Sanity check: #OpHits 7 >= 7
OpFlash PE 108.99 with purity 0.40 time -14.35 plane 0
 - Vertex (0.05, 31.14, 660.30)
	*** 1st Sanity check: Ratio 0.78 <= 1.00
	*** 2nd Sanity check: #OpHits 9 >= 9
OpFlash PE 5.00 with purity 0.17 time -13.98 plane 0
 - Vertex (0.05, 73.98, 487.39)
	*** 1st Sanity check: Ratio 0.34 <= 1.00
	*** 2nd Sanity check: #OpHits 3 >= 3
OpFlash PE 3.65 with purity 1.00 time -13.54 plane 0
 - Vertex (-0.01, 91.53, 329.88)
	*** 1st Sanity check: Ratio 0.56 <= 1.00
	*** 2nd Sanity check: #OpHits 2 >= 2
OpFlash PE 7.59 with purity 0.79 time -12.90 plane 0
 - Vertex (-0.01, -84.87, 566.91)
	*** 1st Sanity check: Ratio 0.41 <= 1.00
	*** 2nd Sanity check: #OpHits 3 >= 3
OpFlash PE 1.94 with purity 1.00 time -10.67 plane 0
 - Vertex (0.05, 217.95, 379.11)
	*** 1st Sanity check: Ratio 1.00 <= 1.00
	*** 2nd Sanity check: #OpHits 1 >= 1
OpFlash PE 5.41 with purity 0.65 time -9.27 plane 0
 - Vertex (-0.05, -93.41, 568.40)
	*** 1st Sanity check: Ratio 0.35 <= 1.00
	*** 2nd Sanity check: #OpHits 3 >= 3
OpFlash PE 9.29 with purity 0.41 time -8.31 plane 0
 - Vertex (-0.05, -155.68, 499.90)
	*** 1st Sanity check: Ratio 0.81 <= 1.00
	*** 2nd Sanity check: #OpHits 2 >= 2

# of OpHits (ophitspe): 69930 for times between -2262.56 us and 2227.71 us.
# of OpFlashes (solarflash): 9605 for times between -2262.56 us and 2227.55 us.

%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  10-Jan-2026 18:40:14 UTC run: 5064 subRun: 0 event: 35500

# OpHits (ophitspe) in full geometry: 69930
# OpFlashes (solarflash) in full geometry: 9605
# Hits (hitfd) in each view: 207, 209, 473, 0
# Cluster from the hits: 127, 129, 424, 0
# Tracks (pmtracktc) in full geometry: 2
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  10-Jan-2026 18:40:42 UTC run: 5064 subRun: 0 event: 35500

# ClusterReco: Looping over 268 matched clusters
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  10-Jan-2026 18:40:42 UTC run: 5064 subRun: 0 event: 35500
*** Matched preselection cluster: 68 from 268
 - MainTrackID 32757
 - Gen 1 -> marley TPC 9
 - Purity 1.00 Hits 8
 - Charge 2447.06 ( MaxHit 709.60 )
 - #AdjCl 0 ( 0 signal ):
 - RecoCol  Time,Y,Z ( 548.01, -81.42, 503.13 )
 - RecoInd0 Time,Y,Z ( 548.62, -81.32, 512.82 )
 - RecoInd1 Time,Y,Z ( 547.85, -81.53, 580.10 )
*** Matched flash: 4813 from 9605
 - Purity 38.12 % Plane 0 #Hits 21
 - MainOpHitPE 373.49 (PE);  TotalPE 453.06 vs expected 779.67 (PE)
 - Time -17.68 (us) Fast 82.66 % Residual 6840.78
 - Reco Time,Y,Z ( -17.68, -31.14, 548.70 )
 - Found valid waveform: false
*** Reconstructed Interaction Vertex: 
 - True X,Y,Z ( 89.21, -81.39, 502.65 )
 - Main X,Y,Z ( 91.76, -82.12, 502.11 )
 - EDep X,Y,Z ( 90.09, -81.00, 502.98 )
 - Reco X,Y,Z ( 90.83, -81.42, 503.13 )
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  10-Jan-2026 18:40:42 UTC run: 5064 subRun: 0 event: 35500
-----------------------------------------------------------------------------------------

%MSG
10-Jan-2026 18:40:43 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/6a/e2/prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250205T171500Z_gen_000710_supernova_g4_detsim_reco_20250724T083615Z_reco2.root"
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@EndJob 10-Jan-2026 18:40:43 UTC  ModuleEndJob
Finished running the SolarNuAna module
%MSG

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.402412       22.4481       71.3283      0.969512       28.2068        50     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.00173766    0.00634682     0.0423224    0.00283756    0.00784141       50     
end_path:solarnuana:SolarNuAna     0.399855       22.4415       71.3248      0.961933       28.2076        50     
====================================================================================================================
%MSG-i NuRandomService:  SolarNuAna:solarnuana@EndJob 10-Jan-2026 18:40:43 UTC  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1

%MSG

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1551.46 MB
  Peak resident set size usage (VmHWM): 905.363 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 50 passed = 50 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport         50         50          0 solarnuana

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 1167.342449 Real = 1185.816695

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 1551.46 VmHWM = 905.363

Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
justIN time: 2026-02-06 13:00:44 UTC       justIN version: 01.06.00