justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 264308.0@dunegpschedd01.fnal.gov

Jobsub ID264308.0@dunegpschedd01.fnal.gov
Workflow ID10952
Stage ID1
User namesmanthey@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-06 09:21:49
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-12-06 23:29:09
From worker nodeHostnamedunegli-7832046-0-fnpc18007.fnal.gov
cpuinfoAMD EPYC 7551 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-06 09:22:36
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250129T082441Z_gen_001928_supernova_g4stage1_g4stage2_detsim_reco_20250729T042816Z_reco2.root
JobscriptExit code0
Real time14h (50776s)
CPU time13h (49665s = 97%)
Max RSS bytes2557186048 (2438 MiB)
Outputting started2025-12-06 23:28:52
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/smanthey/ana/solar_ana_marley_flash_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_nominal/fnal/10952/1/001/fdhd_ana_2025-12-06T_092300Z_hist.root
Finished2025-12-06 23:29:09
Saved logsjustin-logs:264308.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

y	1000180400	0.01	(-12.50, 549.62, 1750.34)	133012	132932
marley	1000180400	0.02	(37.66, 562.06, 1717.32)	133011	132932
marley	1000180400	0.04	(46.51, 529.42, 1714.19)	133010	132932
marley	1000180400	0.03	(47.09, 549.40, 1717.63)	133009	132932
marley	1000180400	0.02	(48.89, 524.97, 1773.77)	133008	132932
marley	1000180400	0.03	(49.11, 525.87, 1774.29)	133007	132932
marley	1000180400	0.00	(51.77, 523.96, 1775.03)	133006	132932
marley	1000180400	0.04	(106.73, 522.81, 1788.62)	133005	132932
marley	1000180400	0.06	(2.09, 429.87, 1929.52)	133004	132932
marley	1000180400	0.00	(37.65, 463.74, 1899.07)	133003	132932
marley	1000180400	0.01	(43.57, 475.15, 1882.69)	133002	132932
marley	1000180400	0.03	(98.40, 493.66, 1850.92)	133001	132932
marley	1000180400	0.01	(95.42, 516.75, 1852.41)	133000	132932
marley	1000180400	0.02	(97.92, 523.05, 1850.51)	132999	132932
marley	1000180400	0.06	(132.01, 528.62, 1856.60)	132998	132932
marley	1000180400	0.03	(101.26, 536.90, 1816.00)	132997	132932
marley	1000180400	0.03	(102.40, 541.99, 1815.22)	132996	132932
marley	1000180400	0.00	(37.64, 559.19, 1754.92)	132995	132932
marley	1000180400	0.08	(-9.89, 565.48, 1709.38)	132994	132932
marley	1000180400	0.00	(-2.57, 566.63, 1715.59)	132993	132932
marley	1000180400	0.00	(12.48, 564.57, 1730.33)	132992	132932
marley	1000180400	0.05	(29.05, 561.02, 1738.84)	132991	132932
marley	1000180400	0.06	(14.97, 558.54, 1759.61)	132990	132932
marley	1000180400	0.10	(15.75, 558.21, 1759.94)	132989	132932
marley	1000180400	0.03	(8.25, 559.50, 1759.10)	132988	132932
marley	22		7.26	(6.00, 556.96, 1759.85)	132933	0
marley	2112		1.14	(6.00, 556.96, 1759.85)	132932	0
marley	1000190390	0.05	(6.00, 556.96, 1759.85)	132931	0
marley	11		11.28	(6.00, 556.96, 1759.85)	132930	0
Signal Track IDs: 132930; 132931; 132932; 132933; 132988; 132989; 132990; 132991; 132992; 132993; 132994; 132995; 132996; 132997; 132998; 132999; 133000; 133001; 133002; 133003; 133004; 133005; 133006; 133007; 133008; 133009; 133010; 133011; 133012; 133013; 133014; 133015; 133016; 133017; 133018; 133019; 133020; 133021; 133022; 133023; 133024; 133025; 133026; 133027; 133028; 133029; 133030; 133031; 133032; 133033; 133034; 133035; 133036; 133037; 133038; 133039; 133040; 133041; 133042; 133043; 133044; 133045; 133046; 133047; 133048; 133049; 133050; 133051; 133052; 133053; 133054; 133056; 133119; 133120; 133122; 133123; 
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  06-Dec-2025 21:57:50 UTC run: 4980 subRun: 0 event: 19280

Kinetic energy of signal particle is below threshold of 30.00 MeV. Selecting event.

%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  06-Dec-2025 21:59:38 UTC run: 4980 subRun: 0 event: 19280
OpFlash PE 513.56 with purity 0.48 time 0.27 plane 0
 - Vertex (-327.50, 540.40, 1771.86)
	*** 1st Sanity check: Ratio 0.10 <= 1.00
	*** 2nd Sanity check: #OpHits 29 >= 29
OpFlash PE 438.79 with purity 0.47 time 0.60 plane 1
 - Vertex (76.96, 743.30, 1721.09)
	*** 1st Sanity check: Ratio 0.17 <= 1.00
	*** 2nd Sanity check: #OpHits 24 >= 24
OpFlash PE 53.52 with purity 0.20 time 1.67 plane 3
 - Vertex (174.98, -95.99, 2188.38)
	*** 1st Sanity check: Ratio 0.20 <= 1.00
	*** 2nd Sanity check: #OpHits 7 >= 7
OpFlash PE 36.36 with purity 0.15 time 4.00 plane 0
 - Vertex (-327.50, 474.44, 1828.04)
	*** 1st Sanity check: Ratio 0.25 <= 1.00
	*** 2nd Sanity check: #OpHits 5 >= 5
OpFlash PE 55.32 with purity 0.07 time 7.90 plane 3
 - Vertex (210.07, 220.00, 2188.38)
	*** 1st Sanity check: Ratio 0.40 <= 1.00
	*** 2nd Sanity check: #OpHits 4 >= 4
OpFlash PE 32.19 with purity 0.12 time 11.12 plane 3
 - Vertex (83.97, 220.00, 2188.38)
	*** 1st Sanity check: Ratio 0.60 <= 1.00
	*** 2nd Sanity check: #OpHits 3 >= 3

# of OpHits (ophit10ppm): 52062 for times between -4253.66 us and 295147919198126080.00 us.
# of OpFlashes (solarflash): 12191 for times between -4253.66 us and 295147919198126080.00 us.

%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  06-Dec-2025 21:59:42 UTC run: 4980 subRun: 0 event: 19280

# OpHits (ophit10ppm) in full geometry: 52062
# OpFlashes (solarflash) in full geometry: 12191
# Hits (gaushit) in each view: 6454, 6443, 6454, 0
# Cluster from the hits: 2720, 2735, 5492, 0
# Tracks (pmtracktc) in full geometry: 457
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  06-Dec-2025 23:28:51 UTC run: 4980 subRun: 0 event: 19280

# ClusterReco: Looping over 5482 matched clusters
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  06-Dec-2025 23:28:51 UTC run: 4980 subRun: 0 event: 19280
*** Matched preselection cluster: 2631 from 5482
 - MainTrackID 132930
 - Gen 1 -> marley TPC 95
 - Purity 1.00 Hits 7
 - Charge 1305.87 ( MaxHit 299.42 )
 - #AdjCl 4 ( 1 signal ):
    - Cluster 2174 at distance 83.51 with time 1677.03 and charge 315.30 in TPC 103 and hits 2
    - Cluster 2685 at distance 10.39 with time 2035.60 and charge 119.53 in TPC 95 and hits 2
    - Cluster 2755 at distance 60.70 with time 2085.91 and charge 64.04 in TPC 95 and hits 1
    - Cluster 3159 at distance 85.56 with time 2415.23 and charge 108.73 in TPC 95 and hits 1
 - RecoCol  Time,Y,Z ( 1991.77, 557.80, 1757.98 )
 - RecoInd0 Time,Y,Z ( 1996.90, 558.17, 1718.08 )
 - RecoInd1 Time,Y,Z ( 1991.70, 557.42, 1729.46 )
*** Matched flash: 6104 from 12191
 - Purity 48.38 % Plane 0 #Hits 29 PE 513.56 MainOpHitPE 53.02
 - Time 0.27 (us) Fast 25.75 % Residual 245.51
 - Reco Time,Y,Z ( 0.27, 540.40, 1771.86 )
 - Found valid waveform: true
*** Reconstructed Interaction Vertex: 
 - True X,Y,Z ( 6.00, 556.96, 1759.85 )
 - Main X,Y,Z ( 4.57, 556.83, 1756.77 )
 - EDep X,Y,Z ( 4.92, 557.31, 1757.11 )
 - Reco X,Y,Z ( 5.27, 557.80, 1757.98 )
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  06-Dec-2025 23:28:51 UTC run: 4980 subRun: 0 event: 19280
*** Matched preselection cluster: 2650 from 5482
 - MainTrackID 132933
 - Gen 1 -> marley TPC 95
 - Purity 1.00 Hits 3
 - Charge 538.73 ( MaxHit 248.08 )
 - #AdjCl 7 ( 0 signal ):
    - Cluster 2200 at distance 93.99 with time 1699.18 and charge 41.87 in TPC 101 and hits 1
    - Cluster 2229 at distance 96.17 with time 1719.72 and charge 55.43 in TPC 101 and hits 1
    - Cluster 2247 at distance 94.17 with time 1735.39 and charge 29.37 in TPC 101 and hits 1
    - Cluster 2538 at distance 86.26 with time 1928.06 and charge 30.83 in TPC 95 and hits 1
    - Cluster 2741 at distance 89.52 with time 2077.25 and charge 137.25 in TPC 93 and hits 1
    - Cluster 2954 at distance 72.12 with time 2227.44 and charge 261.07 in TPC 93 and hits 2
    - Cluster 3105 at distance 99.61 with time 2373.24 and charge 29.51 in TPC 91 and hits 1
 - RecoCol  Time,Y,Z ( 2008.58, 363.91, 1759.71 )
 - RecoInd0 Time,Y,Z ( 2010.49, 168.94, 1868.00 )
 - RecoInd1 Time,Y,Z ( 2008.65, 558.88, 1729.06 )
*** Matched flash: 7983 from 12191
 - Purity 0.00 % Plane 0 #Hits 6 PE 374.34 MainOpHitPE 80.10
 - Time 1300.98 (us) Fast 60.38 % Residual 1193.19
 - Reco Time,Y,Z ( 1300.98, 324.45, 1802.54 )
 - Found valid waveform: true
*** Reconstructed Interaction Vertex: 
 - True X,Y,Z ( 6.00, 556.96, 1759.85 )
 - Main X,Y,Z ( -31.03, 531.01, 1771.72 )
 - EDep X,Y,Z ( 2.25, 559.21, 1759.44 )
 - Reco X,Y,Z ( 211.42, 363.91, 1759.71 )
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  06-Dec-2025 23:28:51 UTC run: 4980 subRun: 0 event: 19280
-----------------------------------------------------------------------------------------

%MSG
06-Dec-2025 23:28:51 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/75/ed/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250129T082441Z_gen_001928_supernova_g4stage1_g4stage2_detsim_reco_20250729T042816Z_reco2.root"
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@EndJob 06-Dec-2025 23:28:51 UTC  ModuleEndJob
Finished running the SolarNuAna module
%MSG

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                          1.8693        5039.92       9819.72       5503.58       2999.38        10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.00207734      14.1435       74.1775     0.00903403      28.3119        10     
end_path:solarnuana:SolarNuAna      1.6173        5025.77       9819.69       5466.49       3009.39        10     
====================================================================================================================
%MSG-i NuRandomService:  SolarNuAna:solarnuana@EndJob 06-Dec-2025 23:28:51 UTC  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1

%MSG

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3173.29 MB
  Peak resident set size usage (VmHWM): 2557.19 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 10 passed = 10 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport         10         10          0 solarnuana

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 49624.505648 Real = 50719.877571

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 3173.29 VmHWM = 2557.19

Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
justIN time: 2026-02-07 23:44:47 UTC       justIN version: 01.06.00