justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 300626.4@dunegpschedd01.fnal.gov

Jobsub ID300626.4@dunegpschedd01.fnal.gov
Workflow ID12372
Stage ID1
User namesmanthey@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-01-25 12:56:50
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2026-01-26 02:26:26
From worker nodeHostnamedunegli-8270269-0-fnpc18001.fnal.gov
cpuinfoAMD EPYC 7551 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-25 12:57:45
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250130T200659Z_gen_003113_supernova_g4stage1_g4stage2_detsim_reco_20250624T154437Z_reco2.root
JobscriptExit code0
Real time13h (48494s)
CPU time13h (47635s = 98%)
Max RSS bytes2580721664 (2461 MiB)
Outputting started2026-01-26 02:25:59
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/smanthey/ana/solar_ana_marley_flash_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_nominal/fnal/12372/1/001/fdhd_ana_2026-01-25T_125805Z_hist.root
Finished2026-01-26 02:26:26
Saved logsjustin-logs:300626.4-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

133083	0
marley	1000190390	0.01	(-233.07, 712.36, 1373.43)	133082	0
marley	11		10.17	(-233.07, 712.36, 1373.43)	133081	0
Signal Track IDs: 133081; 133082; 133083; 133084; 133085; 133086; 133087; 133088; 133089; 133090; 133091; 133092; 133093; 133094; 133095; 133096; 133097; 133098; 133099; 133100; 133101; 133102; 133103; 133104; 133105; 133106; 133107; 133108; 133109; 133110; 133111; 133112; 133113; 133114; 133115; 133116; 133117; 133118; 133119; 133120; 133121; 133122; 133123; 133124; 133125; 133126; 133127; 133128; 133129; 133130; 133131; 133132; 133133; 133134; 133135; 133136; 133137; 133138; 133139; 133140; 133141; 133142; 133143; 133144; 133145; 133146; 133147; 133149; 133150; 133151; 133152; 133153; 133154; 133155; 133156; 133157; 133160; 133161; 133162; 133163; 133164; 133165; 133167; 133168; 133169; 133173; 133175; 133176; 133177; 133178; 133179; 133252; 133253; 133254; 133255; 133256; 133257; 133258; 133259; 133260; 
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Jan-2026 01:02:33 UTC run: 4992 subRun: 0 event: 31130

Kinetic energy of signal particle is below threshold of 30.00 MeV. Selecting event.

%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Jan-2026 01:04:48 UTC run: 4992 subRun: 0 event: 31130
OpFlash PE 73.91 with purity 0.11 time -0.59 plane 2
 - Vertex (210.07, -743.30, 152.42)
	*** 1st Sanity check: Ratio 0.39 <= 1.00
	*** 2nd Sanity check: #OpHits 5 >= 5
OpFlash PE 22.70 with purity 0.27 time 0.77 plane 0
 - Vertex (-327.50, -220.90, 1595.67)
	*** 1st Sanity check: Ratio 0.38 <= 1.00
	*** 2nd Sanity check: #OpHits 3 >= 3
OpFlash PE 9.44 with purity 0.12 time 1.34 plane 2
 - Vertex (210.07, -743.30, 152.42)
	*** 1st Sanity check: Ratio 1.00 <= 1.00
	*** 2nd Sanity check: #OpHits 1 >= 1
OpFlash PE 52.17 with purity 0.72 time 1.68 plane 0
 - Vertex (-327.50, 471.85, 1417.13)
	*** 1st Sanity check: Ratio 0.18 <= 1.00
	*** 2nd Sanity check: #OpHits 7 >= 7
OpFlash PE 20.32 with purity 0.17 time 506.44 plane 0
 - Vertex (-327.50, 293.40, 1302.78)
	*** 1st Sanity check: Ratio 0.68 <= 1.00
	*** 2nd Sanity check: #OpHits 2 >= 2
OpFlash PE 181.41 with purity 0.33 time 509.04 plane 0
 - Vertex (-327.50, 465.70, 1578.42)
	*** 1st Sanity check: Ratio 0.13 <= 1.00
	*** 2nd Sanity check: #OpHits 15 >= 15
OpFlash PE 33.82 with purity 0.78 time 510.25 plane 1
 - Vertex (111.81, 743.30, 1576.73)
	*** 1st Sanity check: Ratio 0.31 <= 1.00
	*** 2nd Sanity check: #OpHits 4 >= 4
OpFlash PE 48.28 with purity 0.18 time 510.90 plane 0
 - Vertex (-327.50, 437.66, 1613.07)
	*** 1st Sanity check: Ratio 0.23 <= 1.00
	*** 2nd Sanity check: #OpHits 6 >= 6

# of OpHits (ophit10ppm): 52940 for times between -4253.85 us and 295147919198126080.00 us.
# of OpFlashes (solarflash): 18212 for times between -4253.54 us and 295147919198126080.00 us.

%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Jan-2026 01:04:52 UTC run: 4992 subRun: 0 event: 31130

# OpHits (ophit10ppm) in full geometry: 52940
# OpFlashes (solarflash) in full geometry: 18212
# Hits (gaushit) in each view: 6532, 6377, 6452, 0
# Cluster from the hits: 2728, 2741, 5501, 0
# Tracks (pmtracktc) in full geometry: 449
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Jan-2026 02:25:57 UTC run: 4992 subRun: 0 event: 31130

# ClusterReco: Looping over 5494 matched clusters
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Jan-2026 02:25:57 UTC run: 4992 subRun: 0 event: 31130
*** Matched preselection cluster: 4071 from 5494
 - MainTrackID 133254
 - Gen 1 -> marley TPC 86
 - Purity 1.00 Hits 3
 - Charge 339.76 ( MaxHit 202.74 )
 - #AdjCl 7 ( 1 signal ):
    - Cluster 3670 at distance 62.08 with time 2861.21 and charge 111.39 in TPC 86 and hits 2
    - Cluster 3734 at distance 84.40 with time 2899.64 and charge 115.10 in TPC 86 and hits 1
    - Cluster 4033 at distance 85.62 with time 3122.09 and charge 105.62 in TPC 86 and hits 1
    - Cluster 4106 at distance 94.99 with time 3172.32 and charge 129.81 in TPC 86 and hits 1
    - Cluster 4121 at distance 89.82 with time 3187.35 and charge 77.22 in TPC 86 and hits 1
    - Cluster 4264 at distance 41.69 with time 3305.17 and charge 54.62 in TPC 86 and hits 1
    - Cluster 4409 at distance 48.72 with time 3411.44 and charge 313.63 in TPC 86 and hits 4
 - RecoCol  Time,Y,Z ( 3151.06, 477.82, 1577.50 )
 - RecoInd0 Time,Y,Z ( 3150.97, 477.79, 1587.28 )
 - RecoInd1 Time,Y,Z ( 3151.01, 477.86, 1559.31 )
*** Matched flash: 10233 from 18212
 - Purity 33.01 % Plane 0 #Hits 15
 - MainOpHitPE 24.48 (PE);  TotalPE 181.41 vs expected 88.33 (PE)
 - Time 509.04 (us) Fast 14.05 % Residual 315.18
 - Reco Time,Y,Z ( 509.04, 465.70, 1578.42 )
 - Found valid waveform: true
*** Reconstructed Interaction Vertex: 
 - True X,Y,Z ( -233.07, 712.36, 1373.43 )
 - Main X,Y,Z ( -142.73, 500.18, 1590.79 )
 - EDep X,Y,Z ( -99.10, 477.85, 1577.39 )
 - Reco X,Y,Z ( -99.18, 477.82, 1577.50 )
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Jan-2026 02:25:57 UTC run: 4992 subRun: 0 event: 31130
*** Matched preselection cluster: 4466 from 5494
 - MainTrackID 133253
 - Gen 1 -> marley TPC 87
 - Purity 1.00 Hits 2
 - Charge 82.57 ( MaxHit 59.78 )
 - #AdjCl 1 ( 0 signal ):
    - Cluster 4517 at distance 85.54 with time 3503.66 and charge 29.28 in TPC 83 and hits 1
 - RecoCol  Time,Y,Z ( 3457.79, 129.38, 1583.04 )
 - RecoInd0 Time,Y,Z ( 3457.06, -262.76, 791.91 )
 - RecoInd1 Time,Y,Z ( 3457.92, 521.51, 1598.63 )
*** Matched flash: 11939 from 18212
 - Purity 0.00 % Plane 0 #Hits 6
 - MainOpHitPE 12.94 (PE);  TotalPE 50.22 vs expected 32.62 (PE)
 - Time 1300.72 (us) Fast 27.76 % Residual 1174.75
 - Reco Time,Y,Z ( 1300.72, 129.91, 1538.65 )
 - Found valid waveform: true
*** Reconstructed Interaction Vertex: 
 - True X,Y,Z ( -233.07, 712.36, 1373.43 )
 - Main X,Y,Z ( -142.73, 500.18, 1590.79 )
 - EDep X,Y,Z ( -148.47, 521.47, 1583.09 )
 - Reco X,Y,Z ( -21.31, 129.38, 1583.04 )
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Jan-2026 02:25:57 UTC run: 4992 subRun: 0 event: 31130
*** Matched preselection cluster: 4468 from 5494
 - MainTrackID 90417
 - Gen 1 -> marley TPC 87
 - Purity 0.78 Hits 3
 - Charge 132.78 ( MaxHit 62.31 )
 - #AdjCl 4 ( 0 signal ):
    - Cluster 4055 at distance 64.55 with time 3140.63 and charge 89.29 in TPC 87 and hits 1
    - Cluster 4200 at distance 84.62 with time 3248.62 and charge 115.14 in TPC 87 and hits 1
    - Cluster 4925 at distance 80.88 with time 3806.29 and charge 48.77 in TPC 87 and hits 2
    - Cluster 4964 at distance 88.86 with time 3836.95 and charge 46.91 in TPC 87 and hits 1
 - RecoCol  Time,Y,Z ( 3458.13, 521.00, 1586.94 )
 - RecoInd0 Time,Y,Z ( 3458.19, 518.23, 1552.68 )
 - RecoInd1 Time,Y,Z ( 3457.92, 523.76, 1598.63 )
*** Matched flash: 8660 from 18212
 - Purity 0.00 % Plane 0 #Hits 19
 - MainOpHitPE 133.72 (PE);  TotalPE 487.49 vs expected 194.28 (PE)
 - Time -217.82 (us) Fast 64.18 % Residual 6806.82
 - Reco Time,Y,Z ( -217.82, 609.05, 1632.41 )
 - Found valid waveform: true
*** Reconstructed Interaction Vertex: 
 - True X,Y,Z ( 0.00, 0.00, 0.00 )
 - Main X,Y,Z ( 0.00, 0.00, 0.00 )
 - EDep X,Y,Z ( 0.00, 0.00, 0.00 )
 - Reco X,Y,Z ( -265.19, 521.00, 1586.94 )
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  26-Jan-2026 02:25:57 UTC run: 4992 subRun: 0 event: 31130
-----------------------------------------------------------------------------------------

%MSG
26-Jan-2026 02:25:58 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/5c/6b/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250130T200659Z_gen_003113_supernova_g4stage1_g4stage2_detsim_reco_20250624T154437Z_reco2.root"
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@EndJob 26-Jan-2026 02:25:58 UTC  ModuleEndJob
Finished running the SolarNuAna module
%MSG

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                          2.45036       4837.4        8186.5         6104         3311.43        10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)             0.0022813     0.0138472     0.0310774     0.0108807     0.0108465       10     
end_path:solarnuana:SolarNuAna      2.42531       4837.38       8186.48       6103.97       3311.42        10     
====================================================================================================================
%MSG-i NuRandomService:  SolarNuAna:solarnuana@EndJob 26-Jan-2026 02:25:58 UTC  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1

%MSG

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3188.99 MB
  Peak resident set size usage (VmHWM): 2580.72 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 10 passed = 10 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport         10         10          0 solarnuana

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 47596.133867 Real = 48445.270960

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 3188.99 VmHWM = 2580.72

Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
justIN time: 2026-02-05 01:21:20 UTC       justIN version: 01.06.00