Jobsub ID 239247.0@dunegpschedd01.fnal.gov

Jobsub ID	239247.0@dunegpschedd01.fnal.gov
Workflow ID	9406
Stage ID	1
User name	smanthey@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-10-28 12:22:14
Site	US_FNAL-FermiGrid
Entry	FNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat	2025-10-28 12:47:10
From worker node	Hostname	dunegli-7551892-0-fnpc19102.fnal.gov
	cpuinfo	AMD EPYC 7502 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	172800 (48 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-10-28 12:23:24
Input files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T162436Z_gen_001197_supernova_g4_detsim_reco_20250724T032624Z_reco2.root
Jobscript	Exit code	0
	Real time	11m (718s)
	CPU time	11m (704s = 98%)
	Max RSS bytes	906047488 (864 MiB)
Outputting started	2025-10-28 12:35:22
Output files	https://fndcadoor.fnal.gov:2880/dune/scratch/users/smanthey/ana/solar_ana_marley_flash_radiological_decay0_dune10kt_1x2x6_centralAPA/fnal/09406/1/001/fdhd_ana_2025-10-28T_122331Z_hist.root
Finished	2025-10-28 12:47:10
Saved logs	justin-logs:239247.0-dunegpschedd01.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

0400	0.00	(32.40, 254.43, 1141.26)	32792	32700
marley	1000180400	0.00	(12.06, 264.24, 1153.06)	32791	32700
marley	1000180400	0.02	(-23.29, 287.04, 1154.19)	32790	32700
marley	1000180400	0.02	(-14.75, 278.50, 1149.79)	32789	32700
marley	1000180400	0.01	(-16.52, 279.40, 1149.01)	32788	32700
marley	1000180400	0.02	(-14.97, 280.16, 1146.41)	32787	32700
marley	1000180400	0.02	(-11.77, 281.49, 1149.68)	32786	32700
marley	1000180400	0.03	(-30.19, 277.41, 1154.30)	32785	32700
marley	1000180400	0.03	(0.47, 293.59, 1128.60)	32784	32700
marley	1000180400	0.01	(-0.17, 300.91, 1150.31)	32783	32700
marley	1000180400	0.04	(-75.46, 290.76, 1216.11)	32782	32700
marley	1000180400	0.01	(-67.12, 281.58, 1195.86)	32781	32700
marley	1000180400	0.04	(-75.71, 275.30, 1186.61)	32780	32700
marley	1000180400	0.03	(-69.08, 277.76, 1187.08)	32779	32700
marley	1000180400	0.04	(-104.84, 324.90, 1177.03)	32778	32700
marley	1000180400	0.04	(-104.37, 324.98, 1177.33)	32777	32700
marley	1000180400	0.05	(-104.50, 324.20, 1177.26)	32776	32700
marley	1000180400	0.03	(-112.77, 331.66, 1177.15)	32775	32700
marley	1000180400	0.03	(-116.78, 328.84, 1173.84)	32774	32700
marley	1000180400	0.01	(-148.26, 344.71, 1192.49)	32773	32700
marley	1000180400	0.05	(-150.38, 348.21, 1193.35)	32772	32700
marley	1000180400	0.06	(-152.42, 343.69, 1174.06)	32771	32700
marley	1000180400	0.03	(-149.76, 359.18, 1181.61)	32770	32700
marley	1000180400	0.03	(-156.69, 354.97, 1200.46)	32769	32700
marley	1000180400	0.07	(-125.05, 363.52, 1225.84)	32768	32700
marley	1000180400	0.09	(-181.67, 379.59, 1238.27)	32767	32700
marley	1000180400	0.03	(-110.55, 345.25, 1262.65)	32766	32700
marley	1000180400	0.01	(-111.14, 332.48, 1264.99)	32765	32700
marley	1000180400	0.05	(-111.39, 327.62, 1262.42)	32764	32700
marley	1000180400	0.02	(-107.08, 336.71, 1243.03)	32763	32700
marley	1000180400	0.00	(-108.26, 334.31, 1234.64)	32762	32700
marley	1000180400	0.10	(-108.08, 333.09, 1232.95)	32761	32700
marley	1000180400	0.11	(-74.70, 355.22, 1255.11)	32760	32700
marley	1000180400	0.04	(-103.67, 361.34, 1246.54)	32759	32700
marley	1000180400	0.00	(-104.34, 360.64, 1244.63)	32758	32700
marley	1000180400	0.14	(-105.22, 359.00, 1242.65)	32757	32700
marley	1000180400	0.09	(-105.00, 359.44, 1242.85)	32756	32700
marley	22		4.93	(-84.32, 358.52, 1205.17)	32702	0
marley	22		1.90	(-84.32, 358.52, 1205.17)	32701	0
marley	2112		1.61	(-84.32, 358.52, 1205.17)	32700	0
marley	1000190390	0.02	(-84.32, 358.52, 1205.17)	32699	0
marley	11		11.51	(-84.32, 358.52, 1205.17)	32698	0
Signal Track IDs: 32698; 32699; 32700; 32701; 32702; 32756; 32757; 32758; 32759; 32760; 32761; 32762; 32763; 32764; 32765; 32766; 32767; 32768; 32769; 32770; 32771; 32772; 32773; 32774; 32775; 32776; 32777; 32778; 32779; 32780; 32781; 32782; 32783; 32784; 32785; 32786; 32787; 32788; 32789; 32790; 32791; 32792; 32793; 32794; 32795; 32796; 32797; 32798; 32799; 32800; 32801; 32802; 32803; 32804; 32805; 32806; 32807; 32808; 32809; 32810; 32811; 32812; 32813; 32814; 32815; 32816; 32817; 32818; 32819; 32820; 32821; 32822; 32823; 32824; 32825; 32826; 32827; 32828; 32830; 32832; 32845; 32846; [0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  28-Oct-2025 12:34:57 UTC run: 5092 subRun: 0 event: 59850
[33m
Kinetic energy of signal particle is below threshold of 30.00 MeV. Selecting event.
[0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  28-Oct-2025 12:35:05 UTC run: 5092 subRun: 0 event: 59850
[34mOpFlash PE 43.94 with purity 0.24 time -0.25 plane 0
 - Vertex (0.05, 591.57, 1357.47)
	*** 1st Sanity check: Ratio 0.96 <= 1.00
	*** 2nd Sanity check: #OpHits 2 >= 2
OpFlash PE 9.24 with purity 0.29 time -0.00 plane 0
 - Vertex (0.05, 438.64, 510.35)
	*** 1st Sanity check: Ratio 0.21 <= 1.00
	*** 2nd Sanity check: #OpHits 5 >= 5
OpFlash PE 644.75 with purity 0.94 time 0.00 plane 0
 - Vertex (0.05, 355.15, 1200.12)
	*** 1st Sanity check: Ratio 0.14 <= 1.00
	*** 2nd Sanity check: #OpHits 57 >= 57
OpFlash PE 11.52 with purity 0.85 time 0.01 plane 0
 - Vertex (0.01, 68.03, 938.40)
	*** 1st Sanity check: Ratio 0.18 <= 1.00
	*** 2nd Sanity check: #OpHits 6 >= 6
OpFlash PE 1.90 with purity 0.50 time 0.00 plane 0
 - Vertex (-0.05, -342.49, 1013.48)
	*** 1st Sanity check: Ratio 1.00 <= 1.00
	*** 2nd Sanity check: #OpHits 1 >= 1
OpFlash PE 3.96 with purity 0.64 time 0.01 plane 0
 - Vertex (-0.05, -191.53, 1278.85)
	*** 1st Sanity check: Ratio 0.53 <= 1.00
	*** 2nd Sanity check: #OpHits 2 >= 2
OpFlash PE 71.53 with purity 0.98 time 0.02 plane 0
 - Vertex (0.05, 459.67, 1229.05)
	*** 1st Sanity check: Ratio 0.16 <= 1.00
	*** 2nd Sanity check: #OpHits 13 >= 13
OpFlash PE 42.88 with purity 0.97 time 0.03 plane 0
 - Vertex (0.05, 336.62, 1213.01)
	*** 1st Sanity check: Ratio 0.13 <= 1.00
	*** 2nd Sanity check: #OpHits 14 >= 14
OpFlash PE 52.65 with purity 0.96 time 0.05 plane 0
 - Vertex (0.05, 319.32, 1238.36)
	*** 1st Sanity check: Ratio 0.18 <= 1.00
	*** 2nd Sanity check: #OpHits 16 >= 16
OpFlash PE 39.97 with purity 0.81 time 0.07 plane 0
 - Vertex (0.05, 350.29, 1132.13)
	*** 1st Sanity check: Ratio 0.11 <= 1.00
	*** 2nd Sanity check: #OpHits 16 >= 16
OpFlash PE 10.25 with purity 0.87 time 0.09 plane 0
 - Vertex (0.05, 404.76, 1197.07)
	*** 1st Sanity check: Ratio 0.67 <= 1.00
	*** 2nd Sanity check: #OpHits 3 >= 3
OpFlash PE 1.61 with purity 1.00 time 0.11 plane 0
 - Vertex (0.05, 467.03, 1308.67)
	*** 1st Sanity check: Ratio 1.00 <= 1.00
	*** 2nd Sanity check: #OpHits 1 >= 1
OpFlash PE 3.67 with purity 0.25 time 0.13 plane 0
 - Vertex (0.05, 124.26, 927.91)
	*** 1st Sanity check: Ratio 0.50 <= 1.00
	*** 2nd Sanity check: #OpHits 2 >= 2
OpFlash PE 16.39 with purity 0.14 time 0.14 plane 0
 - Vertex (0.05, 422.31, 1196.36)
	*** 1st Sanity check: Ratio 0.29 <= 1.00
	*** 2nd Sanity check: #OpHits 6 >= 6

# of OpFlashes (solarflash) in full geometry: 8595
[0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  28-Oct-2025 12:35:05 UTC run: 5092 subRun: 0 event: 59850
[36m
# OpHits (ophitspe) in full geometry: 70209
# OpFlashes (solarflash) in full geometry: 8595
# Hits (hitfd) in each view: 190, 196, 429, 0
# Cluster from the hits: 115, 115, 389, 0
# Tracks (pmtracktc) in full geometry: 5[0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  28-Oct-2025 12:35:22 UTC run: 5092 subRun: 0 event: 59850
[32m
# ClusterReco: Looping over 242 matched clusters[0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  28-Oct-2025 12:35:22 UTC run: 5092 subRun: 0 event: 59850
[32m*** Matched preselection cluster: 47
 - MainTrackID 32698
 - Gen 1 -> marley TPC 22
 - Purity 1.00 Hits 6
 - Charge 1616.85 ( MaxHit 453.34 )
 - #AdjCl 1 ( 1 signal ):
    - Cluster 45 at distance 15.95 with time 477.87 and charge 500.63 in TPC 22 and hits 2
 - RecoCol  Time,Y,Z ( 489.59, 358.49, 1203.94 )
 - RecoInd0 Time,Y,Z ( 489.30, 358.39, 1277.27 )
 - RecoInd1 Time,Y,Z ( 488.31, 358.60, 1269.87 )
*** Matched flash: 
 - Purity 0.94 Plane 0 #Hits 57 PE 644.75 MaxPE 92.48
 - Time -18.10 Fast 0.86 Residual 217.51
 - Reco Time,Y,Z ( -18.10, 355.15, 1200.12 )
*** Reconstructed Interaction Vertex: 
 - True X,Y,Z ( -84.32, 358.52, 1205.17 )
 - Main X,Y,Z ( -79.86, 358.67, 1202.54 )
 - EDep X,Y,Z ( -79.56, 358.56, 1202.82 )
 - Reco X,Y,Z ( -81.52, 358.49, 1203.94 )[0m
%MSG
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@BeginModule  28-Oct-2025 12:35:22 UTC run: 5092 subRun: 0 event: 59850
[32m-----------------------------------------------------------------------------------------
[0m
%MSG
28-Oct-2025 12:35:22 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/87/36/prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T162436Z_gen_001197_supernova_g4_detsim_reco_20250724T032624Z_reco2.root"
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@EndJob 28-Oct-2025 12:35:22 UTC  ModuleEndJob
[35mFinished running the SolarNuAna module[0m
%MSG

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.220923       13.2968       43.3649      0.670652       15.6089        50     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.00138029    0.00729323     0.0252479     0.0041785    0.00613498       50     
end_path:solarnuana:SolarNuAna     0.219242       13.2894       43.3616      0.663602       15.6114        50     
====================================================================================================================
%MSG-i NuRandomService:  SolarNuAna:solarnuana@EndJob 28-Oct-2025 12:35:22 UTC  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1

%MSG

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1550.72 MB
  Peak resident set size usage (VmHWM): 902.738 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 50 passed = 50 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport         50         50          0 solarnuana

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 686.049187 Real = 696.931989

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 1550.72 VmHWM = 902.738

Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0