Jobsub ID 17837.167@dunegpschedd01.fnal.gov

Jobsub ID	17837.167@dunegpschedd01.fnal.gov
Workflow ID	60
Stage ID	1
User name	higuera@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-07-25 02:26:47
Site	US_UChicago
Entry	Engage_US_MWT2_uct2_gk02_condce_mcore
Last heartbeat	2025-07-25 10:34:55
From worker node	Hostname	mwt2-c182.campuscluster.illinois.edu
	cpuinfo	Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	86400 (24 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-07-25 04:18:59
Input files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250209T172836Z_gen_001450_supernova_g4_halfActiveVol_detsim_reco.root
Jobscript	Exit code	0
	Real time	6h (22496s)
	CPU time	6h (21966s = 97%)
	Max RSS bytes	1638031360 (1562 MiB)
Outputting started	2025-07-25 10:33:56
Output files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250209T172836Z_gen_001450_supernova_g4_halfActiveVol_detsim_reco_20250725T041903Z_reco2.root
Finished	2025-07-25 10:34:55
Saved logs	justin-logs:17837.167-dunegpschedd01.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

42761!
Genrating OpFlash 0 with 3 hits
Genrating OpFlash 1 with 8 hits
Genrating OpFlash 2 with 7 hits
Genrating OpFlash 3 with 5 hits
Genrating OpFlash 4 with 10 hits
Genrating OpFlash 5 with 5 hits
Genrating OpFlash 6 with 3 hits
Genrating OpFlash 7 with 4 hits
Genrating OpFlash 8 with 6 hits
Genrating OpFlash 9 with 3 hits
...
Generated 1243 OpFlashes
Begin processing the 50th record. run: 5100 subRun: 0 event: 72500 at 25-Jul-2025 05:26:21 CDT
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
Printing U view candidate length: 25
Printing V view candidate length: 24
Printing W view candidate length: 11
Printing 2D view candidate length: 60
Printing background CaloHitList length: 446
Printing New CaloHitList Names: 
SCaloHitListU | SCaloHitListV | SCaloHitListW | SCaloHitList2D | BBCaloHitList
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
Printing U view candidate length: 18
Printing V view candidate length: 24
Printing W view candidate length: 9
Printing 2D view candidate length: 51
Printing background CaloHitList length: 4
Printing New CaloHitList Names: 
CaloHitListU | CaloHitListV | CaloHitListW | CaloHitList2D | BCaloHitList
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 05:28:50 CDT run: 5100 subRun: 0 event: 72500
Comparing two wires in the same plane: return failure
%MSG
33362
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 40766!
Genrating OpFlash 0 with 5 hits
Genrating OpFlash 1 with 3 hits
Genrating OpFlash 2 with 6 hits
Genrating OpFlash 3 with 3 hits
Genrating OpFlash 4 with 4 hits
Genrating OpFlash 5 with 4 hits
Genrating OpFlash 6 with 10 hits
Genrating OpFlash 7 with 4 hits
Genrating OpFlash 8 with 7 hits
Genrating OpFlash 9 with 4 hits
...
Generated 1156 OpFlashes
25-Jul-2025 05:33:50 CDT  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250209T172836Z_gen_001450_supernova_g4_halfActiveVol_detsim_reco_20250725T041903Z_reco2.root"
25-Jul-2025 05:33:50 CDT  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/8d/53/prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250209T172836Z_gen_001450_supernova_g4_halfActiveVol_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              236.351       446.698       575.334       458.91        92.0282        50     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00656076     0.0257355     0.0658711     0.0236366     0.0119462       50     
reco:linecluster:LineCluster                           0.0689669     0.111989      0.332699      0.106073      0.0376103       50     
reco:trajcluster:TrajCluster                           0.0183701     0.0283566     0.0707091     0.025135     0.00923701       50     
reco:pandora:StandardPandora                            75.1782       170.083       250.242       165.352       48.8082        50     
reco:pandoraTrack:LArPandoraTrackCreation             0.000261436    0.0081985     0.0171303     0.008477     0.00496567       50     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000285743   0.00227004     0.0150135    0.00170993     0.0024983       50     
reco:pandoracalo:Calorimetry                          0.00017001    0.000674489    0.0039259    0.000568888   0.000522105      50     
reco:pandorapid:Chi2ParticleID                        4.6867e-05    0.000187222   0.00159665    0.000147879   0.00021059       50     
reco:pmtracktc:PMAlgTrackMaker                        0.00364397     0.0311169     0.121835      0.019223      0.0287293       50     
reco:pmtracktccalo:Calorimetry                        0.000251355   0.000612376   0.00124005    0.00058469    0.000220985      50     
reco:pmtracktcpid:Chi2ParticleID                      7.4161e-05    0.000152298   0.000316861   0.000141706   5.78129e-05      50     
reco:emtrkmichelid:EmTrackMichelId                      5.54648        11.89        18.0737       11.9317       3.67473        50     
reco:opdec:Deconvolution                                13.415        19.6717       24.4535       20.5951        3.197         50     
reco:ophitspe:OpHitFinderDeco                           133.107       234.146       303.413       246.601       44.9862        50     
reco:opflash:OpFlashFinder                              5.26548       10.2505       16.1381       9.99898       2.61833        50     
reco:solarflash:SolarOpFlash                           0.134384      0.276362      0.401195      0.304044      0.0785804       50     
reco:rns:RandomNumberSaver                            5.9078e-05    0.000104294   0.000768923   8.5379e-05    9.73127e-05      50     
[art]:TriggerResults:TriggerResultInserter            2.0752e-05    3.89934e-05   0.000100833    3.781e-05    1.34488e-05      50     
end_path:out1:RootOutput                               7.078e-06    1.27289e-05   6.4144e-05    1.07805e-05   8.50482e-06      50     
end_path:out1:RootOutput(write)                        0.096777      0.169212      0.229888      0.180674      0.0379842       50     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3134.08 MB
  Peak resident set size usage (VmHWM): 1638.03 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250209T172836Z_gen_001450_supernova_g4_halfActiveVol_detsim_reco_20250725T041903Z_reco2.root
Extracted metadata
checking file====== fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250209T172836Z_gen_001450_supernova_g4_halfActiveVol_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250209T172836Z_gen_001450_supernova_g4_halfActiveVol_detsim_reco_20250725T041903Z_reco2.root
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250209T172836Z_gen_001450_supernova_g4_halfActiveVol_detsim_reco_20250725T041903Z_reco2.root.ext.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250209T172836Z_gen_001450_supernova_g4_halfActiveVol_detsim_reco_20250725T041903Z_reco2.root.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250209T172836Z_gen_001450_supernova_g4_halfActiveVol_detsim_reco_20250725T041903Z_reco2.root.temp.json
reco2_hist.root