Jobsub ID 11128.52@dunegpschedd02.fnal.gov

Jobsub ID	11128.52@dunegpschedd02.fnal.gov
Workflow ID	60
Stage ID	1
User name	higuera@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-07-25 02:42:48
Site	US_UChicago
Entry	Engage_US_MWT2_uiuc_condce_mcore
Last heartbeat	2025-07-25 10:16:06
From worker node	Hostname	iut2-c298.iu.edu
	cpuinfo	Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	86400 (24 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-07-25 04:24:17
Input files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250210T162027Z_gen_001486_supernova_g4_halfActiveVol_detsim_reco.root
Jobscript	Exit code	0
	Real time	5h (21043s)
	CPU time	5h (20608s = 97%)
	Max RSS bytes	1637806080 (1561 MiB)
Outputting started	2025-07-25 10:15:01
Output files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250210T162027Z_gen_001486_supernova_g4_halfActiveVol_detsim_reco_20250725T042423Z_reco2.root
Finished	2025-07-25 10:16:06
Saved logs	justin-logs:11128.52-dunegpschedd02.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

al Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
Printing U view candidate length: 19
Printing V view candidate length: 21
Printing W view candidate length: 22
Printing 2D view candidate length: 62
Printing background CaloHitList length: 393
Printing New CaloHitList Names: 
SCaloHitListU | SCaloHitListV | SCaloHitListW | SCaloHitList2D | BBCaloHitList
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
Printing U view candidate length: 19
Printing V view candidate length: 21
Printing W view candidate length: 21
Printing 2D view candidate length: 61
Printing background CaloHitList length: 5
Printing New CaloHitList Names: 
CaloHitListU | CaloHitListV | CaloHitListW | CaloHitList2D | BCaloHitList
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 06:11:01 EDT run: 5117 subRun: 0 event: 74300
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 06:11:01 EDT run: 5117 subRun: 0 event: 74300
Comparing two wires in the same plane: return failure
%MSG
33263
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 40857!
Genrating OpFlash 0 with 4 hits
Genrating OpFlash 1 with 11 hits
Genrating OpFlash 2 with 3 hits
Genrating OpFlash 3 with 4 hits
Genrating OpFlash 4 with 5 hits
Genrating OpFlash 5 with 3 hits
Genrating OpFlash 6 with 3 hits
Genrating OpFlash 7 with 3 hits
Genrating OpFlash 8 with 4 hits
Genrating OpFlash 9 with 4 hits
...
Generated 1156 OpFlashes
25-Jul-2025 06:14:56 EDT  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250210T162027Z_gen_001486_supernova_g4_halfActiveVol_detsim_reco_20250725T042423Z_reco2.root"
25-Jul-2025 06:14:56 EDT  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/93/24/prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250210T162027Z_gen_001486_supernova_g4_halfActiveVol_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              316.91        417.054       490.069       421.145       34.0067        50     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00739757     0.0202051     0.113662      0.0179996     0.0160397       50     
reco:linecluster:LineCluster                           0.0728462      0.12546      0.492608      0.102006      0.0687241       50     
reco:trajcluster:TrajCluster                           0.0228453     0.0296357     0.163375      0.0251585     0.0198341       50     
reco:pandora:StandardPandora                            128.666       157.908       175.583       159.641       11.3734        50     
reco:pandoraTrack:LArPandoraTrackCreation             0.000368052   0.00692286     0.0178243    0.00772844    0.00393828       50     
reco:pandoraShower:LArPandoraModularShowerCreation    0.00037941    0.00226108     0.0225266    0.00164698    0.00312598       50     
reco:pandoracalo:Calorimetry                          0.000249907   0.000620521    0.0013954    0.000610656   0.000220169      50     
reco:pandorapid:Chi2ParticleID                        5.9619e-05    0.000159981   0.000754558   0.000147698   9.53329e-05      50     
reco:pmtracktc:PMAlgTrackMaker                        0.00601748     0.0651716     0.387608      0.0185159     0.0854382       50     
reco:pmtracktccalo:Calorimetry                        0.000227345   0.000707544   0.00305051    0.000653762   0.000392889      50     
reco:pmtracktcpid:Chi2ParticleID                      6.5073e-05    0.000172139   0.000823069   0.000153371   0.000102052      50     
reco:emtrkmichelid:EmTrackMichelId                      6.8988        10.2957       13.5612       10.3011       1.52595        50     
reco:opdec:Deconvolution                                18.6778       21.1876       23.2532       21.2749       1.17408        50     
reco:ophitspe:OpHitFinderDeco                           153.055       218.746       272.47        219.746       26.945         50     
reco:opflash:OpFlashFinder                              6.39998       8.17015       11.2901       8.29375       1.30558        50     
reco:solarflash:SolarOpFlash                           0.223266       0.32775      0.476943      0.321061      0.0716057       50     
reco:rns:RandomNumberSaver                            6.3283e-05    0.000108214   0.000933417   8.63485e-05   0.000120037      50     
[art]:TriggerResults:TriggerResultInserter            2.8557e-05    4.67376e-05   0.000130822   4.05075e-05   1.68753e-05      50     
end_path:out1:RootOutput                               6.59e-06     1.36164e-05    4.458e-05     1.216e-05    5.86522e-06      50     
end_path:out1:RootOutput(write)                        0.123721      0.165276       0.23495      0.153904      0.0290636       50     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3134.15 MB
  Peak resident set size usage (VmHWM): 1637.81 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250210T162027Z_gen_001486_supernova_g4_halfActiveVol_detsim_reco_20250725T042423Z_reco2.root
Extracted metadata
checking file====== fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250210T162027Z_gen_001486_supernova_g4_halfActiveVol_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250210T162027Z_gen_001486_supernova_g4_halfActiveVol_detsim_reco_20250725T042423Z_reco2.root
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250210T162027Z_gen_001486_supernova_g4_halfActiveVol_detsim_reco_20250725T042423Z_reco2.root.ext.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250210T162027Z_gen_001486_supernova_g4_halfActiveVol_detsim_reco_20250725T042423Z_reco2.root.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250210T162027Z_gen_001486_supernova_g4_halfActiveVol_detsim_reco_20250725T042423Z_reco2.root.temp.json
reco2_hist.root