Jobsub ID 11128.53@dunegpschedd02.fnal.gov

Jobsub ID	11128.53@dunegpschedd02.fnal.gov
Workflow ID	60
Stage ID	1
User name	higuera@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-07-25 02:42:48
Site	US_UChicago
Entry	Engage_US_MWT2_uiuc_condce_mcore
Last heartbeat	2025-07-25 10:23:20
From worker node	Hostname	iut2-c291.iu.edu
	cpuinfo	Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	86400 (24 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-07-25 04:25:05
Input files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250209T190506Z_gen_003631_supernova_g4_halfActiveVol_detsim_reco.root
Jobscript	Exit code	0
	Real time	5h (21411s)
	CPU time	5h (20827s = 97%)
	Max RSS bytes	1653620736 (1577 MiB)
Outputting started	2025-07-25 10:21:57
Output files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250209T190506Z_gen_003631_supernova_g4_halfActiveVol_detsim_reco_20250725T042521Z_reco2.root
Finished	2025-07-25 10:23:20
Saved logs	justin-logs:11128.53-dunegpschedd02.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

d*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Electron identified*
*Photon identified*
*Electron identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Photon identified*
*Electron identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Photon identified*
*Photon identified*
Printing U view candidate length: 25
Printing V view candidate length: 15
Printing W view candidate length: 14
Printing 2D view candidate length: 54
Printing background CaloHitList length: 9
Printing New CaloHitList Names: 
CaloHitListU | CaloHitListV | CaloHitListW | CaloHitList2D | BCaloHitList
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 06:10:11 EDT run: 5100 subRun: 0 event: 181549
Comparing two wires in the same plane: return failure
%MSG
33436
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 41306!
Genrating OpFlash 0 with 4 hits
Genrating OpFlash 1 with 3 hits
Genrating OpFlash 2 with 5 hits
Genrating OpFlash 3 with 3 hits
Genrating OpFlash 4 with 10 hits
Genrating OpFlash 5 with 3 hits
Genrating OpFlash 6 with 4 hits
Genrating OpFlash 7 with 4 hits
Genrating OpFlash 8 with 4 hits
Genrating OpFlash 9 with 4 hits
...
Generated 1149 OpFlashes
Begin processing the 50th record. run: 5100 subRun: 0 event: 181550 at 25-Jul-2025 06:14:59 EDT
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
Printing U view candidate length: 12
Printing V view candidate length: 10
Printing W view candidate length: 5
Printing 2D view candidate length: 27
Printing background CaloHitList length: 376
Printing New CaloHitList Names: 
SCaloHitListU | SCaloHitListV | SCaloHitListW | SCaloHitList2D | BBCaloHitList
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Photon identified*
*Photon identified*
Printing U view candidate length: 10
Printing V view candidate length: 10
Printing W view candidate length: 5
Printing 2D view candidate length: 25
Printing background CaloHitList length: 10
Printing New CaloHitList Names: 
CaloHitListU | CaloHitListV | CaloHitListW | CaloHitList2D | BCaloHitList
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 06:17:18 EDT run: 5100 subRun: 0 event: 181550
Comparing two wires in the same plane: return failure
%MSG
32541
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 39948!
Genrating OpFlash 0 with 3 hits
Genrating OpFlash 1 with 3 hits
Genrating OpFlash 2 with 3 hits
Genrating OpFlash 3 with 3 hits
Genrating OpFlash 4 with 3 hits
Genrating OpFlash 5 with 4 hits
Genrating OpFlash 6 with 4 hits
Genrating OpFlash 7 with 3 hits
Genrating OpFlash 8 with 4 hits
Genrating OpFlash 9 with 3 hits
...
Generated 1135 OpFlashes
25-Jul-2025 06:21:50 EDT  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250209T190506Z_gen_003631_supernova_g4_halfActiveVol_detsim_reco_20250725T042521Z_reco2.root"
25-Jul-2025 06:21:50 EDT  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/c6/6f/prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250209T190506Z_gen_003631_supernova_g4_halfActiveVol_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              374.192       423.053       465.709       429.523       20.9938        50     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0075472     0.0386801     0.744656      0.0233743     0.102581        50     
reco:linecluster:LineCluster                           0.0743493      0.10096      0.270899      0.0942621     0.0318541       50     
reco:trajcluster:TrajCluster                           0.0220862     0.031888      0.0868051     0.026169      0.0144115       50     
reco:pandora:StandardPandora                            126.355       142.642       160.628       142.736       7.69276        50     
reco:pandoraTrack:LArPandoraTrackCreation             0.000336249   0.00685041     0.0155613    0.00788854     0.0036317       50     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000340726   0.00191814     0.0260238    0.00137525    0.00350647       50     
reco:pandoracalo:Calorimetry                          0.000231123   0.000644633   0.00602024    0.000516064   0.000799874      50     
reco:pandorapid:Chi2ParticleID                        5.7741e-05    0.00018529    0.00290047    0.000119515   0.000391199      50     
reco:pmtracktc:PMAlgTrackMaker                        0.00300968     0.0423394     0.257796      0.026654      0.051749        50     
reco:pmtracktccalo:Calorimetry                        0.000169166   0.000571278   0.00120478    0.000548182   0.000229076      50     
reco:pmtracktcpid:Chi2ParticleID                      4.8222e-05    0.000133842   0.000262667   0.000123446   4.87419e-05      50     
reco:emtrkmichelid:EmTrackMichelId                      8.01788       10.2538       12.8972       10.2122       1.11959        50     
reco:opdec:Deconvolution                                16.904        20.4488       22.9581       20.3343       1.22236        50     
reco:ophitspe:OpHitFinderDeco                           193.583       240.991       275.019       246.351       17.4777        50     
reco:opflash:OpFlashFinder                              6.8408        8.05693       9.32515       7.9782       0.700876        50     
reco:solarflash:SolarOpFlash                           0.151101      0.274212      0.381091      0.272268      0.0498496       50     
reco:rns:RandomNumberSaver                            6.2049e-05    9.41434e-05   0.000583382   8.0189e-05    7.27699e-05      50     
[art]:TriggerResults:TriggerResultInserter            1.7939e-05    3.12419e-05   5.8406e-05    2.8925e-05    7.41663e-06      50     
end_path:out1:RootOutput                               6.014e-06    9.14836e-06   2.7283e-05     8.366e-06    3.36648e-06      50     
end_path:out1:RootOutput(write)                        0.100394      0.160567      0.282561      0.157972      0.0304455       50     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3133.21 MB
  Peak resident set size usage (VmHWM): 1653.62 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250209T190506Z_gen_003631_supernova_g4_halfActiveVol_detsim_reco_20250725T042521Z_reco2.root
Extracted metadata
checking file====== fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250209T190506Z_gen_003631_supernova_g4_halfActiveVol_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250209T190506Z_gen_003631_supernova_g4_halfActiveVol_detsim_reco_20250725T042521Z_reco2.root
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250209T190506Z_gen_003631_supernova_g4_halfActiveVol_detsim_reco_20250725T042521Z_reco2.root.ext.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250209T190506Z_gen_003631_supernova_g4_halfActiveVol_detsim_reco_20250725T042521Z_reco2.root.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250209T190506Z_gen_003631_supernova_g4_halfActiveVol_detsim_reco_20250725T042521Z_reco2.root.temp.json
reco2_hist.root