Jobsub ID 17624.29@dunegpschedd01.fnal.gov

Jobsub ID	17624.29@dunegpschedd01.fnal.gov
Workflow ID	55
Stage ID	1
User name	higuera@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-07-24 09:29:52
Site	US_FNAL-T1
Entry	CMSHTPC_T1_US_FNAL_condce_opp1_whole
Last heartbeat	2025-07-24 22:40:02
From worker node	Hostname	dunegli-44332-0-cmswn3007.fnal.gov
	cpuinfo	AMD EPYC 7542 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-07-24 13:03:10
Input files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250206T095008Z_gen_000861_supernova_g4_detsim_reco.root
Jobscript	Exit code	0
	Real time	9h (34508s)
	CPU time	9h (34112s = 98%)
	Max RSS bytes	1729794048 (1649 MiB)
Outputting started	2025-07-24 22:38:18
Output files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250206T095008Z_gen_000861_supernova_g4_detsim_reco_20250724T130321Z_reco2.root
Finished	2025-07-24 22:40:02
Saved logs	justin-logs:17624.29-dunegpschedd01.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ting W view candidate length: 6
Printing 2D view candidate length: 20
Printing background CaloHitList length: 677
Printing New CaloHitList Names: 
SCaloHitListU | SCaloHitListV | SCaloHitListW | SCaloHitList2D | BBCaloHitList
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Photon identified*
*Photon identified*
*Photon identified*
*Photon identified*
*Photon identified*
*Electron identified*
*Photon identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
Printing U view candidate length: 8
Printing V view candidate length: 4
Printing W view candidate length: 6
Printing 2D view candidate length: 18
Printing background CaloHitList length: 3
Printing New CaloHitList Names: 
CaloHitListU | CaloHitListV | CaloHitListW | CaloHitList2D | BCaloHitList
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  24-Jul-2025 22:23:08 UTC run: 5082 subRun: 0 event: 43049
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 22:23:08 UTC run: 5082 subRun: 0 event: 43049
Comparing two wires in the same plane: return failure
%MSG
54869
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 71737!
Genrating OpFlash 0 with 7 hits
Genrating OpFlash 1 with 4 hits
Genrating OpFlash 2 with 4 hits
Genrating OpFlash 3 with 3 hits
Genrating OpFlash 4 with 3 hits
Genrating OpFlash 5 with 3 hits
Genrating OpFlash 6 with 3 hits
Genrating OpFlash 7 with 4 hits
Genrating OpFlash 8 with 6 hits
Genrating OpFlash 9 with 3 hits
...
Generated 2299 OpFlashes
Begin processing the 50th record. run: 5082 subRun: 0 event: 43050 at 24-Jul-2025 22:28:02 UTC
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
Printing U view candidate length: 12
Printing V view candidate length: 6
Printing W view candidate length: 8
Printing 2D view candidate length: 26
Printing background CaloHitList length: 670
Printing New CaloHitList Names: 
SCaloHitListU | SCaloHitListV | SCaloHitListW | SCaloHitList2D | BBCaloHitList
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Photon identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
Printing U view candidate length: 8
Printing V view candidate length: 6
Printing W view candidate length: 5
Printing 2D view candidate length: 19
Printing background CaloHitList length: 7
Printing New CaloHitList Names: 
CaloHitListU | CaloHitListV | CaloHitListW | CaloHitList2D | BCaloHitList
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  24-Jul-2025 22:34:15 UTC run: 5082 subRun: 0 event: 43050
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 22:34:15 UTC run: 5082 subRun: 0 event: 43050
Comparing two wires in the same plane: return failure
%MSG
54909
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 70947!
Genrating OpFlash 0 with 3 hits
Genrating OpFlash 1 with 3 hits
Genrating OpFlash 2 with 3 hits
Genrating OpFlash 3 with 5 hits
Genrating OpFlash 4 with 3 hits
Genrating OpFlash 5 with 4 hits
Genrating OpFlash 6 with 3 hits
Genrating OpFlash 7 with 3 hits
Genrating OpFlash 8 with 3 hits
Genrating OpFlash 9 with 3 hits
...
Generated 2144 OpFlashes
24-Jul-2025 22:38:11 UTC  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250206T095008Z_gen_000861_supernova_g4_detsim_reco_20250724T130321Z_reco2.root"
24-Jul-2025 22:38:11 UTC  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/f7/d0/prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250206T095008Z_gen_000861_supernova_g4_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              530.51        685.736       777.245       697.057       57.6519        50     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00569065     0.0224097     0.0509792     0.0208109     0.0094159       50     
reco:linecluster:LineCluster                           0.0386634     0.0781946     0.709804      0.065734      0.0912805       50     
reco:trajcluster:TrajCluster                          0.00993338     0.0200431     0.0481767     0.0185053    0.00852394       50     
reco:pandora:StandardPandora                            246.488       398.809       481.59        409.725       54.3559        50     
reco:pandoraTrack:LArPandoraTrackCreation             0.000599016    0.0179443     0.036216      0.0197287     0.0104082       50     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000639894   0.00340347     0.0281994    0.00218109    0.00465372       50     
reco:pandoracalo:Calorimetry                          0.000348647    0.0010238    0.00491387    0.00075027    0.000793311      50     
reco:pandorapid:Chi2ParticleID                         7.861e-05    0.000367646   0.00364892    0.000195354   0.000554701      50     
reco:pmtracktc:PMAlgTrackMaker                        0.00324617     0.0532882      0.61028      0.026121      0.0896153       50     
reco:pmtracktccalo:Calorimetry                        0.000211404    0.0010747    0.00331083    0.000795516   0.000661853      50     
reco:pmtracktcpid:Chi2ParticleID                      7.0705e-05    0.000225857   0.000567375   0.00018851    0.000114208      50     
reco:emtrkmichelid:EmTrackMichelId                      17.4286       30.1267       40.3261       31.3812       5.94268        50     
reco:opdec:Deconvolution                                19.957        26.3244       34.184        26.5269       3.85837        50     
reco:ophitspe:OpHitFinderDeco                           154.76        176.964       206.205       173.889       12.3261        50     
reco:opflash:OpFlashFinder                              32.4572       52.4372       74.852        51.1755       10.1345        50     
reco:solarflash:SolarOpFlash                           0.376189      0.592269       1.25327      0.596699      0.174132        50     
reco:rns:RandomNumberSaver                            8.0013e-05    0.00020428    0.00206335    0.000140273   0.000280277      50     
[art]:TriggerResults:TriggerResultInserter            2.7864e-05    6.11276e-05   0.000321315   4.85275e-05   4.6611e-05       50     
end_path:out1:RootOutput                              1.3065e-05    2.37871e-05   7.9162e-05    2.04635e-05   1.26393e-05      50     
end_path:out1:RootOutput(write)                        0.184234      0.279811      0.420709      0.267307      0.0535093       50     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3259.78 MB
  Peak resident set size usage (VmHWM): 1729.79 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250206T095008Z_gen_000861_supernova_g4_detsim_reco_20250724T130321Z_reco2.root
Extracted metadata
checking file====== fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250206T095008Z_gen_000861_supernova_g4_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
old_md.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250206T095008Z_gen_000861_supernova_g4_detsim_reco_20250724T130321Z_reco2.root
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250206T095008Z_gen_000861_supernova_g4_detsim_reco_20250724T130321Z_reco2.root.ext.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250206T095008Z_gen_000861_supernova_g4_detsim_reco_20250724T130321Z_reco2.root.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250206T095008Z_gen_000861_supernova_g4_detsim_reco_20250724T130321Z_reco2.root.temp.json
reco2_hist.root