Jobsub ID 17684.137@dunegpschedd01.fnal.gov

Jobsub ID	17684.137@dunegpschedd01.fnal.gov
Workflow ID	58
Stage ID	1
User name	epennacc@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-07-24 13:26:07
Site	UK_RAL-Tier1
Entry	LIGO_UK_RAL_arc_ce02
Last heartbeat	2025-07-25 03:08:49
From worker node	Hostname	dune001-7388867.0-lcg2462.gridpp.rl.ac.uk
	cpuinfo	AMD EPYC 7452 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	216000 (60 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-07-24 14:10:47
Input files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T171715Z_gen_002635_supernova_g4_detsim_reco.root
Jobscript	Exit code	0
	Real time	12h (44022s)
	CPU time	4h (16984s = 38%)
	Max RSS bytes	1619419136 (1544 MiB)
Outputting started	2025-07-25 02:24:30
Output files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T171715Z_gen_002635_supernova_g4_detsim_reco_20250724T141054Z_reco2.root
Finished	2025-07-25 03:08:49
Saved logs	justin-logs:17684.137-dunegpschedd01.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

U view candidate length: 15
Printing V view candidate length: 12
Printing W view candidate length: 11
Printing 2D view candidate length: 38
Printing background CaloHitList length: 662
Printing New CaloHitList Names: 
SCaloHitListU | SCaloHitListV | SCaloHitListW | SCaloHitList2D | BBCaloHitList
*Photon identified*
*Photon identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Photon identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
Printing U view candidate length: 9
Printing V view candidate length: 10
Printing W view candidate length: 7
Printing 2D view candidate length: 26
Printing background CaloHitList length: 12
Printing New CaloHitList Names: 
CaloHitListU | CaloHitListV | CaloHitListW | CaloHitList2D | BCaloHitList
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 02:12:39 UTC run: 5092 subRun: 0 event: 131749
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 02:12:39 UTC run: 5092 subRun: 0 event: 131749
Comparing two wires in the same plane: return failure
%MSG
56139
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 74052!
Genrating OpFlash 0 with 5 hits
Genrating OpFlash 1 with 3 hits
Genrating OpFlash 2 with 3 hits
Genrating OpFlash 3 with 4 hits
Genrating OpFlash 4 with 6 hits
Genrating OpFlash 5 with 3 hits
Genrating OpFlash 6 with 5 hits
Genrating OpFlash 7 with 5 hits
Genrating OpFlash 8 with 3 hits
Genrating OpFlash 9 with 3 hits
...
Generated 2227 OpFlashes
Begin processing the 50th record. run: 5092 subRun: 0 event: 131750 at 25-Jul-2025 02:16:50 UTC
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
Printing U view candidate length: 5
Printing V view candidate length: 8
Printing W view candidate length: 7
Printing 2D view candidate length: 20
Printing background CaloHitList length: 692
Printing New CaloHitList Names: 
SCaloHitListU | SCaloHitListV | SCaloHitListW | SCaloHitList2D | BBCaloHitList
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Photon identified*
Printing U view candidate length: 4
Printing V view candidate length: 8
Printing W view candidate length: 6
Printing 2D view candidate length: 18
Printing background CaloHitList length: 2
Printing New CaloHitList Names: 
CaloHitListU | CaloHitListV | CaloHitListW | CaloHitList2D | BCaloHitList
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  25-Jul-2025 02:19:51 UTC run: 5092 subRun: 0 event: 131750
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 02:19:51 UTC run: 5092 subRun: 0 event: 131750
Comparing two wires in the same plane: return failure
%MSG
55533
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 72876!
Genrating OpFlash 0 with 6 hits
Genrating OpFlash 1 with 3 hits
Genrating OpFlash 2 with 3 hits
Genrating OpFlash 3 with 3 hits
Genrating OpFlash 4 with 4 hits
Genrating OpFlash 5 with 4 hits
Genrating OpFlash 6 with 3 hits
Genrating OpFlash 7 with 5 hits
Genrating OpFlash 8 with 3 hits
Genrating OpFlash 9 with 7 hits
...
Generated 2296 OpFlashes
25-Jul-2025 02:24:22 UTC  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T171715Z_gen_002635_supernova_g4_detsim_reco_20250724T141054Z_reco2.root"
25-Jul-2025 02:24:22 UTC  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/a8/f5/prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T171715Z_gen_002635_supernova_g4_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              346.758       374.299       485.277       368.438       22.6242        50     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.133711       0.17222      0.297239      0.136005      0.0585949       50     
reco:linecluster:LineCluster                            1.21813       1.37973       2.23253       1.28782      0.222183        50     
reco:trajcluster:TrajCluster                           0.400203      0.407153       0.53583      0.403394      0.019146        50     
reco:pandora:StandardPandora                            122.801       139.643       251.342       135.26        17.9262        50     
reco:pandoraTrack:LArPandoraTrackCreation             0.000234622   0.00705054     0.0125777    0.00733242    0.00241826       50     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000211559   0.00142147     0.0109785    0.00126929    0.00141546       50     
reco:pandoracalo:Calorimetry                          0.000167386   0.000543629   0.00352798    0.000483206   0.000447536      50     
reco:pandorapid:Chi2ParticleID                        3.9715e-05    0.000136307   0.00122131    0.000117205   0.000157306      50     
reco:pmtracktc:PMAlgTrackMaker                        0.00303029     0.0375236     0.208792      0.0181009     0.0473051       50     
reco:pmtracktccalo:Calorimetry                        0.00017511    0.000501043   0.000881542   0.000487034   0.000144018      50     
reco:pmtracktcpid:Chi2ParticleID                      6.3259e-05    0.000119886   0.000311768   0.000114621   3.69735e-05      50     
reco:emtrkmichelid:EmTrackMichelId                      8.7081        10.2379       13.1236       10.1208      0.795001        50     
reco:opdec:Deconvolution                                22.6224       36.091        84.2379       32.0575       13.262         50     
reco:ophitspe:OpHitFinderDeco                           137.417       170.594       192.44        171.069       8.40362        50     
reco:opflash:OpFlashFinder                              13.5937       15.1601       17.4216       15.0943       0.62691        50     
reco:solarflash:SolarOpFlash                           0.285284      0.381678      0.451554      0.384875      0.0346395       50     
reco:rns:RandomNumberSaver                            5.8781e-05    0.000117363   0.000924944   9.2375e-05    0.000135409      50     
[art]:TriggerResults:TriggerResultInserter            2.5999e-05    6.38782e-05   0.000478713   3.6364e-05    9.83894e-05      50     
end_path:out1:RootOutput                               6.683e-06    9.44568e-06   2.8283e-05    8.6865e-06    3.04809e-06      50     
end_path:out1:RootOutput(write)                        0.153604       0.18352      0.226733      0.184436      0.013336        50     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3284.24 MB
  Peak resident set size usage (VmHWM): 1619.42 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T171715Z_gen_002635_supernova_g4_detsim_reco_20250724T141054Z_reco2.root
Extracted metadata
checking file====== fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T171715Z_gen_002635_supernova_g4_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T171715Z_gen_002635_supernova_g4_detsim_reco_20250724T141054Z_reco2.root
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T171715Z_gen_002635_supernova_g4_detsim_reco_20250724T141054Z_reco2.root.ext.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T171715Z_gen_002635_supernova_g4_detsim_reco_20250724T141054Z_reco2.root.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T171715Z_gen_002635_supernova_g4_detsim_reco_20250724T141054Z_reco2.root.temp.json
reco2_hist.root