Jobsub ID 17624.28@dunegpschedd01.fnal.gov

Jobsub ID	17624.28@dunegpschedd01.fnal.gov
Workflow ID	55
Stage ID	1
User name	higuera@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-07-24 09:29:52
Site	US_FNAL-T1
Entry	CMSHTPC_T1_US_FNAL_condce_opp1_whole
Last heartbeat	2025-07-24 22:38:16
From worker node	Hostname	dunegli-44332-0-cmswn3007.fnal.gov
	cpuinfo	AMD EPYC 7542 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-07-24 13:03:05
Input files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T084711Z_gen_001222_supernova_g4_detsim_reco.root
Jobscript	Exit code	0
	Real time	9h (34393s)
	CPU time	9h (33981s = 98%)
	Max RSS bytes	1726595072 (1646 MiB)
Outputting started	2025-07-24 22:36:19
Output files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T084711Z_gen_001222_supernova_g4_detsim_reco_20250724T130316Z_reco2.root
Finished	2025-07-24 22:38:16
Saved logs	justin-logs:17624.28-dunegpschedd01.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ntified*
*Electron identified*
*Electron identified*
*Photon identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
Printing U view candidate length: 16
Printing V view candidate length: 13
Printing W view candidate length: 10
Printing 2D view candidate length: 39
Printing background CaloHitList length: 6
Printing New CaloHitList Names: 
CaloHitListU | CaloHitListV | CaloHitListW | CaloHitList2D | BCaloHitList
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 22:20:18 UTC run: 5089 subRun: 0 event: 61099
Comparing two wires in the same plane: return failure
%MSG
55251
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 71986!
Genrating OpFlash 0 with 3 hits
Genrating OpFlash 1 with 3 hits
Genrating OpFlash 2 with 5 hits
Genrating OpFlash 3 with 4 hits
Genrating OpFlash 4 with 3 hits
Genrating OpFlash 5 with 9 hits
Genrating OpFlash 6 with 5 hits
Genrating OpFlash 7 with 4 hits
Genrating OpFlash 8 with 4 hits
Genrating OpFlash 9 with 3 hits
...
Generated 2225 OpFlashes
Begin processing the 50th record. run: 5089 subRun: 0 event: 61100 at 24-Jul-2025 22:24:53 UTC
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
Printing U view candidate length: 5
Printing V view candidate length: 4
Printing W view candidate length: 9
Printing 2D view candidate length: 18
Printing background CaloHitList length: 544
Printing New CaloHitList Names: 
SCaloHitListU | SCaloHitListV | SCaloHitListW | SCaloHitList2D | BBCaloHitList
*Photon identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Photon identified*
*Photon identified*
*Photon identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
Printing U view candidate length: 5
Printing V view candidate length: 4
Printing W view candidate length: 8
Printing 2D view candidate length: 17
Printing background CaloHitList length: 4
Printing New CaloHitList Names: 
CaloHitListU | CaloHitListV | CaloHitListW | CaloHitList2D | BCaloHitList
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  24-Jul-2025 22:32:07 UTC run: 5089 subRun: 0 event: 61100
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  24-Jul-2025 22:32:07 UTC run: 5089 subRun: 0 event: 61100
Trying to add data product: ShowerPCA. This element does not exist in the element holder
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 22:32:07 UTC run: 5089 subRun: 0 event: 61100
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 22:32:07 UTC run: 5089 subRun: 0 event: 61100
Comparing two wires in the same plane: return failure
%MSG
54099
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 69543!
Genrating OpFlash 0 with 4 hits
Genrating OpFlash 1 with 7 hits
Genrating OpFlash 2 with 3 hits
Genrating OpFlash 3 with 6 hits
Genrating OpFlash 4 with 5 hits
Genrating OpFlash 5 with 3 hits
Genrating OpFlash 6 with 5 hits
Genrating OpFlash 7 with 7 hits
Genrating OpFlash 8 with 3 hits
Genrating OpFlash 9 with 5 hits
...
Generated 2092 OpFlashes
24-Jul-2025 22:36:11 UTC  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T084711Z_gen_001222_supernova_g4_detsim_reco_20250724T130316Z_reco2.root"
24-Jul-2025 22:36:11 UTC  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/49/d2/prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T084711Z_gen_001222_supernova_g4_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              544.72        683.29        778.653       687.545       48.9686        50     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00364474     0.0236239     0.0428542      0.02086     0.00812398       50     
reco:linecluster:LineCluster                           0.0317188     0.0601831     0.124435      0.0589424     0.0136219       50     
reco:trajcluster:TrajCluster                           0.0092698     0.0191634     0.101928      0.0162847     0.0140482       50     
reco:pandora:StandardPandora                            277.333       399.328       495.854       403.741       46.144         50     
reco:pandoraTrack:LArPandoraTrackCreation             0.000431084    0.0188807     0.0383496     0.0200317     0.0102844       50     
reco:pandoraShower:LArPandoraModularShowerCreation    0.00037114    0.00330789     0.0287171    0.00241254    0.00463613       50     
reco:pandoracalo:Calorimetry                          0.000269225   0.000964089   0.00672767    0.000703215   0.000932653      50     
reco:pandorapid:Chi2ParticleID                        7.1938e-05    0.000249263   0.00233398    0.000190129   0.000310883      50     
reco:pmtracktc:PMAlgTrackMaker                         0.0053259     0.0588791     0.464678      0.0311726     0.0748278       50     
reco:pmtracktccalo:Calorimetry                        0.000159465   0.000972151   0.00242133    0.000805871   0.000480951      50     
reco:pmtracktcpid:Chi2ParticleID                      5.8532e-05    0.000213117   0.000555142   0.000182674   0.000103533      50     
reco:emtrkmichelid:EmTrackMichelId                      14.3708       28.0594       44.2167       27.9033       5.98737        50     
reco:opdec:Deconvolution                                20.6635       25.6461       37.3313       24.7758       4.45656        50     
reco:ophitspe:OpHitFinderDeco                           159.122       177.716       195.178       176.782       8.20951        50     
reco:opflash:OpFlashFinder                              26.6117       51.4642       68.7961       51.2751       10.0363        50     
reco:solarflash:SolarOpFlash                           0.348083      0.601094       1.49508      0.529589       0.21356        50     
reco:rns:RandomNumberSaver                            8.5093e-05    0.000178401   0.000992388   0.000146115   0.000129539      50     
[art]:TriggerResults:TriggerResultInserter            3.5197e-05    6.79782e-05   0.000124728   5.97295e-05   2.55339e-05      50     
end_path:out1:RootOutput                              1.2153e-05    2.19961e-05   6.0556e-05    1.86105e-05   8.99444e-06      50     
end_path:out1:RootOutput(write)                        0.192884      0.284545      0.459894      0.274911      0.0562505       50     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3253.68 MB
  Peak resident set size usage (VmHWM): 1726.6 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T084711Z_gen_001222_supernova_g4_detsim_reco_20250724T130316Z_reco2.root
Extracted metadata
checking file====== fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T084711Z_gen_001222_supernova_g4_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
old_md.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T084711Z_gen_001222_supernova_g4_detsim_reco_20250724T130316Z_reco2.root
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T084711Z_gen_001222_supernova_g4_detsim_reco_20250724T130316Z_reco2.root.ext.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T084711Z_gen_001222_supernova_g4_detsim_reco_20250724T130316Z_reco2.root.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T084711Z_gen_001222_supernova_g4_detsim_reco_20250724T130316Z_reco2.root.temp.json
reco2_hist.root