Jobsub ID 17624.74@dunegpschedd01.fnal.gov

Jobsub ID	17624.74@dunegpschedd01.fnal.gov
Workflow ID	55
Stage ID	1
User name	higuera@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-07-24 09:29:52
Site	US_FNAL-T1
Entry	CMSHTPC_T1_US_FNAL_condce_opp1_whole
Last heartbeat	2025-07-24 22:42:53
From worker node	Hostname	dunegli-44332-0-cmswn3007.fnal.gov
	cpuinfo	AMD EPYC 7542 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-07-24 13:04:33
Input files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T091325Z_gen_001581_supernova_g4_detsim_reco.root
Jobscript	Exit code	0
	Real time	9h (34574s)
	CPU time	9h (34173s = 98%)
	Max RSS bytes	1721126912 (1641 MiB)
Outputting started	2025-07-24 22:40:48
Output files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T091325Z_gen_001581_supernova_g4_detsim_reco_20250724T130444Z_reco2.root
Finished	2025-07-24 22:42:53
Saved logs	justin-logs:17624.74-dunegpschedd01.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

3 hits
Genrating OpFlash 3 with 4 hits
Genrating OpFlash 4 with 3 hits
Genrating OpFlash 5 with 3 hits
Genrating OpFlash 6 with 3 hits
Genrating OpFlash 7 with 3 hits
Genrating OpFlash 8 with 4 hits
Genrating OpFlash 9 with 3 hits
...
Generated 2186 OpFlashes
Begin processing the 50th record. run: 5089 subRun: 0 event: 79050 at 24-Jul-2025 22:31:00 UTC
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
Printing U view candidate length: 16
Printing V view candidate length: 18
Printing W view candidate length: 14
Printing 2D view candidate length: 48
Printing background CaloHitList length: 686
Printing New CaloHitList Names: 
SCaloHitListU | SCaloHitListV | SCaloHitListW | SCaloHitList2D | BBCaloHitList
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Electron identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Photon identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
Printing U view candidate length: 12
Printing V view candidate length: 15
Printing W view candidate length: 13
Printing 2D view candidate length: 40
Printing background CaloHitList length: 5
Printing New CaloHitList Names: 
CaloHitListU | CaloHitListV | CaloHitListW | CaloHitList2D | BCaloHitList
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 22:36:38 UTC run: 5089 subRun: 0 event: 79050
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 22:36:38 UTC run: 5089 subRun: 0 event: 79050
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 22:36:38 UTC run: 5089 subRun: 0 event: 79050
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 22:36:38 UTC run: 5089 subRun: 0 event: 79050
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 22:36:38 UTC run: 5089 subRun: 0 event: 79050
Comparing two wires in the same plane: return failure
%MSG
54503
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 70949!
Genrating OpFlash 0 with 3 hits
Genrating OpFlash 1 with 3 hits
Genrating OpFlash 2 with 6 hits
Genrating OpFlash 3 with 4 hits
Genrating OpFlash 4 with 3 hits
Genrating OpFlash 5 with 12 hits
Genrating OpFlash 6 with 3 hits
Genrating OpFlash 7 with 3 hits
Genrating OpFlash 8 with 5 hits
Genrating OpFlash 9 with 3 hits
...
Generated 2222 OpFlashes
24-Jul-2025 22:40:40 UTC  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T091325Z_gen_001581_supernova_g4_detsim_reco_20250724T130444Z_reco2.root"
24-Jul-2025 22:40:40 UTC  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/34/48/prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T091325Z_gen_001581_supernova_g4_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              521.552       686.808       797.172       690.055       50.4542        50     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0057942     0.0242331     0.0463714     0.0236847    0.00816049       50     
reco:linecluster:LineCluster                           0.0325978     0.0685872     0.104164      0.0683075     0.0146421       50     
reco:trajcluster:TrajCluster                          0.00991973     0.0186078     0.0908021     0.0159519     0.0120532       50     
reco:pandora:StandardPandora                            249.063       403.566       490.751       406.834       47.2646        50     
reco:pandoraTrack:LArPandoraTrackCreation             0.000435483    0.0197889     0.0456073     0.0203306     0.011217        50     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000379736   0.00302908     0.0207039    0.00203052    0.00368426       50     
reco:pandoracalo:Calorimetry                          0.000272361   0.000997293   0.00803493    0.000712503   0.00110675       50     
reco:pandorapid:Chi2ParticleID                        6.4794e-05    0.000295557   0.00306074    0.000186356   0.00042244       50     
reco:pmtracktc:PMAlgTrackMaker                        0.00288714     0.0517111     0.200786       0.03007      0.0482575       50     
reco:pmtracktccalo:Calorimetry                         0.0001625    0.000982381   0.00198858    0.000848523   0.00046968       50     
reco:pmtracktcpid:Chi2ParticleID                      5.4574e-05    0.000218498   0.000796303    0.0001749    0.000137702      50     
reco:emtrkmichelid:EmTrackMichelId                      15.2813       29.1782       46.4231       29.5668       6.62838        50     
reco:opdec:Deconvolution                                19.8978       26.5728       37.7081       26.2477       4.32425        50     
reco:ophitspe:OpHitFinderDeco                           152.406       174.47        199.45        173.09        9.83016        50     
reco:opflash:OpFlashFinder                              21.6291       51.9838       72.6292       55.7557       11.9842        50     
reco:solarflash:SolarOpFlash                           0.323931       0.56654       1.06977      0.517633      0.167481        50     
reco:rns:RandomNumberSaver                            7.4673e-05    0.000218833   0.00271461    0.000129964   0.000366664      50     
[art]:TriggerResults:TriggerResultInserter            2.9186e-05    6.79005e-05   0.000246371   5.2035e-05    4.42272e-05      50     
end_path:out1:RootOutput                              1.2695e-05    2.7927e-05    0.00021927    2.07345e-05   2.98644e-05      50     
end_path:out1:RootOutput(write)                        0.189011      0.277698      0.423331      0.263819      0.057804        50     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3253.35 MB
  Peak resident set size usage (VmHWM): 1721.13 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T091325Z_gen_001581_supernova_g4_detsim_reco_20250724T130444Z_reco2.root
Extracted metadata
checking file====== fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T091325Z_gen_001581_supernova_g4_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
old_md.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T091325Z_gen_001581_supernova_g4_detsim_reco_20250724T130444Z_reco2.root
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T091325Z_gen_001581_supernova_g4_detsim_reco_20250724T130444Z_reco2.root.ext.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T091325Z_gen_001581_supernova_g4_detsim_reco_20250724T130444Z_reco2.root.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T091325Z_gen_001581_supernova_g4_detsim_reco_20250724T130444Z_reco2.root.temp.json
reco2_hist.root