Jobsub ID 17624.42@dunegpschedd01.fnal.gov

Jobsub ID	17624.42@dunegpschedd01.fnal.gov
Workflow ID	55
Stage ID	1
User name	higuera@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-07-24 09:29:52
Site	US_FNAL-T1
Entry	CMSHTPC_T1_US_FNAL_condce_opp1_whole
Last heartbeat	2025-07-24 22:40:00
From worker node	Hostname	dunegli-44332-0-cmswn3007.fnal.gov
	cpuinfo	AMD EPYC 7542 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-07-24 13:03:49
Input files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T082306Z_gen_000848_supernova_g4_detsim_reco.root
Jobscript	Exit code	0
	Real time	9h (34468s)
	CPU time	9h (34070s = 98%)
	Max RSS bytes	1721012224 (1641 MiB)
Outputting started	2025-07-24 22:38:18
Output files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T082306Z_gen_000848_supernova_g4_detsim_reco_20250724T130401Z_reco2.root
Finished	2025-07-24 22:40:00
Saved logs	justin-logs:17624.42-dunegpschedd01.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

nal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
Printing U view candidate length: 24
Printing V view candidate length: 33
Printing W view candidate length: 19
Printing 2D view candidate length: 76
Printing background CaloHitList length: 672
Printing New CaloHitList Names: 
SCaloHitListU | SCaloHitListV | SCaloHitListW | SCaloHitList2D | BBCaloHitList
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Photon identified*
*Photon identified*
*Photon identified*
*Photon identified*
*Photon identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
Printing U view candidate length: 24
Printing V view candidate length: 33
Printing W view candidate length: 19
Printing 2D view candidate length: 76
Printing background CaloHitList length: 0
Printing New CaloHitList Names: 
CaloHitListU | CaloHitListV | CaloHitListW | CaloHitList2D | BCaloHitList
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 22:34:03 UTC run: 5089 subRun: 0 event: 42400
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 22:34:03 UTC run: 5089 subRun: 0 event: 42400
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 22:34:03 UTC run: 5089 subRun: 0 event: 42400
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 22:34:03 UTC run: 5089 subRun: 0 event: 42400
Comparing two wires in the same plane: return failure
%MSG
55581
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 72801!
Genrating OpFlash 0 with 3 hits
Genrating OpFlash 1 with 5 hits
Genrating OpFlash 2 with 3 hits
Genrating OpFlash 3 with 3 hits
Genrating OpFlash 4 with 3 hits
Genrating OpFlash 5 with 3 hits
Genrating OpFlash 6 with 3 hits
Genrating OpFlash 7 with 3 hits
Genrating OpFlash 8 with 3 hits
Genrating OpFlash 9 with 3 hits
...
Generated 2291 OpFlashes
24-Jul-2025 22:38:11 UTC  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T082306Z_gen_000848_supernova_g4_detsim_reco_20250724T130401Z_reco2.root"
24-Jul-2025 22:38:11 UTC  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/f8/39/prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T082306Z_gen_000848_supernova_g4_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              578.165       684.814       785.362       687.416       48.1477        50     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00546902     0.0183038     0.0351048     0.0179376    0.00641449       50     
reco:linecluster:LineCluster                           0.0310124     0.0614633     0.113316      0.0602116     0.0143237       50     
reco:trajcluster:TrajCluster                           0.0101223     0.0193592     0.104286      0.015717      0.0143335       50     
reco:pandora:StandardPandora                            294.759       398.352       478.55        403.419       39.4508        50     
reco:pandoraTrack:LArPandoraTrackCreation             0.000479307    0.0195413     0.0370544     0.0212643     0.0095822       50     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000465299   0.00284937     0.0213753    0.00215449    0.00290846       50     
reco:pandoracalo:Calorimetry                          0.000327316   0.00108874    0.00810759    0.000835839   0.00110978       50     
reco:pandorapid:Chi2ParticleID                        8.0063e-05    0.000312841   0.00360787    0.000216069   0.00048702       50     
reco:pmtracktc:PMAlgTrackMaker                        0.00395646     0.0635701     0.330214      0.0333238     0.0794944       50     
reco:pmtracktccalo:Calorimetry                        0.000241863   0.00114848    0.00369909    0.000970937   0.000655008      50     
reco:pmtracktcpid:Chi2ParticleID                      6.7839e-05    0.000235433   0.000655854   0.000183546   0.000134658      50     
reco:emtrkmichelid:EmTrackMichelId                      17.1713       29.3724       42.1154       29.6691       5.53597        50     
reco:opdec:Deconvolution                                20.2826       26.9869       40.1569       26.7642       4.40442        50     
reco:ophitspe:OpHitFinderDeco                           159.802       178.674       203.303       177.596       10.4523        50     
reco:opflash:OpFlashFinder                              26.0547       50.3956       68.1531       53.1605       10.9408        50     
reco:solarflash:SolarOpFlash                           0.349258      0.559087       1.04008      0.562055      0.165591        50     
reco:rns:RandomNumberSaver                            9.2898e-05    0.000177995   0.00104106    0.000138018   0.000139417      50     
[art]:TriggerResults:TriggerResultInserter            2.8775e-05    6.16378e-05   0.000240059   4.57725e-05   3.87295e-05      50     
end_path:out1:RootOutput                               8.647e-06    2.23123e-05   7.8871e-05    1.91115e-05   1.06604e-05      50     
end_path:out1:RootOutput(write)                        0.196095      0.282399      0.437889      0.268183       0.05824        50     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3258.99 MB
  Peak resident set size usage (VmHWM): 1721.01 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T082306Z_gen_000848_supernova_g4_detsim_reco_20250724T130401Z_reco2.root
Extracted metadata
checking file====== fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T082306Z_gen_000848_supernova_g4_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
old_md.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T082306Z_gen_000848_supernova_g4_detsim_reco_20250724T130401Z_reco2.root
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T082306Z_gen_000848_supernova_g4_detsim_reco_20250724T130401Z_reco2.root.ext.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T082306Z_gen_000848_supernova_g4_detsim_reco_20250724T130401Z_reco2.root.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T082306Z_gen_000848_supernova_g4_detsim_reco_20250724T130401Z_reco2.root.temp.json
reco2_hist.root