Jobsub ID 17624.71@dunegpschedd01.fnal.gov

Jobsub ID	17624.71@dunegpschedd01.fnal.gov
Workflow ID	55
Stage ID	1
User name	higuera@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-07-24 09:29:52
Site	US_FNAL-T1
Entry	CMSHTPC_T1_US_FNAL_condce_opp1_whole
Last heartbeat	2025-07-24 22:44:18
From worker node	Hostname	dunegli-44332-0-cmswn3007.fnal.gov
	cpuinfo	AMD EPYC 7542 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-07-24 13:04:30
Input files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250208T121420Z_gen_002239_supernova_g4_detsim_reco.root
Jobscript	Exit code	0
	Real time	9h (34664s)
	CPU time	9h (34264s = 98%)
	Max RSS bytes	1724194816 (1644 MiB)
Outputting started	2025-07-24 22:42:15
Output files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250208T121420Z_gen_002239_supernova_g4_detsim_reco_20250724T130441Z_reco2.root
Finished	2025-07-24 22:44:18
Saved logs	justin-logs:17624.71-dunegpschedd01.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

tified*
*Photon identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
Printing U view candidate length: 12
Printing V view candidate length: 14
Printing W view candidate length: 11
Printing 2D view candidate length: 37
Printing background CaloHitList length: 0
Printing New CaloHitList Names: 
CaloHitListU | CaloHitListV | CaloHitListW | CaloHitList2D | BCaloHitList
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 22:29:31 UTC run: 5097 subRun: 0 event: 111949
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 22:29:31 UTC run: 5097 subRun: 0 event: 111949
Comparing two wires in the same plane: return failure
%MSG
54074
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 69862!
Genrating OpFlash 0 with 3 hits
Genrating OpFlash 1 with 5 hits
Genrating OpFlash 2 with 4 hits
Genrating OpFlash 3 with 4 hits
Genrating OpFlash 4 with 3 hits
Genrating OpFlash 5 with 7 hits
Genrating OpFlash 6 with 3 hits
Genrating OpFlash 7 with 3 hits
Genrating OpFlash 8 with 4 hits
Genrating OpFlash 9 with 4 hits
...
Generated 2038 OpFlashes
Begin processing the 50th record. run: 5097 subRun: 0 event: 111950 at 24-Jul-2025 22:33:43 UTC
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
Printing U view candidate length: 12
Printing V view candidate length: 10
Printing W view candidate length: 11
Printing 2D view candidate length: 33
Printing background CaloHitList length: 617
Printing New CaloHitList Names: 
SCaloHitListU | SCaloHitListV | SCaloHitListW | SCaloHitList2D | BBCaloHitList
*Electron identified*
*Photon identified*
*Photon identified*
*Photon identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Electron identified*
*Photon identified*
*Photon identified*
*Electron identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Electron identified*
*Photon identified*
*Electron identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
Printing U view candidate length: 12
Printing V view candidate length: 9
Printing W view candidate length: 11
Printing 2D view candidate length: 32
Printing background CaloHitList length: 1
Printing New CaloHitList Names: 
CaloHitListU | CaloHitListV | CaloHitListW | CaloHitList2D | BCaloHitList
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  24-Jul-2025 22:37:52 UTC run: 5097 subRun: 0 event: 111950
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 22:37:52 UTC run: 5097 subRun: 0 event: 111950
Comparing two wires in the same plane: return failure
%MSG
54948
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 71144!
Genrating OpFlash 0 with 3 hits
Genrating OpFlash 1 with 4 hits
Genrating OpFlash 2 with 3 hits
Genrating OpFlash 3 with 3 hits
Genrating OpFlash 4 with 4 hits
Genrating OpFlash 5 with 4 hits
Genrating OpFlash 6 with 4 hits
Genrating OpFlash 7 with 4 hits
Genrating OpFlash 8 with 3 hits
Genrating OpFlash 9 with 3 hits
...
Generated 2162 OpFlashes
24-Jul-2025 22:42:07 UTC  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250208T121420Z_gen_002239_supernova_g4_detsim_reco_20250724T130441Z_reco2.root"
24-Jul-2025 22:42:07 UTC  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/a8/30/prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250208T121420Z_gen_002239_supernova_g4_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                                502         688.715       774.902       690.549       52.4887        50     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00559536     0.0327615     0.204737      0.0239142     0.0379946       50     
reco:linecluster:LineCluster                           0.0334908     0.0688175     0.109065      0.0667267     0.0139794       50     
reco:trajcluster:TrajCluster                           0.0101029     0.0197257      0.05932      0.017705     0.00881417       50     
reco:pandora:StandardPandora                            247.601       404.821       487.444       408.186       48.2474        50     
reco:pandoraTrack:LArPandoraTrackCreation             0.000382311    0.0191796     0.0392156     0.0196442     0.0116022       50     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000327156   0.00352439     0.0338592    0.00205244    0.00570363       50     
reco:pandoracalo:Calorimetry                          0.000245499   0.00111294     0.0104935    0.000727561   0.00158426       50     
reco:pandorapid:Chi2ParticleID                        5.3703e-05    0.000294474   0.00383081    0.000181637   0.000524267      50     
reco:pmtracktc:PMAlgTrackMaker                        0.00476385     0.0455121     0.219128      0.0286514     0.0489007       50     
reco:pmtracktccalo:Calorimetry                        0.000170766   0.000914609   0.00227903    0.000805851   0.000455578      50     
reco:pmtracktcpid:Chi2ParticleID                      4.8994e-05    0.00019098    0.000534873   0.000181016   9.14083e-05      50     
reco:emtrkmichelid:EmTrackMichelId                      14.9743       27.8077       37.281        28.8988       5.52694        50     
reco:opdec:Deconvolution                                20.3194       26.6497       35.1301       26.1445       4.34469        50     
reco:ophitspe:OpHitFinderDeco                           159.108       177.627       204.616       174.096       12.116         50     
reco:opflash:OpFlashFinder                              30.2471       50.7137       75.5954       50.9753       9.80176        50     
reco:solarflash:SolarOpFlash                           0.312865      0.611124       1.36162       0.56139      0.208108        50     
reco:rns:RandomNumberSaver                            8.7958e-05    0.000198291   0.00158996    0.000150102   0.000209649      50     
[art]:TriggerResults:TriggerResultInserter            2.5399e-05    6.42626e-05   0.000180756   5.45395e-05   3.3545e-05       50     
end_path:out1:RootOutput                              1.3045e-05    2.85384e-05   0.000124398   2.3946e-05    1.68335e-05      50     
end_path:out1:RootOutput(write)                        0.177021       0.28871      0.446007      0.269918      0.0636915       50     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3252.88 MB
  Peak resident set size usage (VmHWM): 1724.19 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250208T121420Z_gen_002239_supernova_g4_detsim_reco_20250724T130441Z_reco2.root
Extracted metadata
checking file====== fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250208T121420Z_gen_002239_supernova_g4_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
old_md.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250208T121420Z_gen_002239_supernova_g4_detsim_reco_20250724T130441Z_reco2.root
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250208T121420Z_gen_002239_supernova_g4_detsim_reco_20250724T130441Z_reco2.root.ext.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250208T121420Z_gen_002239_supernova_g4_detsim_reco_20250724T130441Z_reco2.root.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250208T121420Z_gen_002239_supernova_g4_detsim_reco_20250724T130441Z_reco2.root.temp.json
reco2_hist.root