Jobsub ID 17623.177@dunegpschedd01.fnal.gov

Jobsub ID	17623.177@dunegpschedd01.fnal.gov
Workflow ID	55
Stage ID	1
User name	higuera@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-07-24 09:21:52
Site	US_FNAL-T1
Entry	CMSHTPC_T1_US_FNAL_condce_opp1_whole
Last heartbeat	2025-07-24 22:59:00
From worker node	Hostname	dunegli-44332-0-cmswn3007.fnal.gov
	cpuinfo	AMD EPYC 7542 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-07-24 13:01:30
Input files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250208T114518Z_gen_001676_supernova_g4_detsim_reco.root
Jobscript	Exit code	0
	Real time	9h (35733s)
	CPU time	9h (35366s = 98%)
	Max RSS bytes	1726595072 (1646 MiB)
Outputting started	2025-07-24 22:57:04
Output files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250208T114518Z_gen_001676_supernova_g4_detsim_reco_20250724T130140Z_reco2.root
Finished	2025-07-24 22:59:00
Saved logs	justin-logs:17623.177-dunegpschedd01.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ed*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
Printing U view candidate length: 29
Printing V view candidate length: 30
Printing W view candidate length: 24
Printing 2D view candidate length: 83
Printing background CaloHitList length: 732
Printing New CaloHitList Names: 
SCaloHitListU | SCaloHitListV | SCaloHitListW | SCaloHitList2D | BBCaloHitList
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
Printing U view candidate length: 25
Printing V view candidate length: 25
Printing W view candidate length: 21
Printing 2D view candidate length: 71
Printing background CaloHitList length: 0
Printing New CaloHitList Names: 
CaloHitListU | CaloHitListV | CaloHitListW | CaloHitList2D | BCaloHitList
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 22:52:52 UTC run: 5097 subRun: 0 event: 83800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 22:52:52 UTC run: 5097 subRun: 0 event: 83800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 22:52:52 UTC run: 5097 subRun: 0 event: 83800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 22:52:52 UTC run: 5097 subRun: 0 event: 83800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 22:52:52 UTC run: 5097 subRun: 0 event: 83800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 22:52:52 UTC run: 5097 subRun: 0 event: 83800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 22:52:52 UTC run: 5097 subRun: 0 event: 83800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 22:52:52 UTC run: 5097 subRun: 0 event: 83800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 22:52:52 UTC run: 5097 subRun: 0 event: 83800
Comparing two wires in the same plane: return failure
%MSG
54879
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 71250!
Genrating OpFlash 0 with 4 hits
Genrating OpFlash 1 with 3 hits
Genrating OpFlash 2 with 9 hits
Genrating OpFlash 3 with 3 hits
Genrating OpFlash 4 with 4 hits
Genrating OpFlash 5 with 7 hits
Genrating OpFlash 6 with 3 hits
Genrating OpFlash 7 with 4 hits
Genrating OpFlash 8 with 3 hits
Genrating OpFlash 9 with 3 hits
...
Generated 2236 OpFlashes
24-Jul-2025 22:56:56 UTC  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250208T114518Z_gen_001676_supernova_g4_detsim_reco_20250724T130140Z_reco2.root"
24-Jul-2025 22:56:56 UTC  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/00/d3/prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250208T114518Z_gen_001676_supernova_g4_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              590.778       710.548       822.538       720.286       61.0034        50     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00511559     0.0196913     0.0466742     0.0192267    0.00858188       50     
reco:linecluster:LineCluster                           0.0332554     0.062497      0.137569      0.0588961     0.0167136       50     
reco:trajcluster:TrajCluster                           0.0103692     0.0195415     0.0605345     0.0154045     0.0105559       50     
reco:pandora:StandardPandora                            287.434       419.424       512.564       430.499       56.4094        50     
reco:pandoraTrack:LArPandoraTrackCreation             0.000501649    0.0200811     0.0420408     0.0197316     0.0110074       50     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000492271   0.00391878     0.0410445    0.00221422    0.00715969       50     
reco:pandoracalo:Calorimetry                          0.000332757   0.00115292     0.0101345    0.000794916   0.00144161       50     
reco:pandorapid:Chi2ParticleID                        7.2468e-05    0.000378707   0.00520954    0.000212987   0.000715993      50     
reco:pmtracktc:PMAlgTrackMaker                        0.00648843     0.0384531      0.19863      0.0244277     0.037715        50     
reco:pmtracktccalo:Calorimetry                        0.000249557   0.00108725    0.00282282    0.000874828   0.000598774      50     
reco:pmtracktcpid:Chi2ParticleID                      8.9341e-05    0.000268515   0.00156731    0.000196957   0.000234898      50     
reco:emtrkmichelid:EmTrackMichelId                      18.1447       30.7123       42.3221       31.0302       6.11511        50     
reco:opdec:Deconvolution                                20.1229       28.7521       38.8922       28.3926       5.40929        50     
reco:ophitspe:OpHitFinderDeco                           157.294       178.717       208.223       177.576       13.2009        50     
reco:opflash:OpFlashFinder                              30.7071       51.9188       74.1986       52.9147       10.8686        50     
reco:solarflash:SolarOpFlash                           0.280111      0.575198       1.49142      0.554382      0.232336        50     
reco:rns:RandomNumberSaver                            9.3038e-05    0.000179895   0.00101293    0.000154832   0.000136559      50     
[art]:TriggerResults:TriggerResultInserter            2.7192e-05    6.80276e-05   0.000269896   5.4955e-05    4.34878e-05      50     
end_path:out1:RootOutput                               9.428e-06    2.13939e-05   7.2228e-05    1.8816e-05    1.03986e-05      50     
end_path:out1:RootOutput(write)                        0.164821      0.276888      0.463791      0.274937      0.0611367       50     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3261.37 MB
  Peak resident set size usage (VmHWM): 1726.6 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250208T114518Z_gen_001676_supernova_g4_detsim_reco_20250724T130140Z_reco2.root
Extracted metadata
checking file====== fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250208T114518Z_gen_001676_supernova_g4_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
old_md.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250208T114518Z_gen_001676_supernova_g4_detsim_reco_20250724T130140Z_reco2.root
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250208T114518Z_gen_001676_supernova_g4_detsim_reco_20250724T130140Z_reco2.root.ext.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250208T114518Z_gen_001676_supernova_g4_detsim_reco_20250724T130140Z_reco2.root.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250208T114518Z_gen_001676_supernova_g4_detsim_reco_20250724T130140Z_reco2.root.temp.json
reco2_hist.root