Jobsub ID 17624.30@dunegpschedd01.fnal.gov

Jobsub ID	17624.30@dunegpschedd01.fnal.gov
Workflow ID	55
Stage ID	1
User name	higuera@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-07-24 09:29:52
Site	US_FNAL-T1
Entry	CMSHTPC_T1_US_FNAL_condce_opp1_whole
Last heartbeat	2025-07-24 22:44:10
From worker node	Hostname	dunegli-44332-0-cmswn3007.fnal.gov
	cpuinfo	AMD EPYC 7542 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-07-24 13:02:42
Input files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T105833Z_gen_002988_supernova_g4_detsim_reco.root
Jobscript	Exit code	0
	Real time	9h (34762s)
	CPU time	9h (34353s = 98%)
	Max RSS bytes	1738194944 (1657 MiB)
Outputting started	2025-07-24 22:42:05
Output files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T105833Z_gen_002988_supernova_g4_detsim_reco_20250724T130253Z_reco2.root
Finished	2025-07-24 22:44:10
Saved logs	justin-logs:17624.30-dunegpschedd01.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
Printing U view candidate length: 34
Printing V view candidate length: 41
Printing W view candidate length: 21
Printing 2D view candidate length: 96
Printing background CaloHitList length: 696
Printing New CaloHitList Names: 
SCaloHitListU | SCaloHitListV | SCaloHitListW | SCaloHitList2D | BBCaloHitList
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Photon identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Photon identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Photon identified*
Printing U view candidate length: 34
Printing V view candidate length: 41
Printing W view candidate length: 21
Printing 2D view candidate length: 96
Printing background CaloHitList length: 1
Printing New CaloHitList Names: 
CaloHitListU | CaloHitListV | CaloHitListW | CaloHitList2D | BCaloHitList
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 22:37:51 UTC run: 5089 subRun: 0 event: 149400
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 22:37:51 UTC run: 5089 subRun: 0 event: 149400
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 22:37:51 UTC run: 5089 subRun: 0 event: 149400
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 22:37:51 UTC run: 5089 subRun: 0 event: 149400
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 22:37:51 UTC run: 5089 subRun: 0 event: 149400
Comparing two wires in the same plane: return failure
%MSG
54510
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 70078!
Genrating OpFlash 0 with 3 hits
Genrating OpFlash 1 with 3 hits
Genrating OpFlash 2 with 3 hits
Genrating OpFlash 3 with 4 hits
Genrating OpFlash 4 with 4 hits
Genrating OpFlash 5 with 4 hits
Genrating OpFlash 6 with 4 hits
Genrating OpFlash 7 with 5 hits
Genrating OpFlash 8 with 3 hits
Genrating OpFlash 9 with 3 hits
...
Generated 2108 OpFlashes
24-Jul-2025 22:41:56 UTC  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T105833Z_gen_002988_supernova_g4_detsim_reco_20250724T130253Z_reco2.root"
24-Jul-2025 22:41:56 UTC  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/33/31/prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T105833Z_gen_002988_supernova_g4_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              512.646       690.646       817.301       699.434       60.3298        50     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00365476     0.0119761     0.0301838    0.00961695    0.00634635       50     
reco:linecluster:LineCluster                           0.0341378     0.0654344     0.153326      0.062035      0.0219841       50     
reco:trajcluster:TrajCluster                          0.00887374     0.0175407     0.0366682     0.0154196    0.00588954       50     
reco:pandora:StandardPandora                            269.735       405.784       490.677       411.354       52.5787        50     
reco:pandoraTrack:LArPandoraTrackCreation             0.000458647    0.018608      0.0397137     0.0186496     0.0102393       50     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000362724   0.00409224     0.0508469    0.00196845    0.00828524       50     
reco:pandoracalo:Calorimetry                          0.000282391   0.000975484   0.00337955    0.000743823   0.000696758      50     
reco:pandorapid:Chi2ParticleID                        6.7329e-05    0.000245579   0.00146954    0.000180274   0.000230707      50     
reco:pmtracktc:PMAlgTrackMaker                        0.00864238     0.0484164      0.28852      0.0352039     0.0465416       50     
reco:pmtracktccalo:Calorimetry                        0.000252182    0.0010797    0.00682952    0.000868186   0.000918395      50     
reco:pmtracktcpid:Chi2ParticleID                      6.1547e-05    0.000256114   0.00170674    0.000181893   0.000247486      50     
reco:emtrkmichelid:EmTrackMichelId                      17.7139       29.1122       38.9926       29.0416       5.11264        50     
reco:opdec:Deconvolution                                20.2684       26.8095       39.2061       25.3584       4.86506        50     
reco:ophitspe:OpHitFinderDeco                           156.121       177.42        205.543       176.666       10.8159        50     
reco:opflash:OpFlashFinder                              28.7348       50.4731       75.7835       49.9521       10.795         50     
reco:solarflash:SolarOpFlash                           0.302185      0.586861       1.12557      0.593291      0.177968        50     
reco:rns:RandomNumberSaver                            6.6327e-05    0.000168215   0.00111395    0.000141556   0.000143524      50     
[art]:TriggerResults:TriggerResultInserter            3.0197e-05    6.7784e-05    0.00014663    6.0275e-05    2.83448e-05      50     
end_path:out1:RootOutput                              1.3235e-05    2.4461e-05    7.4562e-05    2.07945e-05   1.05971e-05      50     
end_path:out1:RootOutput(write)                        0.189119      0.288526      0.429791      0.283835      0.0522207       50     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3253.64 MB
  Peak resident set size usage (VmHWM): 1738.19 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T105833Z_gen_002988_supernova_g4_detsim_reco_20250724T130253Z_reco2.root
Extracted metadata
checking file====== fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T105833Z_gen_002988_supernova_g4_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
old_md.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T105833Z_gen_002988_supernova_g4_detsim_reco_20250724T130253Z_reco2.root
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T105833Z_gen_002988_supernova_g4_detsim_reco_20250724T130253Z_reco2.root.ext.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T105833Z_gen_002988_supernova_g4_detsim_reco_20250724T130253Z_reco2.root.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T105833Z_gen_002988_supernova_g4_detsim_reco_20250724T130253Z_reco2.root.temp.json
reco2_hist.root