Jobsub ID 17436.0@dunegpschedd01.fnal.gov

Jobsub ID	17436.0@dunegpschedd01.fnal.gov
Workflow ID	54
Stage ID	1
User name	epennacc@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-07-23 16:02:55
Site	BR_CBPF
Entry	DUNE_BR_CBPF_ce01
Last heartbeat	2025-07-24 04:56:48
From worker node	Hostname	wn55
	cpuinfo	Intel(R) Xeon(R) CPU E5-2640 v4 @ 2.40GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	257400 (71 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-07-23 16:04:37
Input files	fardet-hd:fardet-hd_mc_le_mc_2024a_list-4397_input.list_out1_reco1_supernova_dune10kt_1x2x6_hit-reconstructed_merged_skip000200_lim000025_mergeLar_20241218T191342.root
Jobscript	Exit code	0
	Real time	12h (46165s)
	CPU time	3h (14026s = 30%)
	Max RSS bytes	1475330048 (1406 MiB)
Outputting started	2025-07-24 04:54:03
Output files	fardet-hd:fardet-hd_mc_le_mc_2024a_list-4397_input.list_out1_reco1_supernova_dune10kt_1x2x6_hit-reconstructed_merged_skip000200_lim000025_mergeLar_20241218T191342_20250723T160814Z_reco2.root
Finished	2025-07-24 04:56:48
Saved logs	justin-logs:17436.0-dunegpschedd01.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
Printing U view candidate length: 27
Printing V view candidate length: 24
Printing W view candidate length: 28
Printing 2D view candidate length: 79
Printing background CaloHitList length: 3
Printing New CaloHitList Names: 
CaloHitListU | CaloHitListV | CaloHitListW | CaloHitList2D | BCaloHitList
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 01:53:30 -03 run: 4397 subRun: 0 event: 22499
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 01:53:30 -03 run: 4397 subRun: 0 event: 22499
Comparing two wires in the same plane: return failure
%MSG
239
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 459!
Genrating OpFlash 0 with 24 hits
Genrating OpFlash 1 with 5 hits
Genrating OpFlash 2 with 3 hits
Genrating OpFlash 3 with 18 hits
Genrating OpFlash 4 with 4 hits
Genrating OpFlash 5 with 11 hits
Genrating OpFlash 6 with 7 hits
Genrating OpFlash 7 with 16 hits
Genrating OpFlash 8 with 8 hits
Genrating OpFlash 9 with 3 hits
...
Generated 15 OpFlashes
Begin processing the 2500th record. run: 4397 subRun: 0 event: 22500 at 24-Jul-2025 01:53:34 -03
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
Printing U view candidate length: 7
Printing V view candidate length: 7
Printing W view candidate length: 10
Printing 2D view candidate length: 24
Printing background CaloHitList length: 7
Printing New CaloHitList Names: 
SCaloHitListU | SCaloHitListV | SCaloHitListW | SCaloHitList2D | BBCaloHitList
*Electron identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Electron identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Photon identified*
*Electron identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
Printing U view candidate length: 10
Printing V view candidate length: 11
Printing W view candidate length: 10
Printing 2D view candidate length: 31
Printing background CaloHitList length: 0
Printing New CaloHitList Names: 
CaloHitListU | CaloHitListV | CaloHitListW | CaloHitList2D | BCaloHitList
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 01:53:42 -03 run: 4397 subRun: 0 event: 22500
Comparing two wires in the same plane: return failure
%MSG
127
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 260!
Genrating OpFlash 0 with 44 hits
Genrating OpFlash 1 with 4 hits
Genrating OpFlash 2 with 8 hits
Genrating OpFlash 3 with 3 hits
Genrating OpFlash 4 with 18 hits
Genrating OpFlash 5 with 5 hits
Genrating OpFlash 6 with 3 hits
Genrating OpFlash 7 with 8 hits
Genrating OpFlash 8 with 4 hits
...
Generated 9 OpFlashes
24-Jul-2025 01:53:46 -03  Closed output file "fardet-hd_mc_le_mc_2024a_list-4397_input.list_out1_reco1_supernova_dune10kt_1x2x6_hit-reconstructed_merged_skip000200_lim000025_mergeLar_20241218T191342_20250723T160814Z_reco2.root"
24-Jul-2025 01:53:46 -03  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/84/f8/fardet-hd_mc_le_mc_2024a_list-4397_input.list_out1_reco1_supernova_dune10kt_1x2x6_hit-reconstructed_merged_skip000200_lim000025_mergeLar_20241218T191342.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2.50197       8.76019       543.902       8.73876       10.7819       2500    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000381799    0.0181037      1.21727     0.000664086    0.0578855      2500    
reco:linecluster:LineCluster                           0.816923       1.33868       1.74851       1.37793      0.0809302      2500    
reco:trajcluster:TrajCluster                           0.408848       0.41621      0.480217      0.412333     0.00867926      2500    
reco:pandora:StandardPandora                           0.420068       4.33999       538.946       4.28549       10.7146       2500    
reco:pandoraTrack:LArPandoraTrackCreation             0.000128936   0.000653357   0.00642388    0.000660751   0.000267617     2500    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000180801   0.00136226     0.0135726    0.00136066    0.000591909     2500    
reco:pandoracalo:Calorimetry                          0.000102304   0.000475248   0.00448917    0.000461172   0.000211283     2500    
reco:pandorapid:Chi2ParticleID                        3.9011e-05    0.00012961    0.00169986    0.000132508   5.19209e-05     2500    
reco:pmtracktc:PMAlgTrackMaker                        0.00148931     0.0566875      1.1286       0.0288025     0.0898402      2500    
reco:pmtracktccalo:Calorimetry                         8.927e-05    0.000459561   0.00138538    0.000459971   0.000194975     2500    
reco:pmtracktcpid:Chi2ParticleID                      3.4012e-05    0.000120798   0.00067639    0.000123462   3.69486e-05     2500    
reco:emtrkmichelid:EmTrackMichelId                    0.00018615     0.810054       2.16447      0.780029      0.380647       2500    
reco:opdec:Deconvolution                               0.411083      0.854547       1.25299      0.918884      0.132839       2500    
reco:ophitspe:OpHitFinderDeco                         0.000298521    0.895905       2.55533       0.87319      0.399814       2500    
reco:opflash:OpFlashFinder                            0.000169959   0.000930179   0.00718816    0.00083496    0.000524799     2500    
reco:solarflash:SolarOpFlash                          7.3869e-05    0.00785471     0.0516414    0.00694964    0.00538828      2500    
reco:rns:RandomNumberSaver                            1.8596e-05    7.32874e-05   0.000935541   4.8956e-05    0.000122265     2500    
[art]:TriggerResults:TriggerResultInserter             1.152e-05    2.5553e-05    0.000666906   2.27555e-05   1.89575e-05     2500    
end_path:out1:RootOutput                               2.644e-06    8.98614e-06   5.7993e-05     8.77e-06     3.71333e-06     2500    
end_path:out1:RootOutput(write)                       0.00691904     0.0166332     0.096766      0.0164878     0.0039773      2500    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 2959.34 MB
  Peak resident set size usage (VmHWM): 1475.33 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  fardet-hd_mc_le_mc_2024a_list-4397_input.list_out1_reco1_supernova_dune10kt_1x2x6_hit-reconstructed_merged_skip000200_lim000025_mergeLar_20241218T191342_20250723T160814Z_reco2.root
Extracted metadata
checking file====== fardet-hd:fardet-hd_mc_le_mc_2024a_list-4397_input.list_out1_reco1_supernova_dune10kt_1x2x6_hit-reconstructed_merged_skip000200_lim000025_mergeLar_20241218T191342.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
fardet-hd_mc_le_mc_2024a_list-4397_input.list_out1_reco1_supernova_dune10kt_1x2x6_hit-reconstructed_merged_skip000200_lim000025_mergeLar_20241218T191342_20250723T160814Z_reco2.root
fardet-hd_mc_le_mc_2024a_list-4397_input.list_out1_reco1_supernova_dune10kt_1x2x6_hit-reconstructed_merged_skip000200_lim000025_mergeLar_20241218T191342_20250723T160814Z_reco2.root.ext.json
fardet-hd_mc_le_mc_2024a_list-4397_input.list_out1_reco1_supernova_dune10kt_1x2x6_hit-reconstructed_merged_skip000200_lim000025_mergeLar_20241218T191342_20250723T160814Z_reco2.root.json
fardet-hd_mc_le_mc_2024a_list-4397_input.list_out1_reco1_supernova_dune10kt_1x2x6_hit-reconstructed_merged_skip000200_lim000025_mergeLar_20241218T191342_20250723T160814Z_reco2.root.temp.json
jobscript.log
justin-processed-pfns.txt
old_md.json
reco2_hist.root