Jobsub ID 17439.3@dunegpschedd01.fnal.gov

Jobsub ID	17439.3@dunegpschedd01.fnal.gov
Workflow ID	54
Stage ID	1
User name	epennacc@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-07-23 17:12:58
Site	NL_NIKHEF
Entry	VIRGO_NL_NIKHEF_brug
Last heartbeat	2025-07-24 05:07:01
From worker node	Hostname	wn-sate-045.farm.nikhef.nl
	cpuinfo	AMD EPYC 7551P 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	129600 (36 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-07-23 17:15:02
Input files	fardet-hd:fardet-hd_mc_le_mc_2024a_workflow0000003786_out1_reco1_supernova_dune10kt_1x2x6_hit-reconstructed_merged_skip000400_lim000025_mergeLar_20241217T150015.root
Jobscript	Exit code	0
	Real time	11h (42604s)
	CPU time	2h (10004s = 23%)
	Max RSS bytes	1504489472 (1434 MiB)
Outputting started	2025-07-24 05:05:07
Output files	fardet-hd:fardet-hd_mc_le_mc_2024a_workflow0000003786_out1_reco1_supernova_dune10kt_1x2x6_hit-reconstructed_merged_skip000400_lim000025_mergeLar_20241217T150015_20250723T171509Z_reco2.root
Finished	2025-07-24 05:07:01
Saved logs	justin-logs:17439.3-dunegpschedd01.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

der with InputDigiType = 'recob'
Found hits: 640!
Genrating OpFlash 0 with 71 hits
Genrating OpFlash 1 with 6 hits
Genrating OpFlash 2 with 16 hits
Genrating OpFlash 3 with 7 hits
Genrating OpFlash 4 with 10 hits
Genrating OpFlash 5 with 3 hits
Genrating OpFlash 6 with 7 hits
Genrating OpFlash 7 with 7 hits
Genrating OpFlash 8 with 4 hits
Genrating OpFlash 9 with 6 hits
...
Generated 25 OpFlashes
Begin processing the 2500th record. run: 3786 subRun: 0 event: 49700 at 24-Jul-2025 07:04:45 CEST
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
Printing U view candidate length: 15
Printing V view candidate length: 16
Printing W view candidate length: 9
Printing 2D view candidate length: 40
Printing background CaloHitList length: 11
Printing New CaloHitList Names: 
SCaloHitListU | SCaloHitListV | SCaloHitListW | SCaloHitList2D | BBCaloHitList
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Electron identified*
*Photon identified*
*Electron identified*
*Electron identified*
Printing U view candidate length: 19
Printing V view candidate length: 19
Printing W view candidate length: 12
Printing 2D view candidate length: 50
Printing background CaloHitList length: 1
Printing New CaloHitList Names: 
CaloHitListU | CaloHitListV | CaloHitListW | CaloHitList2D | BCaloHitList
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 07:04:51 CEST run: 3786 subRun: 0 event: 49700
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 07:04:51 CEST run: 3786 subRun: 0 event: 49700
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 07:04:51 CEST run: 3786 subRun: 0 event: 49700
Comparing two wires in the same plane: return failure
%MSG
146
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 334!
Genrating OpFlash 0 with 37 hits
Genrating OpFlash 1 with 3 hits
Genrating OpFlash 2 with 3 hits
Genrating OpFlash 3 with 14 hits
Genrating OpFlash 4 with 3 hits
Genrating OpFlash 5 with 4 hits
Genrating OpFlash 6 with 3 hits
Genrating OpFlash 7 with 3 hits
Genrating OpFlash 8 with 3 hits
Genrating OpFlash 9 with 4 hits
...
Generated 16 OpFlashes
24-Jul-2025 07:05:03 CEST  Closed output file "fardet-hd_mc_le_mc_2024a_workflow0000003786_out1_reco1_supernova_dune10kt_1x2x6_hit-reconstructed_merged_skip000400_lim000025_mergeLar_20241217T150015_20250723T171509Z_reco2.root"
24-Jul-2025 07:05:03 CEST  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/53/0b/fardet-hd_mc_le_mc_2024a_workflow0000003786_out1_reco1_supernova_dune10kt_1x2x6_hit-reconstructed_merged_skip000400_lim000025_mergeLar_20241217T150015.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              11.0063       15.4661       19.8725       15.609        0.94994       2500    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000371468    0.0176461     0.582953     0.000581463    0.0499856      2500    
reco:linecluster:LineCluster                           0.752225       1.14385       2.08251       1.14031      0.0512575      2500    
reco:trajcluster:TrajCluster                           0.376416      0.378552      0.569115      0.377819     0.00691531      2500    
reco:pandora:StandardPandora                           0.383566       3.21955       7.24086       3.33864      0.517347       2500    
reco:pandoraTrack:LArPandoraTrackCreation             0.000124083   0.000366458   0.00537112    0.000371559   0.000163026     2500    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000177433   0.00080571     0.014062     0.000792265    0.0004874      2500    
reco:pandoracalo:Calorimetry                          9.7974e-05    0.000254241   0.00337666    0.000253191   0.000106631     2500    
reco:pandorapid:Chi2ParticleID                        3.8032e-05    7.01718e-05   0.00144367    7.1725e-05    3.13202e-05     2500    
reco:pmtracktc:PMAlgTrackMaker                        0.00146095     0.0329935     0.486806      0.016714      0.0522463      2500    
reco:pmtracktccalo:Calorimetry                        7.8849e-05    0.000274364   0.00135636    0.000276389   0.000111685     2500    
reco:pmtracktcpid:Chi2ParticleID                       3.208e-05    7.11878e-05   0.000540075   7.21855e-05   2.19338e-05     2500    
reco:emtrkmichelid:EmTrackMichelId                    0.000120226    0.443085       1.21322      0.419881      0.213296       2500    
reco:opdec:Deconvolution                                0.37827      0.759708       1.42065      0.824378      0.121861       2500    
reco:ophitspe:OpHitFinderDeco                         0.000156394    0.500648       2.14402      0.485472      0.233607       2500    
reco:opflash:OpFlashFinder                            8.2265e-05    0.000445941   0.00882857    0.000424418   0.000224288     2500    
reco:solarflash:SolarOpFlash                          4.0627e-05    0.00492464     0.0352225    0.00425468    0.00349156      2500    
reco:rns:RandomNumberSaver                            1.5939e-05    2.15738e-05   0.000217108   2.12445e-05   5.20073e-06     2500    
[art]:TriggerResults:TriggerResultInserter            1.0469e-05    1.23476e-05   9.5209e-05    1.2063e-05    2.36143e-06     2500    
end_path:out1:RootOutput                               2.434e-06    3.16725e-06   1.5589e-05     3.096e-06    5.63498e-07     2500    
end_path:out1:RootOutput(write)                         8.72359        8.962        10.9239       8.95272      0.126084       2500    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 2983.47 MB
  Peak resident set size usage (VmHWM): 1504.49 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  fardet-hd_mc_le_mc_2024a_workflow0000003786_out1_reco1_supernova_dune10kt_1x2x6_hit-reconstructed_merged_skip000400_lim000025_mergeLar_20241217T150015_20250723T171509Z_reco2.root
Extracted metadata
checking file====== fardet-hd:fardet-hd_mc_le_mc_2024a_workflow0000003786_out1_reco1_supernova_dune10kt_1x2x6_hit-reconstructed_merged_skip000400_lim000025_mergeLar_20241217T150015.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
fardet-hd_mc_le_mc_2024a_workflow0000003786_out1_reco1_supernova_dune10kt_1x2x6_hit-reconstructed_merged_skip000400_lim000025_mergeLar_20241217T150015_20250723T171509Z_reco2.root
fardet-hd_mc_le_mc_2024a_workflow0000003786_out1_reco1_supernova_dune10kt_1x2x6_hit-reconstructed_merged_skip000400_lim000025_mergeLar_20241217T150015_20250723T171509Z_reco2.root.ext.json
fardet-hd_mc_le_mc_2024a_workflow0000003786_out1_reco1_supernova_dune10kt_1x2x6_hit-reconstructed_merged_skip000400_lim000025_mergeLar_20241217T150015_20250723T171509Z_reco2.root.json
fardet-hd_mc_le_mc_2024a_workflow0000003786_out1_reco1_supernova_dune10kt_1x2x6_hit-reconstructed_merged_skip000400_lim000025_mergeLar_20241217T150015_20250723T171509Z_reco2.root.temp.json
jobscript.log
justin-processed-pfns.txt
old_md.json
reco2_hist.root