Jobsub ID 17624.48@dunegpschedd01.fnal.gov

Jobsub ID	17624.48@dunegpschedd01.fnal.gov
Workflow ID	55
Stage ID	1
User name	higuera@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-07-24 09:29:52
Site	US_FNAL-T1
Entry	CMSHTPC_T1_US_FNAL_condce_opp1_whole
Last heartbeat	2025-07-24 22:45:17
From worker node	Hostname	dunegli-44332-0-cmswn3007.fnal.gov
	cpuinfo	AMD EPYC 7542 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-07-24 13:04:17
Input files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250206T151634Z_gen_004833_supernova_g4_detsim_reco.root
Jobscript	Exit code	0
	Real time	9h (34731s)
	CPU time	9h (34378s = 98%)
	Max RSS bytes	1732128768 (1651 MiB)
Outputting started	2025-07-24 22:43:08
Output files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250206T151634Z_gen_004833_supernova_g4_detsim_reco_20250724T130428Z_reco2.root
Finished	2025-07-24 22:45:17
Saved logs	justin-logs:17624.48-dunegpschedd01.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

l identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
Printing U view candidate length: 22
Printing V view candidate length: 21
Printing W view candidate length: 17
Printing 2D view candidate length: 60
Printing background CaloHitList length: 617
Printing New CaloHitList Names: 
SCaloHitListU | SCaloHitListV | SCaloHitListW | SCaloHitList2D | BBCaloHitList
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
Printing U view candidate length: 22
Printing V view candidate length: 21
Printing W view candidate length: 17
Printing 2D view candidate length: 60
Printing background CaloHitList length: 0
Printing New CaloHitList Names: 
CaloHitListU | CaloHitListV | CaloHitListW | CaloHitList2D | BCaloHitList
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 22:38:25 UTC run: 5082 subRun: 0 event: 241650
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 22:38:25 UTC run: 5082 subRun: 0 event: 241650
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 22:38:25 UTC run: 5082 subRun: 0 event: 241650
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 22:38:25 UTC run: 5082 subRun: 0 event: 241650
Comparing two wires in the same plane: return failure
%MSG
54601
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 70485!
Genrating OpFlash 0 with 3 hits
Genrating OpFlash 1 with 3 hits
Genrating OpFlash 2 with 3 hits
Genrating OpFlash 3 with 3 hits
Genrating OpFlash 4 with 4 hits
Genrating OpFlash 5 with 6 hits
Genrating OpFlash 6 with 4 hits
Genrating OpFlash 7 with 6 hits
Genrating OpFlash 8 with 6 hits
Genrating OpFlash 9 with 4 hits
...
Generated 2127 OpFlashes
24-Jul-2025 22:43:00 UTC  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250206T151634Z_gen_004833_supernova_g4_detsim_reco_20250724T130428Z_reco2.root"
24-Jul-2025 22:43:00 UTC  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/d2/b3/prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250206T151634Z_gen_004833_supernova_g4_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              499.566       689.551       761.86        697.688       48.2091        50     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00489675     0.0224649     0.0724159     0.0219252     0.0134285       50     
reco:linecluster:LineCluster                           0.0291372     0.0658724     0.224556      0.0586329     0.0323705       50     
reco:trajcluster:TrajCluster                           0.0104343     0.0192384     0.0599368     0.0176799    0.00959436       50     
reco:pandora:StandardPandora                            242.195       403.761       484.456       410.086       46.1406        50     
reco:pandoraTrack:LArPandoraTrackCreation             0.000452164    0.0189638     0.0372004     0.0187762     0.0105822       50     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000401708   0.00390885     0.0284843    0.00208726    0.00557849       50     
reco:pandoracalo:Calorimetry                          0.000279585   0.00100223    0.00306948    0.000744629   0.000684012      50     
reco:pandorapid:Chi2ParticleID                        6.4203e-05    0.000239279   0.000880223   0.000185335   0.000161433      50     
reco:pmtracktc:PMAlgTrackMaker                        0.00405883     0.0704851      1.15369      0.0308995     0.164621        50     
reco:pmtracktccalo:Calorimetry                        0.000274976   0.00109306    0.00408519    0.000898073   0.000788542      50     
reco:pmtracktcpid:Chi2ParticleID                      6.6558e-05    0.000242376   0.00105607    0.000183736   0.000181456      50     
reco:emtrkmichelid:EmTrackMichelId                      14.1558       29.8757       40.1253       30.2475       6.30616        50     
reco:opdec:Deconvolution                                19.8619       27.1867        35.37        27.8209       4.14832        50     
reco:ophitspe:OpHitFinderDeco                           150.798       177.702       200.671       176.105        12.21         50     
reco:opflash:OpFlashFinder                              30.5639       49.9234       72.6066       49.1616       9.95354        50     
reco:solarflash:SolarOpFlash                           0.294073      0.614664       1.57894      0.547119      0.251227        50     
reco:rns:RandomNumberSaver                            8.2808e-05    0.000196315   0.00146787    0.000156825   0.000193342      50     
[art]:TriggerResults:TriggerResultInserter            2.8885e-05    6.40033e-05   0.000221263   5.1899e-05    3.51636e-05      50     
end_path:out1:RootOutput                               1.008e-05    2.52224e-05   5.5977e-05     2.336e-05    1.02127e-05      50     
end_path:out1:RootOutput(write)                        0.180493      0.280347      0.466326      0.257938      0.0627544       50     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3325.09 MB
  Peak resident set size usage (VmHWM): 1732.13 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250206T151634Z_gen_004833_supernova_g4_detsim_reco_20250724T130428Z_reco2.root
Extracted metadata
checking file====== fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250206T151634Z_gen_004833_supernova_g4_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
old_md.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250206T151634Z_gen_004833_supernova_g4_detsim_reco_20250724T130428Z_reco2.root
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250206T151634Z_gen_004833_supernova_g4_detsim_reco_20250724T130428Z_reco2.root.ext.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250206T151634Z_gen_004833_supernova_g4_detsim_reco_20250724T130428Z_reco2.root.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250206T151634Z_gen_004833_supernova_g4_detsim_reco_20250724T130428Z_reco2.root.temp.json
reco2_hist.root