Jobsub ID 11118.152@dunegpschedd02.fnal.gov

Jobsub ID	11118.152@dunegpschedd02.fnal.gov
Workflow ID	60
Stage ID	1
User name	higuera@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-07-25 01:22:46
Site	US_UChicago
Entry	Engage_US_MWT2_iut2_gk02_condce_mcore
Last heartbeat	2025-07-25 10:26:34
From worker node	Hostname	iut2-c278.iu.edu
	cpuinfo	Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	86400 (24 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-07-25 04:00:43
Input files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250210T165738Z_gen_002657_supernova_g4_halfActiveVol_detsim_reco.root
Jobscript	Exit code	0
	Real time	6h (23082s)
	CPU time	6h (22584s = 97%)
	Max RSS bytes	1631121408 (1555 MiB)
Outputting started	2025-07-25 10:25:26
Output files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250210T165738Z_gen_002657_supernova_g4_halfActiveVol_detsim_reco_20250725T040049Z_reco2.root
Finished	2025-07-25 10:26:34
Saved logs	justin-logs:11118.152-dunegpschedd02.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

xel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
Printing U view candidate length: 26
Printing V view candidate length: 28
Printing W view candidate length: 26
Printing 2D view candidate length: 80
Printing background CaloHitList length: 468
Printing New CaloHitList Names: 
SCaloHitListU | SCaloHitListV | SCaloHitListW | SCaloHitList2D | BBCaloHitList
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
Printing U view candidate length: 25
Printing V view candidate length: 22
Printing W view candidate length: 18
Printing 2D view candidate length: 65
Printing background CaloHitList length: 11
Printing New CaloHitList Names: 
CaloHitListU | CaloHitListV | CaloHitListW | CaloHitList2D | BCaloHitList
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 06:20:34 EDT run: 5117 subRun: 0 event: 132850
Comparing two wires in the same plane: return failure
%MSG
33276
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 40622!
Genrating OpFlash 0 with 3 hits
Genrating OpFlash 1 with 5 hits
Genrating OpFlash 2 with 3 hits
Genrating OpFlash 3 with 12 hits
Genrating OpFlash 4 with 4 hits
Genrating OpFlash 5 with 3 hits
Genrating OpFlash 6 with 3 hits
Genrating OpFlash 7 with 4 hits
Genrating OpFlash 8 with 5 hits
Genrating OpFlash 9 with 3 hits
...
Generated 1116 OpFlashes
25-Jul-2025 06:25:19 EDT  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250210T165738Z_gen_002657_supernova_g4_halfActiveVol_detsim_reco_20250725T040049Z_reco2.root"
25-Jul-2025 06:25:19 EDT  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/d3/a3/prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250210T165738Z_gen_002657_supernova_g4_halfActiveVol_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              363.681       457.288       515.834       459.297       33.5277        50     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00843728      0.0251       0.201808      0.0220703     0.0263957       50     
reco:linecluster:LineCluster                           0.0785775     0.126996      0.291561      0.111984      0.0415767       50     
reco:trajcluster:TrajCluster                           0.0234544     0.0292596     0.0656849     0.0273205    0.00677235       50     
reco:pandora:StandardPandora                            110.109       152.004       174.583       154.971       14.4143        50     
reco:pandoraTrack:LArPandoraTrackCreation             0.000432977    0.0067171     0.0240036     0.0079011    0.00417472       50     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000411029   0.00254546     0.028316     0.00187112    0.00389417       50     
reco:pandoracalo:Calorimetry                          0.000249544   0.000828979   0.00826168    0.000673145   0.00108819       50     
reco:pandorapid:Chi2ParticleID                        6.8377e-05    0.000241785   0.00333577    0.000179744   0.000444975      50     
reco:pmtracktc:PMAlgTrackMaker                        0.00295662     0.0686394     0.562639      0.0307368     0.106283        50     
reco:pmtracktccalo:Calorimetry                        0.000164143   0.000697835   0.00220407    0.000673581   0.000298811      50     
reco:pmtracktcpid:Chi2ParticleID                      5.8508e-05    0.000175263   0.000355211   0.000172229   5.6116e-05       50     
reco:emtrkmichelid:EmTrackMichelId                      7.77105       11.2951       14.8018       11.3664       1.53223        50     
reco:opdec:Deconvolution                                15.5441       20.9551       24.1478       20.9263       1.8555         50     
reco:ophitspe:OpHitFinderDeco                           184.864       263.29        306.796       265.019       27.2701        50     
reco:opflash:OpFlashFinder                              7.34042       9.00473       10.748        9.28186      0.837553        50     
reco:solarflash:SolarOpFlash                           0.160589      0.297648      0.536746      0.308021      0.0770973       50     
reco:rns:RandomNumberSaver                             9.066e-05     0.000132     0.000535554   0.000123082   6.09182e-05      50     
[art]:TriggerResults:TriggerResultInserter            4.0058e-05    5.56582e-05   7.7957e-05    5.50555e-05   9.12118e-06      50     
end_path:out1:RootOutput                               8.722e-06    1.30903e-05   3.4765e-05    1.24015e-05   4.45011e-06      50     
end_path:out1:RootOutput(write)                        0.113076      0.177948      0.249262      0.183092      0.0363572       50     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3132.78 MB
  Peak resident set size usage (VmHWM): 1631.12 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250210T165738Z_gen_002657_supernova_g4_halfActiveVol_detsim_reco_20250725T040049Z_reco2.root
Extracted metadata
checking file====== fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250210T165738Z_gen_002657_supernova_g4_halfActiveVol_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250210T165738Z_gen_002657_supernova_g4_halfActiveVol_detsim_reco_20250725T040049Z_reco2.root
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250210T165738Z_gen_002657_supernova_g4_halfActiveVol_detsim_reco_20250725T040049Z_reco2.root.ext.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250210T165738Z_gen_002657_supernova_g4_halfActiveVol_detsim_reco_20250725T040049Z_reco2.root.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250210T165738Z_gen_002657_supernova_g4_halfActiveVol_detsim_reco_20250725T040049Z_reco2.root.temp.json
reco2_hist.root