Jobsub ID 11264.0@dunegpschedd02.fnal.gov

Jobsub ID	11264.0@dunegpschedd02.fnal.gov
Workflow ID	59
Stage ID	1
User name	epennacc@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-07-25 16:59:31
Site	BR_CBPF
Entry	DUNE_BR_CBPF_ce02
Last heartbeat	2025-07-25 22:00:40
From worker node	Hostname	wn31
	cpuinfo	Intel(R) Xeon(R) CPU E5-2660 0 @ 2.20GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	257400 (71 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-07-25 17:00:55
Input files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250209T194808Z_gen_004465_supernova_g4_halfActiveVol_detsim_reco.root
Jobscript	Exit code	0
	Real time	4h (17797s)
	CPU time	4h (15449s = 86%)
	Max RSS bytes	1646080000 (1569 MiB)
Outputting started	2025-07-25 21:57:33
Output files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250209T194808Z_gen_004465_supernova_g4_halfActiveVol_detsim_reco_20250725T170105Z_reco2.root
Finished	2025-07-25 22:00:40
Saved logs	justin-logs:11264.0-dunegpschedd02.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

| CaloHitList2D | BCaloHitList
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 18:49:21 -03 run: 5100 subRun: 0 event: 223249
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 18:49:21 -03 run: 5100 subRun: 0 event: 223249
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 18:49:21 -03 run: 5100 subRun: 0 event: 223249
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 18:49:21 -03 run: 5100 subRun: 0 event: 223249
Comparing two wires in the same plane: return failure
%MSG
32339
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 39013!
Genrating OpFlash 0 with 3 hits
Genrating OpFlash 1 with 7 hits
Genrating OpFlash 2 with 3 hits
Genrating OpFlash 3 with 4 hits
Genrating OpFlash 4 with 3 hits
Genrating OpFlash 5 with 3 hits
Genrating OpFlash 6 with 4 hits
Genrating OpFlash 7 with 3 hits
Genrating OpFlash 8 with 3 hits
Genrating OpFlash 9 with 3 hits
...
Generated 1063 OpFlashes
Begin processing the 50th record. run: 5100 subRun: 0 event: 223250 at 25-Jul-2025 18:52:24 -03
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
Printing U view candidate length: 13
Printing V view candidate length: 11
Printing W view candidate length: 10
Printing 2D view candidate length: 34
Printing background CaloHitList length: 486
Printing New CaloHitList Names: 
SCaloHitListU | SCaloHitListV | SCaloHitListW | SCaloHitList2D | BBCaloHitList
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
Printing U view candidate length: 11
Printing V view candidate length: 11
Printing W view candidate length: 10
Printing 2D view candidate length: 32
Printing background CaloHitList length: 4
Printing New CaloHitList Names: 
CaloHitListU | CaloHitListV | CaloHitListW | CaloHitList2D | BCaloHitList
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 18:54:18 -03 run: 5100 subRun: 0 event: 223250
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 18:54:18 -03 run: 5100 subRun: 0 event: 223250
Comparing two wires in the same plane: return failure
%MSG
32737
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 39976!
Genrating OpFlash 0 with 3 hits
Genrating OpFlash 1 with 3 hits
Genrating OpFlash 2 with 5 hits
Genrating OpFlash 3 with 3 hits
Genrating OpFlash 4 with 4 hits
Genrating OpFlash 5 with 4 hits
Genrating OpFlash 6 with 4 hits
Genrating OpFlash 7 with 3 hits
Genrating OpFlash 8 with 9 hits
Genrating OpFlash 9 with 5 hits
...
Generated 1156 OpFlashes
25-Jul-2025 18:57:24 -03  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250209T194808Z_gen_004465_supernova_g4_halfActiveVol_detsim_reco_20250725T170105Z_reco2.root"
25-Jul-2025 18:57:24 -03  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/2e/00/prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250209T194808Z_gen_004465_supernova_g4_halfActiveVol_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              275.025       329.836       446.014       337.717       27.8984        50     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.137552      0.181709      0.302829      0.150797      0.0585376       50     
reco:linecluster:LineCluster                            1.24973       1.31287       2.33458       1.28251       0.16144        50     
reco:trajcluster:TrajCluster                           0.414086      0.424803      0.519445      0.419494      0.0195558       50     
reco:pandora:StandardPandora                            106.257       151.378       277.988       154.795       23.6134        50     
reco:pandoraTrack:LArPandoraTrackCreation             0.000231208   0.00388184     0.0352371    0.00364454    0.00480444       50     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000280945    0.002381      0.0316912    0.00176203    0.00427064       50     
reco:pandoracalo:Calorimetry                          0.000181105   0.000654006    0.0057398    0.000587906   0.00075407       50     
reco:pandorapid:Chi2ParticleID                        6.1091e-05    0.000202058    0.0022312    0.000176051   0.000294088      50     
reco:pmtracktc:PMAlgTrackMaker                        0.00255361     0.0503582     0.491916      0.0204982     0.0828471       50     
reco:pmtracktccalo:Calorimetry                        0.000148642   0.000607797   0.00123618    0.000589318   0.000242187      50     
reco:pmtracktcpid:Chi2ParticleID                      5.9702e-05    0.00016603    0.000280387   0.000169678   5.04512e-05      50     
reco:emtrkmichelid:EmTrackMichelId                      5.3196        8.42045       11.4418       8.86585       1.59246        50     
reco:opdec:Deconvolution                                23.7497       24.9356       27.7376       24.6362      0.942124        50     
reco:ophitspe:OpHitFinderDeco                           104.861       137.268       154.709       144.017       13.8004        50     
reco:opflash:OpFlashFinder                              4.45288       5.28768       6.76736       5.32101      0.446792        50     
reco:solarflash:SolarOpFlash                           0.219152      0.405395      0.528473      0.476432      0.117463        50     
reco:rns:RandomNumberSaver                            6.1898e-05    8.9753e-05    0.000369985   8.77555e-05   4.22193e-05      50     
[art]:TriggerResults:TriggerResultInserter            2.9349e-05    4.58494e-05   7.0031e-05    4.8061e-05    9.03069e-06      50     
end_path:out1:RootOutput                               8.427e-06    1.18465e-05   2.7616e-05    1.19145e-05   3.37512e-06      50     
end_path:out1:RootOutput(write)                        0.113464      0.161336      0.286784      0.172164      0.0326201       50     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3132.82 MB
  Peak resident set size usage (VmHWM): 1646.08 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250209T194808Z_gen_004465_supernova_g4_halfActiveVol_detsim_reco_20250725T170105Z_reco2.root
Extracted metadata
checking file====== fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250209T194808Z_gen_004465_supernova_g4_halfActiveVol_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250209T194808Z_gen_004465_supernova_g4_halfActiveVol_detsim_reco_20250725T170105Z_reco2.root
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250209T194808Z_gen_004465_supernova_g4_halfActiveVol_detsim_reco_20250725T170105Z_reco2.root.ext.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250209T194808Z_gen_004465_supernova_g4_halfActiveVol_detsim_reco_20250725T170105Z_reco2.root.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_lateralAPA_20250209T194808Z_gen_004465_supernova_g4_halfActiveVol_detsim_reco_20250725T170105Z_reco2.root.temp.json
reco2_hist.root