Jobsub ID 17663.111@dunegpschedd01.fnal.gov

Jobsub ID	17663.111@dunegpschedd01.fnal.gov
Workflow ID	58
Stage ID	1
User name	epennacc@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-07-24 11:58:03
Site	UK_Oxford
Entry	DUNE_UK_SGrid_Oxford_arc01
Last heartbeat	2025-07-25 01:24:22
From worker node	Hostname	t2wn140.physics.ox.ac.uk
	cpuinfo	Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	257400 (71 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-07-24 12:39:43
Input files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T121644Z_gen_004144_supernova_g4_detsim_reco.root
Jobscript	Exit code	0
	Real time	12h (45535s)
	CPU time	11h (42665s = 93%)
	Max RSS bytes	1768972288 (1687 MiB)
Outputting started	2025-07-25 01:18:40
Output files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T121644Z_gen_004144_supernova_g4_detsim_reco_20250724T123959Z_reco2.root
Finished	2025-07-25 01:24:22
Saved logs	justin-logs:17663.111-dunegpschedd01.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

s in the same plane: return failure
%MSG
%MSG-e pma::Track3D:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 01:52:04 BST run: 5089 subRun: 0 event: 207199
Flip, endpoint closer to vStart.
%MSG
54996
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 71935!
Genrating OpFlash 0 with 5 hits
Genrating OpFlash 1 with 8 hits
Genrating OpFlash 2 with 4 hits
Genrating OpFlash 3 with 6 hits
Genrating OpFlash 4 with 3 hits
Genrating OpFlash 5 with 3 hits
Genrating OpFlash 6 with 3 hits
Genrating OpFlash 7 with 5 hits
Genrating OpFlash 8 with 7 hits
Genrating OpFlash 9 with 3 hits
...
Generated 2210 OpFlashes
Begin processing the 50th record. run: 5089 subRun: 0 event: 207200 at 25-Jul-2025 02:02:18 BST
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
Printing U view candidate length: 18
Printing V view candidate length: 13
Printing W view candidate length: 13
Printing 2D view candidate length: 44
Printing background CaloHitList length: 682
Printing New CaloHitList Names: 
SCaloHitListU | SCaloHitListV | SCaloHitListW | SCaloHitList2D | BBCaloHitList
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Photon identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
Printing U view candidate length: 18
Printing V view candidate length: 13
Printing W view candidate length: 13
Printing 2D view candidate length: 44
Printing background CaloHitList length: 1
Printing New CaloHitList Names: 
CaloHitListU | CaloHitListV | CaloHitListW | CaloHitList2D | BCaloHitList
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 02:07:41 BST run: 5089 subRun: 0 event: 207200
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 02:07:41 BST run: 5089 subRun: 0 event: 207200
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 02:07:41 BST run: 5089 subRun: 0 event: 207200
Comparing two wires in the same plane: return failure
%MSG
54572
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 71088!
Genrating OpFlash 0 with 3 hits
Genrating OpFlash 1 with 3 hits
Genrating OpFlash 2 with 7 hits
Genrating OpFlash 3 with 4 hits
Genrating OpFlash 4 with 4 hits
Genrating OpFlash 5 with 5 hits
Genrating OpFlash 6 with 5 hits
Genrating OpFlash 7 with 3 hits
Genrating OpFlash 8 with 6 hits
Genrating OpFlash 9 with 3 hits
...
Generated 2148 OpFlashes
25-Jul-2025 02:18:22 BST  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T121644Z_gen_004144_supernova_g4_detsim_reco_20250724T123959Z_reco2.root"
25-Jul-2025 02:18:22 BST  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/e1/65/prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T121644Z_gen_004144_supernova_g4_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              593.204       886.341       1056.25       931.779       139.75         50     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                  0.13026      0.184883      0.599831      0.138265      0.092755        50     
reco:linecluster:LineCluster                            1.1698        1.2272        1.50886       1.21669      0.0561252       50     
reco:trajcluster:TrajCluster                           0.382198      0.401575      0.669785      0.392446      0.0399557       50     
reco:pandora:StandardPandora                            201.721       291.474       340.594       311.834       42.355         50     
reco:pandoraTrack:LArPandoraTrackCreation             0.000488246   0.00864293     0.0280112    0.00909156    0.00491878       50     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000561533   0.00451995     0.0568672     0.0028775    0.00855033       50     
reco:pandoracalo:Calorimetry                          0.000348381   0.00114159    0.00974604    0.000918381   0.00127498       50     
reco:pandorapid:Chi2ParticleID                        0.000117266   0.000337946   0.00379517    0.000273278   0.000499266      50     
reco:pmtracktc:PMAlgTrackMaker                        0.00520512     0.148989       2.51092      0.0740828     0.351553        50     
reco:pmtracktccalo:Calorimetry                        0.00021541    0.00280008     0.0434871    0.00112655    0.00813388       50     
reco:pmtracktcpid:Chi2ParticleID                      8.0734e-05    0.000306152   0.000502343   0.000306976   0.000101881      50     
reco:emtrkmichelid:EmTrackMichelId                      17.1685       26.5755       33.8401       28.4438       4.75376        50     
reco:opdec:Deconvolution                                47.401        65.9343       76.2782       67.7673       7.24964        50     
reco:ophitspe:OpHitFinderDeco                           288.853       466.893       577.672       503.23        89.1544        50     
reco:opflash:OpFlashFinder                              25.5428       31.8236       37.0682       32.0727       2.49887        50     
reco:solarflash:SolarOpFlash                           0.571904       1.0871        1.52632       1.18465      0.285873        50     
reco:rns:RandomNumberSaver                            9.0113e-05    0.000166669   0.000643829   0.000153966   8.22852e-05      50     
[art]:TriggerResults:TriggerResultInserter            3.5603e-05    8.23395e-05   0.000222518   7.25865e-05   3.70852e-05      50     
end_path:out1:RootOutput                              1.0994e-05    2.3121e-05    7.0663e-05    1.8629e-05    1.22686e-05      50     
end_path:out1:RootOutput(write)                        0.359837      0.567548      0.816839      0.590629      0.115552        50     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3270.09 MB
  Peak resident set size usage (VmHWM): 1768.97 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T121644Z_gen_004144_supernova_g4_detsim_reco_20250724T123959Z_reco2.root
Extracted metadata
checking file====== fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T121644Z_gen_004144_supernova_g4_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T121644Z_gen_004144_supernova_g4_detsim_reco_20250724T123959Z_reco2.root
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T121644Z_gen_004144_supernova_g4_detsim_reco_20250724T123959Z_reco2.root.ext.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T121644Z_gen_004144_supernova_g4_detsim_reco_20250724T123959Z_reco2.root.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T121644Z_gen_004144_supernova_g4_detsim_reco_20250724T123959Z_reco2.root.temp.json
reco2_hist.root