Jobsub ID 271859.8@dunegpschedd01.fnal.gov

Jobsub ID	271859.8@dunegpschedd01.fnal.gov
Workflow ID	11534
Stage ID	1
User name	avizcaya@fnal.gov
Requested	Processors	1
	GPU	No
	RSS bytes	5242880000 (5000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2026-01-05 17:58:37
Site	US_BNL
Entry	DUNE_US_BNL_spoolce01
Last heartbeat	2026-01-05 18:56:47
From worker node	Hostname	spool0207.sdcc.bnl.gov
	cpuinfo	AMD EPYC 7351 16-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	5242880000 (5000 MiB)
	Wall seconds limit	171000 (47 hours)
	GPU
	Inner Apptainer?	True
Job state	jobscript_error
Started	2026-01-05 18:50:54
Input files	usertests:pdhd_prod_beam__261612_29_1_20251212T155519Z_gen_g4_IonScintPDExt.root_268535_4_1_20251215T213114Z_PDInt.root_269119_96_1_20251218T203858Z_detsim_reco1.root
Jobscript	Exit code	1
	Real time	0m (0s)
	CPU time	0m (0s = 0%)
	Max RSS bytes	0 (0 MiB)
Outputting started
Output files
Finished	2026-01-05 18:56:47
Saved logs	justin-logs:271859.8-dunegpschedd01.fnal.gov.logs.tgz
List job events Cached HTCondor job logs
Jobscript log (last 10,000 characters)

rvice::ctor:     Found tool pdspTailPedRemovalZKe @ 0xaa5f830
ToolBasedRawDigitPrepService::ctor: Fetching wire building service.
ToolBasedRawDigitPrepService::ctor:   Wire building service: @0x6383e40
ToolBasedRawDigitPrepService::ctor: ToolBasedRawDigitPrepService:
ToolBasedRawDigitPrepService::ctor:                     LogLevel: 3
ToolBasedRawDigitPrepService::ctor:                      DoWires: 1
ToolBasedRawDigitPrepService::ctor:      ADC channel tools:
ToolBasedRawDigitPrepService::ctor:            digitReader
ToolBasedRawDigitPrepService::ctor:            adcSampleFiller
ToolBasedRawDigitPrepService::ctor:            adcScaleAdcToKe
ToolBasedRawDigitPrepService::ctor:            pdspTailPedRemovalZKe
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
05-Jan-2026 13:55:30 EST  Initiating request to open input file "root://dcdndoor.sdcc.bnl.gov:1094//pnfs/sdcc.bnl.gov/data/dune/RSE/usertests/b7/8d/pdhd_prod_beam__261612_29_1_20251212T155519Z_gen_g4_IonScintPDExt.root_268535_4_1_20251215T213114Z_PDInt.root_269119_96_1_20251218T203858Z_detsim_reco1.root"
05-Jan-2026 13:55:30 EST  Initiating request to open input file "root://dcdndoor.sdcc.bnl.gov:1094//pnfs/sdcc.bnl.gov/data/dune/RSE/usertests/b7/8d/pdhd_prod_beam__261612_29_1_20251212T155519Z_gen_g4_IonScintPDExt.root_268535_4_1_20251215T213114Z_PDInt.root_269119_96_1_20251218T203858Z_detsim_reco1.root"
05-Jan-2026 13:55:46 EST  Opened input file "root://dcdndoor.sdcc.bnl.gov:1094//pnfs/sdcc.bnl.gov/data/dune/RSE/usertests/b7/8d/pdhd_prod_beam__261612_29_1_20251212T155519Z_gen_g4_IonScintPDExt.root_268535_4_1_20251215T213114Z_PDInt.root_269119_96_1_20251218T203858Z_detsim_reco1.root"
05-Jan-2026 13:55:46 EST  Opened input file "root://dcdndoor.sdcc.bnl.gov:1094//pnfs/sdcc.bnl.gov/data/dune/RSE/usertests/b7/8d/pdhd_prod_beam__261612_29_1_20251212T155519Z_gen_g4_IonScintPDExt.root_268535_4_1_20251215T213114Z_PDInt.root_269119_96_1_20251218T203858Z_detsim_reco1.root"
Setting applicationFamily: "art"
Setting process_name: "PDHDAna"
db: runtime0
%MSG-w PhotonBackTrackerService:  ProcessEvent 05-Jan-2026 13:56:04 EST  run: 20250627 subRun: 1 event: 170
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
########## EvtNo.170
%MSG-w PDSPAnalyzer:  PDSPAnalyzer:pduneana@BeginModule  05-Jan-2026 13:56:07 EST run: 20250627 subRun: 1 event: 170 PDSPAnalyzer_module.cc:4729
pandora2Track object not found, moving on
%MSG
%MSG-w PDSPAnalyzer:  PDSPAnalyzer:pduneana@BeginModule  05-Jan-2026 13:56:07 EST run: 20250627 subRun: 1 event: 170 PDSPAnalyzer_module.cc:4850
pandora2Shower object not found, moving on
%MSG
%MSG-w PDSPAnalyzer:  PDSPAnalyzer:pduneana@BeginModule  05-Jan-2026 13:56:07 EST run: 20250627 subRun: 1 event: 170 PDSPAnalyzer_module.cc:4729
pandora2Track object not found, moving on
%MSG
%MSG-w PDSPAnalyzer:  PDSPAnalyzer:pduneana@BeginModule  05-Jan-2026 13:56:07 EST run: 20250627 subRun: 1 event: 170 PDSPAnalyzer_module.cc:4850
pandora2Shower object not found, moving on
%MSG
%MSG-w PDSPAnalyzer:  PDSPAnalyzer:pduneana@BeginModule  05-Jan-2026 13:56:08 EST run: 20250627 subRun: 1 event: 170 PDSPAnalyzer_module.cc:4729
pandora2Track object not found, moving on
%MSG
%MSG-w PDSPAnalyzer:  PDSPAnalyzer:pduneana@BeginModule  05-Jan-2026 13:56:08 EST run: 20250627 subRun: 1 event: 170 PDSPAnalyzer_module.cc:4850
pandora2Shower object not found, moving on
%MSG
%MSG-w PDSPAnalyzer:  PDSPAnalyzer:pduneana@BeginModule  05-Jan-2026 13:56:08 EST run: 20250627 subRun: 1 event: 170 PDSPAnalyzer_module.cc:5239
beam pandora2Track object not found, moving on
%MSG
%MSG-w PhotonBackTrackerService:  ProcessEvent 05-Jan-2026 13:56:28 EST  run: 20250627 subRun: 1 event: 171
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
########## EvtNo.171
05-Jan-2026 13:56:28 EST  Closed input file "root://dcdndoor.sdcc.bnl.gov:1094//pnfs/sdcc.bnl.gov/data/dune/RSE/usertests/b7/8d/pdhd_prod_beam__261612_29_1_20251212T155519Z_gen_g4_IonScintPDExt.root_268535_4_1_20251215T213114Z_PDInt.root_269119_96_1_20251218T203858Z_detsim_reco1.root"
05-Jan-2026 13:56:28 EST  Closed input file "root://dcdndoor.sdcc.bnl.gov:1094//pnfs/sdcc.bnl.gov/data/dune/RSE/usertests/b7/8d/pdhd_prod_beam__261612_29_1_20251212T155519Z_gen_g4_IonScintPDExt.root_268535_4_1_20251215T213114Z_PDInt.root_269119_96_1_20251218T203858Z_detsim_reco1.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                        0.000442113     5.93364       11.8668       5.93364       5.9332          2     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000442113    0.0195126     0.038583      0.0195126     0.0190704        2     
end_path:pduneana:PDSPAnalyzer      11.8277       11.8277       11.8277       11.8277          0            1     
====================================================================================================================

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                        0.000442113     5.93364       11.8668       5.93364       5.9332          2     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000442113    0.0195126     0.038583      0.0195126     0.0190704        2     
end_path:pduneana:PDSPAnalyzer      11.8277       11.8277       11.8277       11.8277          0            1     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3622.21 MB
  Peak resident set size usage (VmHWM): 2553.72 MB
====================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3622.21 MB
  Peak resident set size usage (VmHWM): 2553.72 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 2 passed = 2 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport          2          1          1 pduneana

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 19.782562 Real = 38.866914

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 3622.21 VmHWM = 2553.72

ToolBasedRawDigitPrepService:dtor: Event count: 0
ToolBasedRawDigitPrepService:dtor:  Call count: 0
ToolBasedRawDigitPrepService:dtor: Time report for 4 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.00 sec
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.00 sec
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.00 sec
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   0.00 sec
%MSG-s ArtException:  PostEndJob 05-Jan-2026 13:56:28 EST ModuleEndJob
---- EventProcessorFailure BEGIN
  EventProcessor: an exception occurred during current event processing
  ---- EventProcessorFailure BEGIN
    EndPathExecutor: an exception occurred during current event processing
    ---- ScheduleExecutionFailure BEGIN
      Path: ProcessingStopped.
      ---- ProductNotFound BEGIN
        Found zero products matching all selection criteria
          C++ type: std::vector<recob::SpacePoint>
          Module label: 'reco3d'
          Product instance name: ''
          Process name: (empty)
        The above exception was thrown while processing module PDSPAnalyzer/pduneana run: 20250627 subRun: 1 event: 171
      ---- ProductNotFound END
      Exception going through path end_path
    ---- ScheduleExecutionFailure END
  ---- EventProcessorFailure END
---- EventProcessorFailure END
%MSG
%MSG-s ArtException:  PostEndJob 05-Jan-2026 13:56:28 EST ModuleEndJob
---- EventProcessorFailure BEGIN
  EventProcessor: an exception occurred during current event processing
  ---- EventProcessorFailure BEGIN
    EndPathExecutor: an exception occurred during current event processing
    ---- ScheduleExecutionFailure BEGIN
      Path: ProcessingStopped.
      ---- ProductNotFound BEGIN
        Found zero products matching all selection criteria
          C++ type: std::vector<recob::SpacePoint>
          Module label: 'reco3d'
          Product instance name: ''
          Process name: (empty)
        The above exception was thrown while processing module PDSPAnalyzer/pduneana run: 20250627 subRun: 1 event: 171
      ---- ProductNotFound END
      Exception going through path end_path
    ---- ScheduleExecutionFailure END
  ---- EventProcessorFailure END
---- EventProcessorFailure END
%MSG
Art has completed and will exit with status 1.