Jobsub ID 23077.3@dunegpschedd01.fnal.gov

Jobsub ID	23077.3@dunegpschedd01.fnal.gov
Workflow ID	759
Stage ID	1
User name	ichong@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-08-11 01:38:00
Site	UK_Edinburgh
Entry	DUNE_UK_SGridECDF_ce1_multicore
Last heartbeat	2025-08-11 02:21:09
From worker node	Hostname	node2b04.ecdf.ed.ac.uk
	cpuinfo	Intel(R) Xeon(R) Gold 6338 CPU @ 2.00GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	GPU
	Inner Apptainer?	True
Job state	outputting_failed
Started	2025-08-11 01:39:04
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50609765_359_20231207T064900Z_gen_g4_detsim_hitreco__20240510T063909Z_reco2.root
Jobscript	Exit code	0
	Real time	41m (2507s)
	CPU time	24m (1450s = 57%)
	Max RSS bytes	2280722432 (2175 MiB)
Outputting started	2025-08-11 02:20:52
Output files
Finished	2025-08-11 02:21:09
List job events Cached HTCondor job logs
Jobscript log (last 10,000 characters)

GetTrack.
%MSG
%MSG-w DUNEAna:  EnergyReco:energyrecnc@BeginModule 11-Aug-2025 03:20:47 BST  run: 50609765 subRun: 1 event: 36000
 Please note: accessing PFParticle showers through this method is not the recommended workflow.
 Please use DUNEAnaEventUtils::GetPFParticles and access the tracks with DUNEAnaPFParticleUtils::GetShower.
%MSG
%MSG-w DUNEAna:  NuAngularReco:anglereconue@BeginModule  11-Aug-2025 03:20:47 BST run: 50609765 subRun: 1 event: 36000
 Please note: accessing PFParticle tracks through this method is not the recommended workflow.
 Please use DUNEAnaEventUtils::GetPFParticles and access the tracks with DUNEAnaPFParticleUtils::GetTrack.
%MSG
%MSG-w DUNEAna:  NuAngularReco:anglereconue@BeginModule  11-Aug-2025 03:20:47 BST run: 50609765 subRun: 1 event: 36000
 Please note: accessing PFParticle showers through this method is not the recommended workflow.
 Please use DUNEAnaEventUtils::GetPFParticles and access the tracks with DUNEAnaPFParticleUtils::GetShower.
%MSG
%MSG-w DUNEAna:  NuAngularReco:anglereconumu@BeginModule  11-Aug-2025 03:20:47 BST run: 50609765 subRun: 1 event: 36000
 Please note: accessing PFParticle tracks through this method is not the recommended workflow.
 Please use DUNEAnaEventUtils::GetPFParticles and access the tracks with DUNEAnaPFParticleUtils::GetTrack.
%MSG
%MSG-w DUNEAna:  NuAngularReco:anglereconumu@BeginModule  11-Aug-2025 03:20:47 BST run: 50609765 subRun: 1 event: 36000
 Please note: accessing PFParticle showers through this method is not the recommended workflow.
 Please use DUNEAnaEventUtils::GetPFParticles and access the tracks with DUNEAnaPFParticleUtils::GetShower.
%MSG
%MSG-w DUNEAna:  NuAngularReco:anglereconuepfps@BeginModule  11-Aug-2025 03:20:47 BST run: 50609765 subRun: 1 event: 36000
 Please note: accessing PFParticle tracks through this method is not the recommended workflow.
 Please use DUNEAnaEventUtils::GetPFParticles and access the tracks with DUNEAnaPFParticleUtils::GetTrack.
%MSG
%MSG-w DUNEAna:  NuAngularReco:anglereconuepfps@BeginModule  11-Aug-2025 03:20:47 BST run: 50609765 subRun: 1 event: 36000
 Please note: accessing PFParticle showers through this method is not the recommended workflow.
 Please use DUNEAnaEventUtils::GetPFParticles and access the tracks with DUNEAnaPFParticleUtils::GetShower.
%MSG
%MSG-w DUNEAna:  NuAngularReco:anglereconumupfps@BeginModule  11-Aug-2025 03:20:47 BST run: 50609765 subRun: 1 event: 36000
 Please note: accessing PFParticle tracks through this method is not the recommended workflow.
 Please use DUNEAnaEventUtils::GetPFParticles and access the tracks with DUNEAnaPFParticleUtils::GetTrack.
%MSG
%MSG-w DUNEAna:  NuAngularReco:anglereconumupfps@BeginModule  11-Aug-2025 03:20:47 BST run: 50609765 subRun: 1 event: 36000
 Please note: accessing PFParticle showers through this method is not the recommended workflow.
 Please use DUNEAnaEventUtils::GetPFParticles and access the tracks with DUNEAnaPFParticleUtils::GetShower.
%MSG
%MSG-w DUNEAna:  NuAngularReco:anglereconumupfps@BeginModule  11-Aug-2025 03:20:47 BST run: 50609765 subRun: 1 event: 36000
 Please note: accessing PFParticle tracks through this method is not the recommended workflow.
 Please use DUNEAnaEventUtils::GetPFParticles and access the tracks with DUNEAnaPFParticleUtils::GetTrack.
%MSG
%MSG-i CAFMaker:  CAFMaker:cafmaker@BeginModule 11-Aug-2025 03:20:47 BST  run: 50609765 subRun: 1 event: 36000
Geo name is: dune10kt_v5_1x2x6
%MSG
%MSG-i CAFMaker:  CAFMaker:cafmaker@BeginModule 11-Aug-2025 03:20:47 BST  run: 50609765 subRun: 1 event: 36000
Assuming the FD HD detector
%MSG
%MSG-w CAFMaker:  CAFMaker:cafmaker@BeginModule 11-Aug-2025 03:20:47 BST  run: 50609765 subRun: 1 event: 36000
InputTag: label = 'regcnneval', instance = 'regcnnresult' does not correspond to a valid RegCNNResult product
%MSG
11-Aug-2025 03:20:48 BST  Closed output file "caf_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50609765_359_20231207T064900Z_gen_g4_detsim_hitreco__20240510T063909Z_reco2_truthvtx_2025-08-11T_021935Z.root"
11-Aug-2025 03:20:48 BST  Closed input file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50609765_359_20231207T064900Z_gen_g4_detsim_hitreco__20240510T063909Z_reco2_truthvtx_2025-08-11T_013908Z.root"
%MSG-i NuRandomService:  RootOutput:out@EndJob 11-Aug-2025 03:20:48 BST  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1

%MSG

TrigReport ---------- Event summary -------------
TrigReport Events total = 100 passed = 100 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport        100        100          0 cafmaker
TrigReport        100        100          0 out

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 55.733928 Real = 56.396739

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 10842.1 VmHWM = 1194.61

Art has completed and will exit with status 0.
=== End last  100 lines of lar log file ===
[2025-08-11 03:20:48.803069 +0100][Debug  ][Utility           ] Initializing xrootd client version: v5.5.5
[2025-08-11 03:20:48.803328 +0100][Debug  ][Utility           ] Unable to process user config file: [ERROR] OS Error: no such file or directory
[2025-08-11 03:20:48.803494 +0100][Debug  ][PlugInMgr         ] Initializing plug-in manager...
[2025-08-11 03:20:48.803502 +0100][Debug  ][PlugInMgr         ] No default plug-in, loading plug-in configs...
[2025-08-11 03:20:48.803507 +0100][Debug  ][PlugInMgr         ] Processing plug-in definitions in /etc/xrootd/client.plugins.d...
[2025-08-11 03:20:48.803670 +0100][Debug  ][PlugInMgr         ] Trying to disable plug-in for '*'
[2025-08-11 03:20:48.803699 +0100][Debug  ][PlugInMgr         ] Processing plug-in definitions in /home/.xrootd/client.plugins.d...
[2025-08-11 03:20:48.803707 +0100][Debug  ][PlugInMgr         ] Unable to process directory /home/.xrootd/client.plugins.d: [ERROR] OS Error: no such file or directory
[2025-08-11 03:20:48.805017 +0100][Debug  ][Utility           ] In the prepare fork handler for process 1346
[2025-08-11 03:20:48.805040 +0100][Debug  ][Utility           ] Running the prepare fork handler for process 1346
[2025-08-11 03:20:48.805044 +0100][Debug  ][Utility           ] Locking File and FileSystem objects for process: 1346
[2025-08-11 03:20:48.805378 +0100][Debug  ][Utility           ] In the parent fork handler for process 1346
[2025-08-11 03:20:48.805399 +0100][Debug  ][Utility           ][ 1346] Running the parent fork handler for process 1346
[2025-08-11 03:20:48.805412 +0100][Debug  ][Utility           ][ 1346] Unlocking File and FileSystem objects for process:  1346
[2025-08-11 03:20:48.805650 +0100][Debug  ][Utility           ] In the child fork handler for process 1348
[2025-08-11 03:20:48.805742 +0100][Debug  ][Utility           ][ 1348] Running the child fork handler for process 1348
[2025-08-11 03:20:48.805764 +0100][Debug  ][Utility           ][ 1348] Unlocking File and FileSystem objects for process:  1348
[2025-08-11 03:20:48.843777 +0100][Debug  ][Utility           ] Initializing xrootd client version: v5.5.5
[2025-08-11 03:20:48.844058 +0100][Debug  ][Utility           ] Unable to process user config file: [ERROR] OS Error: no such file or directory
[2025-08-11 03:20:48.844235 +0100][Debug  ][PlugInMgr         ] Initializing plug-in manager...
[2025-08-11 03:20:48.844245 +0100][Debug  ][PlugInMgr         ] No default plug-in, loading plug-in configs...
[2025-08-11 03:20:48.844250 +0100][Debug  ][PlugInMgr         ] Processing plug-in definitions in /etc/xrootd/client.plugins.d...
[2025-08-11 03:20:48.844412 +0100][Debug  ][PlugInMgr         ] Trying to disable plug-in for '*'
[2025-08-11 03:20:48.844442 +0100][Debug  ][PlugInMgr         ] Processing plug-in definitions in /home/.xrootd/client.plugins.d...
[2025-08-11 03:20:48.844451 +0100][Debug  ][PlugInMgr         ] Unable to process directory /home/.xrootd/client.plugins.d: [ERROR] OS Error: no such file or directory
=== Generated output files ===
[2025-08-11 03:20:49.380685 +0100][Debug  ][Utility           ] Initializing xrootd client version: v5.5.5
[2025-08-11 03:20:49.380946 +0100][Debug  ][Utility           ] Unable to process user config file: [ERROR] OS Error: no such file or directory
[2025-08-11 03:20:49.381129 +0100][Debug  ][PlugInMgr         ] Initializing plug-in manager...
[2025-08-11 03:20:49.381140 +0100][Debug  ][PlugInMgr         ] No default plug-in, loading plug-in configs...
[2025-08-11 03:20:49.381145 +0100][Debug  ][PlugInMgr         ] Processing plug-in definitions in /etc/xrootd/client.plugins.d...
[2025-08-11 03:20:49.381317 +0100][Debug  ][PlugInMgr         ] Trying to disable plug-in for '*'
[2025-08-11 03:20:49.381348 +0100][Debug  ][PlugInMgr         ] Processing plug-in definitions in /home/.xrootd/client.plugins.d...
[2025-08-11 03:20:49.381356 +0100][Debug  ][PlugInMgr         ] Unable to process directory /home/.xrootd/client.plugins.d: [ERROR] OS Error: no such file or directory
23077.3_dunegpschedd01.fnal.gov.logs.tgz
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50609765_359_20231207T064900Z_gen_g4_detsim_hitreco__20240510T063909Z_reco2_truthvtx_2025-08-11T_013908Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50609765_359_20231207T064900Z_gen_g4_detsim_hitreco__20240510T063909Z_reco2_truthvtx_2025-08-11T_013908Z.root
caf_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50609765_359_20231207T064900Z_gen_g4_detsim_hitreco__20240510T063909Z_reco2_truthvtx_2025-08-11T_021935Z.log
caf_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50609765_359_20231207T064900Z_gen_g4_detsim_hitreco__20240510T063909Z_reco2_truthvtx_2025-08-11T_021935Z.root
caf_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50609765_359_20231207T064900Z_gen_g4_detsim_hitreco__20240510T063909Z_reco2_truthvtx_2025-08-11T_021935Z_caf.root
debugprod.log
flatcaf.root
jobscript.log
justin-processed-pfns.txt
reco_hist.root