Jobsub ID 30642.40@dunegpschedd02.fnal.gov

Jobsub ID	30642.40@dunegpschedd02.fnal.gov
Workflow ID	1901
Stage ID	1
User name	ichong@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-08-31 10:21:06
Site	FR_CCIN2P3
Entry	DUNE_FR_CCIN2P3_cccondorce01
Last heartbeat	2025-08-31 10:28:10
From worker node	Hostname	ccwcondor0384
	cpuinfo	AMD EPYC 9334 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	106200 (29 hours)
	GPU
	Inner Apptainer?	True
Job state	outputting_failed
Started	2025-08-31 10:21:45
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6400837_427_20231202T082512Z_gen_g4_detsim_hitreco__20240507T205315Z_reco2.root
Jobscript	Exit code	0
	Real time	6m (362s)
	CPU time	5m (320s = 88%)
	Max RSS bytes	2047934464 (1953 MiB)
Outputting started	2025-08-31 10:27:48
Output files
Finished	2025-08-31 10:28:10
List job events Cached HTCondor job logs
Jobscript log (last 10,000 characters)

ergyReco                  0.000581093    0.027098      0.951763     0.00290955      0.11053        100    
reco:opdec:Deconvolution                               0.0281285     0.149788      0.295564      0.151137      0.0644145       100    
reco:ophitspe:OpHitFinderDeco                         0.00405424     0.0093185     0.0183866    0.00922627     0.0029908       100    
reco:opflash:OpFlashFinder                            0.000287497   0.00134726     0.0074789    0.00126784    0.000846927      100    
reco:opslicer:OpSlicer                                0.00135488     0.233418      0.859384       0.20999       0.1942         100    
reco:rns:RandomNumberSaver                            2.3296e-05    5.10881e-05   0.000851114   3.98455e-05   8.12839e-05      100    
reco:anglereconue:NuAngularReco                       0.000172001   0.00133611     0.0313136    0.000422627   0.00344052       100    
reco:anglereconumu:NuAngularReco                      5.1839e-05    0.00105912     0.0290694    0.000214647   0.00321271       100    
reco:anglereconuepfps:NuAngularReco                   0.00011252    0.00124645     0.0294728    0.000352611   0.00336022       100    
reco:anglereconumupfps:NuAngularReco                   8.589e-05     0.0011306     0.0296218    0.000293416    0.0032868       100    
reco:anglerecohits:NuAngularReco                      0.000344353   0.00185399     0.0347582    0.00068152    0.00410269       100    
[art]:TriggerResults:TriggerResultInserter             9.124e-06    1.58405e-05   0.000113071   1.4522e-05    1.01786e-05      100    
end_path:out1:RootOutput                               3.315e-06    5.18451e-06    2.642e-05     4.627e-06     2.468e-06       100    
end_path:out1:RootOutput(write)                        0.016836      0.223808      0.596225      0.277031      0.136647        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 20786.6 MB
  Peak resident set size usage (VmHWM): 2047.93 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
%MSG
%MSG-w DUNEAna:  EnergyReco:energyrecnc@BeginModule 31-Aug-2025 12:27:46 CEST  run: 6400837 subRun: 1 event: 42800
 Please note: accessing PFParticle tracks through this method is not the recommended workflow.
 Please use DUNEAnaEventUtils::GetPFParticles and access the tracks with DUNEAnaPFParticleUtils::GetTrack.
%MSG
%MSG-w DUNEAna:  EnergyReco:energyrecnc@BeginModule 31-Aug-2025 12:27:46 CEST  run: 6400837 subRun: 1 event: 42800
 Please note: accessing PFParticle showers through this method is not the recommended workflow.
 Please use DUNEAnaEventUtils::GetPFParticles and access the tracks with DUNEAnaPFParticleUtils::GetShower.
%MSG
%MSG-w DUNEAna:  NuAngularReco:anglereconue@BeginModule  31-Aug-2025 12:27:46 CEST run: 6400837 subRun: 1 event: 42800
 Please note: accessing PFParticle tracks through this method is not the recommended workflow.
 Please use DUNEAnaEventUtils::GetPFParticles and access the tracks with DUNEAnaPFParticleUtils::GetTrack.
%MSG
%MSG-w DUNEAna:  NuAngularReco:anglereconue@BeginModule  31-Aug-2025 12:27:46 CEST run: 6400837 subRun: 1 event: 42800
 Please note: accessing PFParticle showers through this method is not the recommended workflow.
 Please use DUNEAnaEventUtils::GetPFParticles and access the tracks with DUNEAnaPFParticleUtils::GetShower.
%MSG
%MSG-w NeutrinoAngularRecoAlg:  NuAngularReco:anglereconue@BeginModule  31-Aug-2025 12:27:46 CEST run: 6400837 subRun: 1 event: 42800
 Cannot access the electron shower which is needed for this angular reconstruction method.
Not able to calculate neutrino direction
%MSG
%MSG-w DUNEAna:  NuAngularReco:anglereconumu@BeginModule  31-Aug-2025 12:27:46 CEST run: 6400837 subRun: 1 event: 42800
 Please note: accessing PFParticle tracks through this method is not the recommended workflow.
 Please use DUNEAnaEventUtils::GetPFParticles and access the tracks with DUNEAnaPFParticleUtils::GetTrack.
%MSG
%MSG-w DUNEAna:  NuAngularReco:anglereconumu@BeginModule  31-Aug-2025 12:27:46 CEST run: 6400837 subRun: 1 event: 42800
 Please note: accessing PFParticle showers through this method is not the recommended workflow.
 Please use DUNEAnaEventUtils::GetPFParticles and access the tracks with DUNEAnaPFParticleUtils::GetShower.
%MSG
%MSG-w NeutrinoAngularRecoAlg:  NuAngularReco:anglereconumu@BeginModule  31-Aug-2025 12:27:46 CEST run: 6400837 subRun: 1 event: 42800
 Cannot access the muon track which is needed for this angular reconstruction method.
Not able to calculate neutrino direction
%MSG
%MSG-w DUNEAna:  NuAngularReco:anglereconuepfps@BeginModule  31-Aug-2025 12:27:46 CEST run: 6400837 subRun: 1 event: 42800
 Please note: accessing PFParticle tracks through this method is not the recommended workflow.
 Please use DUNEAnaEventUtils::GetPFParticles and access the tracks with DUNEAnaPFParticleUtils::GetTrack.
%MSG
%MSG-w DUNEAna:  NuAngularReco:anglereconuepfps@BeginModule  31-Aug-2025 12:27:46 CEST run: 6400837 subRun: 1 event: 42800
 Please note: accessing PFParticle showers through this method is not the recommended workflow.
 Please use DUNEAnaEventUtils::GetPFParticles and access the tracks with DUNEAnaPFParticleUtils::GetShower.
%MSG
%MSG-w NeutrinoAngularRecoAlg:  NuAngularReco:anglereconuepfps@BeginModule  31-Aug-2025 12:27:46 CEST run: 6400837 subRun: 1 event: 42800
 No track or shower selected.
Not able to calculate neutrino direction
%MSG
%MSG-w DUNEAna:  NuAngularReco:anglereconumupfps@BeginModule  31-Aug-2025 12:27:46 CEST run: 6400837 subRun: 1 event: 42800
 Please note: accessing PFParticle tracks through this method is not the recommended workflow.
 Please use DUNEAnaEventUtils::GetPFParticles and access the tracks with DUNEAnaPFParticleUtils::GetTrack.
%MSG
%MSG-w DUNEAna:  NuAngularReco:anglereconumupfps@BeginModule  31-Aug-2025 12:27:46 CEST run: 6400837 subRun: 1 event: 42800
 Please note: accessing PFParticle showers through this method is not the recommended workflow.
 Please use DUNEAnaEventUtils::GetPFParticles and access the tracks with DUNEAnaPFParticleUtils::GetShower.
%MSG
%MSG-w DUNEAna:  NuAngularReco:anglereconumupfps@BeginModule  31-Aug-2025 12:27:46 CEST run: 6400837 subRun: 1 event: 42800
 Please note: accessing PFParticle tracks through this method is not the recommended workflow.
 Please use DUNEAnaEventUtils::GetPFParticles and access the tracks with DUNEAnaPFParticleUtils::GetTrack.
%MSG
%MSG-w NeutrinoAngularRecoAlg:  NuAngularReco:anglereconumupfps@BeginModule  31-Aug-2025 12:27:46 CEST run: 6400837 subRun: 1 event: 42800
 Cannot access the muon track which is needed for this angular reconstruction method.
Not able to calculate neutrino direction
%MSG
%MSG-w DUNEAna:  NuAngularReco:anglerecohits@BeginModule  31-Aug-2025 12:27:46 CEST run: 6400837 subRun: 1 event: 42800
 Please note: accessing PFParticle tracks through this method is not the recommended workflow.
 Please use DUNEAnaEventUtils::GetPFParticles and access the tracks with DUNEAnaPFParticleUtils::GetTrack.
%MSG
%MSG-w DUNEAna:  NuAngularReco:anglerecohits@BeginModule  31-Aug-2025 12:27:46 CEST run: 6400837 subRun: 1 event: 42800
 Please note: accessing PFParticle showers through this method is not the recommended workflow.
 Please use DUNEAnaEventUtils::GetPFParticles and access the tracks with DUNEAnaPFParticleUtils::GetShower.
%MSG
%MSG-i CAFMaker:  CAFMaker:cafmaker@BeginModule 31-Aug-2025 12:27:46 CEST  run: 6400837 subRun: 1 event: 42800
Geo name is: dune10kt_v5_1x2x6
%MSG
%MSG-i CAFMaker:  CAFMaker:cafmaker@BeginModule 31-Aug-2025 12:27:46 CEST  run: 6400837 subRun: 1 event: 42800
Assuming the FD HD detector
%MSG
31-Aug-2025 12:27:46 CEST  Closed output file "caf_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6400837_427_20231202T082512Z_gen_g4_detsim_hitreco__20240507T205315Z_reco2_truthvtx_2025-08-31T_102701Z.root"
31-Aug-2025 12:27:46 CEST  Closed input file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6400837_427_20231202T082512Z_gen_g4_detsim_hitreco__20240507T205315Z_reco2_truthvtx_2025-08-31T_102151Z.root"
%MSG-i NuRandomService:  RootOutput:out@EndJob 31-Aug-2025 12:27:46 CEST  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1

%MSG

TrigReport ---------- Event summary -------------
TrigReport Events total = 100 passed = 100 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport        100        100          0 cafmaker
TrigReport        100        100          0 out

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 22.275633 Real = 23.412696

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 19318.4 VmHWM = 1060.75

Art has completed and will exit with status 0.
=== End last  100 lines of lar log file ===
=== Generated output files ===
30642.40_dunegpschedd02.fnal.gov.logs.tgz
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6400837_427_20231202T082512Z_gen_g4_detsim_hitreco__20240507T205315Z_reco2_truthvtx_2025-08-31T_102151Z.root
caf_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6400837_427_20231202T082512Z_gen_g4_detsim_hitreco__20240507T205315Z_reco2_truthvtx_2025-08-31T_102701Z.log
caf_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6400837_427_20231202T082512Z_gen_g4_detsim_hitreco__20240507T205315Z_reco2_truthvtx_2025-08-31T_102701Z.root
caf_truthvtx_2025-08-31T_102701Z_caf.root
debugprod.log
flatcaf.root
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
truthvtx_2025-08-31T_102151Z_vtx.log