Jobsub ID 29125.46@dunegpschedd02.fnal.gov

Jobsub ID	29125.46@dunegpschedd02.fnal.gov
Workflow ID	1892
Stage ID	1
User name	ichong@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-08-29 17:25:53
Site	CERN
Entry	CMSHTPC_T2_CH_CERN_ce504
Last heartbeat	2025-08-29 18:13:22
From worker node	Hostname	b9p10p4449.cern.ch
	cpuinfo	AMD EPYC 7543 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	343800 (95 hours)
	GPU
	Inner Apptainer?	True
Job state	outputting_failed
Started	2025-08-29 18:03:55
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66020489_363_20231205T025320Z_gen_g4_detsim_hitreco__20240510T020204Z_reco2.root
Jobscript	Exit code	0
	Real time	9m (545s)
	CPU time	6m (373s = 68%)
	Max RSS bytes	1851527168 (1765 MiB)
Outputting started	2025-08-29 18:13:01
Output files
Finished	2025-08-29 18:13:22
List job events Cached HTCondor job logs
Jobscript log (last 10,000 characters)

0.000899398    0.0550587      1.13352     0.00463874     0.157291        100    
reco:opdec:Deconvolution                               0.0761207     0.207422      0.392717      0.201089      0.0707901       100    
reco:ophitspe:OpHitFinderDeco                          0.142801      0.148796      0.178551      0.148526     0.00413467       100    
reco:opflash:OpFlashFinder                            0.000319389   0.00139582    0.00568187    0.00126603    0.000741244      100    
reco:opslicer:OpSlicer                                 0.0019505     0.303391      0.936296      0.228691      0.237843        100    
reco:rns:RandomNumberSaver                             2.125e-05    3.61327e-05   0.000472139   2.9855e-05    4.43736e-05      100    
reco:anglereconue:NuAngularReco                       0.00017029    0.00184242     0.0248743    0.000466524   0.00361775       100    
reco:anglereconumu:NuAngularReco                       8.773e-05    0.00164755     0.0243327    0.00031725    0.00351423       100    
reco:anglereconuepfps:NuAngularReco                   0.00012962    0.00178485     0.0246794    0.000397939   0.00355861       100    
reco:anglereconumupfps:NuAngularReco                  0.00011052    0.00173215     0.024557     0.000390254   0.00358279       100    
reco:anglerecohits:NuAngularReco                      0.000354389   0.00251694     0.0279208    0.000754798   0.00426995       100    
[art]:TriggerResults:TriggerResultInserter             9.93e-06     1.34959e-05    5.704e-05    1.2455e-05    5.27831e-06      100    
end_path:out1:RootOutput                               2.82e-06     3.80049e-06    2.22e-05       3.5e-06     1.91922e-06      100    
end_path:out1:RootOutput(write)                        0.0161569     0.270599      0.552891      0.315497      0.144802        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 20690.9 MB
  Peak resident set size usage (VmHWM): 1851.53 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
%MSG
%MSG-w DUNEAna:  EnergyReco:energyrecnc@BeginModule 29-Aug-2025 20:12:59 CEST  run: 66020489 subRun: 1 event: 36400
 Please note: accessing PFParticle tracks through this method is not the recommended workflow.
 Please use DUNEAnaEventUtils::GetPFParticles and access the tracks with DUNEAnaPFParticleUtils::GetTrack.
%MSG
%MSG-w DUNEAna:  EnergyReco:energyrecnc@BeginModule 29-Aug-2025 20:12:59 CEST  run: 66020489 subRun: 1 event: 36400
 Please note: accessing PFParticle showers through this method is not the recommended workflow.
 Please use DUNEAnaEventUtils::GetPFParticles and access the tracks with DUNEAnaPFParticleUtils::GetShower.
%MSG
%MSG-w DUNEAna:  NuAngularReco:anglereconue@BeginModule  29-Aug-2025 20:12:59 CEST run: 66020489 subRun: 1 event: 36400
 Please note: accessing PFParticle tracks through this method is not the recommended workflow.
 Please use DUNEAnaEventUtils::GetPFParticles and access the tracks with DUNEAnaPFParticleUtils::GetTrack.
%MSG
%MSG-w DUNEAna:  NuAngularReco:anglereconue@BeginModule  29-Aug-2025 20:12:59 CEST run: 66020489 subRun: 1 event: 36400
 Please note: accessing PFParticle showers through this method is not the recommended workflow.
 Please use DUNEAnaEventUtils::GetPFParticles and access the tracks with DUNEAnaPFParticleUtils::GetShower.
%MSG
%MSG-w NeutrinoAngularRecoAlg:  NuAngularReco:anglereconue@BeginModule  29-Aug-2025 20:12:59 CEST run: 66020489 subRun: 1 event: 36400
 Cannot access the electron shower which is needed for this angular reconstruction method.
Not able to calculate neutrino direction
%MSG
%MSG-w DUNEAna:  NuAngularReco:anglereconumu@BeginModule  29-Aug-2025 20:12:59 CEST run: 66020489 subRun: 1 event: 36400
 Please note: accessing PFParticle tracks through this method is not the recommended workflow.
 Please use DUNEAnaEventUtils::GetPFParticles and access the tracks with DUNEAnaPFParticleUtils::GetTrack.
%MSG
%MSG-w DUNEAna:  NuAngularReco:anglereconumu@BeginModule  29-Aug-2025 20:12:59 CEST run: 66020489 subRun: 1 event: 36400
 Please note: accessing PFParticle showers through this method is not the recommended workflow.
 Please use DUNEAnaEventUtils::GetPFParticles and access the tracks with DUNEAnaPFParticleUtils::GetShower.
%MSG
%MSG-w NeutrinoAngularRecoAlg:  NuAngularReco:anglereconumu@BeginModule  29-Aug-2025 20:12:59 CEST run: 66020489 subRun: 1 event: 36400
 Cannot access the muon track which is needed for this angular reconstruction method.
Not able to calculate neutrino direction
%MSG
%MSG-w DUNEAna:  NuAngularReco:anglereconuepfps@BeginModule  29-Aug-2025 20:12:59 CEST run: 66020489 subRun: 1 event: 36400
 Please note: accessing PFParticle tracks through this method is not the recommended workflow.
 Please use DUNEAnaEventUtils::GetPFParticles and access the tracks with DUNEAnaPFParticleUtils::GetTrack.
%MSG
%MSG-w DUNEAna:  NuAngularReco:anglereconuepfps@BeginModule  29-Aug-2025 20:12:59 CEST run: 66020489 subRun: 1 event: 36400
 Please note: accessing PFParticle showers through this method is not the recommended workflow.
 Please use DUNEAnaEventUtils::GetPFParticles and access the tracks with DUNEAnaPFParticleUtils::GetShower.
%MSG
%MSG-w NeutrinoAngularRecoAlg:  NuAngularReco:anglereconuepfps@BeginModule  29-Aug-2025 20:12:59 CEST run: 66020489 subRun: 1 event: 36400
 No track or shower selected.
Not able to calculate neutrino direction
%MSG
%MSG-w DUNEAna:  NuAngularReco:anglereconumupfps@BeginModule  29-Aug-2025 20:12:59 CEST run: 66020489 subRun: 1 event: 36400
 Please note: accessing PFParticle tracks through this method is not the recommended workflow.
 Please use DUNEAnaEventUtils::GetPFParticles and access the tracks with DUNEAnaPFParticleUtils::GetTrack.
%MSG
%MSG-w DUNEAna:  NuAngularReco:anglereconumupfps@BeginModule  29-Aug-2025 20:12:59 CEST run: 66020489 subRun: 1 event: 36400
 Please note: accessing PFParticle showers through this method is not the recommended workflow.
 Please use DUNEAnaEventUtils::GetPFParticles and access the tracks with DUNEAnaPFParticleUtils::GetShower.
%MSG
%MSG-w DUNEAna:  NuAngularReco:anglereconumupfps@BeginModule  29-Aug-2025 20:12:59 CEST run: 66020489 subRun: 1 event: 36400
 Please note: accessing PFParticle tracks through this method is not the recommended workflow.
 Please use DUNEAnaEventUtils::GetPFParticles and access the tracks with DUNEAnaPFParticleUtils::GetTrack.
%MSG
%MSG-w NeutrinoAngularRecoAlg:  NuAngularReco:anglereconumupfps@BeginModule  29-Aug-2025 20:12:59 CEST run: 66020489 subRun: 1 event: 36400
 Cannot access the muon track which is needed for this angular reconstruction method.
Not able to calculate neutrino direction
%MSG
%MSG-w DUNEAna:  NuAngularReco:anglerecohits@BeginModule  29-Aug-2025 20:12:59 CEST run: 66020489 subRun: 1 event: 36400
 Please note: accessing PFParticle tracks through this method is not the recommended workflow.
 Please use DUNEAnaEventUtils::GetPFParticles and access the tracks with DUNEAnaPFParticleUtils::GetTrack.
%MSG
%MSG-w DUNEAna:  NuAngularReco:anglerecohits@BeginModule  29-Aug-2025 20:12:59 CEST run: 66020489 subRun: 1 event: 36400
 Please note: accessing PFParticle showers through this method is not the recommended workflow.
 Please use DUNEAnaEventUtils::GetPFParticles and access the tracks with DUNEAnaPFParticleUtils::GetShower.
%MSG
%MSG-i CAFMaker:  CAFMaker:cafmaker@BeginModule 29-Aug-2025 20:12:59 CEST  run: 66020489 subRun: 1 event: 36400
Geo name is: dune10kt_v5_1x2x6
%MSG
%MSG-i CAFMaker:  CAFMaker:cafmaker@BeginModule 29-Aug-2025 20:12:59 CEST  run: 66020489 subRun: 1 event: 36400
Assuming the FD HD detector
%MSG
29-Aug-2025 20:13:00 CEST  Closed output file "caf_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66020489_363_20231205T025320Z_gen_g4_detsim_hitreco__20240510T020204Z_reco2_truthvtx_2025-08-29T_181202Z.root"
29-Aug-2025 20:13:00 CEST  Closed input file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66020489_363_20231205T025320Z_gen_g4_detsim_hitreco__20240510T020204Z_reco2_truthvtx_2025-08-29T_180402Z.root"
%MSG-i NuRandomService:  RootOutput:out@EndJob 29-Aug-2025 20:13:00 CEST  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1

%MSG

TrigReport ---------- Event summary -------------
TrigReport Events total = 100 passed = 100 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport        100        100          0 cafmaker
TrigReport        100        100          0 out

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 35.307400 Real = 36.162363

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 19222.7 VmHWM = 971.612

Art has completed and will exit with status 0.
=== End last  100 lines of lar log file ===
=== Generated output files ===
29125.46_dunegpschedd02.fnal.gov.logs.tgz
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66020489_363_20231205T025320Z_gen_g4_detsim_hitreco__20240510T020204Z_reco2_truthvtx_2025-08-29T_180402Z.root
caf_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66020489_363_20231205T025320Z_gen_g4_detsim_hitreco__20240510T020204Z_reco2_truthvtx_2025-08-29T_181202Z.log
caf_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66020489_363_20231205T025320Z_gen_g4_detsim_hitreco__20240510T020204Z_reco2_truthvtx_2025-08-29T_181202Z.root
caf_truthvtx_2025-08-29T_181202Z_caf.root
debugprod.log
flatcaf.root
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
truthvtx_2025-08-29T_180402Z_vtx.log