Jobsub ID 37270.79@dunegpschedd01.fnal.gov

Jobsub ID	37270.79@dunegpschedd01.fnal.gov
Workflow ID	1901
Stage ID	1
User name	ichong@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-08-31 09:24:05
Site	FR_CCIN2P3
Entry	DUNE_FR_CCIN2P3_cccondorce02
Last heartbeat	2025-08-31 09:50:35
From worker node	Hostname	ccwcondor0401
	cpuinfo	AMD EPYC 9334 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	106200 (29 hours)
	GPU
	Inner Apptainer?	True
Job state	outputting_failed
Started	2025-08-31 09:44:16
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74553259_737_20231207T150220Z_gen_g4_detsim_hitreco__20240510T063535Z_reco2.root
Jobscript	Exit code	0
	Real time	5m (344s)
	CPU time	5m (315s = 91%)
	Max RSS bytes	2028584960 (1934 MiB)
Outputting started	2025-08-31 09:50:01
Output files
Finished	2025-08-31 09:50:35
List job events Cached HTCondor job logs
Jobscript log (last 10,000 characters)

0.0106873     0.0994093       100    
reco:energyrecnue:EnergyReco                          0.000140003    0.0022631     0.0521528    0.000564081   0.00610088       100    
reco:energyrecnc:EnergyReco                           0.00195761     0.0148532     0.0987463     0.0118191     0.016319        100    
reco:energyrecnumurange:EnergyReco                    0.000272544    0.0246747     0.735301     0.00232413     0.0912935       100    
reco:energyrecnumumcs:EnergyReco                      0.000962912    0.0242054     0.723537     0.00291707     0.0892688       100    
reco:energyrecnumumcsllhd:EnergyReco                  0.000757801    0.0243071     0.714297     0.00291265     0.0887267       100    
reco:opdec:Deconvolution                              0.00635584     0.140954      0.262172      0.140696      0.0575645       100    
reco:ophitspe:OpHitFinderDeco                         0.00417109     0.0165809     0.0433576     0.0135484    0.00846892       100    
reco:opflash:OpFlashFinder                            0.000133583   0.00118803    0.00403683    0.00109731    0.000610965      100    
reco:opslicer:OpSlicer                                4.0552e-05     0.227445      0.854235      0.175046       0.18784        100    
reco:rns:RandomNumberSaver                            2.2584e-05    4.59022e-05   0.000463373   3.7406e-05    4.32861e-05      100    
reco:anglereconue:NuAngularReco                       0.00018384    0.00138104     0.0370279    0.000406497   0.00425643       100    
reco:anglereconumu:NuAngularReco                      6.8724e-05    0.00115788     0.0365156    0.000176929   0.00421402       100    
reco:anglereconuepfps:NuAngularReco                   0.000114013   0.00129159     0.0367247    0.000298619   0.00423147       100    
reco:anglereconumupfps:NuAngularReco                  8.7974e-05    0.00118911     0.0354527    0.000225793   0.00412729       100    
reco:anglerecohits:NuAngularReco                      7.5745e-05    0.00182067     0.0398907    0.000581809   0.00473495       100    
[art]:TriggerResults:TriggerResultInserter            1.0326e-05    1.45984e-05   6.3046e-05    1.34005e-05   5.36494e-06      100    
end_path:out1:RootOutput                               4.067e-06    5.36256e-06   1.9339e-05    4.7825e-06    1.84302e-06      100    
end_path:out1:RootOutput(write)                       0.00363316     0.242241      0.562456      0.280227      0.125777        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 20771.6 MB
  Peak resident set size usage (VmHWM): 2028.58 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
%MSG-w CAFMaker:  CAFMaker:cafmaker@BeginModule 31-Aug-2025 11:49:58 CEST  run: 74553259 subRun: 1 event: 73800
Unknown property 'LArThreeDVertexDistanceFeatureTool_VertexDistance' for PFP with ID 1
%MSG
%MSG-w CAFMaker:  CAFMaker:cafmaker@BeginModule 31-Aug-2025 11:49:58 CEST  run: 74553259 subRun: 1 event: 73800
Unknown property 'LArPfoHierarchyFeatureTool_DaughterParentHitRatio' for PFP with ID 2
%MSG
%MSG-w CAFMaker:  CAFMaker:cafmaker@BeginModule 31-Aug-2025 11:49:58 CEST  run: 74553259 subRun: 1 event: 73800
Unknown property 'LArPfoHierarchyFeatureTool_NDaughterHits3D' for PFP with ID 2
%MSG
%MSG-w CAFMaker:  CAFMaker:cafmaker@BeginModule 31-Aug-2025 11:49:58 CEST  run: 74553259 subRun: 1 event: 73800
Unknown property 'LArThreeDChargeFeatureTool_EndFraction' for PFP with ID 2
%MSG
%MSG-w CAFMaker:  CAFMaker:cafmaker@BeginModule 31-Aug-2025 11:49:58 CEST  run: 74553259 subRun: 1 event: 73800
Unknown property 'LArThreeDChargeFeatureTool_FractionalSpread' for PFP with ID 2
%MSG
%MSG-w CAFMaker:  CAFMaker:cafmaker@BeginModule 31-Aug-2025 11:49:58 CEST  run: 74553259 subRun: 1 event: 73800
Unknown property 'LArThreeDLinearFitFeatureTool_DiffStraightLineMean' for PFP with ID 2
%MSG
%MSG-w CAFMaker:  CAFMaker:cafmaker@BeginModule 31-Aug-2025 11:49:58 CEST  run: 74553259 subRun: 1 event: 73800
Unknown property 'LArThreeDLinearFitFeatureTool_Length' for PFP with ID 2
%MSG
%MSG-w CAFMaker:  CAFMaker:cafmaker@BeginModule 31-Aug-2025 11:49:58 CEST  run: 74553259 subRun: 1 event: 73800
Unknown property 'LArThreeDLinearFitFeatureTool_MaxFitGapLength' for PFP with ID 2
%MSG
%MSG-w CAFMaker:  CAFMaker:cafmaker@BeginModule 31-Aug-2025 11:49:58 CEST  run: 74553259 subRun: 1 event: 73800
Unknown property 'LArThreeDLinearFitFeatureTool_SlidingLinearFitRMS' for PFP with ID 2
%MSG
%MSG-w CAFMaker:  CAFMaker:cafmaker@BeginModule 31-Aug-2025 11:49:58 CEST  run: 74553259 subRun: 1 event: 73800
Unknown property 'LArThreeDOpeningAngleFeatureTool_AngleDiff' for PFP with ID 2
%MSG
%MSG-w CAFMaker:  CAFMaker:cafmaker@BeginModule 31-Aug-2025 11:49:58 CEST  run: 74553259 subRun: 1 event: 73800
Unknown property 'LArThreeDPCAFeatureTool_SecondaryPCARatio' for PFP with ID 2
%MSG
%MSG-w CAFMaker:  CAFMaker:cafmaker@BeginModule 31-Aug-2025 11:49:58 CEST  run: 74553259 subRun: 1 event: 73800
Unknown property 'LArThreeDPCAFeatureTool_TertiaryPCARatio' for PFP with ID 2
%MSG
%MSG-w CAFMaker:  CAFMaker:cafmaker@BeginModule 31-Aug-2025 11:49:58 CEST  run: 74553259 subRun: 1 event: 73800
Unknown property 'LArThreeDVertexDistanceFeatureTool_VertexDistance' for PFP with ID 2
%MSG
%MSG-w CAFMaker:  CAFMaker:cafmaker@BeginModule 31-Aug-2025 11:49:58 CEST  run: 74553259 subRun: 1 event: 73800
Unknown property 'LArPfoHierarchyFeatureTool_DaughterParentHitRatio' for PFP with ID 3
%MSG
%MSG-w CAFMaker:  CAFMaker:cafmaker@BeginModule 31-Aug-2025 11:49:58 CEST  run: 74553259 subRun: 1 event: 73800
Unknown property 'LArPfoHierarchyFeatureTool_NDaughterHits3D' for PFP with ID 3
%MSG
%MSG-w CAFMaker:  CAFMaker:cafmaker@BeginModule 31-Aug-2025 11:49:58 CEST  run: 74553259 subRun: 1 event: 73800
Unknown property 'LArThreeDChargeFeatureTool_EndFraction' for PFP with ID 3
%MSG
%MSG-w CAFMaker:  CAFMaker:cafmaker@BeginModule 31-Aug-2025 11:49:58 CEST  run: 74553259 subRun: 1 event: 73800
Unknown property 'LArThreeDChargeFeatureTool_FractionalSpread' for PFP with ID 3
%MSG
%MSG-w CAFMaker:  CAFMaker:cafmaker@BeginModule 31-Aug-2025 11:49:58 CEST  run: 74553259 subRun: 1 event: 73800
Unknown property 'LArThreeDLinearFitFeatureTool_DiffStraightLineMean' for PFP with ID 3
%MSG
%MSG-w CAFMaker:  CAFMaker:cafmaker@BeginModule 31-Aug-2025 11:49:58 CEST  run: 74553259 subRun: 1 event: 73800
Unknown property 'LArThreeDLinearFitFeatureTool_Length' for PFP with ID 3
%MSG
%MSG-w CAFMaker:  CAFMaker:cafmaker@BeginModule 31-Aug-2025 11:49:58 CEST  run: 74553259 subRun: 1 event: 73800
Unknown property 'LArThreeDLinearFitFeatureTool_MaxFitGapLength' for PFP with ID 3
%MSG
%MSG-w CAFMaker:  CAFMaker:cafmaker@BeginModule 31-Aug-2025 11:49:58 CEST  run: 74553259 subRun: 1 event: 73800
Unknown property 'LArThreeDLinearFitFeatureTool_SlidingLinearFitRMS' for PFP with ID 3
%MSG
%MSG-w CAFMaker:  CAFMaker:cafmaker@BeginModule 31-Aug-2025 11:49:58 CEST  run: 74553259 subRun: 1 event: 73800
Unknown property 'LArThreeDOpeningAngleFeatureTool_AngleDiff' for PFP with ID 3
%MSG
%MSG-w CAFMaker:  CAFMaker:cafmaker@BeginModule 31-Aug-2025 11:49:58 CEST  run: 74553259 subRun: 1 event: 73800
Unknown property 'LArThreeDPCAFeatureTool_SecondaryPCARatio' for PFP with ID 3
%MSG
%MSG-w CAFMaker:  CAFMaker:cafmaker@BeginModule 31-Aug-2025 11:49:58 CEST  run: 74553259 subRun: 1 event: 73800
Unknown property 'LArThreeDPCAFeatureTool_TertiaryPCARatio' for PFP with ID 3
%MSG
%MSG-w CAFMaker:  CAFMaker:cafmaker@BeginModule 31-Aug-2025 11:49:58 CEST  run: 74553259 subRun: 1 event: 73800
Unknown property 'LArThreeDVertexDistanceFeatureTool_VertexDistance' for PFP with ID 3
%MSG
31-Aug-2025 11:49:59 CEST  Closed output file "caf_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74553259_737_20231207T150220Z_gen_g4_detsim_hitreco__20240510T063535Z_reco2_truthvtx_2025-08-31T_094908Z.root"
31-Aug-2025 11:49:59 CEST  Closed input file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74553259_737_20231207T150220Z_gen_g4_detsim_hitreco__20240510T063535Z_reco2_truthvtx_2025-08-31T_094423Z.root"
%MSG-i NuRandomService:  RootOutput:out@EndJob 31-Aug-2025 11:49:59 CEST  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1

%MSG

TrigReport ---------- Event summary -------------
TrigReport Events total = 100 passed = 100 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport        100        100          0 cafmaker
TrigReport        100        100          0 out

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 24.688009 Real = 27.017724

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 19288.6 VmHWM = 1024.65

Art has completed and will exit with status 0.
=== End last  100 lines of lar log file ===
=== Generated output files ===
37270.79_dunegpschedd01.fnal.gov.logs.tgz
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74553259_737_20231207T150220Z_gen_g4_detsim_hitreco__20240510T063535Z_reco2_truthvtx_2025-08-31T_094423Z.root
caf_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74553259_737_20231207T150220Z_gen_g4_detsim_hitreco__20240510T063535Z_reco2_truthvtx_2025-08-31T_094908Z.log
caf_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74553259_737_20231207T150220Z_gen_g4_detsim_hitreco__20240510T063535Z_reco2_truthvtx_2025-08-31T_094908Z.root
caf_truthvtx_2025-08-31T_094908Z_caf.root
debugprod.log
flatcaf.root
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
truthvtx_2025-08-31T_094423Z_vtx.log