justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 261322.1@dunegpschedd02.fnal.gov

Jobsub ID261322.1@dunegpschedd02.fnal.gov
Workflow ID11094
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-11 22:38:40
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-12-12 00:23:00
From worker nodeHostnamedunegli-7878630-0-fnpc23006.fnal.gov
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-11 22:39:19
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T202647Z_gen_002862_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time1h (6210s)
CPU time1h (6166s = 99%)
Max RSS bytes2911436800 (2776 MiB)
Outputting started2025-12-12 00:22:49
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11094/1/001/analysisLowE_20251211T223919Z.root
Finished2025-12-12 00:23:00
Saved logsjustin-logs:261322.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

second->Run() throw STATUS_CODE_NOT_INITIALIZED
    in function: RunAlgorithm
    in file:     /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0005, LArDLVertexing, STATUS_CODE_NOT_INITIALIZED
Boundary wire vector sizes: 6705, 6653, 6701
minwire 0: 45
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 1364, 1863
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0962128, 
Output 1: 0.0328391, 0.0561391, 0.112322, 0.7987, 
Output 2: 0.157196, 0.276289, 0.518489, 0.0480262, 
Output 3: 0.958168, 0.0347983, 0.00254789, 0.00448613, 
Output 4: 0.237754, 0.721835, 0.0386221, 0.00178899, 
Output 5: 0.67663, 0.291953, 0.0258753, 0.00554165, 
Output 6: 0.0210364, 0.0701791, 0.0797817, 0.829003, 

12-Dec-2025 00:22:21 UTC  Closed output file "matt_reco2.root"
12-Dec-2025 00:22:21 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/24/99/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T202647Z_gen_002862_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              605.015       612.557       630.823       611.585       6.65871        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00137101    0.00716499     0.0223286    0.00416223    0.00723528       10     
reco:pandora:StandardPandora                            604.048       611.468       629.842       610.613       6.6478         10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000289324    0.0135988     0.0498606     0.0110884     0.0136395       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000747831   0.00371117     0.0103745    0.00344916    0.00298363       10     
reco:pandoracalo:Calorimetry                           9.92e-05     0.00326201     0.0154108    0.000308049   0.00545758       10     
reco:pandorapid:Chi2ParticleID                        3.1981e-05    0.000167199   0.000634825   6.6903e-05    0.000213655      10     
reco:cvnmap:CVNMapper                                  0.0152355     0.0187534     0.0267219     0.0170279    0.00365742       10     
reco:cvneva:CVNEvaluator                               0.666986      0.776957       1.71165      0.672642      0.311605        10     
reco:energyrecnumu:EnergyReco                         0.00249709     0.0368747     0.119809      0.0304171     0.037107        10     
reco:energyrecnue:EnergyReco                          0.00072094    0.00193854    0.00468902    0.000839777   0.00175534       10     
reco:energyrecnc:EnergyReco                           0.00446128     0.0265682     0.0548163     0.0194465     0.0226531       10     
[art]:TriggerResults:TriggerResultInserter             8.978e-06    1.39008e-05   4.2812e-05    1.0771e-05    9.72382e-06      10     
end_path:out1:RootOutput                               4.759e-06    8.5382e-06    2.5719e-05     6.257e-06    6.03257e-06      10     
end_path:out1:RootOutput(write)                         0.19286      0.199314      0.207594      0.198779      0.0033983       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21753.6 MB
  Peak resident set size usage (VmHWM): 2911.44 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 140.876 | hit correction : 1.25745
hit initial : 151.023 | hit correction : 1.25761
hit initial : 22.6696 | hit correction : 1.25771
 *** Total Charge : 2751.27
 Looking at view: 2 | Number of hits in view: 16
Begin processing the 8th record. run: 5026 subRun: 0 event: 28618 at 12-Dec-2025 00:22:43 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
This pfp has size:  46
Added hit list length: 46 | Max was set to: -3.40282e+38
True Electron Energy: 36.2354
Completed MCParticle Loop
True Neutrino Energy: 58.2305
 Size of all hits: 46
hit initial : 116.828 | hit correction : 1.29509
hit initial : 64.0129 | hit correction : 1.29557
hit initial : 176.42 | hit correction : 1.29467
hit initial : 28.228 | hit correction : 1.29836
hit initial : 101.69 | hit correction : 1.29462
hit initial : 24.9422 | hit correction : 1.29464
hit initial : 51.4951 | hit correction : 1.29522
hit initial : 249.303 | hit correction : 1.29554
hit initial : 85.6311 | hit correction : 1.29578
hit initial : 104.832 | hit correction : 1.29733
hit initial : 109.098 | hit correction : 1.29747
hit initial : 236.792 | hit correction : 1.29855
hit initial : 278.069 | hit correction : 1.29917
hit initial : 263.154 | hit correction : 1.29983
hit initial : 212.369 | hit correction : 1.30026
hit initial : 191.547 | hit correction : 1.3006
hit initial : 98.0039 | hit correction : 1.3008
 *** Total Charge : 3104.96
 Looking at view: 0 | Number of hits in view: 17
hit initial : 32.8099 | hit correction : 1.29467
hit initial : 272.137 | hit correction : 1.29564
hit initial : 134.029 | hit correction : 1.29464
hit initial : 136.437 | hit correction : 1.29463
hit initial : 77.9205 | hit correction : 1.29519
hit initial : 57.739 | hit correction : 1.29519
hit initial : 71.2168 | hit correction : 1.29573
hit initial : 56.654 | hit correction : 1.29581
hit initial : 37.3529 | hit correction : 1.29714
hit initial : 187.253 | hit correction : 1.29749
hit initial : 244.512 | hit correction : 1.29806
hit initial : 290.784 | hit correction : 1.29904
hit initial : 375.56 | hit correction : 1.29984
hit initial : 314.122 | hit correction : 1.30041
hit initial : 108.049 | hit correction : 1.30066
 *** Total Charge : 3110.35
 Looking at view: 1 | Number of hits in view: 15
hit initial : 157.124 | hit correction : 1.29346
hit initial : 12.6264 | hit correction : 1.29337
hit initial : 57.2413 | hit correction : 1.30084
hit initial : 235.703 | hit correction : 1.30066
hit initial : 278.358 | hit correction : 1.3002
hit initial : 71.3753 | hit correction : 1.29876
hit initial : 67.361 | hit correction : 1.29896
hit initial : 50.5224 | hit correction : 1.29515
hit initial : 99.9749 | hit correction : 1.29518
hit initial : 124.628 | hit correction : 1.29466
hit initial : 375.608 | hit correction : 1.29574
hit initial : 18.9243 | hit correction : 1.29674
hit initial : 1144.15 | hit correction : 1.29715
hit initial : 152.203 | hit correction : 1.29743
 *** Total Charge : 3691.82
 Looking at view: 2 | Number of hits in view: 14
Begin processing the 9th record. run: 5026 subRun: 0 event: 28619 at 12-Dec-2025 00:22:45 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 940.912
True Electron Energy: 20.7078
Completed MCParticle Loop
True Neutrino Energy: 36.2024
 Size of all hits: 0
Begin processing the 10th record. run: 5026 subRun: 0 event: 28620 at 12-Dec-2025 00:22:47 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.574
True Electron Energy: 30.2847
Completed MCParticle Loop
True Neutrino Energy: 52.2379
 Size of all hits: 0
12-Dec-2025 00:22:49 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0457404     0.0555172     0.0631135     0.0565196    0.00532572       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000284053   0.000489256   0.00118014    0.000403667   0.000247626      10     
end_path:ana:AnalyzeEvents         0.0452624     0.0548679     0.0617436     0.0560583    0.00515894       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19482.7 MB
  Peak resident set size usage (VmHWM): 1544.66 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 16:29:01 UTC       justIN version: 01.06.00