justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 255285.0@dunegpschedd01.fnal.gov

Jobsub ID255285.0@dunegpschedd01.fnal.gov
Workflow ID10432
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-11-22 10:41:29
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-11-22 13:17:24
From worker nodeHostnamedunegli-7693663-0-fnpc23031.fnal.gov
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-22 11:24:39
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241024T000559Z_gen_004603_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time1h (6752s)
CPU time1h (6671s = 98%)
Max RSS bytes3104645120 (2960 MiB)
Outputting started2025-11-22 13:17:11
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/10432/1/001/analysisLowE_20251122T112439Z.root
Finished2025-11-22 13:17:24
Saved logsjustin-logs:255285.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

row STATUS_CODE_NOT_INITIALIZED
    in function: RunAlgorithm
    in file:     /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0005, LArDLVertexing, STATUS_CODE_NOT_INITIALIZED
Boundary wire vector sizes: 6718, 6747, 6718
minwire 0: 18
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 2195, 2694
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.110795, 
Output 1: 0.0919254, 0.0943084, 0.133493, 0.680273, 
Output 2: 0.0530561, 0.214447, 0.69423, 0.0382663, 
Output 3: 0.965479, 0.0224766, 0.00383422, 0.00820999, 
Output 4: 0.642812, 0.32209, 0.0314038, 0.00369402, 
Output 5: 0.501516, 0.3177, 0.136636, 0.0441486, 
Output 6: 0.0336562, 0.0799885, 0.0834924, 0.802863, 

22-Nov-2025 13:16:40 UTC  Closed output file "matt_reco2.root"
22-Nov-2025 13:16:40 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/20/84/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241024T000559Z_gen_004603_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              636.076       665.579       698.524       661.25        17.8882        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00141075    0.00415066    0.00671676    0.00459688    0.00152723       10     
reco:pandora:StandardPandora                            635.089       664.41        697.463       660.233       17.9257        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000285516    0.0131872     0.0689055    0.00366005     0.0202253       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000217607   0.00198216     0.0122014    0.000986589   0.00342481       10     
reco:pandoracalo:Calorimetry                          0.000183571   0.00467344     0.0199159    0.000360786   0.00736454       10     
reco:pandorapid:Chi2ParticleID                        3.8984e-05    0.000181563   0.00123988    7.0565e-05    0.000353168      10     
reco:cvnmap:CVNMapper                                  0.0165438     0.0219787     0.0364535     0.0196802     0.0058565       10     
reco:cvneva:CVNEvaluator                               0.707565      0.848849       1.96726      0.725296      0.373062        10     
reco:energyrecnumu:EnergyReco                         0.00240385     0.0287793     0.0684966     0.0337802     0.0233955       10     
reco:energyrecnue:EnergyReco                          0.000757971   0.00253485    0.00513336    0.00117805    0.00197857       10     
reco:energyrecnc:EnergyReco                           0.00468032     0.0331551     0.0985286     0.0195196     0.0329725       10     
[art]:TriggerResults:TriggerResultInserter            1.0009e-05    2.63105e-05   0.000144026   1.3456e-05    3.92929e-05      10     
end_path:out1:RootOutput                               3.627e-06    1.05652e-05   3.7862e-05     8.211e-06    9.25481e-06      10     
end_path:out1:RootOutput(write)                        0.201725      0.208377      0.224365      0.205642     0.00641037       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21779.9 MB
  Peak resident set size usage (VmHWM): 3104.65 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
 *** Total Charge : 4788.88
 Looking at view: 2 | Number of hits in view: 14
Begin processing the 8th record. run: 3799 subRun: 0 event: 46028 at 22-Nov-2025 13:17:06 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Electron Energy: 1.2848
Completed MCParticle Loop
True Neutrino Energy: 4.56825
 Size of all hits: 0
Begin processing the 9th record. run: 3799 subRun: 0 event: 46029 at 22-Nov-2025 13:17:07 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
This pfp has size:  50
Added hit list length: 50 | Max was set to: -3.40282e+38
True Electron Energy: 39.2147
Completed MCParticle Loop
True Neutrino Energy: 44.7434
 Size of all hits: 50
hit initial : 37.0416 | hit correction : 1.03907
hit initial : 23.191 | hit correction : 1.03908
hit initial : 44.8315 | hit correction : 1.03864
hit initial : 308.217 | hit correction : 1.03802
hit initial : 163.073 | hit correction : 1.03764
hit initial : 47.6209 | hit correction : 1.03755
hit initial : 316.279 | hit correction : 1.03513
hit initial : 656.161 | hit correction : 1.03527
hit initial : 110.942 | hit correction : 1.03554
hit initial : 536.576 | hit correction : 1.03511
hit initial : 321.186 | hit correction : 1.03464
hit initial : 81.9684 | hit correction : 1.03898
hit initial : 92.5488 | hit correction : 1.0345
hit initial : 39.7348 | hit correction : 1.03897
hit initial : 96.488 | hit correction : 1.03844
hit initial : 173.016 | hit correction : 1.03864
hit initial : 323.191 | hit correction : 1.03872
hit initial : 302.476 | hit correction : 1.03873
hit initial : 198.96 | hit correction : 1.03868
 *** Total Charge : 4015.65
 Looking at view: 0 | Number of hits in view: 19
hit initial : 214.219 | hit correction : 1.03457
hit initial : 276.951 | hit correction : 1.03486
hit initial : 327.242 | hit correction : 1.03526
hit initial : 391.172 | hit correction : 1.03546
hit initial : 299.273 | hit correction : 1.03538
hit initial : 230.411 | hit correction : 1.03508
hit initial : 84.8618 | hit correction : 1.03898
hit initial : 22.5421 | hit correction : 1.03749
hit initial : 162.597 | hit correction : 1.035
hit initial : 33.4075 | hit correction : 1.03897
hit initial : 157.769 | hit correction : 1.03765
hit initial : 41.8032 | hit correction : 1.03499
hit initial : 296.414 | hit correction : 1.03803
hit initial : 229.179 | hit correction : 1.03875
hit initial : 37.5181 | hit correction : 1.03867
hit initial : 261.276 | hit correction : 1.03852
hit initial : 223.203 | hit correction : 1.0387
hit initial : 259.145 | hit correction : 1.03873
hit initial : 175.109 | hit correction : 1.03868
 *** Total Charge : 3860.89
 Looking at view: 1 | Number of hits in view: 19
hit initial : 410.239 | hit correction : 1.03511
hit initial : 409.609 | hit correction : 1.03547
hit initial : 352.5 | hit correction : 1.03541
hit initial : 112.245 | hit correction : 1.03746
hit initial : 255.682 | hit correction : 1.03504
hit initial : 399.346 | hit correction : 1.03795
hit initial : 193.163 | hit correction : 1.03497
hit initial : 22.4063 | hit correction : 1.03897
hit initial : 138.295 | hit correction : 1.03898
hit initial : 434.398 | hit correction : 1.03863
hit initial : 76.4256 | hit correction : 1.0387
hit initial : 641.729 | hit correction : 1.03872
 *** Total Charge : 3573.38
 Looking at view: 2 | Number of hits in view: 12
Begin processing the 10th record. run: 3799 subRun: 0 event: 46030 at 22-Nov-2025 13:17:09 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.568
True Electron Energy: 2.98822
Completed MCParticle Loop
True Neutrino Energy: 6.71272
 Size of all hits: 0
22-Nov-2025 13:17:11 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0468873     0.0577846     0.0735609     0.0561276    0.00931447       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000288703   0.000372441   0.000592905   0.000366938   8.34058e-05      10     
end_path:ana:AnalyzeEvents         0.0464649     0.057246      0.073068      0.055703     0.00921586       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19473.3 MB
  Peak resident set size usage (VmHWM): 1536.67 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-09 01:56:45 UTC       justIN version: 01.06.00