justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 267660.1@dunegpschedd01.fnal.gov

Jobsub ID267660.1@dunegpschedd01.fnal.gov
Workflow ID11171
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-13 04:20:12
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-12-13 16:47:25
From worker nodeHostnamedunegli-8047001-0-fnpc19108.fnal.gov
cpuinfoAMD EPYC 7502 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-13 04:21:31
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T193020Z_gen_002367_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time12h (44740s)
CPU time12h (44042s = 98%)
Max RSS bytes2785804288 (2656 MiB)
Outputting started2025-12-13 16:47:11
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11171/1/001/analysisLowE_20251213T042133Z.root
Finished2025-12-13 16:47:25
Saved logsjustin-logs:267660.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ine#: 64
iter->second->Run() throw STATUS_CODE_NOT_INITIALIZED
    in function: RunAlgorithm
    in file:     /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0015, LArNeutrinoEventValidation, STATUS_CODE_NOT_INITIALIZED
Boundary wire vector sizes: 6803, 6676, 6779
minwire 0: 36
minwire 1: 1
minwire 2: 2
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 619, 1118
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.116666, 
Output 1: 0.0570615, 0.144142, 0.138902, 0.659895, 
Output 2: 0.0497579, 0.235177, 0.689945, 0.0251208, 
Output 3: 0.971826, 0.0187322, 0.00271575, 0.0067264, 
Output 4: 0.536246, 0.404417, 0.0530695, 0.00626743, 
Output 5: 0.581682, 0.342743, 0.062565, 0.0130106, 
Output 6: 0.013104, 0.0451201, 0.0627811, 0.878995, 

13-Dec-2025 16:46:31 UTC  Closed output file "matt_reco2.root"
13-Dec-2025 16:46:31 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/a8/37/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T193020Z_gen_002367_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              3335.85       4436.42       4803.61       4550.68       394.934        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000989198   0.00616592     0.024214     0.00234417    0.00815692       10     
reco:pandora:StandardPandora                            3334.45       4433.64       4797.46       4548.62       394.313        10     
reco:pandoraTrack:LArPandoraTrackCreation              0.0023805     0.113697      0.183708      0.137004      0.0537064       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.00149839     0.0103037     0.0495725    0.00484022     0.0136929       10     
reco:pandoracalo:Calorimetry                          0.000554914   0.00210137     0.0125168    0.000735736   0.00350268       10     
reco:pandorapid:Chi2ParticleID                         9.87e-05     0.000725753    0.0054991    0.000184311   0.00159233       10     
reco:cvnmap:CVNMapper                                  0.0641933     0.132101      0.188329      0.132635      0.0323261       10     
reco:cvneva:CVNEvaluator                               0.871327       1.93438       4.99718       1.85821       1.07578        10     
reco:energyrecnumu:EnergyReco                          0.0784622     0.123215      0.228361      0.112587      0.0379341       10     
reco:energyrecnue:EnergyReco                          0.00127703    0.00410593    0.00761572    0.00355388    0.00197043       10     
reco:energyrecnc:EnergyReco                           0.00656351     0.0142437     0.0222242     0.0143133    0.00377019       10     
[art]:TriggerResults:TriggerResultInserter             2.581e-05    6.03995e-05   0.000281262   3.56005e-05   7.41947e-05      10     
end_path:out1:RootOutput                               7.93e-06     2.00724e-05   7.0781e-05    1.32955e-05   1.75608e-05      10     
end_path:out1:RootOutput(write)                        0.336471      0.438365      0.530692      0.441653      0.0475125       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21572 MB
  Peak resident set size usage (VmHWM): 2785.8 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 60.7891 | hit correction : 1.35313
hit initial : 285.088 | hit correction : 1.35311
hit initial : 395.736 | hit correction : 1.35311
hit initial : 229.799 | hit correction : 1.35318
hit initial : 187.163 | hit correction : 1.35336
hit initial : 189.375 | hit correction : 1.35332
hit initial : 26.9641 | hit correction : 1.35339
 *** Total Charge : 1860.52
 Looking at view: 0 | Number of hits in view: 7
hit initial : 32.1685 | hit correction : 1.34572
hit initial : 20.0849 | hit correction : 1.34573
hit initial : 43.9433 | hit correction : 1.35326
hit initial : 162.123 | hit correction : 1.35315
hit initial : 360.345 | hit correction : 1.3531
hit initial : 429.753 | hit correction : 1.35318
hit initial : 160.608 | hit correction : 1.35337
hit initial : 131.115 | hit correction : 1.35339
hit initial : 61.7703 | hit correction : 1.35344
 *** Total Charge : 1896.69
 Looking at view: 1 | Number of hits in view: 9
hit initial : 19.9368 | hit correction : 1.34575
hit initial : 36.0831 | hit correction : 1.34574
hit initial : 232.594 | hit correction : 1.35313
hit initial : 233.406 | hit correction : 1.35302
hit initial : 27.7684 | hit correction : 1.35318
hit initial : 403.415 | hit correction : 1.3532
hit initial : 641.003 | hit correction : 1.35333
hit initial : 97.5384 | hit correction : 1.35338
 *** Total Charge : 2288.9
 Looking at view: 2 | Number of hits in view: 8
Begin processing the 10th record. run: 5026 subRun: 0 event: 23670 at 13-Dec-2025 16:47:08 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  3
This pfp has size:  35
Added hit list length: 35 | Max was set to: -3.40282e+38
True Neutron Energy: 939.57
True Neutron Energy: 939.719
True Electron Energy: 24.1001
Completed MCParticle Loop
True Neutrino Energy: 33.075
 Size of all hits: 35
hit initial : 94.3982 | hit correction : 1.04243
hit initial : 262.418 | hit correction : 1.04282
hit initial : 362.028 | hit correction : 1.04341
hit initial : 29.2734 | hit correction : 1.04489
hit initial : 795.801 | hit correction : 1.04521
hit initial : 30.3235 | hit correction : 1.04544
hit initial : 198.456 | hit correction : 1.04661
hit initial : 303.959 | hit correction : 1.04688
hit initial : 42.3353 | hit correction : 1.04735
 *** Total Charge : 2214.13
 Looking at view: 0 | Number of hits in view: 9
hit initial : 194.927 | hit correction : 1.04416
hit initial : 20.4728 | hit correction : 1.01791
hit initial : 117.198 | hit correction : 1.0426
hit initial : 230.43 | hit correction : 1.04253
hit initial : 146.637 | hit correction : 1.04336
hit initial : 193.928 | hit correction : 1.04486
hit initial : 378.682 | hit correction : 1.04509
hit initial : 322.038 | hit correction : 1.04548
hit initial : 309.56 | hit correction : 1.04595
hit initial : 335.031 | hit correction : 1.04649
hit initial : 280.76 | hit correction : 1.04697
hit initial : 115.242 | hit correction : 1.04723
 *** Total Charge : 2763.88
 Looking at view: 1 | Number of hits in view: 12
hit initial : 33.2859 | hit correction : 1.01793
hit initial : 21.7778 | hit correction : 1.01847
hit initial : 596.01 | hit correction : 1.04276
hit initial : 16.6409 | hit correction : 1.04367
hit initial : 214.75 | hit correction : 1.04375
hit initial : 208.423 | hit correction : 1.04429
hit initial : 220.412 | hit correction : 1.04477
hit initial : 192.916 | hit correction : 1.04501
hit initial : 171.626 | hit correction : 1.04528
hit initial : 273.954 | hit correction : 1.04551
hit initial : 307.055 | hit correction : 1.04592
hit initial : 359.337 | hit correction : 1.04644
hit initial : 346.579 | hit correction : 1.04703
hit initial : 75.9647 | hit correction : 1.04739
 *** Total Charge : 3174.04
 Looking at view: 2 | Number of hits in view: 14
13-Dec-2025 16:47:11 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0812866     0.0894452     0.107861      0.0866198    0.00775634       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000378933   0.000472082   0.000807477   0.000436048   0.000116756      10     
end_path:ana:AnalyzeEvents         0.0807477     0.0888365     0.106688      0.086067     0.00761461       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19475.3 MB
  Peak resident set size usage (VmHWM): 1537.88 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 12:04:39 UTC       justIN version: 01.06.00