justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 268123.0@dunegpschedd01.fnal.gov

Jobsub ID268123.0@dunegpschedd01.fnal.gov
Workflow ID11170
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-13 20:15:00
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-12-14 09:20:12
From worker nodeHostnamedunegli-8066212-0-fnpc17126.fnal.gov
cpuinfoIntel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-14 03:31:53
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T010253Z_gen_002930_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time5h (20887s)
CPU time5h (20215s = 96%)
Max RSS bytes2788507648 (2659 MiB)
Outputting started2025-12-14 09:20:01
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11170/1/001/analysisLowE_20251214T033154Z.root
Finished2025-12-14 09:20:12
Saved logsjustin-logs:268123.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

cc line#: 64
iter->second->Run() throw STATUS_CODE_NOT_INITIALIZED
    in function: RunAlgorithm
    in file:     /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0015, LArNeutrinoEventValidation, STATUS_CODE_NOT_INITIALIZED
Boundary wire vector sizes: 6759, 6664, 6647
minwire 0: 7
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 1050, 1549
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.119, 
Output 1: 0.0712155, 0.0426361, 0.107878, 0.778271, 
Output 2: 0.0720085, 0.294794, 0.590341, 0.0428557, 
Output 3: 0.951789, 0.0279884, 0.006426, 0.0137968, 
Output 4: 0.29578, 0.604496, 0.0922015, 0.00752181, 
Output 5: 0.694589, 0.25877, 0.0387225, 0.00791864, 
Output 6: 0.0211315, 0.0619248, 0.0798793, 0.837064, 

14-Dec-2025 09:18:48 UTC  Closed output file "matt_reco2.root"
14-Dec-2025 09:18:48 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/f2/2e/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T010253Z_gen_002930_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1845.68       2019.98       2407.39       1920.92       191.915        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000749838    0.0936884     0.459812      0.0021946     0.181817        10     
reco:pandora:StandardPandora                            1843.92       2017.4        2404.91       1916.35       191.953        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.00068952     0.0507331     0.0861743     0.0627727     0.0311908       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000439085    0.0110607     0.0824213    0.00312995     0.0238591       10     
reco:pandoracalo:Calorimetry                          0.000340053   0.00807832     0.0723637    0.000751425    0.0214417       10     
reco:pandorapid:Chi2ParticleID                        9.5182e-05    0.000263298   0.00134359    0.000137144   0.000362659      10     
reco:cvnmap:CVNMapper                                  0.117107      0.144948      0.226205      0.126012      0.0346181       10     
reco:cvneva:CVNEvaluator                               0.916738       1.6432        4.49275       1.31589      0.995383        10     
reco:energyrecnumu:EnergyReco                          0.0829047     0.104012      0.123337      0.101343      0.0143363       10     
reco:energyrecnue:EnergyReco                          0.00160066    0.00336102     0.0140523    0.00192105    0.00362002       10     
reco:energyrecnc:EnergyReco                           0.00837561     0.0107128     0.014394     0.00988293    0.00190831       10     
[art]:TriggerResults:TriggerResultInserter            1.9794e-05    3.02873e-05   7.4745e-05    2.72185e-05   1.55149e-05      10     
end_path:out1:RootOutput                               6.835e-06    9.4604e-06    2.0156e-05    8.2105e-06    3.77232e-06      10     
end_path:out1:RootOutput(write)                        0.354041      0.513192      0.998791      0.415887      0.198956        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 14050 MB
  Peak resident set size usage (VmHWM): 2788.51 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
 *** Total Charge : 3309.72
 Looking at view: 2 | Number of hits in view: 9
Begin processing the 10th record. run: 5027 subRun: 0 event: 29300 at 14-Dec-2025 09:19:54 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  4
This pfp has size:  61
Added hit list length: 61 | Max was set to: -3.40282e+38
This pfp has size:  11
True Neutron Energy: 939.57
True Neutron Energy: 943.209
True Neutron Energy: 941.533
True Electron Energy: 36.4871
Completed MCParticle Loop
True Neutrino Energy: 49.1653
 Size of all hits: 61
hit initial : 29.955 | hit correction : 1.03926
hit initial : 16.6654 | hit correction : 1.03929
hit initial : 39.0056 | hit correction : 1.04197
hit initial : 17.4728 | hit correction : 1.04198
hit initial : 83.743 | hit correction : 1.05072
hit initial : 263.571 | hit correction : 1.05421
hit initial : 112.11 | hit correction : 1.05418
hit initial : 495.073 | hit correction : 1.05909
hit initial : 319.22 | hit correction : 1.05907
hit initial : 189.807 | hit correction : 1.05913
hit initial : 140.828 | hit correction : 1.05912
hit initial : 55.1937 | hit correction : 1.06017
hit initial : 280.158 | hit correction : 1.06016
hit initial : 393.904 | hit correction : 1.061
hit initial : 188.896 | hit correction : 1.06165
hit initial : 183.59 | hit correction : 1.06154
hit initial : 46.2535 | hit correction : 1.06241
hit initial : 71.3568 | hit correction : 1.06305
hit initial : 165.303 | hit correction : 1.06306
 *** Total Charge : 3273.53
 Looking at view: 0 | Number of hits in view: 19
hit initial : 35.1269 | hit correction : 1.03924
hit initial : 27.8146 | hit correction : 1.03923
hit initial : 29.4727 | hit correction : 1.04197
hit initial : 64.9279 | hit correction : 1.05073
hit initial : 49.4691 | hit correction : 1.05071
hit initial : 169.401 | hit correction : 1.05413
hit initial : 209.634 | hit correction : 1.05426
hit initial : 246.689 | hit correction : 1.05911
hit initial : 235.197 | hit correction : 1.05899
hit initial : 256.074 | hit correction : 1.05902
hit initial : 271.474 | hit correction : 1.05918
hit initial : 401.417 | hit correction : 1.05968
hit initial : 448.329 | hit correction : 1.06048
hit initial : 555.439 | hit correction : 1.06114
hit initial : 105.709 | hit correction : 1.06164
hit initial : 514.534 | hit correction : 1.06168
hit initial : 158.359 | hit correction : 1.06204
hit initial : 138.32 | hit correction : 1.06305
hit initial : 230.736 | hit correction : 1.06294
 *** Total Charge : 4394.16
 Looking at view: 1 | Number of hits in view: 19
hit initial : 36.8898 | hit correction : 1.05509
hit initial : 26.2284 | hit correction : 1.04061
hit initial : 67.3797 | hit correction : 1.03924
hit initial : 67.1416 | hit correction : 1.04199
hit initial : 127.847 | hit correction : 1.05072
hit initial : 278.085 | hit correction : 1.05409
hit initial : 173.608 | hit correction : 1.0544
hit initial : 402.319 | hit correction : 1.05909
hit initial : 370.811 | hit correction : 1.05897
hit initial : 197.013 | hit correction : 1.05909
hit initial : 157.002 | hit correction : 1.05928
hit initial : 176.695 | hit correction : 1.05955
hit initial : 249.685 | hit correction : 1.05993
hit initial : 178.967 | hit correction : 1.06033
hit initial : 173.325 | hit correction : 1.0606
hit initial : 228.901 | hit correction : 1.06088
hit initial : 361.226 | hit correction : 1.06126
hit initial : 173.431 | hit correction : 1.0617
hit initial : 235.339 | hit correction : 1.06151
hit initial : 247.419 | hit correction : 1.06194
hit initial : 239.768 | hit correction : 1.06251
hit initial : 248.813 | hit correction : 1.06301
hit initial : 94.7342 | hit correction : 1.06331
 *** Total Charge : 4778.76
 Looking at view: 2 | Number of hits in view: 23
14-Dec-2025 09:19:59 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0870125      0.16273      0.504651      0.118475      0.118291        10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.00044409    0.000592197   0.00110351    0.000530435   0.000183081      10     
end_path:ana:AnalyzeEvents         0.0863226     0.161934      0.504053      0.117736      0.118276        10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 11956.5 MB
  Peak resident set size usage (VmHWM): 1535.91 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 12:08:17 UTC       justIN version: 01.06.00