justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 263741.45@dunegpschedd01.fnal.gov

Jobsub ID263741.45@dunegpschedd01.fnal.gov
Workflow ID11025
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-05 10:51:41
SiteUS_FNAL-T1
EntryCMSHTPC_T1_US_FNAL_condce_opp1_whole
Last heartbeat2025-12-06 07:35:08
From worker nodeHostnamedunegli-48727-0-cmswn5027.fnal.gov
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-05 18:14:58
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T230328Z_gen_001122_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time13h (47993s)
CPU time13h (47636s = 99%)
Max RSS bytes2793656320 (2664 MiB)
Outputting started2025-12-06 07:34:51
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11025/1/001/analysisLowE_20251205T181459Z.root
Finished2025-12-06 07:34:59
Saved logsjustin-logs:263741.45-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

nd->Run() throw STATUS_CODE_NOT_INITIALIZED
    in function: RunAlgorithm
    in file:     /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0005, LArDLVertexing, STATUS_CODE_NOT_INITIALIZED
Boundary wire vector sizes: 6649, 6599, 6674
minwire 0: 25
minwire 1: 1
minwire 2: 1
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 2150, 2649
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.12564, 
Output 1: 0.0770351, 0.0398301, 0.0840455, 0.799089, 
Output 2: 0.209636, 0.290031, 0.463932, 0.0364008, 
Output 3: 0.980489, 0.0162972, 0.000728292, 0.00248597, 
Output 4: 0.359641, 0.597574, 0.0394634, 0.00332238, 
Output 5: 0.595472, 0.38372, 0.017836, 0.00297156, 
Output 6: 0.0184477, 0.0734376, 0.0902665, 0.817848, 

06-Dec-2025 07:34:20 UTC  Closed output file "matt_reco2.root"
06-Dec-2025 07:34:20 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/86/7f/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T230328Z_gen_001122_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1094.8        4765.75       6369.45       5475.45       1730.96        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00578801     0.0269483     0.0452239     0.0280505     0.0104801       10     
reco:pandora:StandardPandora                            1093.53       4761.75       6364.83       5466.32       1729.24        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.00498021     0.0947127      0.30405      0.0578962     0.0950739       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.00151734     0.0146743     0.0760103    0.00239534     0.0228411       10     
reco:pandoracalo:Calorimetry                          0.000484097    0.0164661     0.0996437    0.00163244     0.0295611       10     
reco:pandorapid:Chi2ParticleID                        0.000123564   0.00118757    0.00660037    0.000198872   0.00204761       10     
reco:cvnmap:CVNMapper                                  0.0398524     0.130625      0.263625      0.110593      0.0704433       10     
reco:cvneva:CVNEvaluator                               0.777829       2.83716       13.4993       1.86003       3.60013        10     
reco:energyrecnumu:EnergyReco                          0.009634       0.16801      0.408471       0.16338      0.122559        10     
reco:energyrecnue:EnergyReco                          0.00235952    0.00924383     0.0332877    0.00534295     0.0088712       10     
reco:energyrecnc:EnergyReco                           0.00465344     0.0425587     0.186268      0.0205421     0.0551323       10     
[art]:TriggerResults:TriggerResultInserter            2.0518e-05    0.000116607   0.000414937   9.6137e-05    0.000107693      10     
end_path:out1:RootOutput                              1.1562e-05    4.25076e-05   0.000156006    3.171e-05    3.92767e-05      10     
end_path:out1:RootOutput(write)                        0.291136       0.65706       1.06703      0.737075      0.271444        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21750.4 MB
  Peak resident set size usage (VmHWM): 2793.66 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 30.1511 | hit correction : 1.42034
hit initial : 101.958 | hit correction : 1.41317
hit initial : 348.886 | hit correction : 1.41333
hit initial : 229.583 | hit correction : 1.41339
hit initial : 92.8793 | hit correction : 1.41347
hit initial : 103.772 | hit correction : 1.41398
hit initial : 417.311 | hit correction : 1.41484
hit initial : 14.2774 | hit correction : 1.41483
hit initial : 345.987 | hit correction : 1.41525
hit initial : 197.036 | hit correction : 1.41683
hit initial : 208.727 | hit correction : 1.4175
hit initial : 66.5058 | hit correction : 1.41836
hit initial : 113.056 | hit correction : 1.41791
hit initial : 143.184 | hit correction : 1.41834
hit initial : 108.571 | hit correction : 1.41787
hit initial : 213.328 | hit correction : 1.41837
hit initial : 29.9752 | hit correction : 1.41924
hit initial : 216.818 | hit correction : 1.41907
hit initial : 194.384 | hit correction : 1.41974
hit initial : 129.84 | hit correction : 1.42018
 *** Total Charge : 4682.65
 Looking at view: 2 | Number of hits in view: 20
Begin processing the 9th record. run: 5027 subRun: 0 event: 11219 at 06-Dec-2025 07:34:47 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  3
This pfp has size:  37
Added hit list length: 37 | Max was set to: -3.40282e+38
This pfp has size:  24
True Neutron Energy: 940.31
True Electron Energy: 36.719
Completed MCParticle Loop
True Neutrino Energy: 54.0491
 Size of all hits: 37
hit initial : 249.15 | hit correction : 1.36998
hit initial : 249.631 | hit correction : 1.37
hit initial : 105.999 | hit correction : 1.37093
hit initial : 177.358 | hit correction : 1.37161
hit initial : 202.604 | hit correction : 1.37162
hit initial : 49.5007 | hit correction : 1.37261
hit initial : 77.0299 | hit correction : 1.37112
hit initial : 71.1193 | hit correction : 1.37193
hit initial : 269.55 | hit correction : 1.37106
hit initial : 137.391 | hit correction : 1.37193
hit initial : 208.904 | hit correction : 1.37197
hit initial : 49.5327 | hit correction : 1.37119
hit initial : 67.3853 | hit correction : 1.372
 *** Total Charge : 2625.98
 Looking at view: 1 | Number of hits in view: 13
hit initial : 118.448 | hit correction : 1.37987
hit initial : 180.379 | hit correction : 1.3798
hit initial : 38.0958 | hit correction : 1.38174
hit initial : 15.0748 | hit correction : 1.38174
hit initial : 183.871 | hit correction : 1.37007
hit initial : 167.82 | hit correction : 1.37228
hit initial : 302.012 | hit correction : 1.36991
hit initial : 435.043 | hit correction : 1.37239
hit initial : 258.576 | hit correction : 1.36995
hit initial : 201.009 | hit correction : 1.37258
hit initial : 50.6043 | hit correction : 1.37009
hit initial : 28.0204 | hit correction : 1.37261
hit initial : 27.055 | hit correction : 1.37083
hit initial : 231.269 | hit correction : 1.37088
hit initial : 26.0239 | hit correction : 1.37207
hit initial : 159.947 | hit correction : 1.37205
hit initial : 194.795 | hit correction : 1.37202
hit initial : 143.133 | hit correction : 1.37193
hit initial : 164.119 | hit correction : 1.37157
hit initial : 128.926 | hit correction : 1.37175
hit initial : 124.812 | hit correction : 1.37149
hit initial : 126.609 | hit correction : 1.37155
hit initial : 200.725 | hit correction : 1.37117
hit initial : 325.094 | hit correction : 1.37102
 *** Total Charge : 5257.36
 Looking at view: 2 | Number of hits in view: 24
Begin processing the 10th record. run: 5027 subRun: 0 event: 11220 at 06-Dec-2025 07:34:49 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.566
True Electron Energy: 8.01173
Completed MCParticle Loop
True Neutrino Energy: 13.5169
 Size of all hits: 0
06-Dec-2025 07:34:51 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0469872     0.0635727     0.0812194     0.0621304     0.0100862       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000368198   0.000544402   0.000937356    0.0004909    0.000176056      10     
end_path:ana:AnalyzeEvents         0.0465028     0.0628526     0.0806892     0.0613286     0.0101056       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19476.1 MB
  Peak resident set size usage (VmHWM): 1530.9 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 19:09:43 UTC       justIN version: 01.06.00