justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 263750.120@dunegpschedd01.fnal.gov

Jobsub ID263750.120@dunegpschedd01.fnal.gov
Workflow ID11025
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-05 10:59:40
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-12-06 07:33:40
From worker nodeHostnamedunegli-7832480-0-fnpc18005.fnal.gov
cpuinfoAMD EPYC 7551 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-05 23:19:57
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T163835Z_gen_000458_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time8h (29606s)
CPU time8h (29081s = 98%)
Max RSS bytes3050299392 (2908 MiB)
Outputting started2025-12-06 07:33:23
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11025/1/001/analysisLowE_20251205T231959Z.root
Finished2025-12-06 07:33:40
Saved logsjustin-logs:263750.120-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

10_24_Official_LowE_Build/srcs/larpandoracontent/larpandoradlcontent/LArVertex/DlVertexingAlgorithm.cc line#: 181
iter->second->Run() throw STATUS_CODE_NOT_INITIALIZED
    in function: RunAlgorithm
    in file:     /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0005, LArDLVertexing, STATUS_CODE_NOT_INITIALIZED
Boundary wire vector sizes: 6551, 6489, 6518
minwire 0: 26
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 39, 538
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.103968, 
Output 1: 0.0421461, 0.0449396, 0.134662, 0.778252, 
Output 2: 0.0581805, 0.187903, 0.729759, 0.024158, 
Output 3: 0.957284, 0.0371111, 0.00289724, 0.00270732, 
Output 4: 0.224664, 0.666105, 0.101624, 0.0076068, 
Output 5: 0.641362, 0.279768, 0.06322, 0.0156498, 
Output 6: 0.0245237, 0.0934905, 0.0966744, 0.785311, 

06-Dec-2025 07:32:15 UTC  Closed output file "matt_reco2.root"
06-Dec-2025 07:32:15 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/e6/0f/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T163835Z_gen_000458_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2433.85       2930.3        3512.68       2944.9        337.634        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0060198     0.0135573     0.0277661     0.0101616     0.0074576       10     
reco:pandora:StandardPandora                            2427.99       2924.64       3506.95       2939.25       337.278        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.00067947     0.0415838     0.287963      0.0111879     0.0831863       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.00146233    0.00829003     0.0372571    0.00421926     0.0103433       10     
reco:pandoracalo:Calorimetry                          0.000270158    0.0163633     0.0556944    0.00366979     0.0215305       10     
reco:pandorapid:Chi2ParticleID                        0.000112171   0.000777465   0.00277324    0.000229297   0.00100854       10     
reco:cvnmap:CVNMapper                                  0.0445511     0.0773633     0.211786      0.0604516     0.0468075       10     
reco:cvneva:CVNEvaluator                                2.32768       4.64488       10.7656       4.05587       2.2909         10     
reco:energyrecnumu:EnergyReco                         0.00951577     0.086233      0.242732      0.070057      0.0753777       10     
reco:energyrecnue:EnergyReco                           0.002068     0.00789469     0.0182372    0.00323734    0.00648645       10     
reco:energyrecnc:EnergyReco                            0.0130809     0.0735464      0.14872      0.0640474     0.0593319       10     
[art]:TriggerResults:TriggerResultInserter            2.3835e-05    5.93076e-05   0.000186351   4.24195e-05   4.54693e-05      10     
end_path:out1:RootOutput                              1.1431e-05    2.42057e-05   6.0173e-05    1.42825e-05   1.80746e-05      10     
end_path:out1:RootOutput(write)                        0.534188      0.685422      0.989387      0.675037      0.130678        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21750.6 MB
  Peak resident set size usage (VmHWM): 3050.3 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
This pfp has size:  35
Added hit list length: 35 | Max was set to: -3.40282e+38
True Neutron Energy: 939.575
True Neutron Energy: 944.117
True Neutron Energy: 940.971
True Electron Energy: 10.715
Completed MCParticle Loop
True Neutrino Energy: 30.1474
 Size of all hits: 35
hit initial : 49.5406 | hit correction : 1.43972
hit initial : 33.1583 | hit correction : 1.43969
hit initial : 81.1665 | hit correction : 1.43025
hit initial : 204.531 | hit correction : 1.4303
hit initial : 199.145 | hit correction : 1.43037
hit initial : 72.7738 | hit correction : 1.4305
hit initial : 65.9051 | hit correction : 1.43227
hit initial : 32.0898 | hit correction : 1.4323
hit initial : 36.8048 | hit correction : 1.43229
hit initial : 84.0709 | hit correction : 1.43368
hit initial : 129.209 | hit correction : 1.43377
hit initial : 318.472 | hit correction : 1.43397
hit initial : 220.65 | hit correction : 1.43497
hit initial : 164.501 | hit correction : 1.43531
hit initial : 64.9648 | hit correction : 1.43546
 *** Total Charge : 2518.17
 Looking at view: 0 | Number of hits in view: 15
hit initial : 119.008 | hit correction : 1.43029
hit initial : 260.179 | hit correction : 1.43032
hit initial : 147.919 | hit correction : 1.43039
hit initial : 63.6817 | hit correction : 1.43364
hit initial : 160.146 | hit correction : 1.43372
hit initial : 60.5931 | hit correction : 1.43381
hit initial : 53.7076 | hit correction : 1.43379
hit initial : 142.823 | hit correction : 1.43484
hit initial : 166.256 | hit correction : 1.43522
hit initial : 155.23 | hit correction : 1.43526
 *** Total Charge : 1905.06
 Looking at view: 1 | Number of hits in view: 10
hit initial : 28.055 | hit correction : 1.43555
hit initial : 71.3713 | hit correction : 1.43028
hit initial : 319.429 | hit correction : 1.43031
hit initial : 141.447 | hit correction : 1.43046
hit initial : 93.9243 | hit correction : 1.43364
hit initial : 195.746 | hit correction : 1.43366
hit initial : 107.026 | hit correction : 1.43411
hit initial : 237.659 | hit correction : 1.43458
hit initial : 178.037 | hit correction : 1.43521
hit initial : 132.853 | hit correction : 1.4354
 *** Total Charge : 2157.5
 Looking at view: 2 | Number of hits in view: 10
Begin processing the 7th record. run: 5026 subRun: 0 event: 4577 at 06-Dec-2025 07:33:07 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Electron Energy: 2.93524
Completed MCParticle Loop
True Neutrino Energy: 7.76927
 Size of all hits: 0
Begin processing the 8th record. run: 5026 subRun: 0 event: 4578 at 06-Dec-2025 07:33:11 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
True Electron Energy: 8.10012
Completed MCParticle Loop
True Neutrino Energy: 13.4783
 Size of all hits: 0
Begin processing the 9th record. run: 5026 subRun: 0 event: 4579 at 06-Dec-2025 07:33:14 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Electron Energy: 3.60781
Completed MCParticle Loop
True Neutrino Energy: 8.11944
 Size of all hits: 0
Begin processing the 10th record. run: 5026 subRun: 0 event: 4580 at 06-Dec-2025 07:33:18 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.574
True Neutron Energy: 940.821
True Electron Energy: 40.0611
Completed MCParticle Loop
True Neutrino Energy: 57.3433
 Size of all hits: 0
06-Dec-2025 07:33:22 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0937215     0.123508       0.14096      0.123279      0.0139963       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000596614   0.000865519    0.0018799    0.000747768   0.000357953      10     
end_path:ana:AnalyzeEvents         0.0928375     0.122357      0.139805      0.122322      0.0138044       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19475.5 MB
  Peak resident set size usage (VmHWM): 1536.57 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 19:11:00 UTC       justIN version: 01.06.00