justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 263750.93@dunegpschedd01.fnal.gov

Jobsub ID263750.93@dunegpschedd01.fnal.gov
Workflow ID11025
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-05 10:59:40
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-12-06 08:51:33
From worker nodeHostnamedunegli-7989145-0-fnpc17108.fnal.gov
cpuinfoIntel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-05 23:11:52
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T203916Z_gen_003053_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time9h (34765s)
CPU time9h (34262s = 98%)
Max RSS bytes3035766784 (2895 MiB)
Outputting started2025-12-06 08:51:17
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11025/1/001/analysisLowE_20251205T231154Z.root
Finished2025-12-06 08:51:33
Saved logsjustin-logs:263750.93-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ontent/larpandoradlcontent/LArVertex/DlVertexingAlgorithm.cc line#: 181
iter->second->Run() throw STATUS_CODE_NOT_INITIALIZED
    in function: RunAlgorithm
    in file:     /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0005, LArDLVertexing, STATUS_CODE_NOT_INITIALIZED
Boundary wire vector sizes: 6863, 6863, 6762
minwire 0: 34
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0847149, 
Output 1: 0.0534503, 0.117859, 0.0961922, 0.732498, 
Output 2: 0.0716303, 0.227872, 0.672012, 0.0284855, 
Output 3: 0.971364, 0.0202693, 0.00190875, 0.00645775, 
Output 4: 0.634383, 0.332247, 0.0295946, 0.00377492, 
Output 5: 0.616508, 0.303488, 0.0616676, 0.0183368, 
Output 6: 0.0206958, 0.0521308, 0.0650234, 0.86215, 

06-Dec-2025 08:49:55 UTC  Closed output file "matt_reco2.root"
06-Dec-2025 08:49:55 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/77/b3/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T203916Z_gen_003053_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2191.15       3440.58       4822.35       3502.88       691.625        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0119218     0.063286       0.3235       0.014118       0.10092        10     
reco:pandora:StandardPandora                            2188.79       3417.33       4741.88       3500.21       666.44         10     
reco:pandoraTrack:LArPandoraTrackCreation             0.00181445     0.0577718     0.411829      0.0132473     0.119357        10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.00220111     0.0109003     0.0497458    0.00719621     0.0135619       10     
reco:pandoracalo:Calorimetry                          0.000555071    0.0295486     0.0930359    0.00648044     0.0347871       10     
reco:pandorapid:Chi2ParticleID                        0.00014785    0.000699428   0.00505623    0.000220194   0.00145288       10     
reco:cvnmap:CVNMapper                                  0.0674764     0.0997568     0.171598      0.0927519     0.0287031       10     
reco:cvneva:CVNEvaluator                                1.50534       2.40765       9.40199       1.64449       2.33314        10     
reco:energyrecnumu:EnergyReco                          0.0137233      13.3356       69.3511      0.129651       26.5327        10     
reco:energyrecnue:EnergyReco                          0.00209174    0.00804582     0.0172606    0.00553481    0.00592207       10     
reco:energyrecnc:EnergyReco                            0.0105185      6.55935       65.2908      0.0163306      19.5772        10     
[art]:TriggerResults:TriggerResultInserter            2.4231e-05    5.47697e-05   0.00023357    3.45235e-05   6.04172e-05      10     
end_path:out1:RootOutput                               7.847e-06    1.81947e-05   6.1132e-05    1.43795e-05   1.45815e-05      10     
end_path:out1:RootOutput(write)                        0.420805       0.67659       1.02282      0.725851      0.177443        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 14278.3 MB
  Peak resident set size usage (VmHWM): 3035.77 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 138.56 | hit correction : 1.3368
hit initial : 428.98 | hit correction : 1.33045
hit initial : 24.8238 | hit correction : 1.33045
hit initial : 1333.42 | hit correction : 1.33335
hit initial : 589.012 | hit correction : 1.33319
hit initial : 104.691 | hit correction : 1.33336
hit initial : 240.017 | hit correction : 1.33471
 *** Total Charge : 3994.1
 Looking at view: 2 | Number of hits in view: 9
Begin processing the 8th record. run: 5026 subRun: 0 event: 30528 at 06-Dec-2025 08:51:00 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
This pfp has size:  43
Added hit list length: 43 | Max was set to: -3.40282e+38
True Electron Energy: 28.196
Completed MCParticle Loop
True Neutrino Energy: 33.5837
 Size of all hits: 43
hit initial : 41.7142 | hit correction : 1.46683
hit initial : 93.1952 | hit correction : 1.46645
hit initial : 181.407 | hit correction : 1.46625
hit initial : 110.508 | hit correction : 1.46602
hit initial : 34.115 | hit correction : 1.46676
hit initial : 229.524 | hit correction : 1.46686
hit initial : 134.67 | hit correction : 1.46684
hit initial : 185.983 | hit correction : 1.46689
hit initial : 160.455 | hit correction : 1.46685
hit initial : 178.226 | hit correction : 1.46671
hit initial : 164.132 | hit correction : 1.46681
hit initial : 188.01 | hit correction : 1.46656
hit initial : 265.474 | hit correction : 1.46655
hit initial : 321.792 | hit correction : 1.46661
hit initial : 115.222 | hit correction : 1.46681
hit initial : 224.485 | hit correction : 1.46671
 *** Total Charge : 3855.71
 Looking at view: 0 | Number of hits in view: 16
hit initial : 93.2698 | hit correction : 1.4669
hit initial : 132.224 | hit correction : 1.46682
hit initial : 58.0132 | hit correction : 1.46685
hit initial : 30.7831 | hit correction : 1.46816
hit initial : 49.2552 | hit correction : 1.46813
hit initial : 382.992 | hit correction : 1.46685
hit initial : 519.243 | hit correction : 1.46679
hit initial : 155.754 | hit correction : 1.46674
hit initial : 299.028 | hit correction : 1.46656
hit initial : 202.614 | hit correction : 1.46659
hit initial : 400.009 | hit correction : 1.46666
 *** Total Charge : 3407.61
 Looking at view: 1 | Number of hits in view: 11
hit initial : 46.467 | hit correction : 1.46657
hit initial : 218.421 | hit correction : 1.46635
hit initial : 59.9137 | hit correction : 1.46688
hit initial : 138.534 | hit correction : 1.46593
hit initial : 97.3894 | hit correction : 1.46665
hit initial : 461.605 | hit correction : 1.46654
hit initial : 46.7655 | hit correction : 1.46629
hit initial : 23.5463 | hit correction : 1.46683
hit initial : 108.585 | hit correction : 1.46681
hit initial : 114.74 | hit correction : 1.46686
hit initial : 126.354 | hit correction : 1.46678
hit initial : 124.602 | hit correction : 1.46697
hit initial : 123.743 | hit correction : 1.4669
hit initial : 150.144 | hit correction : 1.46683
hit initial : 264.886 | hit correction : 1.4668
hit initial : 586.815 | hit correction : 1.46666
 *** Total Charge : 3948.95
 Looking at view: 2 | Number of hits in view: 16
Begin processing the 9th record. run: 5026 subRun: 0 event: 30529 at 06-Dec-2025 08:51:06 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Electron Energy: 8.87195
Completed MCParticle Loop
True Neutrino Energy: 14.2506
 Size of all hits: 0
Begin processing the 10th record. run: 5026 subRun: 0 event: 30530 at 06-Dec-2025 08:51:12 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.577
True Electron Energy: 47.4169
Completed MCParticle Loop
True Neutrino Energy: 55.312
 Size of all hits: 0
06-Dec-2025 08:51:16 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0917727     0.177369      0.221887      0.190657      0.038987        10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.00052969    0.00152234     0.0075851    0.000905366   0.00202673       10     
end_path:ana:AnalyzeEvents         0.0910904     0.175604      0.220692      0.189512      0.038445        10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 11964.9 MB
  Peak resident set size usage (VmHWM): 1542.55 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 19:11:01 UTC       justIN version: 01.06.00