justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 264317.0@dunegpschedd01.fnal.gov

Jobsub ID264317.0@dunegpschedd01.fnal.gov
Workflow ID11023
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-06 09:43:50
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-12-06 15:32:14
From worker nodeHostnamedunegli-7832314-0-fnpc18010.fnal.gov
cpuinfoAMD EPYC 7551 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-06 09:44:36
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T020045Z_gen_003477_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time5h (20839s)
CPU time5h (20558s = 98%)
Max RSS bytes3071086592 (2928 MiB)
Outputting started2025-12-06 15:31:56
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11023/1/001/analysisLowE_20251206T094437Z.root
Finished2025-12-06 15:32:14
Saved logsjustin-logs:264317.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

in function: RunAlgorithm
    in file:     /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0005, LArDLVertexing, STATUS_CODE_NOT_INITIALIZED
Boundary wire vector sizes: 6865, 6791, 6779
minwire 0: 32
minwire 1: 1
minwire 2: 4
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 2259, 2758
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0627625, 
Output 1: 0.0903498, 0.108713, 0.0871049, 0.713833, 
Output 2: 0.0468045, 0.190336, 0.727645, 0.0352148, 
Output 3: 0.956697, 0.014874, 0.00458153, 0.0238473, 
Output 4: 0.946152, 0.045505, 0.00584018, 0.00250323, 
Output 5: 0.70658, 0.193292, 0.0690165, 0.031112, 
Output 6: 0.0136959, 0.0299974, 0.0409839, 0.915323, 

06-Dec-2025 15:30:30 UTC  Closed output file "matt_reco2.root"
06-Dec-2025 15:30:30 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/eb/9e/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T020045Z_gen_003477_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1866.22       2061.47       2709.37       1991.7        242.781        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00226507    0.00883681     0.0213783    0.00682453    0.00519471       10     
reco:pandora:StandardPandora                            1863.76       2058.48       2706.85       1989.18       242.755        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000491468    0.035262      0.175512      0.0247187     0.0487316       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000412058    0.0064994     0.021084      0.0037018    0.00650055       10     
reco:pandoracalo:Calorimetry                          0.000427607    0.0095327     0.0379748    0.00238318     0.0140405       10     
reco:pandorapid:Chi2ParticleID                        9.8436e-05    0.000488738    0.0024082    0.00014822    0.000736978      10     
reco:cvnmap:CVNMapper                                  0.0330594     0.0499055      0.10169      0.0439997     0.0190272       10     
reco:cvneva:CVNEvaluator                                1.85505       2.29576       4.15112       1.93987      0.739589        10     
reco:energyrecnumu:EnergyReco                          0.0129837     0.0747943     0.153306      0.0943612     0.048173        10     
reco:energyrecnue:EnergyReco                          0.00152014    0.00566225     0.0145077    0.00235951    0.00470743       10     
reco:energyrecnc:EnergyReco                           0.00963069     0.0475959     0.120912      0.0124201     0.0464654       10     
[art]:TriggerResults:TriggerResultInserter            2.1179e-05    3.92988e-05   0.000149291   2.7006e-05    3.6993e-05       10     
end_path:out1:RootOutput                               8.716e-06    1.50485e-05   5.2439e-05    9.6335e-06    1.28328e-05      10     
end_path:out1:RootOutput(write)                         0.38001      0.453454      0.674336      0.405772      0.0937771       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21755.7 MB
  Peak resident set size usage (VmHWM): 3071.09 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 88.6061 | hit correction : 1.04607
hit initial : 193.737 | hit correction : 1.0462
hit initial : 39.5457 | hit correction : 1.04623
hit initial : 487.582 | hit correction : 1.04683
hit initial : 84.8714 | hit correction : 1.04685
hit initial : 91.3176 | hit correction : 1.04685
hit initial : 639.88 | hit correction : 1.0477
hit initial : 44.9298 | hit correction : 1.04781
hit initial : 140.38 | hit correction : 1.04813
hit initial : 67.1886 | hit correction : 1.04826
 *** Total Charge : 3719.48
 Looking at view: 0 | Number of hits in view: 22
hit initial : 45.9711 | hit correction : 1.04827
hit initial : 12.6774 | hit correction : 1.04006
hit initial : 226.403 | hit correction : 1.0407
hit initial : 326.703 | hit correction : 1.04073
hit initial : 51.3672 | hit correction : 1.04007
hit initial : 29.8238 | hit correction : 1.04007
hit initial : 80.5372 | hit correction : 1.04042
hit initial : 60.1379 | hit correction : 1.04041
hit initial : 93.216 | hit correction : 1.04045
hit initial : 124.095 | hit correction : 1.04075
hit initial : 44.1268 | hit correction : 1.0409
hit initial : 66.8869 | hit correction : 1.04096
hit initial : 258.355 | hit correction : 1.041
hit initial : 65.9828 | hit correction : 1.04104
hit initial : 35.6929 | hit correction : 1.04444
hit initial : 49.6691 | hit correction : 1.04446
hit initial : 161.43 | hit correction : 1.04618
hit initial : 371.53 | hit correction : 1.04624
hit initial : 114.103 | hit correction : 1.0462
hit initial : 232.633 | hit correction : 1.04672
hit initial : 246.372 | hit correction : 1.04683
hit initial : 249.873 | hit correction : 1.04694
hit initial : 260.161 | hit correction : 1.04714
hit initial : 277.511 | hit correction : 1.04745
hit initial : 296.887 | hit correction : 1.04773
hit initial : 266.622 | hit correction : 1.048
hit initial : 168.626 | hit correction : 1.0482
 *** Total Charge : 4406.62
 Looking at view: 1 | Number of hits in view: 27
hit initial : 345.877 | hit correction : 1.04102
hit initial : 218.88 | hit correction : 1.04091
hit initial : 32.8369 | hit correction : 1.04522
hit initial : 23.5283 | hit correction : 1.04526
hit initial : 34.1837 | hit correction : 1.04082
hit initial : 362.286 | hit correction : 1.04075
hit initial : 397.453 | hit correction : 1.04067
hit initial : 55.4739 | hit correction : 1.04007
hit initial : 106.829 | hit correction : 1.04009
hit initial : 116.71 | hit correction : 1.04623
hit initial : 16.5132 | hit correction : 1.04442
hit initial : 82.8077 | hit correction : 1.04444
hit initial : 270.438 | hit correction : 1.04045
hit initial : 103.547 | hit correction : 1.04042
hit initial : 21.2573 | hit correction : 1.04622
hit initial : 236.307 | hit correction : 1.04606
hit initial : 139.59 | hit correction : 1.0462
hit initial : 258.244 | hit correction : 1.04659
hit initial : 150.066 | hit correction : 1.04677
hit initial : 171.763 | hit correction : 1.04683
hit initial : 174.103 | hit correction : 1.04689
hit initial : 144.58 | hit correction : 1.04697
hit initial : 149.955 | hit correction : 1.0471
hit initial : 159.334 | hit correction : 1.04726
hit initial : 177.421 | hit correction : 1.04746
hit initial : 230.255 | hit correction : 1.04766
hit initial : 162.165 | hit correction : 1.04785
hit initial : 148.718 | hit correction : 1.04802
hit initial : 150.785 | hit correction : 1.04816
hit initial : 124.527 | hit correction : 1.04829
 *** Total Charge : 4978.42
 Looking at view: 2 | Number of hits in view: 30
Begin processing the 10th record. run: 5027 subRun: 0 event: 34770 at 06-Dec-2025 15:31:52 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.567
True Neutron Energy: 939.726
True Electron Energy: 20.5454
Completed MCParticle Loop
True Neutrino Energy: 29.5089
 Size of all hits: 0
06-Dec-2025 15:31:55 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0999254     0.157738       0.18868      0.167834      0.0281303       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.00057232    0.000840473   0.00166066    0.000754233   0.000301483      10     
end_path:ana:AnalyzeEvents         0.0991932     0.156623      0.187701       0.16695      0.0279718       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19478 MB
  Peak resident set size usage (VmHWM): 1542.66 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 19:10:31 UTC       justIN version: 01.06.00