justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 262169.1@dunegpschedd02.fnal.gov

Jobsub ID262169.1@dunegpschedd02.fnal.gov
Workflow ID11175
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-13 10:44:31
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-12-13 21:17:50
From worker nodeHostnamedunegli-7890501-0-fnpc18005.fnal.gov
cpuinfoAMD EPYC 7551 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-13 10:46:35
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T232306Z_gen_001663_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time10h (37859s)
CPU time10h (37200s = 98%)
Max RSS bytes2827911168 (2696 MiB)
Outputting started2025-12-13 21:17:35
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11175/1/001/analysisLowE_20251213T104638Z.root
Finished2025-12-13 21:17:50
Saved logsjustin-logs:262169.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

in function: Run
    in file:     /exp/dune/app/users/osbiston/18_10_24_Official_LowE_Build/srcs/larpandoracontent/larpandoracontent/LArMonitoring/EventValidationBaseAlgorithm.cc line#: 64
iter->second->Run() throw STATUS_CODE_NOT_INITIALIZED
    in function: RunAlgorithm
    in file:     /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0015, LArNeutrinoEventValidation, STATUS_CODE_NOT_INITIALIZED
Boundary wire vector sizes: 6729, 6790, 6831
minwire 0: 40
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 237, 736
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.028024, 
Output 1: 0.0452232, 0.116586, 0.0429298, 0.795261, 
Output 2: 0.112814, 0.291845, 0.561387, 0.0339539, 
Output 3: 0.805684, 0.16828, 0.0160435, 0.00999246, 
Output 4: 0.940589, 0.055621, 0.00298238, 0.000807856, 
Output 5: 0.15939, 0.707801, 0.1188, 0.0140091, 
Output 6: 0.0496744, 0.396421, 0.210286, 0.343619, 

13-Dec-2025 21:16:26 UTC  Closed output file "matt_reco2.root"
13-Dec-2025 21:16:26 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/10/2d/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T232306Z_gen_001663_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2455.78       3712.71       5987.78       3675.22       874.125        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00147275    0.00931391     0.0627295    0.00179444     0.0183385       10     
reco:pandora:StandardPandora                            2453.01       3708.75       5978.88       3671.3        872.638        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.00151428     0.0808304     0.156498      0.0923125     0.0556862       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000543634    0.0103756     0.0489356    0.00449355     0.0136062       10     
reco:pandoracalo:Calorimetry                          0.000368995    0.0136139     0.116865     0.00100793     0.0345386       10     
reco:pandorapid:Chi2ParticleID                        0.000127219   0.000573003   0.00374885    0.000201159   0.00106129       10     
reco:cvnmap:CVNMapper                                  0.0875131      0.12875      0.229607       0.1218       0.0404763       10     
reco:cvneva:CVNEvaluator                                1.98416       3.00964       7.29364       2.35389       1.50505        10     
reco:energyrecnumu:EnergyReco                          0.0971463     0.124371      0.240514      0.104067      0.0427091       10     
reco:energyrecnue:EnergyReco                          0.00261752    0.00561562     0.0121542    0.00410724    0.00342244       10     
reco:energyrecnc:EnergyReco                           0.00983209     0.0138906     0.0228376     0.012434      0.0037772       10     
[art]:TriggerResults:TriggerResultInserter             2.685e-05    5.60285e-05   0.000206328   3.67645e-05   5.17325e-05      10     
end_path:out1:RootOutput                              1.2674e-05    3.33769e-05   0.00017023    1.9457e-05    4.5717e-05       10     
end_path:out1:RootOutput(write)                        0.391816       0.56003       0.88412       0.45387      0.188294        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21560.3 MB
  Peak resident set size usage (VmHWM): 2827.91 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 228.444 | hit correction : 1.00635
hit initial : 266.285 | hit correction : 1.00589
hit initial : 377.659 | hit correction : 1.00552
 *** Total Charge : 3311.53
 Looking at view: 2 | Number of hits in view: 17
Begin processing the 8th record. run: 5027 subRun: 0 event: 16628 at 13-Dec-2025 21:17:19 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  6
This pfp has size:  15
Added hit list length: 15 | Max was set to: -3.40282e+38
This pfp has size:  12
This pfp has size:  6
This pfp has size:  4
True Electron Energy: 13.1706
Completed MCParticle Loop
True Neutrino Energy: 18.6797
 Size of all hits: 15
hit initial : 57.1602 | hit correction : 1.14005
hit initial : 94.5622 | hit correction : 1.14057
hit initial : 271.497 | hit correction : 1.14059
hit initial : 433.155 | hit correction : 1.14071
hit initial : 548.885 | hit correction : 1.14072
 *** Total Charge : 1602.91
 Looking at view: 0 | Number of hits in view: 5
hit initial : 56.8514 | hit correction : 1.14058
hit initial : 216.988 | hit correction : 1.14058
hit initial : 276.49 | hit correction : 1.14064
hit initial : 268.507 | hit correction : 1.14076
hit initial : 273.235 | hit correction : 1.14076
hit initial : 262.406 | hit correction : 1.14042
 *** Total Charge : 1544.96
 Looking at view: 1 | Number of hits in view: 6
hit initial : 347.505 | hit correction : 1.14017
hit initial : 339.413 | hit correction : 1.14063
hit initial : 294.725 | hit correction : 1.14068
hit initial : 549.26 | hit correction : 1.14072
 *** Total Charge : 1746.1
 Looking at view: 2 | Number of hits in view: 4
Begin processing the 9th record. run: 5027 subRun: 0 event: 16629 at 13-Dec-2025 21:17:24 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  5
This pfp has size:  19
Added hit list length: 19 | Max was set to: -3.40282e+38
This pfp has size:  8
This pfp has size:  4
True Electron Energy: 14.3492
Completed MCParticle Loop
True Neutrino Energy: 18.0767
 Size of all hits: 19
hit initial : 125.865 | hit correction : 1.07051
hit initial : 303.896 | hit correction : 1.071
hit initial : 149.966 | hit correction : 1.07065
hit initial : 625.068 | hit correction : 1.07117
hit initial : 357.221 | hit correction : 1.07158
hit initial : 157.858 | hit correction : 1.07181
 *** Total Charge : 1842.31
 Looking at view: 0 | Number of hits in view: 6
hit initial : 119.196 | hit correction : 1.07063
hit initial : 166.634 | hit correction : 1.07052
hit initial : 298.479 | hit correction : 1.071
hit initial : 71.0538 | hit correction : 1.07051
hit initial : 479.26 | hit correction : 1.07117
hit initial : 353.479 | hit correction : 1.07143
hit initial : 216.515 | hit correction : 1.07174
hit initial : 104.919 | hit correction : 1.07185
 *** Total Charge : 1938.34
 Looking at view: 1 | Number of hits in view: 8
hit initial : 296.442 | hit correction : 1.07076
hit initial : 223.584 | hit correction : 1.0705
hit initial : 733.808 | hit correction : 1.07116
hit initial : 517.204 | hit correction : 1.07169
hit initial : 50.0188 | hit correction : 1.07189
 *** Total Charge : 1950.69
 Looking at view: 2 | Number of hits in view: 5
Begin processing the 10th record. run: 5027 subRun: 0 event: 16630 at 13-Dec-2025 21:17:28 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
True Neutron Energy: 939.571
True Electron Energy: 37.2926
Completed MCParticle Loop
True Neutrino Energy: 41.0538
 Size of all hits: 0
13-Dec-2025 21:17:33 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0965014     0.131182      0.198092      0.122341      0.0280283       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000678478    0.0011062    0.00381973    0.000806063   0.000908422      10     
end_path:ana:AnalyzeEvents         0.0955788     0.129773      0.193444      0.121237      0.0271514       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19472.2 MB
  Peak resident set size usage (VmHWM): 1533.55 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 12:23:37 UTC       justIN version: 01.06.00