justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 264365.5@dunegpschedd01.fnal.gov

Jobsub ID264365.5@dunegpschedd01.fnal.gov
Workflow ID11030
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-06 11:05:54
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-12-06 19:25:37
From worker nodeHostnamedunegli-7832544-0-fnpc7304.fnal.gov
cpuinfoIntel(R) Xeon(R) CPU E5-2650 v3 @ 2.30GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-06 11:06:16
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T015338Z_gen_001500_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time8h (29945s)
CPU time8h (29488s = 98%)
Max RSS bytes3034554368 (2893 MiB)
Outputting started2025-12-06 19:25:21
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11030/1/001/analysisLowE_20251206T110616Z.root
Finished2025-12-06 19:25:37
Saved logsjustin-logs:264365.5-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ATUS_CODE_NOT_INITIALIZED
    in function: RunAlgorithm
    in file:     /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0005, LArDLVertexing, STATUS_CODE_NOT_INITIALIZED
Boundary wire vector sizes: 6675, 6639, 6502
minwire 0: 47
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 2059, 2558
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0826801, 
Output 1: 0.0565376, 0.0670448, 0.134639, 0.741779, 
Output 2: 0.0843486, 0.207549, 0.689115, 0.018987, 
Output 3: 0.941841, 0.0517257, 0.00302717, 0.00340636, 
Output 4: 0.292275, 0.621966, 0.078307, 0.00745191, 
Output 5: 0.567622, 0.390581, 0.0359686, 0.0058286, 
Output 6: 0.0342269, 0.110141, 0.104157, 0.751475, 

06-Dec-2025 19:24:15 UTC  Closed output file "matt_reco2.root"
06-Dec-2025 19:24:15 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/d8/b1/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T015338Z_gen_001500_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2264.93       2967.97       3504.87       2963.91       343.771        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00278523     0.0116667     0.0235705     0.011283     0.00545175       10     
reco:pandora:StandardPandora                            2262.06       2964.01       3499.51       2960.6        343.023        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.00118775     0.0420746     0.168588      0.0280575     0.0462072       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000526465    0.0104472     0.0567656    0.00624724     0.0158232       10     
reco:pandoracalo:Calorimetry                          0.000411713   0.00783803     0.0494044    0.00156726     0.0142256       10     
reco:pandorapid:Chi2ParticleID                        0.000124465   0.000739221   0.00350168    0.000193933   0.00114523       10     
reco:cvnmap:CVNMapper                                  0.0496235     0.076273       0.12565      0.0782323     0.0205478       10     
reco:cvneva:CVNEvaluator                                1.29627       3.01224       6.66039       2.50375       1.48104        10     
reco:energyrecnumu:EnergyReco                          0.0106992     0.0948407     0.171603      0.122647      0.0644271       10     
reco:energyrecnue:EnergyReco                          0.00170026     0.0062058     0.0151339    0.00324156     0.0052845       10     
reco:energyrecnc:EnergyReco                            0.0118248     0.0603084     0.138016      0.0166703     0.0562636       10     
[art]:TriggerResults:TriggerResultInserter            2.8385e-05    5.02461e-05   0.000154138   4.0671e-05    3.50865e-05      10     
end_path:out1:RootOutput                              1.0918e-05    1.71722e-05   5.1145e-05    1.21745e-05   1.17967e-05      10     
end_path:out1:RootOutput(write)                        0.400122      0.635185      0.756499       0.68075       0.10054        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 9932.33 MB
  Peak resident set size usage (VmHWM): 3034.55 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 191.505 | hit correction : 1.40427
hit initial : 201.64 | hit correction : 1.40395
hit initial : 225.638 | hit correction : 1.40366
hit initial : 207.929 | hit correction : 1.4035
hit initial : 239.91 | hit correction : 1.40342
hit initial : 257.054 | hit correction : 1.40337
hit initial : 165.707 | hit correction : 1.40335
 *** Total Charge : 4689.62
 Looking at view: 0 | Number of hits in view: 24
hit initial : 71.8458 | hit correction : 1.40836
hit initial : 63.1098 | hit correction : 1.4051
hit initial : 232 | hit correction : 1.40524
hit initial : 222.088 | hit correction : 1.40519
hit initial : 200.03 | hit correction : 1.40523
hit initial : 158.455 | hit correction : 1.40513
hit initial : 229.179 | hit correction : 1.4051
hit initial : 246.799 | hit correction : 1.40479
hit initial : 210.466 | hit correction : 1.40433
hit initial : 183.886 | hit correction : 1.40398
hit initial : 174.283 | hit correction : 1.4037
hit initial : 161.4 | hit correction : 1.40354
hit initial : 136.455 | hit correction : 1.40344
hit initial : 147.551 | hit correction : 1.40341
hit initial : 180.4 | hit correction : 1.40338
hit initial : 180.789 | hit correction : 1.40335
hit initial : 117.743 | hit correction : 1.40334
hit initial : 47.1202 | hit correction : 1.40334
 *** Total Charge : 4162.14
 Looking at view: 1 | Number of hits in view: 18
Begin processing the 9th record. run: 5027 subRun: 0 event: 14999 at 06-Dec-2025 19:25:10 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
This pfp has size:  32
Added hit list length: 32 | Max was set to: -3.40282e+38
True Electron Energy: 38.4119
Completed MCParticle Loop
True Neutrino Energy: 60.4611
 Size of all hits: 32
hit initial : 74.3705 | hit correction : 1.40076
hit initial : 51.323 | hit correction : 1.40076
hit initial : 214.126 | hit correction : 1.40384
hit initial : 449.124 | hit correction : 1.40411
hit initial : 500.449 | hit correction : 1.40459
hit initial : 604.735 | hit correction : 1.40493
hit initial : 700.033 | hit correction : 1.40502
hit initial : 823.487 | hit correction : 1.40491
hit initial : 277.958 | hit correction : 1.4042
hit initial : 243.526 | hit correction : 1.40519
hit initial : 75.8808 | hit correction : 1.40414
hit initial : 69.333 | hit correction : 1.40704
hit initial : 161.145 | hit correction : 1.40632
hit initial : 108.307 | hit correction : 1.40717
hit initial : 71.3147 | hit correction : 1.40702
 *** Total Charge : 6216.29
 Looking at view: 0 | Number of hits in view: 15
hit initial : 69.2754 | hit correction : 1.40371
hit initial : 103.678 | hit correction : 1.40526
hit initial : 219.335 | hit correction : 1.40514
hit initial : 238.499 | hit correction : 1.40488
hit initial : 86.368 | hit correction : 1.40465
hit initial : 188.504 | hit correction : 1.40378
hit initial : 262.071 | hit correction : 1.40395
hit initial : 363.598 | hit correction : 1.40421
hit initial : 471.446 | hit correction : 1.40439
hit initial : 567.365 | hit correction : 1.40469
hit initial : 686.145 | hit correction : 1.40491
hit initial : 177.824 | hit correction : 1.40692
hit initial : 280.576 | hit correction : 1.40477
hit initial : 198.402 | hit correction : 1.40553
hit initial : 107.147 | hit correction : 1.40656
hit initial : 57.0438 | hit correction : 1.40722
hit initial : 223.504 | hit correction : 1.4052
 *** Total Charge : 6041.91
 Looking at view: 1 | Number of hits in view: 17
Begin processing the 10th record. run: 5027 subRun: 0 event: 15000 at 06-Dec-2025 19:25:16 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.566
True Electron Energy: 50.2503
Completed MCParticle Loop
True Neutrino Energy: 55.6715
 Size of all hits: 0
06-Dec-2025 19:25:21 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0971297     0.126406      0.162123      0.120023      0.0257583       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000386786   0.00060158    0.00145793    0.000483978   0.000296936      10     
end_path:ana:AnalyzeEvents         0.0951975     0.125617      0.161313      0.119335      0.0258376       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 7662.86 MB
  Peak resident set size usage (VmHWM): 1536.37 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 19:10:33 UTC       justIN version: 01.06.00