justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 265373.0@dunegpschedd01.fnal.gov

Jobsub ID265373.0@dunegpschedd01.fnal.gov
Workflow ID11086
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-09 12:59:45
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-12-10 18:12:24
From worker nodeHostnamedunegli-8024619-0-fnpc8016.fnal.gov
cpuinfoIntel(R) Xeon(R) CPU E5-2670 v3 @ 2.30GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-10 11:58:46
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T001241Z_gen_001948_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time6h (22395s)
CPU time6h (22135s = 98%)
Max RSS bytes3048599552 (2907 MiB)
Outputting started2025-12-10 18:12:02
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11086/1/001/analysisLowE_20251210T115847Z.root
Finished2025-12-10 18:12:24
Saved logsjustin-logs:265373.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

IALIZED
    in function: RunAlgorithm
    in file:     /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0005, LArDLVertexing, STATUS_CODE_NOT_INITIALIZED
Boundary wire vector sizes: 6908, 6788, 6799
minwire 0: 21
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 2220, 2719
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.123924, 
Output 1: 0.0692891, 0.0375172, 0.0804921, 0.812702, 
Output 2: 0.092044, 0.296992, 0.573887, 0.037077, 
Output 3: 0.97494, 0.014196, 0.00217915, 0.00868515, 
Output 4: 0.361009, 0.558127, 0.0738745, 0.0069895, 
Output 5: 0.6362, 0.323719, 0.0340565, 0.00602494, 
Output 6: 0.0207468, 0.0524283, 0.070633, 0.856192, 

10-Dec-2025 18:09:49 UTC  Closed output file "matt_reco2.root"
10-Dec-2025 18:09:49 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/38/31/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T001241Z_gen_001948_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1685.73       2202.75       2643.69       2236.24       352.416        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00142735    0.00946749     0.0239797    0.00873816    0.00614489       10     
reco:pandora:StandardPandora                            1680.06       2200.05       2641.12       2233.96       352.891        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000383105    0.0270342     0.0960879    0.00949369     0.0299548       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000368473    0.0105658     0.0517098    0.00580668     0.0147117       10     
reco:pandoracalo:Calorimetry                          0.000197463   0.00843271     0.0434492    0.000727671    0.0134752       10     
reco:pandorapid:Chi2ParticleID                        6.6563e-05    0.000388643   0.00237064    0.000173406   0.000666021      10     
reco:cvnmap:CVNMapper                                  0.0360958     0.0961321     0.227141      0.0738942     0.0542369       10     
reco:cvneva:CVNEvaluator                                1.21045       1.82763       4.74183       1.45601      0.992715        10     
reco:energyrecnumu:EnergyReco                          0.0134243     0.0676106     0.135071      0.0662658     0.0475436       10     
reco:energyrecnue:EnergyReco                          0.00196922    0.00624656     0.0139285    0.00262668    0.00487355       10     
reco:energyrecnc:EnergyReco                            0.0101325     0.0652871     0.184707      0.0508783     0.0583828       10     
[art]:TriggerResults:TriggerResultInserter            1.9807e-05    3.72418e-05   0.00010876    2.92875e-05   2.46293e-05      10     
end_path:out1:RootOutput                               6.652e-06    1.24336e-05   3.3207e-05    1.02005e-05   7.13517e-06      10     
end_path:out1:RootOutput(write)                        0.418795      0.581195      0.759002      0.614627      0.122582        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 11047.5 MB
  Peak resident set size usage (VmHWM): 3048.6 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
Begin processing the 8th record. run: 5027 subRun: 0 event: 19478 at 10-Dec-2025 18:11:36 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
This pfp has size:  52
Added hit list length: 52 | Max was set to: -3.40282e+38
True Electron Energy: 49.0538
Completed MCParticle Loop
True Neutrino Energy: 66.4414
 Size of all hits: 52
hit initial : 73.8722 | hit correction : 1.21738
hit initial : 256.334 | hit correction : 1.21745
hit initial : 160.145 | hit correction : 1.21681
hit initial : 223.227 | hit correction : 1.21753
hit initial : 409.888 | hit correction : 1.21681
hit initial : 138.518 | hit correction : 1.21312
hit initial : 177.919 | hit correction : 1.21251
hit initial : 45.8333 | hit correction : 1.21675
hit initial : 96.1205 | hit correction : 1.21593
hit initial : 32.3452 | hit correction : 1.21582
hit initial : 235.177 | hit correction : 1.21504
hit initial : 1137.07 | hit correction : 1.21446
hit initial : 10.1688 | hit correction : 1.21267
hit initial : 61.0604 | hit correction : 1.21204
hit initial : 42.7378 | hit correction : 1.21151
hit initial : 96.3395 | hit correction : 1.21138
 *** Total Charge : 3884.65
 Looking at view: 0 | Number of hits in view: 16
hit initial : 30.2498 | hit correction : 1.21137
hit initial : 27.4669 | hit correction : 1.21685
hit initial : 80.886 | hit correction : 1.21684
hit initial : 12.5169 | hit correction : 1.21744
hit initial : 113.878 | hit correction : 1.21682
hit initial : 15.396 | hit correction : 1.21745
hit initial : 129.491 | hit correction : 1.21679
hit initial : 102.912 | hit correction : 1.2175
hit initial : 126.197 | hit correction : 1.21684
hit initial : 68.3689 | hit correction : 1.21776
hit initial : 50.0175 | hit correction : 1.21788
hit initial : 12.4868 | hit correction : 1.21582
hit initial : 317.665 | hit correction : 1.21508
hit initial : 574.017 | hit correction : 1.21391
hit initial : 173.016 | hit correction : 1.21344
hit initial : 41.5772 | hit correction : 1.21279
hit initial : 292.575 | hit correction : 1.2119
hit initial : 168.274 | hit correction : 1.21153
 *** Total Charge : 2838.42
 Looking at view: 1 | Number of hits in view: 18
hit initial : 27.5514 | hit correction : 1.21122
hit initial : 89.563 | hit correction : 1.21746
hit initial : 102.718 | hit correction : 1.21741
hit initial : 170.486 | hit correction : 1.21684
hit initial : 369.522 | hit correction : 1.21689
hit initial : 165.106 | hit correction : 1.21748
hit initial : 338.977 | hit correction : 1.21677
hit initial : 33.6112 | hit correction : 1.21772
hit initial : 39.9274 | hit correction : 1.21789
hit initial : 57.5914 | hit correction : 1.21792
hit initial : 84.0504 | hit correction : 1.21641
hit initial : 369.609 | hit correction : 1.21592
hit initial : 330.285 | hit correction : 1.21516
hit initial : 295.382 | hit correction : 1.2145
hit initial : 268.341 | hit correction : 1.21379
hit initial : 261.456 | hit correction : 1.21306
hit initial : 294.291 | hit correction : 1.21224
hit initial : 275.092 | hit correction : 1.21147
 *** Total Charge : 4342.38
 Looking at view: 2 | Number of hits in view: 18
Begin processing the 9th record. run: 5027 subRun: 0 event: 19479 at 10-Dec-2025 18:11:45 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
True Electron Energy: 12.8852
Completed MCParticle Loop
True Neutrino Energy: 20.0058
 Size of all hits: 0
Begin processing the 10th record. run: 5027 subRun: 0 event: 19480 at 10-Dec-2025 18:11:53 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.567
True Electron Energy: 1.53855
Completed MCParticle Loop
True Neutrino Energy: 4.82229
 Size of all hits: 0
10-Dec-2025 18:12:00 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                           0.143       0.363788       1.34731      0.278458      0.339523        10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.00066251     0.0572239     0.211541     0.000905488    0.0751373       10     
end_path:ana:AnalyzeEvents         0.140589      0.306185       1.18919      0.177469      0.305231        10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 8737.33 MB
  Peak resident set size usage (VmHWM): 1538.78 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 16:28:45 UTC       justIN version: 01.06.00