justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 249059.1@dunegpschedd02.fnal.gov

Jobsub ID249059.1@dunegpschedd02.fnal.gov
Workflow ID10411
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-11-21 00:19:44
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-11-21 11:04:24
From worker nodeHostnamedunegli-7833860-0-fnpc17005.fnal.gov
cpuinfoIntel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-21 06:59:17
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T193308Z_gen_000229_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time4h (14696s)
CPU time4h (14438s = 98%)
Max RSS bytes3052974080 (2911 MiB)
Outputting started2025-11-21 11:04:13
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/10411/1/001/analysisLowE_20251121T065917Z.root
Finished2025-11-21 11:04:24
Saved logsjustin-logs:249059.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

in file:     /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0005, LArDLVertexing, STATUS_CODE_NOT_INITIALIZED
Boundary wire vector sizes: 6626, 6549, 6552
minwire 0: 46
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 1201, 1700
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.127624, 
Output 1: 0.0682652, 0.0515669, 0.0935586, 0.786609, 
Output 2: 0.0629775, 0.264293, 0.635329, 0.0374001, 
Output 3: 0.961862, 0.0179515, 0.00494057, 0.0152456, 
Output 4: 0.645977, 0.295696, 0.0498922, 0.00843486, 
Output 5: 0.410223, 0.478183, 0.097358, 0.0142364, 
Output 6: 0.0206139, 0.0479168, 0.0652659, 0.866203, 

21-Nov-2025 11:03:11 UTC  Closed output file "matt_reco2.root"
21-Nov-2025 11:03:11 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/49/3d/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T193308Z_gen_000229_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1373.66       1433.87       1476.04       1435.68       26.4764        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00130855    0.00851414     0.0205794    0.00694611    0.00629437       10     
reco:pandora:StandardPandora                            1371.5        1431.61       1473.93       1433.59       26.5359        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.00141611     0.021968      0.0632816     0.0197808     0.0181089       10     
reco:pandoraShower:LArPandoraModularShowerCreation     0.0015654    0.00568837     0.0196992    0.00443671    0.00482723       10     
reco:pandoracalo:Calorimetry                          0.000212492   0.00422484     0.0283086    0.000473731   0.00828651       10     
reco:pandorapid:Chi2ParticleID                        6.2212e-05    0.000289948   0.00124561    0.000110327   0.000352224      10     
reco:cvnmap:CVNMapper                                  0.0385872     0.0462044     0.0605428     0.0431315    0.00767182       10     
reco:cvneva:CVNEvaluator                                1.45055       1.75198       2.91238       1.64216      0.391707        10     
reco:energyrecnumu:EnergyReco                          0.0097878     0.0469821     0.0919369     0.0457252     0.0344129       10     
reco:energyrecnue:EnergyReco                          0.00103025    0.00378316    0.00778839    0.00157123    0.00310926       10     
reco:energyrecnc:EnergyReco                           0.00736349     0.0446803     0.111661      0.0394978     0.0384573       10     
[art]:TriggerResults:TriggerResultInserter            1.5788e-05    2.37746e-05   6.6902e-05    1.8125e-05    1.47291e-05      10     
end_path:out1:RootOutput                               6.151e-06    9.8144e-06    2.6876e-05     7.465e-06    5.89216e-06      10     
end_path:out1:RootOutput(write)                        0.304446      0.327295      0.394521      0.311879      0.0287138       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 12086.1 MB
  Peak resident set size usage (VmHWM): 3052.97 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 75.053 | hit correction : 1.09172
hit initial : 64.7613 | hit correction : 1.09169
hit initial : 214.576 | hit correction : 1.09178
hit initial : 43.9756 | hit correction : 1.09197
 *** Total Charge : 7176.55
 Looking at view: 0 | Number of hits in view: 38
hit initial : 92.3467 | hit correction : 1.09171
hit initial : 71.17 | hit correction : 1.0917
hit initial : 57.2998 | hit correction : 1.07583
hit initial : 155.722 | hit correction : 1.07586
hit initial : 34.5258 | hit correction : 1.09106
hit initial : 98.7189 | hit correction : 1.07674
hit initial : 250.63 | hit correction : 1.07686
hit initial : 616.306 | hit correction : 1.07695
hit initial : 160.583 | hit correction : 1.07709
hit initial : 53.0194 | hit correction : 1.0855
hit initial : 220.889 | hit correction : 1.08555
hit initial : 156.857 | hit correction : 1.08558
hit initial : 51.2326 | hit correction : 1.08608
hit initial : 54.2148 | hit correction : 1.08698
hit initial : 43.2112 | hit correction : 1.08716
hit initial : 24.2205 | hit correction : 1.08759
hit initial : 63.064 | hit correction : 1.08758
hit initial : 258.365 | hit correction : 1.08765
hit initial : 32.4787 | hit correction : 1.08782
hit initial : 33.9241 | hit correction : 1.08832
hit initial : 52.4882 | hit correction : 1.08924
hit initial : 88.6712 | hit correction : 1.08933
hit initial : 61.6308 | hit correction : 1.08961
hit initial : 371.471 | hit correction : 1.08995
hit initial : 73.5168 | hit correction : 1.09013
hit initial : 35.3091 | hit correction : 1.09015
hit initial : 9.64556 | hit correction : 1.09066
hit initial : 36.5418 | hit correction : 1.09067
hit initial : 140.688 | hit correction : 1.09147
hit initial : 81.2158 | hit correction : 1.09153
hit initial : 73.9326 | hit correction : 1.09176
hit initial : 83.0395 | hit correction : 1.09201
 *** Total Charge : 3944.02
 Looking at view: 1 | Number of hits in view: 32
hit initial : 37.0337 | hit correction : 1.0862
hit initial : 23.8534 | hit correction : 1.08624
hit initial : 68.9794 | hit correction : 1.09173
hit initial : 132.849 | hit correction : 1.09171
hit initial : 45.5163 | hit correction : 1.06803
hit initial : 186.482 | hit correction : 1.06805
hit initial : 32.4528 | hit correction : 1.07532
hit initial : 167.561 | hit correction : 1.07587
hit initial : 1023.98 | hit correction : 1.07661
hit initial : 22.7127 | hit correction : 1.07688
hit initial : 285.867 | hit correction : 1.07685
hit initial : 231.641 | hit correction : 1.07721
hit initial : 96.676 | hit correction : 1.07746
hit initial : 265.682 | hit correction : 1.08546
hit initial : 34.9825 | hit correction : 1.08542
hit initial : 39.3992 | hit correction : 1.08566
hit initial : 366.418 | hit correction : 1.08595
hit initial : 430.926 | hit correction : 1.08683
hit initial : 361.184 | hit correction : 1.08749
hit initial : 319.607 | hit correction : 1.08781
hit initial : 181.045 | hit correction : 1.0882
hit initial : 170.814 | hit correction : 1.08848
hit initial : 34.2766 | hit correction : 1.08848
hit initial : 177.397 | hit correction : 1.0888
hit initial : 189.378 | hit correction : 1.08905
hit initial : 144.662 | hit correction : 1.08925
hit initial : 204.049 | hit correction : 1.08944
hit initial : 213.285 | hit correction : 1.08961
hit initial : 361.191 | hit correction : 1.09005
hit initial : 391.733 | hit correction : 1.09055
hit initial : 396.632 | hit correction : 1.09138
hit initial : 218.437 | hit correction : 1.09197
 *** Total Charge : 7437.92
 Looking at view: 2 | Number of hits in view: 32
Begin processing the 10th record. run: 3799 subRun: 0 event: 2290 at 21-Nov-2025 11:04:10 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.574
True Electron Energy: 22.775
Completed MCParticle Loop
True Neutrino Energy: 26.5309
 Size of all hits: 0
21-Nov-2025 11:04:13 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0674382     0.0911348     0.152665      0.0838382     0.0262455       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000356157   0.000515363   0.000852922   0.000436519   0.000160368      10     
end_path:ana:AnalyzeEvents         0.0669172     0.0904578     0.151758      0.083315      0.0261841       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 9811.86 MB
  Peak resident set size usage (VmHWM): 1537.62 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-09 03:26:56 UTC       justIN version: 01.06.00