justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 249761.18@dunegpschedd02.fnal.gov

Jobsub ID249761.18@dunegpschedd02.fnal.gov
Workflow ID10523
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-11-22 19:23:06
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-11-22 21:50:11
From worker nodeHostnamedunegli-7850002-0-fnpc22001.fnal.gov
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-22 19:44:42
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T194331Z_gen_000990_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time1h (6448s)
CPU time1h (6381s = 98%)
Max RSS bytes3059269632 (2917 MiB)
Outputting started2025-11-22 21:32:12
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/10523/1/001/analysisLowE_20251122T194443Z.root
Finished2025-11-22 21:50:11
Saved logsjustin-logs:249761.18-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ArDLVertexing, STATUS_CODE_NOT_INITIALIZED
PandoraContentApi::GetList(*this, m_inputVertexListName, pVertexList) throw STATUS_CODE_NOT_INITIALIZED
    in function: GetHitRegion
    in file:     /exp/dune/app/users/osbiston/18_10_24_Official_LowE_Build/srcs/larpandoracontent/larpandoradlcontent/LArVertex/DlVertexingAlgorithm.cc line#: 624
Failure in algorithm Alg0005, LArDLVertexing, STATUS_CODE_NOT_INITIALIZED
Boundary wire vector sizes: 6712, 6676, 6638
minwire 0: 10
minwire 1: 1
minwire 2: 3
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 1818, 2317
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.160374, 
Output 1: 0.0860275, 0.0484675, 0.111144, 0.754361, 
Output 2: 0.13221, 0.243946, 0.571912, 0.0519327, 
Output 3: 0.976871, 0.00737741, 0.00150745, 0.0142438, 
Output 4: 0.395591, 0.557903, 0.0418738, 0.00463254, 
Output 5: 0.584251, 0.361687, 0.0424992, 0.0115622, 
Output 6: 0.0172115, 0.0435197, 0.0621455, 0.877123, 

22-Nov-2025 21:31:44 UTC  Closed output file "matt_reco2.root"
22-Nov-2025 21:31:44 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/ea/93/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T194331Z_gen_000990_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              622.258       636.049       663.321       630.42        13.0878        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00129005    0.00688072     0.0237325    0.00385681    0.00712419       10     
reco:pandora:StandardPandora                            620.087       634.937       662.316       629.441       13.2173        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.00108081    0.00784976     0.0568215    0.00208786     0.0163748       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.00178261    0.00411027     0.0111343    0.00328046    0.00282662       10     
reco:pandoracalo:Calorimetry                          0.000121831   0.00101906    0.00353543    0.000268465   0.00127531       10     
reco:pandorapid:Chi2ParticleID                        3.1801e-05    0.000165989   0.000853752   6.53945e-05   0.000238937      10     
reco:cvnmap:CVNMapper                                  0.0152242     0.0195947     0.0280438     0.0190668    0.00336014       10     
reco:cvneva:CVNEvaluator                               0.678201      0.789433       1.73204      0.684743      0.314229        10     
reco:energyrecnumu:EnergyReco                         0.00328677     0.0403089     0.0721118     0.0569279     0.0271156       10     
reco:energyrecnue:EnergyReco                          0.000755756   0.00239459    0.00474613    0.00098187    0.00185204       10     
reco:energyrecnc:EnergyReco                           0.00454015     0.0345814     0.112397     0.00467207     0.039182        10     
[art]:TriggerResults:TriggerResultInserter             8.056e-06    1.31311e-05   4.6138e-05    9.5185e-06    1.10386e-05      10     
end_path:out1:RootOutput                               2.184e-06    6.6587e-06    1.6651e-05     5.876e-06    3.55303e-06      10     
end_path:out1:RootOutput(write)                        0.199953      0.205318      0.213845      0.204457     0.00443963       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21777.6 MB
  Peak resident set size usage (VmHWM): 3059.27 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 115.194 | hit correction : 1.22239
hit initial : 81.3332 | hit correction : 1.22238
hit initial : 61.1959 | hit correction : 1.2216
hit initial : 273.964 | hit correction : 1.22156
hit initial : 419.574 | hit correction : 1.2213
 *** Total Charge : 1955.25
 Looking at view: 0 | Number of hits in view: 11
hit initial : 49.0868 | hit correction : 1.22244
hit initial : 50.5628 | hit correction : 1.2224
hit initial : 89.2146 | hit correction : 1.22064
hit initial : 476.372 | hit correction : 1.22141
hit initial : 131.46 | hit correction : 1.22053
hit initial : 118.143 | hit correction : 1.22144
hit initial : 134.735 | hit correction : 1.21956
hit initial : 195.125 | hit correction : 1.21956
hit initial : 57.9543 | hit correction : 1.21957
hit initial : 41.9297 | hit correction : 1.22156
hit initial : 68.8632 | hit correction : 1.22152
hit initial : 81.4131 | hit correction : 1.22066
 *** Total Charge : 1824.99
 Looking at view: 1 | Number of hits in view: 12
hit initial : 55.9507 | hit correction : 1.22034
hit initial : 229.087 | hit correction : 1.22052
hit initial : 180.712 | hit correction : 1.22083
hit initial : 158.534 | hit correction : 1.22238
hit initial : 413.046 | hit correction : 1.22141
hit initial : 306.729 | hit correction : 1.2217
hit initial : 172.061 | hit correction : 1.2195
hit initial : 229.272 | hit correction : 1.21959
 *** Total Charge : 2130.97
 Looking at view: 2 | Number of hits in view: 8
Begin processing the 7th record. run: 3799 subRun: 0 event: 9897 at 22-Nov-2025 21:32:04 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Electron Energy: 44.8933
Completed MCParticle Loop
True Neutrino Energy: 48.2065
 Size of all hits: 0
Begin processing the 8th record. run: 3799 subRun: 0 event: 9898 at 22-Nov-2025 21:32:06 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Electron Energy: 1.94785
Completed MCParticle Loop
True Neutrino Energy: 5.2315
 Size of all hits: 0
Begin processing the 9th record. run: 3799 subRun: 0 event: 9899 at 22-Nov-2025 21:32:08 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
This pfp has size:  14
Added hit list length: 14 | Max was set to: -3.40282e+38
True Electron Energy: 30.7601
Completed MCParticle Loop
True Neutrino Energy: 53.7817
 Size of all hits: 14
hit initial : 73.7087 | hit correction : 1.08912
hit initial : 133.872 | hit correction : 1.08988
hit initial : 311.15 | hit correction : 1.08938
hit initial : 85.3987 | hit correction : 1.08917
 *** Total Charge : 658.158
 Looking at view: 0 | Number of hits in view: 4
hit initial : 49.4263 | hit correction : 1.08994
hit initial : 235.219 | hit correction : 1.08974
hit initial : 238.828 | hit correction : 1.08948
hit initial : 57.0154 | hit correction : 1.08898
hit initial : 49.8171 | hit correction : 1.08953
hit initial : 54.811 | hit correction : 1.08897
 *** Total Charge : 746.45
 Looking at view: 1 | Number of hits in view: 6
hit initial : 111.215 | hit correction : 1.08995
hit initial : 20.8219 | hit correction : 1.08891
hit initial : 403.862 | hit correction : 1.0895
hit initial : 111.834 | hit correction : 1.089
 *** Total Charge : 705.688
 Looking at view: 2 | Number of hits in view: 4
Begin processing the 10th record. run: 3799 subRun: 0 event: 9900 at 22-Nov-2025 21:32:09 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.569
True Electron Energy: 7.03872
Completed MCParticle Loop
True Neutrino Energy: 10.3224
 Size of all hits: 0
22-Nov-2025 21:32:11 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0452187     0.0527711     0.0683426     0.0508773    0.00799018       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000270795   0.00038029    0.000728153   0.000331645   0.000127675      10     
end_path:ana:AnalyzeEvents         0.0446863     0.0522468     0.0672304     0.050457      0.0078833       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19470.8 MB
  Peak resident set size usage (VmHWM): 1532.65 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-08 21:38:16 UTC       justIN version: 01.06.00