justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 258497.6@dunegpschedd02.fnal.gov

Jobsub ID258497.6@dunegpschedd02.fnal.gov
Workflow ID11030
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-06 10:33:54
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-12-06 19:06:25
From worker nodeHostnamedunegli-7988749-0-fnpc18001.fnal.gov
cpuinfoAMD EPYC 7551 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-06 10:34:50
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T215313Z_gen_004528_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time8h (30679s)
CPU time8h (30270s = 98%)
Max RSS bytes3059937280 (2918 MiB)
Outputting started2025-12-06 19:06:09
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11030/1/001/analysisLowE_20251206T103451Z.root
Finished2025-12-06 19:06:25
Saved logsjustin-logs:258497.6-dunegpschedd02.fnal.gov.logs.tgz
List job events     (HTCondor job logs unavailable)

Jobscript log (last 10,000 characters)

-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0004, LArDLVertexing, STATUS_CODE_NOT_INITIALIZED
PandoraContentApi::GetList(*this, m_inputVertexListName, pVertexList) throw STATUS_CODE_NOT_INITIALIZED
    in function: GetHitRegion
    in file:     /exp/dune/app/users/osbiston/18_10_24_Official_LowE_Build/srcs/larpandoracontent/larpandoradlcontent/LArVertex/DlVertexingAlgorithm.cc line#: 624
Failure in algorithm Alg0005, LArDLVertexing, STATUS_CODE_NOT_INITIALIZED
Boundary wire vector sizes: 6851, 6794, 6800
minwire 0: 36
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.145063, 
Output 1: 0.0843153, 0.0578894, 0.129925, 0.72787, 
Output 2: 0.0416943, 0.245856, 0.677563, 0.0348863, 
Output 3: 0.969514, 0.0183616, 0.00407758, 0.00804659, 
Output 4: 0.504238, 0.422399, 0.0667677, 0.00659541, 
Output 5: 0.51534, 0.341854, 0.118678, 0.0241279, 
Output 6: 0.0177211, 0.0454796, 0.0624141, 0.874385, 

06-Dec-2025 19:04:51 UTC  Closed output file "matt_reco2.root"
06-Dec-2025 19:04:51 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/3a/08/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T215313Z_gen_004528_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2484.98       3040.59       3355.07       3094.54       263.191        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00288821    0.00996434     0.0244567     0.008576     0.00612148       10     
reco:pandora:StandardPandora                            2480.88       3036.26       3348.19       3090.42       262.895        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.00203377     0.0308322     0.174823     0.00587583     0.0509676       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000461721    0.0129854     0.0535604     0.0068473     0.0155753       10     
reco:pandoracalo:Calorimetry                          0.000379607    0.0111387     0.0475256    0.00261845     0.0171677       10     
reco:pandorapid:Chi2ParticleID                        0.000114055   0.000578213   0.00215247    0.00020405    0.000782087      10     
reco:cvnmap:CVNMapper                                  0.0420283     0.061815      0.0901109     0.062483      0.0151794       10     
reco:cvneva:CVNEvaluator                                2.01022       3.31312       5.8678        3.1228       0.968657        10     
reco:energyrecnumu:EnergyReco                          0.0105271     0.110624      0.306204      0.118642       0.0843         10     
reco:energyrecnue:EnergyReco                          0.00226697    0.00882822     0.0165911    0.00906842    0.00601474       10     
reco:energyrecnc:EnergyReco                            0.0137417     0.197774       1.34076      0.0151001       0.399         10     
[art]:TriggerResults:TriggerResultInserter            2.5488e-05    4.82611e-05   0.000177946   3.32425e-05   4.34264e-05      10     
end_path:out1:RootOutput                               9.578e-06    1.56426e-05   5.1467e-05    1.1777e-05    1.19962e-05      10     
end_path:out1:RootOutput(write)                        0.505363      0.571606      0.680002      0.561157      0.0492793       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21737.9 MB
  Peak resident set size usage (VmHWM): 3059.94 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 82.5991 | hit correction : 1.41612
hit initial : 450.51 | hit correction : 1.41618
 *** Total Charge : 1739.75
 Looking at view: 1 | Number of hits in view: 5
hit initial : 48.0627 | hit correction : 1.41708
hit initial : 174.002 | hit correction : 1.41679
hit initial : 344.495 | hit correction : 1.4158
hit initial : 378.574 | hit correction : 1.41582
hit initial : 66.8828 | hit correction : 1.41598
hit initial : 278.438 | hit correction : 1.41625
 *** Total Charge : 1827.41
 Looking at view: 2 | Number of hits in view: 6
Begin processing the 8th record. run: 5026 subRun: 0 event: 45278 at 06-Dec-2025 19:05:54 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 943.278
True Neutron Energy: 941.77
True Neutron Energy: 940.179
True Electron Energy: 39.232
Completed MCParticle Loop
True Neutrino Energy: 56.5378
 Size of all hits: 0
Begin processing the 9th record. run: 5026 subRun: 0 event: 45279 at 06-Dec-2025 19:05:58 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
This pfp has size:  39
Added hit list length: 39 | Max was set to: -3.40282e+38
True Electron Energy: 48.5016
Completed MCParticle Loop
True Neutrino Energy: 65.844
 Size of all hits: 39
hit initial : 26.149 | hit correction : 1.05816
hit initial : 27.6361 | hit correction : 1.05813
hit initial : 75.1786 | hit correction : 1.05815
hit initial : 157.931 | hit correction : 1.05972
hit initial : 82.4507 | hit correction : 1.06086
hit initial : 350.134 | hit correction : 1.05962
hit initial : 401.781 | hit correction : 1.06094
hit initial : 152.328 | hit correction : 1.05956
hit initial : 435.088 | hit correction : 1.06098
hit initial : 297.091 | hit correction : 1.06083
hit initial : 254.374 | hit correction : 1.06055
hit initial : 263.879 | hit correction : 1.06027
hit initial : 248.11 | hit correction : 1.06005
hit initial : 257.077 | hit correction : 1.0598
hit initial : 264.276 | hit correction : 1.0596
hit initial : 257.57 | hit correction : 1.05942
hit initial : 238.858 | hit correction : 1.05926
hit initial : 238.747 | hit correction : 1.0591
hit initial : 307.451 | hit correction : 1.05886
hit initial : 410.158 | hit correction : 1.05864
hit initial : 325.885 | hit correction : 1.05853
hit initial : 253.718 | hit correction : 1.05833
hit initial : 209.251 | hit correction : 1.05819
 *** Total Charge : 5865.2
 Looking at view: 0 | Number of hits in view: 23
hit initial : 114.6 | hit correction : 1.06069
hit initial : 322.791 | hit correction : 1.06084
hit initial : 412.408 | hit correction : 1.06097
hit initial : 465.769 | hit correction : 1.06086
hit initial : 240.16 | hit correction : 1.06011
hit initial : 99.5169 | hit correction : 1.06083
hit initial : 24.6387 | hit correction : 1.06016
hit initial : 369.093 | hit correction : 1.0598
hit initial : 371.501 | hit correction : 1.05957
hit initial : 141.531 | hit correction : 1.05949
hit initial : 50.3732 | hit correction : 1.05992
hit initial : 145.307 | hit correction : 1.05953
hit initial : 97.7046 | hit correction : 1.05939
hit initial : 445.758 | hit correction : 1.0586
hit initial : 338.993 | hit correction : 1.05844
hit initial : 126.832 | hit correction : 1.05841
 *** Total Charge : 3992.47
 Looking at view: 1 | Number of hits in view: 16
Begin processing the 10th record. run: 5026 subRun: 0 event: 45280 at 06-Dec-2025 19:06:03 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.569
True Electron Energy: 3.55175
Completed MCParticle Loop
True Neutrino Energy: 8.92961
 Size of all hits: 0
06-Dec-2025 19:06:08 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.129899      0.156404      0.216097      0.148764      0.0251573       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000484725   0.000725975   0.00139384    0.000684445   0.000235249      10     
end_path:ana:AnalyzeEvents         0.129209      0.155408      0.215283      0.147848      0.025034        10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19472.9 MB
  Peak resident set size usage (VmHWM): 1528.2 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 19:08:59 UTC       justIN version: 01.06.00