justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 273236.188@dunegpschedd02.fnal.gov

Jobsub ID273236.188@dunegpschedd02.fnal.gov
Workflow ID11955
Stage ID1
User nameimawby@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes1048576000 (1000 MiB)
Wall seconds limit7200 (2 hours)
Submitted time2026-01-15 10:27:45
SiteUK_Bristol
EntryCMSHTPC_T2_UK_SGrid_Bristol_lcgce02
Last heartbeat2026-01-15 10:42:48
From worker nodeHostnamehd83.dice.priv
cpuinfoAMD EPYC 7551P 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes1048576000 (1000 MiB)
Wall seconds limit259200 (72 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-15 10:29:35
Input filesfardet-hd:nu_dune10kt_1x2x6_1413_171_20230826T183311Z_gen_g4_detsim_hitreco__20240229T181926Z_reco2.root
JobscriptExit code0
Real time12m (767s)
CPU time11m (672s = 87%)
Max RSS bytes1291063296 (1231 MiB)
Outputting started2026-01-15 10:42:23
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/imawby/metrics_nu_1/fnal/11955/1/001/Validation_nu_dune10kt_1x2x6_1413_171_20230826T183311Z_gen_g4_detsim_hitreco__20240229T181926Z_reco2.root
Finished2026-01-15 10:42:48
Saved logsjustin-logs:273236.188-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ad554f6b0d8af5cd51272760a301f448969f3ed8/localProducts_larsoft_v10_13_00_e26_prof
----------- this block should be empty ------------------
---------------------------------------------------------
Input PFN = root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/fardet-hd/71/e1/nu_dune10kt_1x2x6_1413_171_20230826T183311Z_gen_g4_detsim_hitreco__20240229T181926Z_reco2.root
lar exit code 0
=== Start last 100 lines of lar log file ===
Begin processing the 38th record. run: 1413 subRun: 1 event: 17138 at 15-Jan-2026 10:35:01 UTC
Begin processing the 39th record. run: 1413 subRun: 1 event: 17139 at 15-Jan-2026 10:35:05 UTC
Begin processing the 40th record. run: 1413 subRun: 1 event: 17140 at 15-Jan-2026 10:35:10 UTC
Begin processing the 41st record. run: 1413 subRun: 1 event: 17141 at 15-Jan-2026 10:35:18 UTC
Begin processing the 42nd record. run: 1413 subRun: 1 event: 17142 at 15-Jan-2026 10:35:24 UTC
Begin processing the 43rd record. run: 1413 subRun: 1 event: 17143 at 15-Jan-2026 10:35:28 UTC
Begin processing the 44th record. run: 1413 subRun: 1 event: 17144 at 15-Jan-2026 10:36:01 UTC
Begin processing the 45th record. run: 1413 subRun: 1 event: 17145 at 15-Jan-2026 10:36:07 UTC
Begin processing the 46th record. run: 1413 subRun: 1 event: 17146 at 15-Jan-2026 10:36:11 UTC
Begin processing the 47th record. run: 1413 subRun: 1 event: 17147 at 15-Jan-2026 10:36:16 UTC
Begin processing the 48th record. run: 1413 subRun: 1 event: 17148 at 15-Jan-2026 10:36:21 UTC
Begin processing the 49th record. run: 1413 subRun: 1 event: 17149 at 15-Jan-2026 10:36:26 UTC
Begin processing the 50th record. run: 1413 subRun: 1 event: 17150 at 15-Jan-2026 10:36:30 UTC
Begin processing the 51st record. run: 1413 subRun: 1 event: 17151 at 15-Jan-2026 10:36:35 UTC
Begin processing the 52nd record. run: 1413 subRun: 1 event: 17152 at 15-Jan-2026 10:36:40 UTC
Begin processing the 53rd record. run: 1413 subRun: 1 event: 17153 at 15-Jan-2026 10:36:45 UTC
Begin processing the 54th record. run: 1413 subRun: 1 event: 17154 at 15-Jan-2026 10:36:51 UTC
Begin processing the 55th record. run: 1413 subRun: 1 event: 17155 at 15-Jan-2026 10:36:57 UTC
Begin processing the 56th record. run: 1413 subRun: 1 event: 17156 at 15-Jan-2026 10:37:01 UTC
Begin processing the 57th record. run: 1413 subRun: 1 event: 17157 at 15-Jan-2026 10:37:14 UTC
Begin processing the 58th record. run: 1413 subRun: 1 event: 17158 at 15-Jan-2026 10:37:19 UTC
Begin processing the 59th record. run: 1413 subRun: 1 event: 17159 at 15-Jan-2026 10:37:25 UTC
Begin processing the 60th record. run: 1413 subRun: 1 event: 17160 at 15-Jan-2026 10:37:29 UTC
Begin processing the 61st record. run: 1413 subRun: 1 event: 17161 at 15-Jan-2026 10:37:34 UTC
Begin processing the 62nd record. run: 1413 subRun: 1 event: 17162 at 15-Jan-2026 10:37:38 UTC
Begin processing the 63rd record. run: 1413 subRun: 1 event: 17163 at 15-Jan-2026 10:37:43 UTC
Begin processing the 64th record. run: 1413 subRun: 1 event: 17164 at 15-Jan-2026 10:38:04 UTC
Begin processing the 65th record. run: 1413 subRun: 1 event: 17165 at 15-Jan-2026 10:38:10 UTC
Begin processing the 66th record. run: 1413 subRun: 1 event: 17166 at 15-Jan-2026 10:38:16 UTC
Begin processing the 67th record. run: 1413 subRun: 1 event: 17167 at 15-Jan-2026 10:38:20 UTC
PandoraContentApi::GetList(*this, m_inputHitListName, pCaloHitList) return STATUS_CODE_NOT_INITIALIZED
    in function: GetVolumeIdToHitListMap
    in file:     /exp/dune/app/users/imawby/dunesw_validation/srcs/larpandoracontent/larpandoracontent/LArControlFlow/MasterAlgorithm.cc line#: 271
this->GetVolumeIdToHitListMap(volumeIdToHitListMap) return STATUS_CODE_NOT_INITIALIZED
    in function: Run
    in file:     /exp/dune/app/users/imawby/dunesw_validation/srcs/larpandoracontent/larpandoracontent/LArControlFlow/MasterAlgorithm.cc line#: 165
iter->second->Run() throw STATUS_CODE_NOT_INITIALIZED
    in function: RunAlgorithm
    in file:     /scratch/workspace/build-lar-products/BUILDTYPE/prof/QUAL/e26/label1/swarm/label2/ALMA9/build/pandora/v04_16_02/src/pandora-v04-16-02/PandoraSDK-v04-00-02/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0002, LArDLMaster, STATUS_CODE_NOT_INITIALIZED
Begin processing the 68th record. run: 1413 subRun: 1 event: 17168 at 15-Jan-2026 10:38:20 UTC
PandoraContentApi::GetList(*this, listName, pList) return STATUS_CODE_NOT_INITIALIZED
    in function: GetList
    in file:     /exp/dune/app/users/imawby/dunesw_validation/srcs/larpandoracontent/larpandoradlcontent/LArTwoDReco/DLThreeDClusterSplittingAlgorithm.cc line#: 194
Begin processing the 69th record. run: 1413 subRun: 1 event: 17169 at 15-Jan-2026 10:38:24 UTC
Begin processing the 70th record. run: 1413 subRun: 1 event: 17170 at 15-Jan-2026 10:38:30 UTC
Begin processing the 71st record. run: 1413 subRun: 1 event: 17171 at 15-Jan-2026 10:38:34 UTC
PcaShowerParticleBuildingAlgorithm::OpeningAngle - principal eigenvalue less than or equal to 0.
PcaShowerParticleBuildingAlgorithm::OpeningAngle - principal eigenvalue less than or equal to 0.
Begin processing the 72nd record. run: 1413 subRun: 1 event: 17172 at 15-Jan-2026 10:38:45 UTC
Begin processing the 73rd record. run: 1413 subRun: 1 event: 17173 at 15-Jan-2026 10:39:34 UTC
Begin processing the 74th record. run: 1413 subRun: 1 event: 17174 at 15-Jan-2026 10:39:39 UTC
Begin processing the 75th record. run: 1413 subRun: 1 event: 17175 at 15-Jan-2026 10:39:43 UTC
Begin processing the 76th record. run: 1413 subRun: 1 event: 17176 at 15-Jan-2026 10:39:48 UTC
Begin processing the 77th record. run: 1413 subRun: 1 event: 17177 at 15-Jan-2026 10:40:04 UTC
Begin processing the 78th record. run: 1413 subRun: 1 event: 17178 at 15-Jan-2026 10:40:09 UTC
Begin processing the 79th record. run: 1413 subRun: 1 event: 17179 at 15-Jan-2026 10:40:13 UTC
Begin processing the 80th record. run: 1413 subRun: 1 event: 17180 at 15-Jan-2026 10:40:20 UTC
Begin processing the 81st record. run: 1413 subRun: 1 event: 17181 at 15-Jan-2026 10:40:26 UTC
Begin processing the 82nd record. run: 1413 subRun: 1 event: 17182 at 15-Jan-2026 10:40:31 UTC
Begin processing the 83rd record. run: 1413 subRun: 1 event: 17183 at 15-Jan-2026 10:40:37 UTC
Begin processing the 84th record. run: 1413 subRun: 1 event: 17184 at 15-Jan-2026 10:40:41 UTC
Begin processing the 85th record. run: 1413 subRun: 1 event: 17185 at 15-Jan-2026 10:40:46 UTC
Begin processing the 86th record. run: 1413 subRun: 1 event: 17186 at 15-Jan-2026 10:40:51 UTC
Begin processing the 87th record. run: 1413 subRun: 1 event: 17187 at 15-Jan-2026 10:40:56 UTC
Begin processing the 88th record. run: 1413 subRun: 1 event: 17188 at 15-Jan-2026 10:41:01 UTC
Begin processing the 89th record. run: 1413 subRun: 1 event: 17189 at 15-Jan-2026 10:41:07 UTC
Begin processing the 90th record. run: 1413 subRun: 1 event: 17190 at 15-Jan-2026 10:41:12 UTC
Begin processing the 91st record. run: 1413 subRun: 1 event: 17191 at 15-Jan-2026 10:41:16 UTC
Begin processing the 92nd record. run: 1413 subRun: 1 event: 17192 at 15-Jan-2026 10:41:21 UTC
Begin processing the 93rd record. run: 1413 subRun: 1 event: 17193 at 15-Jan-2026 10:41:27 UTC
Begin processing the 94th record. run: 1413 subRun: 1 event: 17194 at 15-Jan-2026 10:41:32 UTC
Begin processing the 95th record. run: 1413 subRun: 1 event: 17195 at 15-Jan-2026 10:41:37 UTC
Begin processing the 96th record. run: 1413 subRun: 1 event: 17196 at 15-Jan-2026 10:41:42 UTC
Begin processing the 97th record. run: 1413 subRun: 1 event: 17197 at 15-Jan-2026 10:41:58 UTC
Begin processing the 98th record. run: 1413 subRun: 1 event: 17198 at 15-Jan-2026 10:42:03 UTC
Begin processing the 99th record. run: 1413 subRun: 1 event: 17199 at 15-Jan-2026 10:42:10 UTC
Begin processing the 100th record. run: 1413 subRun: 1 event: 17200 at 15-Jan-2026 10:42:15 UTC
15-Jan-2026 10:42:22 UTC  Closed output file "nu_dune10kt_1x2x6_1413_171_20230826T183311Z_gen_g4_detsim_hitreco__20240229T181926Z_reco2_reco2.root"
15-Jan-2026 10:42:22 UTC  Closed input file "root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/fardet-hd/71/e1/nu_dune10kt_1x2x6_1413_171_20230826T183311Z_gen_g4_detsim_hitreco__20240229T181926Z_reco2.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                     0.0649304      6.27621       47.4037       4.65316       5.74881        100    
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                         0.0119161     0.0260685     0.135878      0.0177308     0.0252104       100    
reco:pandora:StandardPandora                   0.0453321      6.22784       47.2379       4.58204       5.72985        100    
[art]:TriggerResults:TriggerResultInserter     1.558e-05    3.60235e-05   0.000163769   2.97165e-05   2.12784e-05      100    
end_path:out1:RootOutput                       3.366e-06    5.0783e-06    2.3915e-05     4.304e-06    2.39189e-06      100    
end_path:out1:RootOutput(write)               0.00132126     0.0220081     0.138384      0.0148307     0.0224055       100    
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 2276.89 MB
  Peak resident set size usage (VmHWM): 1291.06 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
justIN time: 2026-02-04 06:07:36 UTC       justIN version: 01.06.00