justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 288571.6@dunegpschedd02.fnal.gov

Jobsub ID288571.6@dunegpschedd02.fnal.gov
Workflow ID12095
Stage ID1
User nameimawby@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes1048576000 (1000 MiB)
Wall seconds limit7200 (2 hours)
Submitted time2026-01-20 14:50:46
SiteUS_BNL
EntryDUNE_US_BNL_spoolce01
Last heartbeat2026-01-20 15:18:49
From worker nodeHostnamespool0812.sdcc.bnl.gov
cpuinfoIntel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes1048576000 (1000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-20 14:51:41
Input filesfardet-hd:nue_dune10kt_1x2x6_1426_672_20230827T193432Z_gen_g4_detsim_hitreco__20240221T063747Z_reco2.root
JobscriptExit code0
Real time26m (1603s)
CPU time21m (1266s = 78%)
Max RSS bytes1404297216 (1339 MiB)
Outputting started2026-01-20 15:18:25
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/imawby/metrics_nue_1/fnal/12095/1/001/Validation_nue_dune10kt_1x2x6_1426_672_20230827T193432Z_gen_g4_detsim_hitreco__20240221T063747Z_reco2.root
Finished2026-01-20 15:18:49
Saved logsjustin-logs:288571.6-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

------------------
---------------------------------------------------------
Input PFN = root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/07/c3/nue_dune10kt_1x2x6_1426_672_20230827T193432Z_gen_g4_detsim_hitreco__20240221T063747Z_reco2.root
lar exit code 0
=== Start last 100 lines of lar log file ===
    in function: RunAlgorithm
    in file:     /scratch/workspace/build-lar-products/BUILDTYPE/prof/QUAL/e26/label1/swarm/label2/ALMA9/build/pandora/v04_16_02/src/pandora-v04-16-02/PandoraSDK-v04-00-02/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0004, LArDLSecondaryVertexing, STATUS_CODE_NOT_INITIALIZED
PandoraContentApi::GetList(*this, listName, pList) return STATUS_CODE_NOT_INITIALIZED
    in function: GetList
    in file:     /exp/dune/app/users/imawby/dunesw_validation/srcs/larpandoracontent/larpandoradlcontent/LArTwoDReco/DLThreeDClusterSplittingAlgorithm.cc line#: 194
Begin processing the 32nd record. run: 1426 subRun: 1 event: 67232 at 20-Jan-2026 10:09:03 EST
Begin processing the 33rd record. run: 1426 subRun: 1 event: 67233 at 20-Jan-2026 10:09:08 EST
Begin processing the 34th record. run: 1426 subRun: 1 event: 67234 at 20-Jan-2026 10:09:12 EST
Begin processing the 35th record. run: 1426 subRun: 1 event: 67235 at 20-Jan-2026 10:09:16 EST
Begin processing the 36th record. run: 1426 subRun: 1 event: 67236 at 20-Jan-2026 10:09:20 EST
Begin processing the 37th record. run: 1426 subRun: 1 event: 67237 at 20-Jan-2026 10:09:25 EST
Begin processing the 38th record. run: 1426 subRun: 1 event: 67238 at 20-Jan-2026 10:09:34 EST
Begin processing the 39th record. run: 1426 subRun: 1 event: 67239 at 20-Jan-2026 10:09:38 EST
Begin processing the 40th record. run: 1426 subRun: 1 event: 67240 at 20-Jan-2026 10:09:43 EST
Begin processing the 41st record. run: 1426 subRun: 1 event: 67241 at 20-Jan-2026 10:10:16 EST
Begin processing the 42nd record. run: 1426 subRun: 1 event: 67242 at 20-Jan-2026 10:10:21 EST
Begin processing the 43rd record. run: 1426 subRun: 1 event: 67243 at 20-Jan-2026 10:10:26 EST
Begin processing the 44th record. run: 1426 subRun: 1 event: 67244 at 20-Jan-2026 10:10:44 EST
Begin processing the 45th record. run: 1426 subRun: 1 event: 67245 at 20-Jan-2026 10:10:49 EST
Begin processing the 46th record. run: 1426 subRun: 1 event: 67246 at 20-Jan-2026 10:11:18 EST
Begin processing the 47th record. run: 1426 subRun: 1 event: 67247 at 20-Jan-2026 10:11:23 EST
Begin processing the 48th record. run: 1426 subRun: 1 event: 67248 at 20-Jan-2026 10:11:27 EST
Begin processing the 49th record. run: 1426 subRun: 1 event: 67249 at 20-Jan-2026 10:11:35 EST
Begin processing the 50th record. run: 1426 subRun: 1 event: 67250 at 20-Jan-2026 10:11:48 EST
Begin processing the 51st record. run: 1426 subRun: 1 event: 67251 at 20-Jan-2026 10:12:14 EST
Begin processing the 52nd record. run: 1426 subRun: 1 event: 67252 at 20-Jan-2026 10:12:19 EST
Begin processing the 53rd record. run: 1426 subRun: 1 event: 67253 at 20-Jan-2026 10:12:26 EST
Begin processing the 54th record. run: 1426 subRun: 1 event: 67254 at 20-Jan-2026 10:12:30 EST
Begin processing the 55th record. run: 1426 subRun: 1 event: 67255 at 20-Jan-2026 10:12:35 EST
Begin processing the 56th record. run: 1426 subRun: 1 event: 67256 at 20-Jan-2026 10:12:51 EST
Begin processing the 57th record. run: 1426 subRun: 1 event: 67257 at 20-Jan-2026 10:13:01 EST
Begin processing the 58th record. run: 1426 subRun: 1 event: 67258 at 20-Jan-2026 10:13:28 EST
Begin processing the 59th record. run: 1426 subRun: 1 event: 67259 at 20-Jan-2026 10:13:32 EST
Begin processing the 60th record. run: 1426 subRun: 1 event: 67260 at 20-Jan-2026 10:13:36 EST
Begin processing the 61st record. run: 1426 subRun: 1 event: 67261 at 20-Jan-2026 10:13:42 EST
Begin processing the 62nd record. run: 1426 subRun: 1 event: 67262 at 20-Jan-2026 10:13:50 EST
Begin processing the 63rd record. run: 1426 subRun: 1 event: 67263 at 20-Jan-2026 10:13:55 EST
Begin processing the 64th record. run: 1426 subRun: 1 event: 67264 at 20-Jan-2026 10:14:05 EST
Begin processing the 65th record. run: 1426 subRun: 1 event: 67265 at 20-Jan-2026 10:14:09 EST
Begin processing the 66th record. run: 1426 subRun: 1 event: 67266 at 20-Jan-2026 10:14:29 EST
Begin processing the 67th record. run: 1426 subRun: 1 event: 67267 at 20-Jan-2026 10:14:33 EST
Begin processing the 68th record. run: 1426 subRun: 1 event: 67268 at 20-Jan-2026 10:14:55 EST
Begin processing the 69th record. run: 1426 subRun: 1 event: 67269 at 20-Jan-2026 10:15:01 EST
Begin processing the 70th record. run: 1426 subRun: 1 event: 67270 at 20-Jan-2026 10:15:05 EST
Begin processing the 71st record. run: 1426 subRun: 1 event: 67271 at 20-Jan-2026 10:15:09 EST
Begin processing the 72nd record. run: 1426 subRun: 1 event: 67272 at 20-Jan-2026 10:15:28 EST
Begin processing the 73rd record. run: 1426 subRun: 1 event: 67273 at 20-Jan-2026 10:15:32 EST
Begin processing the 74th record. run: 1426 subRun: 1 event: 67274 at 20-Jan-2026 10:15:38 EST
Begin processing the 75th record. run: 1426 subRun: 1 event: 67275 at 20-Jan-2026 10:15:43 EST
Begin processing the 76th record. run: 1426 subRun: 1 event: 67276 at 20-Jan-2026 10:15:51 EST
Begin processing the 77th record. run: 1426 subRun: 1 event: 67277 at 20-Jan-2026 10:15:55 EST
Begin processing the 78th record. run: 1426 subRun: 1 event: 67278 at 20-Jan-2026 10:15:59 EST
PandoraContentApi::GetList(*this, listName, pList) return STATUS_CODE_NOT_INITIALIZED
    in function: GetList
    in file:     /exp/dune/app/users/imawby/dunesw_validation/srcs/larpandoracontent/larpandoradlcontent/LArTwoDReco/DLThreeDClusterSplittingAlgorithm.cc line#: 194
Begin processing the 79th record. run: 1426 subRun: 1 event: 67279 at 20-Jan-2026 10:16:03 EST
Begin processing the 80th record. run: 1426 subRun: 1 event: 67280 at 20-Jan-2026 10:16:07 EST
Begin processing the 81st record. run: 1426 subRun: 1 event: 67281 at 20-Jan-2026 10:16:18 EST
Begin processing the 82nd record. run: 1426 subRun: 1 event: 67282 at 20-Jan-2026 10:16:22 EST
Begin processing the 83rd record. run: 1426 subRun: 1 event: 67283 at 20-Jan-2026 10:16:27 EST
Begin processing the 84th record. run: 1426 subRun: 1 event: 67284 at 20-Jan-2026 10:16:31 EST
Begin processing the 85th record. run: 1426 subRun: 1 event: 67285 at 20-Jan-2026 10:16:47 EST
Begin processing the 86th record. run: 1426 subRun: 1 event: 67286 at 20-Jan-2026 10:16:52 EST
Begin processing the 87th record. run: 1426 subRun: 1 event: 67287 at 20-Jan-2026 10:16:57 EST
Begin processing the 88th record. run: 1426 subRun: 1 event: 67288 at 20-Jan-2026 10:17:03 EST
Begin processing the 89th record. run: 1426 subRun: 1 event: 67289 at 20-Jan-2026 10:17:08 EST
Begin processing the 90th record. run: 1426 subRun: 1 event: 67290 at 20-Jan-2026 10:17:12 EST
Begin processing the 91st record. run: 1426 subRun: 1 event: 67291 at 20-Jan-2026 10:17:29 EST
Begin processing the 92nd record. run: 1426 subRun: 1 event: 67292 at 20-Jan-2026 10:17:34 EST
Begin processing the 93rd record. run: 1426 subRun: 1 event: 67293 at 20-Jan-2026 10:17:38 EST
Begin processing the 94th record. run: 1426 subRun: 1 event: 67294 at 20-Jan-2026 10:17:43 EST
Begin processing the 95th record. run: 1426 subRun: 1 event: 67295 at 20-Jan-2026 10:17:50 EST
Begin processing the 96th record. run: 1426 subRun: 1 event: 67296 at 20-Jan-2026 10:17:56 EST
Begin processing the 97th record. run: 1426 subRun: 1 event: 67297 at 20-Jan-2026 10:18:00 EST
Begin processing the 98th record. run: 1426 subRun: 1 event: 67298 at 20-Jan-2026 10:18:06 EST
Begin processing the 99th record. run: 1426 subRun: 1 event: 67299 at 20-Jan-2026 10:18:10 EST
Begin processing the 100th record. run: 1426 subRun: 1 event: 67300 at 20-Jan-2026 10:18:16 EST
20-Jan-2026 10:18:24 EST  Closed output file "nue_dune10kt_1x2x6_1426_672_20230827T193432Z_gen_g4_detsim_hitreco__20240221T063747Z_reco2_reco2.root"
20-Jan-2026 10:18:24 EST  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/07/c3/nue_dune10kt_1x2x6_1426_672_20230827T193432Z_gen_g4_detsim_hitreco__20240221T063747Z_reco2.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                     0.259204       12.4917       568.673       4.54049       56.2076        100    
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                         0.0385959     0.0797974     0.562193      0.0675918     0.0688633       100    
reco:pandora:StandardPandora                   0.186645       12.3867       568.379       4.45988       56.1866        100    
[art]:TriggerResults:TriggerResultInserter    1.9394e-05    3.80578e-05   0.000114251   3.55915e-05   1.77845e-05      100    
end_path:out1:RootOutput                       3.037e-06    5.38773e-06   2.3842e-05    4.1585e-06    3.4765e-06       100    
end_path:out1:RootOutput(write)                0.0015608     0.0248985     0.222267      0.0158964     0.0311943       100    
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 2427.83 MB
  Peak resident set size usage (VmHWM): 1404.3 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
justIN time: 2026-02-04 07:19:51 UTC       justIN version: 01.06.00