justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 235332.44@dunegpschedd02.fnal.gov

Jobsub ID235332.44@dunegpschedd02.fnal.gov
Workflow ID9467
Stage ID1
User nameimawby@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes1048576000 (1000 MiB)
Wall seconds limit7200 (2 hours)
Submitted time2025-10-30 11:27:38
SiteUS_UCSD
EntryCMSHTPC_T2_US_UCSD_gw6
Last heartbeat2025-10-30 12:28:45
From worker nodeHostnamemh-7763-16.t2.ucsd.edu
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes1048576000 (1000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-30 12:02:02
Input filesfardet-hd:nu_dune10kt_1x2x6_1410_504_20230826T123341Z_gen_g4_detsim_hitreco__20240222T220045Z_reco2.root
JobscriptExit code0
Real time26m (1577s)
CPU time8m (490s = 31%)
Max RSS bytes1251680256 (1193 MiB)
Outputting started2025-10-30 12:28:20
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/imawby/kalmanAlg_nu_0/fnal/09467/1/001/ClusterValidation_WithoutAlg_nu_dune10kt_1x2x6_1410_504_20230826T123341Z_gen_g4_detsim_hitreco__20240222T220045Z_reco2.root
Finished2025-10-30 12:28:45
Saved logsjustin-logs:235332.44-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

-------
---------------------------------------------------------
Input PFN = root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/84/5d/nu_dune10kt_1x2x6_1410_504_20230826T123341Z_gen_g4_detsim_hitreco__20240222T220045Z_reco2.root
lar exit code 0
=== Start last 100 lines of lar log file ===
Begin processing the 33rd record. run: 1410 subRun: 1 event: 50433 at 30-Oct-2025 05:19:45 PDT
Begin processing the 34th record. run: 1410 subRun: 1 event: 50434 at 30-Oct-2025 05:19:49 PDT
Begin processing the 35th record. run: 1410 subRun: 1 event: 50435 at 30-Oct-2025 05:19:54 PDT
Begin processing the 36th record. run: 1410 subRun: 1 event: 50436 at 30-Oct-2025 05:20:00 PDT
Begin processing the 37th record. run: 1410 subRun: 1 event: 50437 at 30-Oct-2025 05:20:06 PDT
Begin processing the 38th record. run: 1410 subRun: 1 event: 50438 at 30-Oct-2025 05:20:09 PDT
Begin processing the 39th record. run: 1410 subRun: 1 event: 50439 at 30-Oct-2025 05:20:14 PDT
Begin processing the 40th record. run: 1410 subRun: 1 event: 50440 at 30-Oct-2025 05:20:21 PDT
Begin processing the 41st record. run: 1410 subRun: 1 event: 50441 at 30-Oct-2025 05:20:28 PDT
Begin processing the 42nd record. run: 1410 subRun: 1 event: 50442 at 30-Oct-2025 05:20:33 PDT
Begin processing the 43rd record. run: 1410 subRun: 1 event: 50443 at 30-Oct-2025 05:20:40 PDT
Begin processing the 44th record. run: 1410 subRun: 1 event: 50444 at 30-Oct-2025 05:20:44 PDT
Begin processing the 45th record. run: 1410 subRun: 1 event: 50445 at 30-Oct-2025 05:20:53 PDT
Begin processing the 46th record. run: 1410 subRun: 1 event: 50446 at 30-Oct-2025 05:21:03 PDT
Begin processing the 47th record. run: 1410 subRun: 1 event: 50447 at 30-Oct-2025 05:21:07 PDT
Begin processing the 48th record. run: 1410 subRun: 1 event: 50448 at 30-Oct-2025 05:21:12 PDT
Begin processing the 49th record. run: 1410 subRun: 1 event: 50449 at 30-Oct-2025 05:21:18 PDT
Begin processing the 50th record. run: 1410 subRun: 1 event: 50450 at 30-Oct-2025 05:21:25 PDT
Begin processing the 51st record. run: 1410 subRun: 1 event: 50451 at 30-Oct-2025 05:21:29 PDT
Begin processing the 52nd record. run: 1410 subRun: 1 event: 50452 at 30-Oct-2025 05:21:34 PDT
Begin processing the 53rd record. run: 1410 subRun: 1 event: 50453 at 30-Oct-2025 05:23:11 PDT
Begin processing the 54th record. run: 1410 subRun: 1 event: 50454 at 30-Oct-2025 05:23:18 PDT
Begin processing the 55th record. run: 1410 subRun: 1 event: 50455 at 30-Oct-2025 05:23:32 PDT
Begin processing the 56th record. run: 1410 subRun: 1 event: 50456 at 30-Oct-2025 05:23:38 PDT
Begin processing the 57th record. run: 1410 subRun: 1 event: 50457 at 30-Oct-2025 05:23:44 PDT
Begin processing the 58th record. run: 1410 subRun: 1 event: 50458 at 30-Oct-2025 05:23:49 PDT
Begin processing the 59th record. run: 1410 subRun: 1 event: 50459 at 30-Oct-2025 05:23:53 PDT
Begin processing the 60th record. run: 1410 subRun: 1 event: 50460 at 30-Oct-2025 05:23:59 PDT
Begin processing the 61st record. run: 1410 subRun: 1 event: 50461 at 30-Oct-2025 05:24:03 PDT
Begin processing the 62nd record. run: 1410 subRun: 1 event: 50462 at 30-Oct-2025 05:24:08 PDT
Begin processing the 63rd record. run: 1410 subRun: 1 event: 50463 at 30-Oct-2025 05:24:15 PDT
Begin processing the 64th record. run: 1410 subRun: 1 event: 50464 at 30-Oct-2025 05:24:19 PDT
Begin processing the 65th record. run: 1410 subRun: 1 event: 50465 at 30-Oct-2025 05:24:24 PDT
Begin processing the 66th record. run: 1410 subRun: 1 event: 50466 at 30-Oct-2025 05:24:38 PDT
PandoraContentApi::GetList(*this, m_inputHitListName, pCaloHitList) return STATUS_CODE_NOT_INITIALIZED
    in function: GetVolumeIdToHitListMap
    in file:     /exp/dune/app/users/imawby/dunesw_splitting/srcs/larpandoracontent/larpandoracontent/LArControlFlow/MasterAlgorithm.cc line#: 271
this->GetVolumeIdToHitListMap(volumeIdToHitListMap) return STATUS_CODE_NOT_INITIALIZED
    in function: Run
    in file:     /exp/dune/app/users/imawby/dunesw_splitting/srcs/larpandoracontent/larpandoracontent/LArControlFlow/MasterAlgorithm.cc line#: 165
iter->second->Run() throw STATUS_CODE_NOT_INITIALIZED
    in function: RunAlgorithm
    in file:     /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0002, LArDLMaster, STATUS_CODE_NOT_INITIALIZED
Begin processing the 67th record. run: 1410 subRun: 1 event: 50467 at 30-Oct-2025 05:24:41 PDT
Begin processing the 68th record. run: 1410 subRun: 1 event: 50468 at 30-Oct-2025 05:24:47 PDT
Begin processing the 69th record. run: 1410 subRun: 1 event: 50469 at 30-Oct-2025 05:24:52 PDT
Begin processing the 70th record. run: 1410 subRun: 1 event: 50470 at 30-Oct-2025 05:24:56 PDT
Begin processing the 71st record. run: 1410 subRun: 1 event: 50471 at 30-Oct-2025 05:25:01 PDT
Begin processing the 72nd record. run: 1410 subRun: 1 event: 50472 at 30-Oct-2025 05:25:06 PDT
Begin processing the 73rd record. run: 1410 subRun: 1 event: 50473 at 30-Oct-2025 05:25:11 PDT
Begin processing the 74th record. run: 1410 subRun: 1 event: 50474 at 30-Oct-2025 05:25:16 PDT
Begin processing the 75th record. run: 1410 subRun: 1 event: 50475 at 30-Oct-2025 05:25:20 PDT
Begin processing the 76th record. run: 1410 subRun: 1 event: 50476 at 30-Oct-2025 05:25:25 PDT
Begin processing the 77th record. run: 1410 subRun: 1 event: 50477 at 30-Oct-2025 05:25:32 PDT
Begin processing the 78th record. run: 1410 subRun: 1 event: 50478 at 30-Oct-2025 05:25:40 PDT
Begin processing the 79th record. run: 1410 subRun: 1 event: 50479 at 30-Oct-2025 05:25:52 PDT
Begin processing the 80th record. run: 1410 subRun: 1 event: 50480 at 30-Oct-2025 05:25:57 PDT
Begin processing the 81st record. run: 1410 subRun: 1 event: 50481 at 30-Oct-2025 05:26:01 PDT
Begin processing the 82nd record. run: 1410 subRun: 1 event: 50482 at 30-Oct-2025 05:26:07 PDT
Begin processing the 83rd record. run: 1410 subRun: 1 event: 50483 at 30-Oct-2025 05:26:18 PDT
Begin processing the 84th record. run: 1410 subRun: 1 event: 50484 at 30-Oct-2025 05:26:25 PDT
Begin processing the 85th record. run: 1410 subRun: 1 event: 50485 at 30-Oct-2025 05:26:31 PDT
Begin processing the 86th record. run: 1410 subRun: 1 event: 50486 at 30-Oct-2025 05:26:37 PDT
Begin processing the 87th record. run: 1410 subRun: 1 event: 50487 at 30-Oct-2025 05:26:43 PDT
Begin processing the 88th record. run: 1410 subRun: 1 event: 50488 at 30-Oct-2025 05:26:47 PDT
Begin processing the 89th record. run: 1410 subRun: 1 event: 50489 at 30-Oct-2025 05:26:53 PDT
Begin processing the 90th record. run: 1410 subRun: 1 event: 50490 at 30-Oct-2025 05:26:58 PDT
Begin processing the 91st record. run: 1410 subRun: 1 event: 50491 at 30-Oct-2025 05:27:04 PDT
Begin processing the 92nd record. run: 1410 subRun: 1 event: 50492 at 30-Oct-2025 05:27:07 PDT
Begin processing the 93rd record. run: 1410 subRun: 1 event: 50493 at 30-Oct-2025 05:27:14 PDT
Begin processing the 94th record. run: 1410 subRun: 1 event: 50494 at 30-Oct-2025 05:27:19 PDT
Begin processing the 95th record. run: 1410 subRun: 1 event: 50495 at 30-Oct-2025 05:27:45 PDT
Begin processing the 96th record. run: 1410 subRun: 1 event: 50496 at 30-Oct-2025 05:27:49 PDT
Begin processing the 97th record. run: 1410 subRun: 1 event: 50497 at 30-Oct-2025 05:28:03 PDT
Begin processing the 98th record. run: 1410 subRun: 1 event: 50498 at 30-Oct-2025 05:28:07 PDT
Begin processing the 99th record. run: 1410 subRun: 1 event: 50499 at 30-Oct-2025 05:28:11 PDT
Begin processing the 100th record. run: 1410 subRun: 1 event: 50500 at 30-Oct-2025 05:28:15 PDT
30-Oct-2025 05:28:19 PDT  Closed output file "nu_dune10kt_1x2x6_1410_504_20230826T123341Z_gen_g4_detsim_hitreco__20240222T220045Z_reco2_reco2.root"
30-Oct-2025 05:28:19 PDT  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/84/5d/nu_dune10kt_1x2x6_1410_504_20230826T123341Z_gen_g4_detsim_hitreco__20240222T220045Z_reco2.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                     0.460708       4.80713       89.8537       3.37481       8.85106        100    
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                         0.0587077     0.119317      0.372513      0.113455      0.0292681       100    
reco:pandora:StandardPandora                    0.32542       4.67357       89.6949       3.2511        8.84708        100    
[art]:TriggerResults:TriggerResultInserter     1.324e-05    2.81116e-05   0.000106378   2.2665e-05    1.57485e-05      100    
end_path:out1:RootOutput                       3.27e-06     5.13113e-06    2.763e-05     4.36e-06     2.95082e-06      100    
end_path:out1:RootOutput(write)               0.00150006     0.0139277     0.0696929     0.011465      0.0115195       100    
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 2299 MB
  Peak resident set size usage (VmHWM): 1251.68 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
mv: cannot stat 'ClusterValidation_WithAlg.root': No such file or directory
justIN time: 2026-02-11 16:34:09 UTC       justIN version: 01.06.00