justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 47035.42@dunegpschedd01.fnal.gov

Jobsub ID47035.42@dunegpschedd01.fnal.gov
Workflow ID2661
Stage ID1
User nameimawby@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit43200 (12 hours)
Submitted time2025-09-16 16:21:11
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce509
Last heartbeat2025-09-16 16:31:01
From worker nodeHostnameb9p11p2682.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-09-16 16:22:18
Input filesfardet-hd:nu_dune10kt_1x2x6_1100_164_20230826T033416Z_gen_g4_detsim_hitreco__20240229T180235Z_reco2.root
JobscriptExit code0
Real time7m (433s)
CPU time4m (270s = 62%)
Max RSS bytes1321484288 (1260 MiB)
Outputting started2025-09-16 16:29:33
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/imawby/splitting_nu_3/fnal/02661/1/001/CheatingKalmanSplittingU_nu_dune10kt_1x2x6_1100_164_20230826T033416Z_gen_g4_detsim_hitreco__20240229T180235Z_reco2.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/imawby/splitting_nu_3/fnal/02661/1/001/CheatingKalmanSplittingV_nu_dune10kt_1x2x6_1100_164_20230826T033416Z_gen_g4_detsim_hitreco__20240229T180235Z_reco2.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/imawby/splitting_nu_3/fnal/02661/1/001/CheatingKalmanSplittingW_nu_dune10kt_1x2x6_1100_164_20230826T033416Z_gen_g4_detsim_hitreco__20240229T180235Z_reco2.root
Finished2025-09-16 16:31:01
Saved logsjustin-logs:47035.42-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

------
---------------------------------------------------------
Input PFN = root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/fardet-hd/01/a1/nu_dune10kt_1x2x6_1100_164_20230826T033416Z_gen_g4_detsim_hitreco__20240229T180235Z_reco2.root
lar exit code 0
=== Start last 100 lines of lar log file ===
PandoraContentApi::GetList(*this, m_inputHitListName, pCaloHitList) return STATUS_CODE_NOT_INITIALIZED
    in function: GetVolumeIdToHitListMap
    in file:     /exp/dune/app/users/imawby/dunesw_splitting/srcs/larpandoracontent/larpandoracontent/LArControlFlow/MasterAlgorithm.cc line#: 271
this->GetVolumeIdToHitListMap(volumeIdToHitListMap) return STATUS_CODE_NOT_INITIALIZED
    in function: Run
    in file:     /exp/dune/app/users/imawby/dunesw_splitting/srcs/larpandoracontent/larpandoracontent/LArControlFlow/MasterAlgorithm.cc line#: 165
iter->second->Run() throw STATUS_CODE_NOT_INITIALIZED
    in function: RunAlgorithm
    in file:     /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0002, LArDLMaster, STATUS_CODE_NOT_INITIALIZED
Begin processing the 33rd record. run: 1100 subRun: 1 event: 16433 at 16-Sep-2025 18:26:21 CEST
Begin processing the 34th record. run: 1100 subRun: 1 event: 16434 at 16-Sep-2025 18:26:24 CEST
Begin processing the 35th record. run: 1100 subRun: 1 event: 16435 at 16-Sep-2025 18:26:26 CEST
Begin processing the 36th record. run: 1100 subRun: 1 event: 16436 at 16-Sep-2025 18:26:29 CEST
Begin processing the 37th record. run: 1100 subRun: 1 event: 16437 at 16-Sep-2025 18:26:31 CEST
Begin processing the 38th record. run: 1100 subRun: 1 event: 16438 at 16-Sep-2025 18:26:34 CEST
Begin processing the 39th record. run: 1100 subRun: 1 event: 16439 at 16-Sep-2025 18:26:36 CEST
Begin processing the 40th record. run: 1100 subRun: 1 event: 16440 at 16-Sep-2025 18:26:39 CEST
Begin processing the 41st record. run: 1100 subRun: 1 event: 16441 at 16-Sep-2025 18:26:44 CEST
Begin processing the 42nd record. run: 1100 subRun: 1 event: 16442 at 16-Sep-2025 18:26:46 CEST
Begin processing the 43rd record. run: 1100 subRun: 1 event: 16443 at 16-Sep-2025 18:26:49 CEST
Begin processing the 44th record. run: 1100 subRun: 1 event: 16444 at 16-Sep-2025 18:26:52 CEST
Begin processing the 45th record. run: 1100 subRun: 1 event: 16445 at 16-Sep-2025 18:26:55 CEST
Begin processing the 46th record. run: 1100 subRun: 1 event: 16446 at 16-Sep-2025 18:26:57 CEST
Begin processing the 47th record. run: 1100 subRun: 1 event: 16447 at 16-Sep-2025 18:27:00 CEST
Begin processing the 48th record. run: 1100 subRun: 1 event: 16448 at 16-Sep-2025 18:27:03 CEST
Begin processing the 49th record. run: 1100 subRun: 1 event: 16449 at 16-Sep-2025 18:27:06 CEST
Begin processing the 50th record. run: 1100 subRun: 1 event: 16450 at 16-Sep-2025 18:27:10 CEST
Begin processing the 51st record. run: 1100 subRun: 1 event: 16451 at 16-Sep-2025 18:27:12 CEST
Begin processing the 52nd record. run: 1100 subRun: 1 event: 16452 at 16-Sep-2025 18:27:15 CEST
Begin processing the 53rd record. run: 1100 subRun: 1 event: 16453 at 16-Sep-2025 18:27:17 CEST
Begin processing the 54th record. run: 1100 subRun: 1 event: 16454 at 16-Sep-2025 18:27:20 CEST
Begin processing the 55th record. run: 1100 subRun: 1 event: 16455 at 16-Sep-2025 18:27:23 CEST
Begin processing the 56th record. run: 1100 subRun: 1 event: 16456 at 16-Sep-2025 18:27:26 CEST
Begin processing the 57th record. run: 1100 subRun: 1 event: 16457 at 16-Sep-2025 18:27:29 CEST
Begin processing the 58th record. run: 1100 subRun: 1 event: 16458 at 16-Sep-2025 18:27:30 CEST
Begin processing the 59th record. run: 1100 subRun: 1 event: 16459 at 16-Sep-2025 18:27:33 CEST
Begin processing the 60th record. run: 1100 subRun: 1 event: 16460 at 16-Sep-2025 18:27:36 CEST
Begin processing the 61st record. run: 1100 subRun: 1 event: 16461 at 16-Sep-2025 18:27:38 CEST
Begin processing the 62nd record. run: 1100 subRun: 1 event: 16462 at 16-Sep-2025 18:27:41 CEST
Begin processing the 63rd record. run: 1100 subRun: 1 event: 16463 at 16-Sep-2025 18:27:44 CEST
Begin processing the 64th record. run: 1100 subRun: 1 event: 16464 at 16-Sep-2025 18:27:47 CEST
Begin processing the 65th record. run: 1100 subRun: 1 event: 16465 at 16-Sep-2025 18:27:50 CEST
Begin processing the 66th record. run: 1100 subRun: 1 event: 16466 at 16-Sep-2025 18:27:52 CEST
Begin processing the 67th record. run: 1100 subRun: 1 event: 16467 at 16-Sep-2025 18:27:55 CEST
Begin processing the 68th record. run: 1100 subRun: 1 event: 16468 at 16-Sep-2025 18:28:00 CEST
Begin processing the 69th record. run: 1100 subRun: 1 event: 16469 at 16-Sep-2025 18:28:05 CEST
Begin processing the 70th record. run: 1100 subRun: 1 event: 16470 at 16-Sep-2025 18:28:08 CEST
Begin processing the 71st record. run: 1100 subRun: 1 event: 16471 at 16-Sep-2025 18:28:11 CEST
Begin processing the 72nd record. run: 1100 subRun: 1 event: 16472 at 16-Sep-2025 18:28:13 CEST
Begin processing the 73rd record. run: 1100 subRun: 1 event: 16473 at 16-Sep-2025 18:28:17 CEST
Begin processing the 74th record. run: 1100 subRun: 1 event: 16474 at 16-Sep-2025 18:28:20 CEST
Begin processing the 75th record. run: 1100 subRun: 1 event: 16475 at 16-Sep-2025 18:28:23 CEST
Begin processing the 76th record. run: 1100 subRun: 1 event: 16476 at 16-Sep-2025 18:28:26 CEST
Begin processing the 77th record. run: 1100 subRun: 1 event: 16477 at 16-Sep-2025 18:28:28 CEST
Begin processing the 78th record. run: 1100 subRun: 1 event: 16478 at 16-Sep-2025 18:28:31 CEST
Begin processing the 79th record. run: 1100 subRun: 1 event: 16479 at 16-Sep-2025 18:28:33 CEST
Begin processing the 80th record. run: 1100 subRun: 1 event: 16480 at 16-Sep-2025 18:28:36 CEST
Begin processing the 81st record. run: 1100 subRun: 1 event: 16481 at 16-Sep-2025 18:28:38 CEST
Begin processing the 82nd record. run: 1100 subRun: 1 event: 16482 at 16-Sep-2025 18:28:41 CEST
Begin processing the 83rd record. run: 1100 subRun: 1 event: 16483 at 16-Sep-2025 18:28:44 CEST
Begin processing the 84th record. run: 1100 subRun: 1 event: 16484 at 16-Sep-2025 18:28:46 CEST
Begin processing the 85th record. run: 1100 subRun: 1 event: 16485 at 16-Sep-2025 18:28:49 CEST
Begin processing the 86th record. run: 1100 subRun: 1 event: 16486 at 16-Sep-2025 18:28:51 CEST
Begin processing the 87th record. run: 1100 subRun: 1 event: 16487 at 16-Sep-2025 18:28:54 CEST
Begin processing the 88th record. run: 1100 subRun: 1 event: 16488 at 16-Sep-2025 18:28:57 CEST
Begin processing the 89th record. run: 1100 subRun: 1 event: 16489 at 16-Sep-2025 18:28:59 CEST
Begin processing the 90th record. run: 1100 subRun: 1 event: 16490 at 16-Sep-2025 18:29:02 CEST
Begin processing the 91st record. run: 1100 subRun: 1 event: 16491 at 16-Sep-2025 18:29:05 CEST
Begin processing the 92nd record. run: 1100 subRun: 1 event: 16492 at 16-Sep-2025 18:29:07 CEST
Begin processing the 93rd record. run: 1100 subRun: 1 event: 16493 at 16-Sep-2025 18:29:10 CEST
Begin processing the 94th record. run: 1100 subRun: 1 event: 16494 at 16-Sep-2025 18:29:13 CEST
Begin processing the 95th record. run: 1100 subRun: 1 event: 16495 at 16-Sep-2025 18:29:16 CEST
Begin processing the 96th record. run: 1100 subRun: 1 event: 16496 at 16-Sep-2025 18:29:18 CEST
Begin processing the 97th record. run: 1100 subRun: 1 event: 16497 at 16-Sep-2025 18:29:21 CEST
Begin processing the 98th record. run: 1100 subRun: 1 event: 16498 at 16-Sep-2025 18:29:24 CEST
Begin processing the 99th record. run: 1100 subRun: 1 event: 16499 at 16-Sep-2025 18:29:26 CEST
Begin processing the 100th record. run: 1100 subRun: 1 event: 16500 at 16-Sep-2025 18:29:28 CEST
16-Sep-2025 18:29:31 CEST  Closed output file "nu_dune10kt_1x2x6_1100_164_20230826T033416Z_gen_g4_detsim_hitreco__20240229T180235Z_reco2_reco2.root"
16-Sep-2025 18:29:31 CEST  Closed input file "root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/fardet-hd/01/a1/nu_dune10kt_1x2x6_1100_164_20230826T033416Z_gen_g4_detsim_hitreco__20240229T180235Z_reco2.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                     0.154424       2.32899       5.72473       2.31629      0.626085        100    
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                         0.0347455     0.0453069     0.153524      0.0453551     0.0131061       100    
reco:pandora:StandardPandora                   0.115748       2.27908       5.67966       2.26363      0.624795        100    
[art]:TriggerResults:TriggerResultInserter     1.385e-05    2.0338e-05    5.0601e-05    1.9145e-05    5.34923e-06      100    
end_path:out1:RootOutput                       3.63e-06     6.1842e-06    1.7921e-05     6.24e-06     1.42545e-06      100    
end_path:out1:RootOutput(write)               0.00108275    0.00433905     0.0167972    0.00354467    0.00271441       100    
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 2314.61 MB
  Peak resident set size usage (VmHWM): 1321.48 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
justIN time: 2025-09-18 23:59:07 UTC       justIN version: 01.05.00