justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 279414.4@dunegpschedd01.fnal.gov

Jobsub ID279414.4@dunegpschedd01.fnal.gov
Workflow ID11975
Stage ID1
User nameimawby@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes1048576000 (1000 MiB)
Wall seconds limit7200 (2 hours)
Submitted time2026-01-16 09:05:11
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2026-01-16 09:51:59
From worker nodeHostnamedunegli-8167448-0-fnpc18001.fnal.gov
cpuinfoAMD EPYC 7551 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes1048576000 (1000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-16 09:09:22
Input filesfardet-hd:nue_dune10kt_1x2x6_1063_824_20230823T132513Z_gen_g4_detsim_hitreco__20240221T070047Z_reco2.root
JobscriptExit code0
Real time42m (2534s)
CPU time18m (1134s = 44%)
Max RSS bytes1357033472 (1294 MiB)
Outputting started2026-01-16 09:51:37
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/imawby/metrics_nue_0/fnal/11975/1/001/Validation_nue_dune10kt_1x2x6_1063_824_20230823T132513Z_gen_g4_detsim_hitreco__20240221T070047Z_reco2.root
Finished2026-01-16 09:51:59
Saved logsjustin-logs:279414.4-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

3_00_e26_prof
----------- this block should be empty ------------------
---------------------------------------------------------
Input PFN = root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/b3/7b/nue_dune10kt_1x2x6_1063_824_20230823T132513Z_gen_g4_detsim_hitreco__20240221T070047Z_reco2.root
lar exit code 0
=== Start last 100 lines of lar log file ===
Begin processing the 32nd record. run: 1063 subRun: 1 event: 82432 at 16-Jan-2026 09:41:10 UTC
Begin processing the 33rd record. run: 1063 subRun: 1 event: 82433 at 16-Jan-2026 09:41:16 UTC
Begin processing the 34th record. run: 1063 subRun: 1 event: 82434 at 16-Jan-2026 09:41:21 UTC
Begin processing the 35th record. run: 1063 subRun: 1 event: 82435 at 16-Jan-2026 09:41:28 UTC
Begin processing the 36th record. run: 1063 subRun: 1 event: 82436 at 16-Jan-2026 09:41:34 UTC
Begin processing the 37th record. run: 1063 subRun: 1 event: 82437 at 16-Jan-2026 09:41:39 UTC
Begin processing the 38th record. run: 1063 subRun: 1 event: 82438 at 16-Jan-2026 09:41:45 UTC
Begin processing the 39th record. run: 1063 subRun: 1 event: 82439 at 16-Jan-2026 09:41:51 UTC
Begin processing the 40th record. run: 1063 subRun: 1 event: 82440 at 16-Jan-2026 09:41:58 UTC
Begin processing the 41st record. run: 1063 subRun: 1 event: 82441 at 16-Jan-2026 09:42:03 UTC
Begin processing the 42nd record. run: 1063 subRun: 1 event: 82442 at 16-Jan-2026 09:42:09 UTC
Begin processing the 43rd record. run: 1063 subRun: 1 event: 82443 at 16-Jan-2026 09:42:14 UTC
Begin processing the 44th record. run: 1063 subRun: 1 event: 82444 at 16-Jan-2026 09:42:19 UTC
Begin processing the 45th record. run: 1063 subRun: 1 event: 82445 at 16-Jan-2026 09:42:24 UTC
Begin processing the 46th record. run: 1063 subRun: 1 event: 82446 at 16-Jan-2026 09:42:29 UTC
Begin processing the 47th record. run: 1063 subRun: 1 event: 82447 at 16-Jan-2026 09:42:34 UTC
PandoraContentApi::GetList(*this, listName, pList) return STATUS_CODE_NOT_INITIALIZED
    in function: GetList
    in file:     /exp/dune/app/users/imawby/dunesw_validation/srcs/larpandoracontent/larpandoradlcontent/LArTwoDReco/DLThreeDClusterSplittingAlgorithm.cc line#: 194
Begin processing the 48th record. run: 1063 subRun: 1 event: 82448 at 16-Jan-2026 09:42:38 UTC
Begin processing the 49th record. run: 1063 subRun: 1 event: 82449 at 16-Jan-2026 09:42:44 UTC
Begin processing the 50th record. run: 1063 subRun: 1 event: 82450 at 16-Jan-2026 09:42:50 UTC
Begin processing the 51st record. run: 1063 subRun: 1 event: 82451 at 16-Jan-2026 09:42:55 UTC
Begin processing the 52nd record. run: 1063 subRun: 1 event: 82452 at 16-Jan-2026 09:43:00 UTC
Begin processing the 53rd record. run: 1063 subRun: 1 event: 82453 at 16-Jan-2026 09:43:06 UTC
Begin processing the 54th record. run: 1063 subRun: 1 event: 82454 at 16-Jan-2026 09:43:18 UTC
Begin processing the 55th record. run: 1063 subRun: 1 event: 82455 at 16-Jan-2026 09:44:12 UTC
Begin processing the 56th record. run: 1063 subRun: 1 event: 82456 at 16-Jan-2026 09:44:18 UTC
Begin processing the 57th record. run: 1063 subRun: 1 event: 82457 at 16-Jan-2026 09:44:23 UTC
PandoraContentApi::GetList(*this, listName, pList) return STATUS_CODE_NOT_INITIALIZED
    in function: GetList
    in file:     /exp/dune/app/users/imawby/dunesw_validation/srcs/larpandoracontent/larpandoradlcontent/LArTwoDReco/DLThreeDClusterSplittingAlgorithm.cc line#: 194
Begin processing the 58th record. run: 1063 subRun: 1 event: 82458 at 16-Jan-2026 09:44:28 UTC
Begin processing the 59th record. run: 1063 subRun: 1 event: 82459 at 16-Jan-2026 09:44:40 UTC
Begin processing the 60th record. run: 1063 subRun: 1 event: 82460 at 16-Jan-2026 09:44:46 UTC
Begin processing the 61st record. run: 1063 subRun: 1 event: 82461 at 16-Jan-2026 09:44:50 UTC
Begin processing the 62nd record. run: 1063 subRun: 1 event: 82462 at 16-Jan-2026 09:44:57 UTC
Begin processing the 63rd record. run: 1063 subRun: 1 event: 82463 at 16-Jan-2026 09:45:02 UTC
Begin processing the 64th record. run: 1063 subRun: 1 event: 82464 at 16-Jan-2026 09:45:08 UTC
Begin processing the 65th record. run: 1063 subRun: 1 event: 82465 at 16-Jan-2026 09:45:13 UTC
Begin processing the 66th record. run: 1063 subRun: 1 event: 82466 at 16-Jan-2026 09:45:18 UTC
Begin processing the 67th record. run: 1063 subRun: 1 event: 82467 at 16-Jan-2026 09:45:23 UTC
Begin processing the 68th record. run: 1063 subRun: 1 event: 82468 at 16-Jan-2026 09:45:29 UTC
Begin processing the 69th record. run: 1063 subRun: 1 event: 82469 at 16-Jan-2026 09:45:34 UTC
Begin processing the 70th record. run: 1063 subRun: 1 event: 82470 at 16-Jan-2026 09:45:39 UTC
Begin processing the 71st record. run: 1063 subRun: 1 event: 82471 at 16-Jan-2026 09:45:44 UTC
Begin processing the 72nd record. run: 1063 subRun: 1 event: 82472 at 16-Jan-2026 09:45:49 UTC
Begin processing the 73rd record. run: 1063 subRun: 1 event: 82473 at 16-Jan-2026 09:45:55 UTC
Begin processing the 74th record. run: 1063 subRun: 1 event: 82474 at 16-Jan-2026 09:46:00 UTC
Begin processing the 75th record. run: 1063 subRun: 1 event: 82475 at 16-Jan-2026 09:46:05 UTC
Begin processing the 76th record. run: 1063 subRun: 1 event: 82476 at 16-Jan-2026 09:46:10 UTC
Begin processing the 77th record. run: 1063 subRun: 1 event: 82477 at 16-Jan-2026 09:46:15 UTC
Begin processing the 78th record. run: 1063 subRun: 1 event: 82478 at 16-Jan-2026 09:46:34 UTC
Begin processing the 79th record. run: 1063 subRun: 1 event: 82479 at 16-Jan-2026 09:46:41 UTC
PandoraContentApi::GetList(*this, listName, pList) return STATUS_CODE_NOT_INITIALIZED
    in function: GetList
    in file:     /exp/dune/app/users/imawby/dunesw_validation/srcs/larpandoracontent/larpandoradlcontent/LArTwoDReco/DLThreeDClusterSplittingAlgorithm.cc line#: 194
Begin processing the 80th record. run: 1063 subRun: 1 event: 82480 at 16-Jan-2026 09:46:46 UTC
Begin processing the 81st record. run: 1063 subRun: 1 event: 82481 at 16-Jan-2026 09:47:04 UTC
Begin processing the 82nd record. run: 1063 subRun: 1 event: 82482 at 16-Jan-2026 09:47:10 UTC
Begin processing the 83rd record. run: 1063 subRun: 1 event: 82483 at 16-Jan-2026 09:47:15 UTC
Begin processing the 84th record. run: 1063 subRun: 1 event: 82484 at 16-Jan-2026 09:47:21 UTC
Begin processing the 85th record. run: 1063 subRun: 1 event: 82485 at 16-Jan-2026 09:47:29 UTC
Begin processing the 86th record. run: 1063 subRun: 1 event: 82486 at 16-Jan-2026 09:47:34 UTC
Begin processing the 87th record. run: 1063 subRun: 1 event: 82487 at 16-Jan-2026 09:47:41 UTC
Begin processing the 88th record. run: 1063 subRun: 1 event: 82488 at 16-Jan-2026 09:47:46 UTC
Begin processing the 89th record. run: 1063 subRun: 1 event: 82489 at 16-Jan-2026 09:47:52 UTC
Begin processing the 90th record. run: 1063 subRun: 1 event: 82490 at 16-Jan-2026 09:47:57 UTC
Begin processing the 91st record. run: 1063 subRun: 1 event: 82491 at 16-Jan-2026 09:48:12 UTC
Begin processing the 92nd record. run: 1063 subRun: 1 event: 82492 at 16-Jan-2026 09:48:18 UTC
Begin processing the 93rd record. run: 1063 subRun: 1 event: 82493 at 16-Jan-2026 09:48:23 UTC
Begin processing the 94th record. run: 1063 subRun: 1 event: 82494 at 16-Jan-2026 09:50:50 UTC
Begin processing the 95th record. run: 1063 subRun: 1 event: 82495 at 16-Jan-2026 09:50:56 UTC
Begin processing the 96th record. run: 1063 subRun: 1 event: 82496 at 16-Jan-2026 09:51:01 UTC
Begin processing the 97th record. run: 1063 subRun: 1 event: 82497 at 16-Jan-2026 09:51:08 UTC
Begin processing the 98th record. run: 1063 subRun: 1 event: 82498 at 16-Jan-2026 09:51:16 UTC
Begin processing the 99th record. run: 1063 subRun: 1 event: 82499 at 16-Jan-2026 09:51:23 UTC
Begin processing the 100th record. run: 1063 subRun: 1 event: 82500 at 16-Jan-2026 09:51:29 UTC
16-Jan-2026 09:51:36 UTC  Closed output file "nue_dune10kt_1x2x6_1063_824_20230823T132513Z_gen_g4_detsim_hitreco__20240221T070047Z_reco2_reco2.root"
16-Jan-2026 09:51:36 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/b3/7b/nue_dune10kt_1x2x6_1063_824_20230823T132513Z_gen_g4_detsim_hitreco__20240221T070047Z_reco2.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                      4.55747       10.4913       145.38        5.53762       20.6668        100    
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.00227835     0.0263718     0.223744      0.0208999     0.0306089       100    
reco:pandora:StandardPandora                    4.55216       10.4309       145.138       5.50122       20.6371        100    
[art]:TriggerResults:TriggerResultInserter    2.5889e-05    6.06453e-05   0.000342125   4.59165e-05   4.74849e-05      100    
end_path:out1:RootOutput                       4.098e-06    6.55768e-06   2.9065e-05     5.265e-06    3.66914e-06      100    
end_path:out1:RootOutput(write)               0.00185763     0.0336609     0.783271      0.0151317     0.0844103       100    
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 2354.95 MB
  Peak resident set size usage (VmHWM): 1357.03 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
justIN time: 2026-02-04 13:00:26 UTC       justIN version: 01.06.00