justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 35435.12@dunegpschedd02.fnal.gov

Jobsub ID35435.12@dunegpschedd02.fnal.gov
Workflow ID2267
Stage ID1
User nameimawby@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit43200 (12 hours)
Submitted time2025-09-07 08:09:46
SiteUS_UCSD
EntryCMSHTPC_T2_US_UCSD_gw7
Last heartbeat2025-09-07 08:26:44
From worker nodeHostnamemh-7763-18.t2.ucsd.edu
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-09-07 08:10:48
Input filesfardet-hd:nu_dune10kt_1x2x6_1431_117_20230828T033339Z_gen_g4_detsim_hitreco__20240223T001154Z_reco2.root
JobscriptExit code0
Real time15m (930s)
CPU time4m (272s = 29%)
Max RSS bytes1303355392 (1242 MiB)
Outputting started2025-09-07 08:26:19
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/imawby/splitting_nu_5/fnal/02267/1/003/CheatingKalmanSplittingU_nu_dune10kt_1x2x6_1431_117_20230828T033339Z_gen_g4_detsim_hitreco__20240223T001154Z_reco2.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/imawby/splitting_nu_5/fnal/02267/1/003/CheatingKalmanSplittingV_nu_dune10kt_1x2x6_1431_117_20230828T033339Z_gen_g4_detsim_hitreco__20240223T001154Z_reco2.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/imawby/splitting_nu_5/fnal/02267/1/003/CheatingKalmanSplittingW_nu_dune10kt_1x2x6_1431_117_20230828T033339Z_gen_g4_detsim_hitreco__20240223T001154Z_reco2.root
Finished2025-09-07 08:26:44
Saved logsjustin-logs:35435.12-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

------- this block should be empty ------------------
---------------------------------------------------------
Input PFN = root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/c9/5e/nu_dune10kt_1x2x6_1431_117_20230828T033339Z_gen_g4_detsim_hitreco__20240223T001154Z_reco2.root
lar exit code 0
=== Start last 100 lines of lar log file ===
Begin processing the 23rd record. run: 1431 subRun: 1 event: 11723 at 07-Sep-2025 01:21:00 PDT
Begin processing the 24th record. run: 1431 subRun: 1 event: 11724 at 07-Sep-2025 01:21:05 PDT
Begin processing the 25th record. run: 1431 subRun: 1 event: 11725 at 07-Sep-2025 01:21:09 PDT
Begin processing the 26th record. run: 1431 subRun: 1 event: 11726 at 07-Sep-2025 01:21:12 PDT
Begin processing the 27th record. run: 1431 subRun: 1 event: 11727 at 07-Sep-2025 01:21:16 PDT
Begin processing the 28th record. run: 1431 subRun: 1 event: 11728 at 07-Sep-2025 01:21:19 PDT
Begin processing the 29th record. run: 1431 subRun: 1 event: 11729 at 07-Sep-2025 01:21:22 PDT
Begin processing the 30th record. run: 1431 subRun: 1 event: 11730 at 07-Sep-2025 01:21:26 PDT
Begin processing the 31st record. run: 1431 subRun: 1 event: 11731 at 07-Sep-2025 01:21:29 PDT
Begin processing the 32nd record. run: 1431 subRun: 1 event: 11732 at 07-Sep-2025 01:21:32 PDT
Begin processing the 33rd record. run: 1431 subRun: 1 event: 11733 at 07-Sep-2025 01:21:38 PDT
Begin processing the 34th record. run: 1431 subRun: 1 event: 11734 at 07-Sep-2025 01:21:42 PDT
Begin processing the 35th record. run: 1431 subRun: 1 event: 11735 at 07-Sep-2025 01:21:49 PDT
Begin processing the 36th record. run: 1431 subRun: 1 event: 11736 at 07-Sep-2025 01:21:52 PDT
Begin processing the 37th record. run: 1431 subRun: 1 event: 11737 at 07-Sep-2025 01:21:55 PDT
Begin processing the 38th record. run: 1431 subRun: 1 event: 11738 at 07-Sep-2025 01:21:58 PDT
Begin processing the 39th record. run: 1431 subRun: 1 event: 11739 at 07-Sep-2025 01:22:01 PDT
Begin processing the 40th record. run: 1431 subRun: 1 event: 11740 at 07-Sep-2025 01:22:05 PDT
Begin processing the 41st record. run: 1431 subRun: 1 event: 11741 at 07-Sep-2025 01:22:08 PDT
Begin processing the 42nd record. run: 1431 subRun: 1 event: 11742 at 07-Sep-2025 01:22:12 PDT
Begin processing the 43rd record. run: 1431 subRun: 1 event: 11743 at 07-Sep-2025 01:22:17 PDT
Begin processing the 44th record. run: 1431 subRun: 1 event: 11744 at 07-Sep-2025 01:22:20 PDT
Begin processing the 45th record. run: 1431 subRun: 1 event: 11745 at 07-Sep-2025 01:22:23 PDT
Begin processing the 46th record. run: 1431 subRun: 1 event: 11746 at 07-Sep-2025 01:22:26 PDT
Begin processing the 47th record. run: 1431 subRun: 1 event: 11747 at 07-Sep-2025 01:22:29 PDT
Begin processing the 48th record. run: 1431 subRun: 1 event: 11748 at 07-Sep-2025 01:22:33 PDT
Begin processing the 49th record. run: 1431 subRun: 1 event: 11749 at 07-Sep-2025 01:22:36 PDT
Begin processing the 50th record. run: 1431 subRun: 1 event: 11750 at 07-Sep-2025 01:22:39 PDT
Begin processing the 51st record. run: 1431 subRun: 1 event: 11751 at 07-Sep-2025 01:22:44 PDT
Begin processing the 52nd record. run: 1431 subRun: 1 event: 11752 at 07-Sep-2025 01:22:48 PDT
Begin processing the 53rd record. run: 1431 subRun: 1 event: 11753 at 07-Sep-2025 01:22:53 PDT
Begin processing the 54th record. run: 1431 subRun: 1 event: 11754 at 07-Sep-2025 01:22:57 PDT
Begin processing the 55th record. run: 1431 subRun: 1 event: 11755 at 07-Sep-2025 01:23:00 PDT
Begin processing the 56th record. run: 1431 subRun: 1 event: 11756 at 07-Sep-2025 01:23:04 PDT
Begin processing the 57th record. run: 1431 subRun: 1 event: 11757 at 07-Sep-2025 01:23:08 PDT
Begin processing the 58th record. run: 1431 subRun: 1 event: 11758 at 07-Sep-2025 01:23:11 PDT
Begin processing the 59th record. run: 1431 subRun: 1 event: 11759 at 07-Sep-2025 01:23:16 PDT
Begin processing the 60th record. run: 1431 subRun: 1 event: 11760 at 07-Sep-2025 01:23:20 PDT
Begin processing the 61st record. run: 1431 subRun: 1 event: 11761 at 07-Sep-2025 01:23:26 PDT
Begin processing the 62nd record. run: 1431 subRun: 1 event: 11762 at 07-Sep-2025 01:23:30 PDT
Begin processing the 63rd record. run: 1431 subRun: 1 event: 11763 at 07-Sep-2025 01:23:33 PDT
Begin processing the 64th record. run: 1431 subRun: 1 event: 11764 at 07-Sep-2025 01:23:36 PDT
Begin processing the 65th record. run: 1431 subRun: 1 event: 11765 at 07-Sep-2025 01:23:40 PDT
Begin processing the 66th record. run: 1431 subRun: 1 event: 11766 at 07-Sep-2025 01:23:43 PDT
Begin processing the 67th record. run: 1431 subRun: 1 event: 11767 at 07-Sep-2025 01:23:47 PDT
Begin processing the 68th record. run: 1431 subRun: 1 event: 11768 at 07-Sep-2025 01:23:50 PDT
Begin processing the 69th record. run: 1431 subRun: 1 event: 11769 at 07-Sep-2025 01:23:56 PDT
Begin processing the 70th record. run: 1431 subRun: 1 event: 11770 at 07-Sep-2025 01:24:01 PDT
Begin processing the 71st record. run: 1431 subRun: 1 event: 11771 at 07-Sep-2025 01:24:04 PDT
Begin processing the 72nd record. run: 1431 subRun: 1 event: 11772 at 07-Sep-2025 01:24:14 PDT
Begin processing the 73rd record. run: 1431 subRun: 1 event: 11773 at 07-Sep-2025 01:24:18 PDT
Begin processing the 74th record. run: 1431 subRun: 1 event: 11774 at 07-Sep-2025 01:24:22 PDT
Begin processing the 75th record. run: 1431 subRun: 1 event: 11775 at 07-Sep-2025 01:24:26 PDT
Begin processing the 76th record. run: 1431 subRun: 1 event: 11776 at 07-Sep-2025 01:24:30 PDT
Begin processing the 77th record. run: 1431 subRun: 1 event: 11777 at 07-Sep-2025 01:24:33 PDT
Begin processing the 78th record. run: 1431 subRun: 1 event: 11778 at 07-Sep-2025 01:24:37 PDT
Begin processing the 79th record. run: 1431 subRun: 1 event: 11779 at 07-Sep-2025 01:24:40 PDT
Begin processing the 80th record. run: 1431 subRun: 1 event: 11780 at 07-Sep-2025 01:24:45 PDT
Begin processing the 81st record. run: 1431 subRun: 1 event: 11781 at 07-Sep-2025 01:24:49 PDT
Begin processing the 82nd record. run: 1431 subRun: 1 event: 11782 at 07-Sep-2025 01:24:52 PDT
Begin processing the 83rd record. run: 1431 subRun: 1 event: 11783 at 07-Sep-2025 01:24:55 PDT
Begin processing the 84th record. run: 1431 subRun: 1 event: 11784 at 07-Sep-2025 01:24:59 PDT
Begin processing the 85th record. run: 1431 subRun: 1 event: 11785 at 07-Sep-2025 01:25:02 PDT
Begin processing the 86th record. run: 1431 subRun: 1 event: 11786 at 07-Sep-2025 01:25:05 PDT
Begin processing the 87th record. run: 1431 subRun: 1 event: 11787 at 07-Sep-2025 01:25:09 PDT
Begin processing the 88th record. run: 1431 subRun: 1 event: 11788 at 07-Sep-2025 01:25:13 PDT
Begin processing the 89th record. run: 1431 subRun: 1 event: 11789 at 07-Sep-2025 01:25:16 PDT
Begin processing the 90th record. run: 1431 subRun: 1 event: 11790 at 07-Sep-2025 01:25:19 PDT
Begin processing the 91st record. run: 1431 subRun: 1 event: 11791 at 07-Sep-2025 01:25:23 PDT
Begin processing the 92nd record. run: 1431 subRun: 1 event: 11792 at 07-Sep-2025 01:25:26 PDT
Begin processing the 93rd record. run: 1431 subRun: 1 event: 11793 at 07-Sep-2025 01:25:29 PDT
Begin processing the 94th record. run: 1431 subRun: 1 event: 11794 at 07-Sep-2025 01:25:34 PDT
Begin processing the 95th record. run: 1431 subRun: 1 event: 11795 at 07-Sep-2025 01:25:37 PDT
Begin processing the 96th record. run: 1431 subRun: 1 event: 11796 at 07-Sep-2025 01:25:41 PDT
Begin processing the 97th record. run: 1431 subRun: 1 event: 11797 at 07-Sep-2025 01:25:45 PDT
Begin processing the 98th record. run: 1431 subRun: 1 event: 11798 at 07-Sep-2025 01:25:53 PDT
Begin processing the 99th record. run: 1431 subRun: 1 event: 11799 at 07-Sep-2025 01:25:57 PDT
Begin processing the 100th record. run: 1431 subRun: 1 event: 11800 at 07-Sep-2025 01:26:13 PDT
07-Sep-2025 01:26:17 PDT  Closed output file "nu_dune10kt_1x2x6_1431_117_20230828T033339Z_gen_g4_detsim_hitreco__20240223T001154Z_reco2_reco2.root"
07-Sep-2025 01:26:17 PDT  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/c9/5e/nu_dune10kt_1x2x6_1431_117_20230828T033339Z_gen_g4_detsim_hitreco__20240223T001154Z_reco2.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                      2.06074       2.52082       6.35911       2.37471      0.544304        100    
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                         0.0611235     0.0985093     0.121622      0.0968261    0.00604891       100    
reco:pandora:StandardPandora                    1.96229       2.41813       6.21218       2.27386      0.540628        100    
[art]:TriggerResults:TriggerResultInserter     1.03e-05     2.0719e-05    0.000159047   1.66245e-05   2.07696e-05      100    
end_path:out1:RootOutput                       2.331e-06    3.25598e-06    2.062e-05     3.045e-06    1.83196e-06      100    
end_path:out1:RootOutput(write)                0.0010118    0.00392111     0.0367884    0.00306137    0.00401721       100    
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 2372.49 MB
  Peak resident set size usage (VmHWM): 1303.36 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
justIN time: 2025-09-19 08:08:45 UTC       justIN version: 01.05.00