justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 247832.0@dunegpschedd02.fnal.gov

Jobsub ID247832.0@dunegpschedd02.fnal.gov
Workflow ID10317
Stage ID1
User nameimawby@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes1048576000 (1000 MiB)
Wall seconds limit7200 (2 hours)
Submitted time2025-11-17 21:55:00
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-11-17 22:05:22
From worker nodeHostnamedunegli-7795803-0-fnpc19108.fnal.gov
cpuinfoAMD EPYC 7502 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes1048576000 (1000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-17 21:58:26
Input filesfardet-hd:nu_dune10kt_1x2x6_1114_526_20230827T153337Z_gen_g4_detsim_hitreco__20240223T011820Z_reco2.root
JobscriptExit code0
Real time6m (397s)
CPU time6m (364s = 91%)
Max RSS bytes1272332288 (1213 MiB)
Outputting started2025-11-17 22:05:03
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/imawby/splitting_nu_5/fnal/10317/1/001/CheatingKalmanSplittingU_nu_dune10kt_1x2x6_1114_526_20230827T153337Z_gen_g4_detsim_hitreco__20240223T011820Z_reco2.root
Finished2025-11-17 22:05:22
Saved logsjustin-logs:247832.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     (HTCondor job logs unavailable)

Jobscript log (last 10,000 characters)

pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/1c/b8/nu_dune10kt_1x2x6_1114_526_20230827T153337Z_gen_g4_detsim_hitreco__20240223T011820Z_reco2.root
lar exit code 0
=== Start last 100 lines of lar log file ===
Begin processing the 23rd record. run: 1114 subRun: 1 event: 52623 at 17-Nov-2025 22:01:06 UTC
Begin processing the 24th record. run: 1114 subRun: 1 event: 52624 at 17-Nov-2025 22:01:09 UTC
Begin processing the 25th record. run: 1114 subRun: 1 event: 52625 at 17-Nov-2025 22:01:12 UTC
Begin processing the 26th record. run: 1114 subRun: 1 event: 52626 at 17-Nov-2025 22:01:15 UTC
Begin processing the 27th record. run: 1114 subRun: 1 event: 52627 at 17-Nov-2025 22:01:18 UTC
Begin processing the 28th record. run: 1114 subRun: 1 event: 52628 at 17-Nov-2025 22:01:21 UTC
Begin processing the 29th record. run: 1114 subRun: 1 event: 52629 at 17-Nov-2025 22:01:24 UTC
Begin processing the 30th record. run: 1114 subRun: 1 event: 52630 at 17-Nov-2025 22:01:27 UTC
Begin processing the 31st record. run: 1114 subRun: 1 event: 52631 at 17-Nov-2025 22:01:29 UTC
Begin processing the 32nd record. run: 1114 subRun: 1 event: 52632 at 17-Nov-2025 22:01:32 UTC
Begin processing the 33rd record. run: 1114 subRun: 1 event: 52633 at 17-Nov-2025 22:01:35 UTC
Begin processing the 34th record. run: 1114 subRun: 1 event: 52634 at 17-Nov-2025 22:01:38 UTC
Begin processing the 35th record. run: 1114 subRun: 1 event: 52635 at 17-Nov-2025 22:01:41 UTC
Begin processing the 36th record. run: 1114 subRun: 1 event: 52636 at 17-Nov-2025 22:01:44 UTC
Begin processing the 37th record. run: 1114 subRun: 1 event: 52637 at 17-Nov-2025 22:01:47 UTC
Begin processing the 38th record. run: 1114 subRun: 1 event: 52638 at 17-Nov-2025 22:01:50 UTC
Begin processing the 39th record. run: 1114 subRun: 1 event: 52639 at 17-Nov-2025 22:01:53 UTC
Begin processing the 40th record. run: 1114 subRun: 1 event: 52640 at 17-Nov-2025 22:01:56 UTC
Begin processing the 41st record. run: 1114 subRun: 1 event: 52641 at 17-Nov-2025 22:01:59 UTC
Begin processing the 42nd record. run: 1114 subRun: 1 event: 52642 at 17-Nov-2025 22:02:02 UTC
Begin processing the 43rd record. run: 1114 subRun: 1 event: 52643 at 17-Nov-2025 22:02:05 UTC
Begin processing the 44th record. run: 1114 subRun: 1 event: 52644 at 17-Nov-2025 22:02:08 UTC
Begin processing the 45th record. run: 1114 subRun: 1 event: 52645 at 17-Nov-2025 22:02:11 UTC
Begin processing the 46th record. run: 1114 subRun: 1 event: 52646 at 17-Nov-2025 22:02:14 UTC
Begin processing the 47th record. run: 1114 subRun: 1 event: 52647 at 17-Nov-2025 22:02:18 UTC
Begin processing the 48th record. run: 1114 subRun: 1 event: 52648 at 17-Nov-2025 22:02:22 UTC
Begin processing the 49th record. run: 1114 subRun: 1 event: 52649 at 17-Nov-2025 22:02:25 UTC
Begin processing the 50th record. run: 1114 subRun: 1 event: 52650 at 17-Nov-2025 22:02:31 UTC
Begin processing the 51st record. run: 1114 subRun: 1 event: 52651 at 17-Nov-2025 22:02:33 UTC
Begin processing the 52nd record. run: 1114 subRun: 1 event: 52652 at 17-Nov-2025 22:02:36 UTC
Begin processing the 53rd record. run: 1114 subRun: 1 event: 52653 at 17-Nov-2025 22:02:39 UTC
Begin processing the 54th record. run: 1114 subRun: 1 event: 52654 at 17-Nov-2025 22:02:42 UTC
Begin processing the 55th record. run: 1114 subRun: 1 event: 52655 at 17-Nov-2025 22:02:45 UTC
Begin processing the 56th record. run: 1114 subRun: 1 event: 52656 at 17-Nov-2025 22:02:48 UTC
Begin processing the 57th record. run: 1114 subRun: 1 event: 52657 at 17-Nov-2025 22:02:51 UTC
Begin processing the 58th record. run: 1114 subRun: 1 event: 52658 at 17-Nov-2025 22:02:54 UTC
Begin processing the 59th record. run: 1114 subRun: 1 event: 52659 at 17-Nov-2025 22:02:57 UTC
Begin processing the 60th record. run: 1114 subRun: 1 event: 52660 at 17-Nov-2025 22:02:59 UTC
Begin processing the 61st record. run: 1114 subRun: 1 event: 52661 at 17-Nov-2025 22:03:03 UTC
Begin processing the 62nd record. run: 1114 subRun: 1 event: 52662 at 17-Nov-2025 22:03:05 UTC
Begin processing the 63rd record. run: 1114 subRun: 1 event: 52663 at 17-Nov-2025 22:03:08 UTC
Begin processing the 64th record. run: 1114 subRun: 1 event: 52664 at 17-Nov-2025 22:03:11 UTC
Begin processing the 65th record. run: 1114 subRun: 1 event: 52665 at 17-Nov-2025 22:03:15 UTC
Begin processing the 66th record. run: 1114 subRun: 1 event: 52666 at 17-Nov-2025 22:03:18 UTC
Begin processing the 67th record. run: 1114 subRun: 1 event: 52667 at 17-Nov-2025 22:03:20 UTC
Begin processing the 68th record. run: 1114 subRun: 1 event: 52668 at 17-Nov-2025 22:03:23 UTC
Begin processing the 69th record. run: 1114 subRun: 1 event: 52669 at 17-Nov-2025 22:03:26 UTC
Begin processing the 70th record. run: 1114 subRun: 1 event: 52670 at 17-Nov-2025 22:03:29 UTC
Begin processing the 71st record. run: 1114 subRun: 1 event: 52671 at 17-Nov-2025 22:03:32 UTC
Begin processing the 72nd record. run: 1114 subRun: 1 event: 52672 at 17-Nov-2025 22:03:35 UTC
Begin processing the 73rd record. run: 1114 subRun: 1 event: 52673 at 17-Nov-2025 22:03:38 UTC
Begin processing the 74th record. run: 1114 subRun: 1 event: 52674 at 17-Nov-2025 22:03:41 UTC
Begin processing the 75th record. run: 1114 subRun: 1 event: 52675 at 17-Nov-2025 22:03:44 UTC
Begin processing the 76th record. run: 1114 subRun: 1 event: 52676 at 17-Nov-2025 22:03:47 UTC
Begin processing the 77th record. run: 1114 subRun: 1 event: 52677 at 17-Nov-2025 22:03:50 UTC
Begin processing the 78th record. run: 1114 subRun: 1 event: 52678 at 17-Nov-2025 22:03:53 UTC
Begin processing the 79th record. run: 1114 subRun: 1 event: 52679 at 17-Nov-2025 22:03:56 UTC
Begin processing the 80th record. run: 1114 subRun: 1 event: 52680 at 17-Nov-2025 22:03:59 UTC
Begin processing the 81st record. run: 1114 subRun: 1 event: 52681 at 17-Nov-2025 22:04:02 UTC
Begin processing the 82nd record. run: 1114 subRun: 1 event: 52682 at 17-Nov-2025 22:04:05 UTC
Begin processing the 83rd record. run: 1114 subRun: 1 event: 52683 at 17-Nov-2025 22:04:08 UTC
Begin processing the 84th record. run: 1114 subRun: 1 event: 52684 at 17-Nov-2025 22:04:11 UTC
Begin processing the 85th record. run: 1114 subRun: 1 event: 52685 at 17-Nov-2025 22:04:14 UTC
Begin processing the 86th record. run: 1114 subRun: 1 event: 52686 at 17-Nov-2025 22:04:17 UTC
Begin processing the 87th record. run: 1114 subRun: 1 event: 52687 at 17-Nov-2025 22:04:20 UTC
Begin processing the 88th record. run: 1114 subRun: 1 event: 52688 at 17-Nov-2025 22:04:22 UTC
Begin processing the 89th record. run: 1114 subRun: 1 event: 52689 at 17-Nov-2025 22:04:25 UTC
Begin processing the 90th record. run: 1114 subRun: 1 event: 52690 at 17-Nov-2025 22:04:28 UTC
Begin processing the 91st record. run: 1114 subRun: 1 event: 52691 at 17-Nov-2025 22:04:31 UTC
Begin processing the 92nd record. run: 1114 subRun: 1 event: 52692 at 17-Nov-2025 22:04:34 UTC
Begin processing the 93rd record. run: 1114 subRun: 1 event: 52693 at 17-Nov-2025 22:04:37 UTC
Begin processing the 94th record. run: 1114 subRun: 1 event: 52694 at 17-Nov-2025 22:04:40 UTC
Begin processing the 95th record. run: 1114 subRun: 1 event: 52695 at 17-Nov-2025 22:04:42 UTC
Begin processing the 96th record. run: 1114 subRun: 1 event: 52696 at 17-Nov-2025 22:04:46 UTC
Begin processing the 97th record. run: 1114 subRun: 1 event: 52697 at 17-Nov-2025 22:04:49 UTC
Begin processing the 98th record. run: 1114 subRun: 1 event: 52698 at 17-Nov-2025 22:04:52 UTC
Begin processing the 99th record. run: 1114 subRun: 1 event: 52699 at 17-Nov-2025 22:04:55 UTC
Begin processing the 100th record. run: 1114 subRun: 1 event: 52700 at 17-Nov-2025 22:04:59 UTC
17-Nov-2025 22:05:02 UTC  Closed output file "nu_dune10kt_1x2x6_1114_526_20230827T153337Z_gen_g4_detsim_hitreco__20240223T011820Z_reco2_reco2.root"
17-Nov-2025 22:05:02 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/1c/b8/nu_dune10kt_1x2x6_1114_526_20230827T153337Z_gen_g4_detsim_hitreco__20240223T011820Z_reco2.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                      2.5904        3.00033       5.96414       2.81962      0.537203        100    
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.00241083     0.0290036     0.203118      0.0207341     0.0257679       100    
reco:pandora:StandardPandora                    2.57654       2.9654        5.92361       2.78811      0.535421        100    
[art]:TriggerResults:TriggerResultInserter    1.4591e-05    2.44844e-05   0.000159081   2.25005e-05   1.5774e-05       100    
end_path:out1:RootOutput                       3.18e-06     4.61271e-06    2.277e-05     4.325e-06    2.36465e-06      100    
end_path:out1:RootOutput(write)                0.0013226    0.00567838     0.0362552    0.00420974    0.00504077       100    
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 2265.7 MB
  Peak resident set size usage (VmHWM): 1272.33 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
mv: cannot stat 'CheatingKalmanSplittingV.root': No such file or directory
mv: cannot stat 'CheatingKalmanSplittingW.root': No such file or directory
justIN time: 2026-02-04 09:12:55 UTC       justIN version: 01.06.00