justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 236994.0@dunegpschedd02.fnal.gov

Jobsub ID236994.0@dunegpschedd02.fnal.gov
Workflow ID9556
Stage ID1
User nameimawby@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit43200 (12 hours)
Submitted time2025-11-03 22:23:21
SiteUS_UChicago
EntryEngage_US_MWT2_uiuc_condce_mcore
Last heartbeat2025-11-03 22:32:08
From worker nodeHostnamemwt2-c019.campuscluster.illinois.edu
cpuinfoAMD EPYC 7302 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-03 22:24:54
Input filesfardet-hd:nu_dune10kt_1x2x6_1431_106_20230828T033335Z_gen_g4_detsim_hitreco__20240223T001114Z_reco2.root
JobscriptExit code0
Real time6m (416s)
CPU time5m (308s = 74%)
Max RSS bytes1314185216 (1253 MiB)
Outputting started2025-11-03 22:31:51
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/imawby/splitting_nu_1/fnal/09556/1/001/CheatingKalmanSplittingU_nu_dune10kt_1x2x6_1431_106_20230828T033335Z_gen_g4_detsim_hitreco__20240223T001114Z_reco2.root
Finished2025-11-03 22:32:08
Saved logsjustin-logs:236994.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

nfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/e9/14/nu_dune10kt_1x2x6_1431_106_20230828T033335Z_gen_g4_detsim_hitreco__20240223T001114Z_reco2.root
lar exit code 0
=== Start last 100 lines of lar log file ===
Begin processing the 23rd record. run: 1431 subRun: 1 event: 10623 at 03-Nov-2025 16:28:06 CST
Begin processing the 24th record. run: 1431 subRun: 1 event: 10624 at 03-Nov-2025 16:28:09 CST
Begin processing the 25th record. run: 1431 subRun: 1 event: 10625 at 03-Nov-2025 16:28:12 CST
Begin processing the 26th record. run: 1431 subRun: 1 event: 10626 at 03-Nov-2025 16:28:14 CST
Begin processing the 27th record. run: 1431 subRun: 1 event: 10627 at 03-Nov-2025 16:28:17 CST
Begin processing the 28th record. run: 1431 subRun: 1 event: 10628 at 03-Nov-2025 16:28:20 CST
Begin processing the 29th record. run: 1431 subRun: 1 event: 10629 at 03-Nov-2025 16:28:23 CST
Begin processing the 30th record. run: 1431 subRun: 1 event: 10630 at 03-Nov-2025 16:28:25 CST
Begin processing the 31st record. run: 1431 subRun: 1 event: 10631 at 03-Nov-2025 16:28:28 CST
Begin processing the 32nd record. run: 1431 subRun: 1 event: 10632 at 03-Nov-2025 16:28:30 CST
Begin processing the 33rd record. run: 1431 subRun: 1 event: 10633 at 03-Nov-2025 16:28:33 CST
Begin processing the 34th record. run: 1431 subRun: 1 event: 10634 at 03-Nov-2025 16:28:36 CST
Begin processing the 35th record. run: 1431 subRun: 1 event: 10635 at 03-Nov-2025 16:28:39 CST
Begin processing the 36th record. run: 1431 subRun: 1 event: 10636 at 03-Nov-2025 16:28:41 CST
Begin processing the 37th record. run: 1431 subRun: 1 event: 10637 at 03-Nov-2025 16:28:44 CST
Begin processing the 38th record. run: 1431 subRun: 1 event: 10638 at 03-Nov-2025 16:28:47 CST
Begin processing the 39th record. run: 1431 subRun: 1 event: 10639 at 03-Nov-2025 16:28:50 CST
Begin processing the 40th record. run: 1431 subRun: 1 event: 10640 at 03-Nov-2025 16:28:52 CST
Begin processing the 41st record. run: 1431 subRun: 1 event: 10641 at 03-Nov-2025 16:28:58 CST
Begin processing the 42nd record. run: 1431 subRun: 1 event: 10642 at 03-Nov-2025 16:29:01 CST
Begin processing the 43rd record. run: 1431 subRun: 1 event: 10643 at 03-Nov-2025 16:29:04 CST
Begin processing the 44th record. run: 1431 subRun: 1 event: 10644 at 03-Nov-2025 16:29:06 CST
Begin processing the 45th record. run: 1431 subRun: 1 event: 10645 at 03-Nov-2025 16:29:10 CST
Begin processing the 46th record. run: 1431 subRun: 1 event: 10646 at 03-Nov-2025 16:29:13 CST
Begin processing the 47th record. run: 1431 subRun: 1 event: 10647 at 03-Nov-2025 16:29:15 CST
Begin processing the 48th record. run: 1431 subRun: 1 event: 10648 at 03-Nov-2025 16:29:18 CST
Begin processing the 49th record. run: 1431 subRun: 1 event: 10649 at 03-Nov-2025 16:29:21 CST
Begin processing the 50th record. run: 1431 subRun: 1 event: 10650 at 03-Nov-2025 16:29:24 CST
Begin processing the 51st record. run: 1431 subRun: 1 event: 10651 at 03-Nov-2025 16:29:26 CST
Begin processing the 52nd record. run: 1431 subRun: 1 event: 10652 at 03-Nov-2025 16:29:29 CST
Begin processing the 53rd record. run: 1431 subRun: 1 event: 10653 at 03-Nov-2025 16:29:32 CST
Begin processing the 54th record. run: 1431 subRun: 1 event: 10654 at 03-Nov-2025 16:29:35 CST
Begin processing the 55th record. run: 1431 subRun: 1 event: 10655 at 03-Nov-2025 16:29:37 CST
Begin processing the 56th record. run: 1431 subRun: 1 event: 10656 at 03-Nov-2025 16:29:40 CST
Begin processing the 57th record. run: 1431 subRun: 1 event: 10657 at 03-Nov-2025 16:29:42 CST
Begin processing the 58th record. run: 1431 subRun: 1 event: 10658 at 03-Nov-2025 16:29:45 CST
Begin processing the 59th record. run: 1431 subRun: 1 event: 10659 at 03-Nov-2025 16:29:48 CST
Begin processing the 60th record. run: 1431 subRun: 1 event: 10660 at 03-Nov-2025 16:29:51 CST
Begin processing the 61st record. run: 1431 subRun: 1 event: 10661 at 03-Nov-2025 16:29:54 CST
Begin processing the 62nd record. run: 1431 subRun: 1 event: 10662 at 03-Nov-2025 16:29:57 CST
Begin processing the 63rd record. run: 1431 subRun: 1 event: 10663 at 03-Nov-2025 16:29:59 CST
Begin processing the 64th record. run: 1431 subRun: 1 event: 10664 at 03-Nov-2025 16:30:02 CST
Begin processing the 65th record. run: 1431 subRun: 1 event: 10665 at 03-Nov-2025 16:30:05 CST
Begin processing the 66th record. run: 1431 subRun: 1 event: 10666 at 03-Nov-2025 16:30:07 CST
Begin processing the 67th record. run: 1431 subRun: 1 event: 10667 at 03-Nov-2025 16:30:10 CST
Begin processing the 68th record. run: 1431 subRun: 1 event: 10668 at 03-Nov-2025 16:30:16 CST
Begin processing the 69th record. run: 1431 subRun: 1 event: 10669 at 03-Nov-2025 16:30:18 CST
Begin processing the 70th record. run: 1431 subRun: 1 event: 10670 at 03-Nov-2025 16:30:21 CST
Begin processing the 71st record. run: 1431 subRun: 1 event: 10671 at 03-Nov-2025 16:30:23 CST
Begin processing the 72nd record. run: 1431 subRun: 1 event: 10672 at 03-Nov-2025 16:30:26 CST
Begin processing the 73rd record. run: 1431 subRun: 1 event: 10673 at 03-Nov-2025 16:30:29 CST
Begin processing the 74th record. run: 1431 subRun: 1 event: 10674 at 03-Nov-2025 16:30:32 CST
Begin processing the 75th record. run: 1431 subRun: 1 event: 10675 at 03-Nov-2025 16:30:35 CST
Begin processing the 76th record. run: 1431 subRun: 1 event: 10676 at 03-Nov-2025 16:30:38 CST
Begin processing the 77th record. run: 1431 subRun: 1 event: 10677 at 03-Nov-2025 16:30:40 CST
Begin processing the 78th record. run: 1431 subRun: 1 event: 10678 at 03-Nov-2025 16:30:43 CST
Begin processing the 79th record. run: 1431 subRun: 1 event: 10679 at 03-Nov-2025 16:30:47 CST
Begin processing the 80th record. run: 1431 subRun: 1 event: 10680 at 03-Nov-2025 16:30:50 CST
Begin processing the 81st record. run: 1431 subRun: 1 event: 10681 at 03-Nov-2025 16:30:53 CST
Begin processing the 82nd record. run: 1431 subRun: 1 event: 10682 at 03-Nov-2025 16:30:56 CST
Begin processing the 83rd record. run: 1431 subRun: 1 event: 10683 at 03-Nov-2025 16:30:58 CST
Begin processing the 84th record. run: 1431 subRun: 1 event: 10684 at 03-Nov-2025 16:31:01 CST
Begin processing the 85th record. run: 1431 subRun: 1 event: 10685 at 03-Nov-2025 16:31:03 CST
Begin processing the 86th record. run: 1431 subRun: 1 event: 10686 at 03-Nov-2025 16:31:06 CST
Begin processing the 87th record. run: 1431 subRun: 1 event: 10687 at 03-Nov-2025 16:31:09 CST
Begin processing the 88th record. run: 1431 subRun: 1 event: 10688 at 03-Nov-2025 16:31:11 CST
Begin processing the 89th record. run: 1431 subRun: 1 event: 10689 at 03-Nov-2025 16:31:14 CST
Begin processing the 90th record. run: 1431 subRun: 1 event: 10690 at 03-Nov-2025 16:31:19 CST
Begin processing the 91st record. run: 1431 subRun: 1 event: 10691 at 03-Nov-2025 16:31:21 CST
Begin processing the 92nd record. run: 1431 subRun: 1 event: 10692 at 03-Nov-2025 16:31:24 CST
Begin processing the 93rd record. run: 1431 subRun: 1 event: 10693 at 03-Nov-2025 16:31:27 CST
Begin processing the 94th record. run: 1431 subRun: 1 event: 10694 at 03-Nov-2025 16:31:30 CST
Begin processing the 95th record. run: 1431 subRun: 1 event: 10695 at 03-Nov-2025 16:31:33 CST
Begin processing the 96th record. run: 1431 subRun: 1 event: 10696 at 03-Nov-2025 16:31:35 CST
Begin processing the 97th record. run: 1431 subRun: 1 event: 10697 at 03-Nov-2025 16:31:37 CST
Begin processing the 98th record. run: 1431 subRun: 1 event: 10698 at 03-Nov-2025 16:31:41 CST
Begin processing the 99th record. run: 1431 subRun: 1 event: 10699 at 03-Nov-2025 16:31:44 CST
Begin processing the 100th record. run: 1431 subRun: 1 event: 10700 at 03-Nov-2025 16:31:47 CST
03-Nov-2025 16:31:50 CST  Closed output file "nu_dune10kt_1x2x6_1431_106_20230828T033335Z_gen_g4_detsim_hitreco__20240223T001114Z_reco2_reco2.root"
03-Nov-2025 16:31:50 CST  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/e9/14/nu_dune10kt_1x2x6_1431_106_20230828T033335Z_gen_g4_detsim_hitreco__20240223T001114Z_reco2.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                      2.00801       2.60745       5.78205        2.498       0.547522        100    
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                         0.0158346     0.0504417     0.616947      0.0353395     0.0793871       100    
reco:pandora:StandardPandora                    1.97022       2.55173       5.75524       2.45607      0.542365        100    
[art]:TriggerResults:TriggerResultInserter    1.2073e-05    2.09809e-05   6.1275e-05    2.0789e-05    6.06865e-06      100    
end_path:out1:RootOutput                       4.097e-06    7.4101e-06    2.2613e-05    7.2935e-06    2.31742e-06      100    
end_path:out1:RootOutput(write)               0.00117134    0.00500883     0.0172129    0.00458145    0.00287743       100    
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 2302.03 MB
  Peak resident set size usage (VmHWM): 1314.19 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
mv: cannot stat 'CheatingKalmanSplittingV.root': No such file or directory
mv: cannot stat 'CheatingKalmanSplittingW.root': No such file or directory
justIN time: 2026-02-11 12:20:18 UTC       justIN version: 01.06.00