justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 236785.84@dunegpschedd02.fnal.gov

Jobsub ID236785.84@dunegpschedd02.fnal.gov
Workflow ID9556
Stage ID1
User nameimawby@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit43200 (12 hours)
Submitted time2025-11-03 17:23:05
SiteUS_Wisconsin
EntryHCCHTPC_US_Wisconsin_osg01_rhel7
Last heartbeat2025-11-03 17:42:20
From worker nodeHostnamee2461
cpuinfoAMD EPYC 7452 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit82800 (23 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-03 17:35:16
Input filesfardet-hd:nu_dune10kt_1x2x6_1427_514_20230827T213352Z_gen_g4_detsim_hitreco__20240223T011946Z_reco2.root
JobscriptExit code0
Real time6m (413s)
CPU time3m (236s = 57%)
Max RSS bytes1257267200 (1199 MiB)
Outputting started2025-11-03 17:42:09
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/imawby/splitting_nu_1/fnal/09556/1/001/CheatingKalmanSplittingU_nu_dune10kt_1x2x6_1427_514_20230827T213352Z_gen_g4_detsim_hitreco__20240223T011946Z_reco2.root
Finished2025-11-03 17:42:20
Saved logsjustin-logs:236785.84-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

nfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/ff/68/nu_dune10kt_1x2x6_1427_514_20230827T213352Z_gen_g4_detsim_hitreco__20240223T011946Z_reco2.root
lar exit code 0
=== Start last 100 lines of lar log file ===
Begin processing the 23rd record. run: 1427 subRun: 1 event: 51423 at 03-Nov-2025 11:38:47 CST
Begin processing the 24th record. run: 1427 subRun: 1 event: 51424 at 03-Nov-2025 11:38:51 CST
Begin processing the 25th record. run: 1427 subRun: 1 event: 51425 at 03-Nov-2025 11:38:54 CST
Begin processing the 26th record. run: 1427 subRun: 1 event: 51426 at 03-Nov-2025 11:38:56 CST
Begin processing the 27th record. run: 1427 subRun: 1 event: 51427 at 03-Nov-2025 11:38:59 CST
Begin processing the 28th record. run: 1427 subRun: 1 event: 51428 at 03-Nov-2025 11:39:01 CST
Begin processing the 29th record. run: 1427 subRun: 1 event: 51429 at 03-Nov-2025 11:39:03 CST
Begin processing the 30th record. run: 1427 subRun: 1 event: 51430 at 03-Nov-2025 11:39:06 CST
Begin processing the 31st record. run: 1427 subRun: 1 event: 51431 at 03-Nov-2025 11:39:08 CST
Begin processing the 32nd record. run: 1427 subRun: 1 event: 51432 at 03-Nov-2025 11:39:10 CST
Begin processing the 33rd record. run: 1427 subRun: 1 event: 51433 at 03-Nov-2025 11:39:12 CST
Begin processing the 34th record. run: 1427 subRun: 1 event: 51434 at 03-Nov-2025 11:39:15 CST
Begin processing the 35th record. run: 1427 subRun: 1 event: 51435 at 03-Nov-2025 11:39:17 CST
Begin processing the 36th record. run: 1427 subRun: 1 event: 51436 at 03-Nov-2025 11:39:19 CST
Begin processing the 37th record. run: 1427 subRun: 1 event: 51437 at 03-Nov-2025 11:39:22 CST
Begin processing the 38th record. run: 1427 subRun: 1 event: 51438 at 03-Nov-2025 11:39:24 CST
Begin processing the 39th record. run: 1427 subRun: 1 event: 51439 at 03-Nov-2025 11:39:27 CST
Begin processing the 40th record. run: 1427 subRun: 1 event: 51440 at 03-Nov-2025 11:39:29 CST
Begin processing the 41st record. run: 1427 subRun: 1 event: 51441 at 03-Nov-2025 11:39:31 CST
Begin processing the 42nd record. run: 1427 subRun: 1 event: 51442 at 03-Nov-2025 11:39:34 CST
Begin processing the 43rd record. run: 1427 subRun: 1 event: 51443 at 03-Nov-2025 11:39:36 CST
Begin processing the 44th record. run: 1427 subRun: 1 event: 51444 at 03-Nov-2025 11:39:38 CST
Begin processing the 45th record. run: 1427 subRun: 1 event: 51445 at 03-Nov-2025 11:39:40 CST
Begin processing the 46th record. run: 1427 subRun: 1 event: 51446 at 03-Nov-2025 11:39:42 CST
Begin processing the 47th record. run: 1427 subRun: 1 event: 51447 at 03-Nov-2025 11:39:46 CST
Begin processing the 48th record. run: 1427 subRun: 1 event: 51448 at 03-Nov-2025 11:39:48 CST
Begin processing the 49th record. run: 1427 subRun: 1 event: 51449 at 03-Nov-2025 11:39:51 CST
Begin processing the 50th record. run: 1427 subRun: 1 event: 51450 at 03-Nov-2025 11:39:54 CST
Begin processing the 51st record. run: 1427 subRun: 1 event: 51451 at 03-Nov-2025 11:39:57 CST
Begin processing the 52nd record. run: 1427 subRun: 1 event: 51452 at 03-Nov-2025 11:39:59 CST
Begin processing the 53rd record. run: 1427 subRun: 1 event: 51453 at 03-Nov-2025 11:40:02 CST
Begin processing the 54th record. run: 1427 subRun: 1 event: 51454 at 03-Nov-2025 11:40:05 CST
Begin processing the 55th record. run: 1427 subRun: 1 event: 51455 at 03-Nov-2025 11:40:07 CST
Begin processing the 56th record. run: 1427 subRun: 1 event: 51456 at 03-Nov-2025 11:40:10 CST
Begin processing the 57th record. run: 1427 subRun: 1 event: 51457 at 03-Nov-2025 11:40:13 CST
Begin processing the 58th record. run: 1427 subRun: 1 event: 51458 at 03-Nov-2025 11:40:15 CST
Begin processing the 59th record. run: 1427 subRun: 1 event: 51459 at 03-Nov-2025 11:40:18 CST
Begin processing the 60th record. run: 1427 subRun: 1 event: 51460 at 03-Nov-2025 11:40:20 CST
Begin processing the 61st record. run: 1427 subRun: 1 event: 51461 at 03-Nov-2025 11:40:23 CST
Begin processing the 62nd record. run: 1427 subRun: 1 event: 51462 at 03-Nov-2025 11:40:25 CST
Begin processing the 63rd record. run: 1427 subRun: 1 event: 51463 at 03-Nov-2025 11:40:28 CST
Begin processing the 64th record. run: 1427 subRun: 1 event: 51464 at 03-Nov-2025 11:40:30 CST
Begin processing the 65th record. run: 1427 subRun: 1 event: 51465 at 03-Nov-2025 11:40:32 CST
Begin processing the 66th record. run: 1427 subRun: 1 event: 51466 at 03-Nov-2025 11:40:35 CST
Begin processing the 67th record. run: 1427 subRun: 1 event: 51467 at 03-Nov-2025 11:40:37 CST
Begin processing the 68th record. run: 1427 subRun: 1 event: 51468 at 03-Nov-2025 11:40:40 CST
Begin processing the 69th record. run: 1427 subRun: 1 event: 51469 at 03-Nov-2025 11:40:42 CST
Begin processing the 70th record. run: 1427 subRun: 1 event: 51470 at 03-Nov-2025 11:40:44 CST
Begin processing the 71st record. run: 1427 subRun: 1 event: 51471 at 03-Nov-2025 11:40:47 CST
Begin processing the 72nd record. run: 1427 subRun: 1 event: 51472 at 03-Nov-2025 11:40:49 CST
Begin processing the 73rd record. run: 1427 subRun: 1 event: 51473 at 03-Nov-2025 11:40:53 CST
Begin processing the 74th record. run: 1427 subRun: 1 event: 51474 at 03-Nov-2025 11:40:56 CST
Begin processing the 75th record. run: 1427 subRun: 1 event: 51475 at 03-Nov-2025 11:40:59 CST
Begin processing the 76th record. run: 1427 subRun: 1 event: 51476 at 03-Nov-2025 11:41:01 CST
Begin processing the 77th record. run: 1427 subRun: 1 event: 51477 at 03-Nov-2025 11:41:05 CST
Begin processing the 78th record. run: 1427 subRun: 1 event: 51478 at 03-Nov-2025 11:41:07 CST
Begin processing the 79th record. run: 1427 subRun: 1 event: 51479 at 03-Nov-2025 11:41:10 CST
Begin processing the 80th record. run: 1427 subRun: 1 event: 51480 at 03-Nov-2025 11:41:13 CST
Begin processing the 81st record. run: 1427 subRun: 1 event: 51481 at 03-Nov-2025 11:41:15 CST
Begin processing the 82nd record. run: 1427 subRun: 1 event: 51482 at 03-Nov-2025 11:41:18 CST
Begin processing the 83rd record. run: 1427 subRun: 1 event: 51483 at 03-Nov-2025 11:41:20 CST
Begin processing the 84th record. run: 1427 subRun: 1 event: 51484 at 03-Nov-2025 11:41:23 CST
Begin processing the 85th record. run: 1427 subRun: 1 event: 51485 at 03-Nov-2025 11:41:25 CST
Begin processing the 86th record. run: 1427 subRun: 1 event: 51486 at 03-Nov-2025 11:41:28 CST
Begin processing the 87th record. run: 1427 subRun: 1 event: 51487 at 03-Nov-2025 11:41:31 CST
Begin processing the 88th record. run: 1427 subRun: 1 event: 51488 at 03-Nov-2025 11:41:34 CST
Begin processing the 89th record. run: 1427 subRun: 1 event: 51489 at 03-Nov-2025 11:41:36 CST
Begin processing the 90th record. run: 1427 subRun: 1 event: 51490 at 03-Nov-2025 11:41:39 CST
Begin processing the 91st record. run: 1427 subRun: 1 event: 51491 at 03-Nov-2025 11:41:41 CST
Begin processing the 92nd record. run: 1427 subRun: 1 event: 51492 at 03-Nov-2025 11:41:44 CST
Begin processing the 93rd record. run: 1427 subRun: 1 event: 51493 at 03-Nov-2025 11:41:47 CST
Begin processing the 94th record. run: 1427 subRun: 1 event: 51494 at 03-Nov-2025 11:41:50 CST
Begin processing the 95th record. run: 1427 subRun: 1 event: 51495 at 03-Nov-2025 11:41:52 CST
Begin processing the 96th record. run: 1427 subRun: 1 event: 51496 at 03-Nov-2025 11:41:55 CST
Begin processing the 97th record. run: 1427 subRun: 1 event: 51497 at 03-Nov-2025 11:41:57 CST
Begin processing the 98th record. run: 1427 subRun: 1 event: 51498 at 03-Nov-2025 11:42:00 CST
Begin processing the 99th record. run: 1427 subRun: 1 event: 51499 at 03-Nov-2025 11:42:03 CST
Begin processing the 100th record. run: 1427 subRun: 1 event: 51500 at 03-Nov-2025 11:42:06 CST
03-Nov-2025 11:42:09 CST  Closed output file "nu_dune10kt_1x2x6_1427_514_20230827T213352Z_gen_g4_detsim_hitreco__20240223T011946Z_reco2_reco2.root"
03-Nov-2025 11:42:09 CST  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/ff/68/nu_dune10kt_1x2x6_1427_514_20230827T213352Z_gen_g4_detsim_hitreco__20240223T011946Z_reco2.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                      1.83038       2.14726       3.42119       2.08549      0.275127        100    
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                         0.0331692     0.0681852     0.622143       0.0564       0.0644195       100    
reco:pandora:StandardPandora                    1.74908       2.07569       3.38378       2.02008      0.265954        100    
[art]:TriggerResults:TriggerResultInserter    1.0059e-05    1.35949e-05   3.6489e-05    1.22325e-05   3.96157e-06      100    
end_path:out1:RootOutput                       2.135e-06    3.27434e-06   1.4557e-05     2.855e-06    1.37336e-06      100    
end_path:out1:RootOutput(write)               0.000833313   0.00320273    0.00988475    0.00279003     0.0019071       100    
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 2248.08 MB
  Peak resident set size usage (VmHWM): 1257.27 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
mv: cannot stat 'CheatingKalmanSplittingV.root': No such file or directory
mv: cannot stat 'CheatingKalmanSplittingW.root': No such file or directory
justIN time: 2025-11-04 01:41:35 UTC       justIN version: 01.05.01