justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 253477.1@dunegpschedd01.fnal.gov

Jobsub ID253477.1@dunegpschedd01.fnal.gov
Workflow ID10317
Stage ID1
User nameimawby@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes1048576000 (1000 MiB)
Wall seconds limit7200 (2 hours)
Submitted time2025-11-17 22:39:02
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-11-17 22:43:27
From worker nodeHostnamedunegli-7795752-0-fnpc23024.fnal.gov
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes1048576000 (1000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-17 22:39:23
Input filesfardet-hd:nu_dune10kt_1x2x6_1098_324_20230826T012211Z_gen_g4_detsim_hitreco__20240222T214640Z_reco2.root
JobscriptExit code0
Real time3m (232s)
CPU time3m (198s = 85%)
Max RSS bytes1420464128 (1354 MiB)
Outputting started2025-11-17 22:43:16
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/imawby/splitting_nu_5/fnal/10317/1/001/CheatingKalmanSplittingU_nu_dune10kt_1x2x6_1098_324_20230826T012211Z_gen_g4_detsim_hitreco__20240222T214640Z_reco2.root
Finished2025-11-17 22:43:27
Saved logsjustin-logs:253477.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     (HTCondor job logs unavailable)

Jobscript log (last 10,000 characters)

nfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/5f/55/nu_dune10kt_1x2x6_1098_324_20230826T012211Z_gen_g4_detsim_hitreco__20240222T214640Z_reco2.root
lar exit code 0
=== Start last 100 lines of lar log file ===
Begin processing the 23rd record. run: 1098 subRun: 1 event: 32423 at 17-Nov-2025 22:40:55 UTC
Begin processing the 24th record. run: 1098 subRun: 1 event: 32424 at 17-Nov-2025 22:40:56 UTC
Begin processing the 25th record. run: 1098 subRun: 1 event: 32425 at 17-Nov-2025 22:40:58 UTC
Begin processing the 26th record. run: 1098 subRun: 1 event: 32426 at 17-Nov-2025 22:40:59 UTC
Begin processing the 27th record. run: 1098 subRun: 1 event: 32427 at 17-Nov-2025 22:41:01 UTC
Begin processing the 28th record. run: 1098 subRun: 1 event: 32428 at 17-Nov-2025 22:41:02 UTC
Begin processing the 29th record. run: 1098 subRun: 1 event: 32429 at 17-Nov-2025 22:41:04 UTC
Begin processing the 30th record. run: 1098 subRun: 1 event: 32430 at 17-Nov-2025 22:41:05 UTC
Begin processing the 31st record. run: 1098 subRun: 1 event: 32431 at 17-Nov-2025 22:41:17 UTC
Begin processing the 32nd record. run: 1098 subRun: 1 event: 32432 at 17-Nov-2025 22:41:20 UTC
Begin processing the 33rd record. run: 1098 subRun: 1 event: 32433 at 17-Nov-2025 22:41:21 UTC
Begin processing the 34th record. run: 1098 subRun: 1 event: 32434 at 17-Nov-2025 22:41:23 UTC
Begin processing the 35th record. run: 1098 subRun: 1 event: 32435 at 17-Nov-2025 22:41:24 UTC
Begin processing the 36th record. run: 1098 subRun: 1 event: 32436 at 17-Nov-2025 22:41:26 UTC
Begin processing the 37th record. run: 1098 subRun: 1 event: 32437 at 17-Nov-2025 22:41:27 UTC
Begin processing the 38th record. run: 1098 subRun: 1 event: 32438 at 17-Nov-2025 22:41:28 UTC
Begin processing the 39th record. run: 1098 subRun: 1 event: 32439 at 17-Nov-2025 22:41:30 UTC
Begin processing the 40th record. run: 1098 subRun: 1 event: 32440 at 17-Nov-2025 22:41:31 UTC
Begin processing the 41st record. run: 1098 subRun: 1 event: 32441 at 17-Nov-2025 22:41:33 UTC
Begin processing the 42nd record. run: 1098 subRun: 1 event: 32442 at 17-Nov-2025 22:41:34 UTC
Begin processing the 43rd record. run: 1098 subRun: 1 event: 32443 at 17-Nov-2025 22:41:35 UTC
Begin processing the 44th record. run: 1098 subRun: 1 event: 32444 at 17-Nov-2025 22:41:37 UTC
Begin processing the 45th record. run: 1098 subRun: 1 event: 32445 at 17-Nov-2025 22:41:39 UTC
Begin processing the 46th record. run: 1098 subRun: 1 event: 32446 at 17-Nov-2025 22:41:44 UTC
Begin processing the 47th record. run: 1098 subRun: 1 event: 32447 at 17-Nov-2025 22:41:46 UTC
Begin processing the 48th record. run: 1098 subRun: 1 event: 32448 at 17-Nov-2025 22:41:47 UTC
Begin processing the 49th record. run: 1098 subRun: 1 event: 32449 at 17-Nov-2025 22:41:49 UTC
Begin processing the 50th record. run: 1098 subRun: 1 event: 32450 at 17-Nov-2025 22:41:50 UTC
Begin processing the 51st record. run: 1098 subRun: 1 event: 32451 at 17-Nov-2025 22:41:52 UTC
Begin processing the 52nd record. run: 1098 subRun: 1 event: 32452 at 17-Nov-2025 22:41:53 UTC
Begin processing the 53rd record. run: 1098 subRun: 1 event: 32453 at 17-Nov-2025 22:41:54 UTC
Begin processing the 54th record. run: 1098 subRun: 1 event: 32454 at 17-Nov-2025 22:41:56 UTC
Begin processing the 55th record. run: 1098 subRun: 1 event: 32455 at 17-Nov-2025 22:41:57 UTC
Begin processing the 56th record. run: 1098 subRun: 1 event: 32456 at 17-Nov-2025 22:41:59 UTC
Begin processing the 57th record. run: 1098 subRun: 1 event: 32457 at 17-Nov-2025 22:42:00 UTC
Begin processing the 58th record. run: 1098 subRun: 1 event: 32458 at 17-Nov-2025 22:42:02 UTC
Begin processing the 59th record. run: 1098 subRun: 1 event: 32459 at 17-Nov-2025 22:42:03 UTC
Begin processing the 60th record. run: 1098 subRun: 1 event: 32460 at 17-Nov-2025 22:42:05 UTC
Begin processing the 61st record. run: 1098 subRun: 1 event: 32461 at 17-Nov-2025 22:42:06 UTC
Begin processing the 62nd record. run: 1098 subRun: 1 event: 32462 at 17-Nov-2025 22:42:08 UTC
Begin processing the 63rd record. run: 1098 subRun: 1 event: 32463 at 17-Nov-2025 22:42:09 UTC
Begin processing the 64th record. run: 1098 subRun: 1 event: 32464 at 17-Nov-2025 22:42:11 UTC
Begin processing the 65th record. run: 1098 subRun: 1 event: 32465 at 17-Nov-2025 22:42:12 UTC
Begin processing the 66th record. run: 1098 subRun: 1 event: 32466 at 17-Nov-2025 22:42:13 UTC
Begin processing the 67th record. run: 1098 subRun: 1 event: 32467 at 17-Nov-2025 22:42:15 UTC
Begin processing the 68th record. run: 1098 subRun: 1 event: 32468 at 17-Nov-2025 22:42:16 UTC
Begin processing the 69th record. run: 1098 subRun: 1 event: 32469 at 17-Nov-2025 22:42:17 UTC
Begin processing the 70th record. run: 1098 subRun: 1 event: 32470 at 17-Nov-2025 22:42:19 UTC
Begin processing the 71st record. run: 1098 subRun: 1 event: 32471 at 17-Nov-2025 22:42:20 UTC
Begin processing the 72nd record. run: 1098 subRun: 1 event: 32472 at 17-Nov-2025 22:42:21 UTC
Begin processing the 73rd record. run: 1098 subRun: 1 event: 32473 at 17-Nov-2025 22:42:23 UTC
Begin processing the 74th record. run: 1098 subRun: 1 event: 32474 at 17-Nov-2025 22:42:24 UTC
Begin processing the 75th record. run: 1098 subRun: 1 event: 32475 at 17-Nov-2025 22:42:26 UTC
Begin processing the 76th record. run: 1098 subRun: 1 event: 32476 at 17-Nov-2025 22:42:27 UTC
Begin processing the 77th record. run: 1098 subRun: 1 event: 32477 at 17-Nov-2025 22:42:29 UTC
Begin processing the 78th record. run: 1098 subRun: 1 event: 32478 at 17-Nov-2025 22:42:30 UTC
Begin processing the 79th record. run: 1098 subRun: 1 event: 32479 at 17-Nov-2025 22:42:31 UTC
Begin processing the 80th record. run: 1098 subRun: 1 event: 32480 at 17-Nov-2025 22:42:33 UTC
Begin processing the 81st record. run: 1098 subRun: 1 event: 32481 at 17-Nov-2025 22:42:34 UTC
Begin processing the 82nd record. run: 1098 subRun: 1 event: 32482 at 17-Nov-2025 22:42:35 UTC
Begin processing the 83rd record. run: 1098 subRun: 1 event: 32483 at 17-Nov-2025 22:42:37 UTC
Begin processing the 84th record. run: 1098 subRun: 1 event: 32484 at 17-Nov-2025 22:42:38 UTC
Begin processing the 85th record. run: 1098 subRun: 1 event: 32485 at 17-Nov-2025 22:42:40 UTC
Begin processing the 86th record. run: 1098 subRun: 1 event: 32486 at 17-Nov-2025 22:42:45 UTC
Begin processing the 87th record. run: 1098 subRun: 1 event: 32487 at 17-Nov-2025 22:42:47 UTC
Begin processing the 88th record. run: 1098 subRun: 1 event: 32488 at 17-Nov-2025 22:42:48 UTC
Begin processing the 89th record. run: 1098 subRun: 1 event: 32489 at 17-Nov-2025 22:42:50 UTC
Begin processing the 90th record. run: 1098 subRun: 1 event: 32490 at 17-Nov-2025 22:42:51 UTC
Begin processing the 91st record. run: 1098 subRun: 1 event: 32491 at 17-Nov-2025 22:42:53 UTC
Begin processing the 92nd record. run: 1098 subRun: 1 event: 32492 at 17-Nov-2025 22:42:54 UTC
Begin processing the 93rd record. run: 1098 subRun: 1 event: 32493 at 17-Nov-2025 22:42:56 UTC
Begin processing the 94th record. run: 1098 subRun: 1 event: 32494 at 17-Nov-2025 22:42:57 UTC
Begin processing the 95th record. run: 1098 subRun: 1 event: 32495 at 17-Nov-2025 22:42:59 UTC
Begin processing the 96th record. run: 1098 subRun: 1 event: 32496 at 17-Nov-2025 22:43:00 UTC
Begin processing the 97th record. run: 1098 subRun: 1 event: 32497 at 17-Nov-2025 22:43:09 UTC
Begin processing the 98th record. run: 1098 subRun: 1 event: 32498 at 17-Nov-2025 22:43:10 UTC
Begin processing the 99th record. run: 1098 subRun: 1 event: 32499 at 17-Nov-2025 22:43:12 UTC
Begin processing the 100th record. run: 1098 subRun: 1 event: 32500 at 17-Nov-2025 22:43:14 UTC
17-Nov-2025 22:43:16 UTC  Closed output file "nu_dune10kt_1x2x6_1098_324_20230826T012211Z_gen_g4_detsim_hitreco__20240222T214640Z_reco2_reco2.root"
17-Nov-2025 22:43:16 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/5f/55/nu_dune10kt_1x2x6_1098_324_20230826T012211Z_gen_g4_detsim_hitreco__20240222T214640Z_reco2.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                     0.0153475      1.60805       11.3356       1.33452       1.26797        100    
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.00150114     0.0231664      0.10341      0.018449      0.0175665       100    
reco:pandora:StandardPandora                   0.0125013       1.582        11.3004       1.30515       1.2646         100    
[art]:TriggerResults:TriggerResultInserter     6.272e-06    1.56874e-05   7.1908e-05    1.3316e-05    7.24681e-06      100    
end_path:out1:RootOutput                       1.202e-06    4.57125e-06   2.3655e-05     4.268e-06    2.0503e-06       100    
end_path:out1:RootOutput(write)               0.000640363   0.00265758     0.0154136      0.00183     0.00239989       100    
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 2406.58 MB
  Peak resident set size usage (VmHWM): 1420.46 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
mv: cannot stat 'CheatingKalmanSplittingV.root': No such file or directory
mv: cannot stat 'CheatingKalmanSplittingW.root': No such file or directory
justIN time: 2026-02-04 09:13:40 UTC       justIN version: 01.06.00