justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 272704.167@dunegpschedd01.fnal.gov

Jobsub ID272704.167@dunegpschedd01.fnal.gov
Workflow ID11624
Stage ID1
User nameimawby@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes1048576000 (1000 MiB)
Wall seconds limit7200 (2 hours)
Submitted time2026-01-08 15:57:13
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc01
Last heartbeat2026-01-08 16:09:57
From worker nodeHostnamewn-da-20.gina.surf.nl
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes1048576000 (1000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job stateoutputting_failed
Started2026-01-08 15:57:58
Input filesfardet-hd:nu_dune10kt_1x2x6_1413_24_20230826T183300Z_gen_g4_detsim_hitreco__20240229T185032Z_reco2.root
JobscriptExit code0
Real time5m (309s)
CPU time5m (303s = 98%)
Max RSS bytes1374330880 (1310 MiB)
Outputting started2026-01-08 16:03:08
Output files
Finished2026-01-08 16:09:57
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

7b09f7c9bd4e00ac25e95dc15/localProducts_larsoft_v10_13_00_e26_prof
----------- this block should be empty ------------------
---------------------------------------------------------
Input PFN = root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/fardet-hd/76/cb/nu_dune10kt_1x2x6_1413_24_20230826T183300Z_gen_g4_detsim_hitreco__20240229T185032Z_reco2.root
lar exit code 0
=== Start last 100 lines of lar log file ===
Begin processing the 23rd record. run: 1413 subRun: 1 event: 2423 at 08-Jan-2026 16:59:30 CET
Begin processing the 24th record. run: 1413 subRun: 1 event: 2424 at 08-Jan-2026 16:59:31 CET
Begin processing the 25th record. run: 1413 subRun: 1 event: 2425 at 08-Jan-2026 16:59:32 CET
Begin processing the 26th record. run: 1413 subRun: 1 event: 2426 at 08-Jan-2026 16:59:34 CET
Begin processing the 27th record. run: 1413 subRun: 1 event: 2427 at 08-Jan-2026 16:59:35 CET
Begin processing the 28th record. run: 1413 subRun: 1 event: 2428 at 08-Jan-2026 16:59:36 CET
Begin processing the 29th record. run: 1413 subRun: 1 event: 2429 at 08-Jan-2026 16:59:38 CET
Begin processing the 30th record. run: 1413 subRun: 1 event: 2430 at 08-Jan-2026 16:59:39 CET
Begin processing the 31st record. run: 1413 subRun: 1 event: 2431 at 08-Jan-2026 16:59:41 CET
Begin processing the 32nd record. run: 1413 subRun: 1 event: 2432 at 08-Jan-2026 16:59:42 CET
Begin processing the 33rd record. run: 1413 subRun: 1 event: 2433 at 08-Jan-2026 16:59:43 CET
Begin processing the 34th record. run: 1413 subRun: 1 event: 2434 at 08-Jan-2026 16:59:45 CET
Begin processing the 35th record. run: 1413 subRun: 1 event: 2435 at 08-Jan-2026 16:59:46 CET
Begin processing the 36th record. run: 1413 subRun: 1 event: 2436 at 08-Jan-2026 16:59:47 CET
Begin processing the 37th record. run: 1413 subRun: 1 event: 2437 at 08-Jan-2026 16:59:48 CET
Begin processing the 38th record. run: 1413 subRun: 1 event: 2438 at 08-Jan-2026 16:59:50 CET
Begin processing the 39th record. run: 1413 subRun: 1 event: 2439 at 08-Jan-2026 16:59:51 CET
Begin processing the 40th record. run: 1413 subRun: 1 event: 2440 at 08-Jan-2026 16:59:53 CET
Begin processing the 41st record. run: 1413 subRun: 1 event: 2441 at 08-Jan-2026 16:59:57 CET
Begin processing the 42nd record. run: 1413 subRun: 1 event: 2442 at 08-Jan-2026 16:59:59 CET
Begin processing the 43rd record. run: 1413 subRun: 1 event: 2443 at 08-Jan-2026 17:00:00 CET
Begin processing the 44th record. run: 1413 subRun: 1 event: 2444 at 08-Jan-2026 17:00:01 CET
Begin processing the 45th record. run: 1413 subRun: 1 event: 2445 at 08-Jan-2026 17:00:02 CET
Begin processing the 46th record. run: 1413 subRun: 1 event: 2446 at 08-Jan-2026 17:00:04 CET
Begin processing the 47th record. run: 1413 subRun: 1 event: 2447 at 08-Jan-2026 17:00:06 CET
Begin processing the 48th record. run: 1413 subRun: 1 event: 2448 at 08-Jan-2026 17:00:07 CET
Begin processing the 49th record. run: 1413 subRun: 1 event: 2449 at 08-Jan-2026 17:00:09 CET
Begin processing the 50th record. run: 1413 subRun: 1 event: 2450 at 08-Jan-2026 17:00:10 CET
Begin processing the 51st record. run: 1413 subRun: 1 event: 2451 at 08-Jan-2026 17:00:11 CET
Begin processing the 52nd record. run: 1413 subRun: 1 event: 2452 at 08-Jan-2026 17:00:13 CET
Begin processing the 53rd record. run: 1413 subRun: 1 event: 2453 at 08-Jan-2026 17:00:14 CET
Begin processing the 54th record. run: 1413 subRun: 1 event: 2454 at 08-Jan-2026 17:00:22 CET
Begin processing the 55th record. run: 1413 subRun: 1 event: 2455 at 08-Jan-2026 17:00:23 CET
Begin processing the 56th record. run: 1413 subRun: 1 event: 2456 at 08-Jan-2026 17:00:25 CET
Begin processing the 57th record. run: 1413 subRun: 1 event: 2457 at 08-Jan-2026 17:00:29 CET
Begin processing the 58th record. run: 1413 subRun: 1 event: 2458 at 08-Jan-2026 17:00:30 CET
Begin processing the 59th record. run: 1413 subRun: 1 event: 2459 at 08-Jan-2026 17:00:32 CET
Begin processing the 60th record. run: 1413 subRun: 1 event: 2460 at 08-Jan-2026 17:00:34 CET
Begin processing the 61st record. run: 1413 subRun: 1 event: 2461 at 08-Jan-2026 17:00:35 CET
Begin processing the 62nd record. run: 1413 subRun: 1 event: 2462 at 08-Jan-2026 17:00:36 CET
Begin processing the 63rd record. run: 1413 subRun: 1 event: 2463 at 08-Jan-2026 17:00:38 CET
Begin processing the 64th record. run: 1413 subRun: 1 event: 2464 at 08-Jan-2026 17:02:14 CET
Begin processing the 65th record. run: 1413 subRun: 1 event: 2465 at 08-Jan-2026 17:02:15 CET
Begin processing the 66th record. run: 1413 subRun: 1 event: 2466 at 08-Jan-2026 17:02:17 CET
Begin processing the 67th record. run: 1413 subRun: 1 event: 2467 at 08-Jan-2026 17:02:19 CET
Begin processing the 68th record. run: 1413 subRun: 1 event: 2468 at 08-Jan-2026 17:02:20 CET
Begin processing the 69th record. run: 1413 subRun: 1 event: 2469 at 08-Jan-2026 17:02:21 CET
Begin processing the 70th record. run: 1413 subRun: 1 event: 2470 at 08-Jan-2026 17:02:23 CET
Begin processing the 71st record. run: 1413 subRun: 1 event: 2471 at 08-Jan-2026 17:02:24 CET
Begin processing the 72nd record. run: 1413 subRun: 1 event: 2472 at 08-Jan-2026 17:02:25 CET
Begin processing the 73rd record. run: 1413 subRun: 1 event: 2473 at 08-Jan-2026 17:02:26 CET
Begin processing the 74th record. run: 1413 subRun: 1 event: 2474 at 08-Jan-2026 17:02:28 CET
Begin processing the 75th record. run: 1413 subRun: 1 event: 2475 at 08-Jan-2026 17:02:29 CET
Begin processing the 76th record. run: 1413 subRun: 1 event: 2476 at 08-Jan-2026 17:02:30 CET
Begin processing the 77th record. run: 1413 subRun: 1 event: 2477 at 08-Jan-2026 17:02:32 CET
Begin processing the 78th record. run: 1413 subRun: 1 event: 2478 at 08-Jan-2026 17:02:33 CET
Begin processing the 79th record. run: 1413 subRun: 1 event: 2479 at 08-Jan-2026 17:02:35 CET
Begin processing the 80th record. run: 1413 subRun: 1 event: 2480 at 08-Jan-2026 17:02:36 CET
Begin processing the 81st record. run: 1413 subRun: 1 event: 2481 at 08-Jan-2026 17:02:37 CET
Begin processing the 82nd record. run: 1413 subRun: 1 event: 2482 at 08-Jan-2026 17:02:39 CET
Begin processing the 83rd record. run: 1413 subRun: 1 event: 2483 at 08-Jan-2026 17:02:40 CET
Begin processing the 84th record. run: 1413 subRun: 1 event: 2484 at 08-Jan-2026 17:02:42 CET
Begin processing the 85th record. run: 1413 subRun: 1 event: 2485 at 08-Jan-2026 17:02:43 CET
Begin processing the 86th record. run: 1413 subRun: 1 event: 2486 at 08-Jan-2026 17:02:45 CET
Begin processing the 87th record. run: 1413 subRun: 1 event: 2487 at 08-Jan-2026 17:02:47 CET
Begin processing the 88th record. run: 1413 subRun: 1 event: 2488 at 08-Jan-2026 17:02:48 CET
Begin processing the 89th record. run: 1413 subRun: 1 event: 2489 at 08-Jan-2026 17:02:50 CET
Begin processing the 90th record. run: 1413 subRun: 1 event: 2490 at 08-Jan-2026 17:02:51 CET
Begin processing the 91st record. run: 1413 subRun: 1 event: 2491 at 08-Jan-2026 17:02:52 CET
Begin processing the 92nd record. run: 1413 subRun: 1 event: 2492 at 08-Jan-2026 17:02:53 CET
Begin processing the 93rd record. run: 1413 subRun: 1 event: 2493 at 08-Jan-2026 17:02:55 CET
Begin processing the 94th record. run: 1413 subRun: 1 event: 2494 at 08-Jan-2026 17:02:56 CET
Begin processing the 95th record. run: 1413 subRun: 1 event: 2495 at 08-Jan-2026 17:02:57 CET
Begin processing the 96th record. run: 1413 subRun: 1 event: 2496 at 08-Jan-2026 17:02:59 CET
Begin processing the 97th record. run: 1413 subRun: 1 event: 2497 at 08-Jan-2026 17:03:00 CET
Begin processing the 98th record. run: 1413 subRun: 1 event: 2498 at 08-Jan-2026 17:03:02 CET
Begin processing the 99th record. run: 1413 subRun: 1 event: 2499 at 08-Jan-2026 17:03:03 CET
Begin processing the 100th record. run: 1413 subRun: 1 event: 2500 at 08-Jan-2026 17:03:05 CET
08-Jan-2026 17:03:07 CET  Closed output file "nu_dune10kt_1x2x6_1413_24_20230826T183300Z_gen_g4_detsim_hitreco__20240229T185032Z_reco2_reco2.root"
08-Jan-2026 17:03:07 CET  Closed input file "root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/fardet-hd/76/cb/nu_dune10kt_1x2x6_1413_24_20230826T183300Z_gen_g4_detsim_hitreco__20240229T185032Z_reco2.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                     0.0186211      2.48306       94.3718       1.29856       9.33545        100    
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.00205372    0.00369946     0.0109019    0.00357978    0.000917991      100    
reco:pandora:StandardPandora                   0.0144403      2.46814       94.2693       1.28779       9.3257         100    
[art]:TriggerResults:TriggerResultInserter    1.0932e-05    1.96539e-05   5.1709e-05    1.81795e-05   8.01251e-06      100    
end_path:out1:RootOutput                       3.306e-06    4.21927e-06   1.8556e-05     3.978e-06    1.76887e-06      100    
end_path:out1:RootOutput(write)               0.000840874    0.0110136     0.0982778    0.00816928     0.0124302       100    
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 2360.71 MB
  Peak resident set size usage (VmHWM): 1374.33 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
justIN time: 2026-02-04 05:59:17 UTC       justIN version: 01.06.00