justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 301448.2@dunegpschedd01.fnal.gov

Jobsub ID301448.2@dunegpschedd01.fnal.gov
Workflow ID12428
Stage ID2
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2026-01-27 09:53:14
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2026-01-27 14:41:01
From worker nodeHostnamedunegli-8449995-0-fnpc17112.fnal.gov
cpuinfoIntel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-27 09:54:42
Input filesusertests:np02vd_raw_run039636_0075_df-s04-d2_dw_0_20250925T202201_reco_stage0_20260127T094921_offline.root
JobscriptExit code0
Real time4h (17048s)
CPU time4h (16781s = 98%)
Max RSS bytes4054462464 (3866 MiB)
Outputting started2026-01-27 14:38:51
Output filesusertests:np02vd_raw_run039636_0075_df-s04-d2_dw_0_20250925T202201_reco_stage0_20260127T094921_offline_reco_stage1_20260127T143837_offline.root
usertests:np02vd_raw_run039636_0075_df-s04-d2_dw_0_20250925T202201_reco_stage0_20260127T094921_offline_reco_stage1_20260127T143837_offline_larcv.root
Finished2026-01-27 14:41:01
Saved logsjustin-logs:301448.2-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

tree entry 46
     [DEBUG]  <IOManager::save_entry> IOManager.cxx::L416 Saving sparse3d_reco_chi2_tree entry 46
     [DEBUG]  <IOManager::save_entry> IOManager.cxx::L416 Saving sparse3d_reco_cryo_tree entry 46
     [DEBUG]  <IOManager::save_entry> IOManager.cxx::L416 Saving sparse3d_reco_tpc_tree entry 46
     [DEBUG]  <IOManager::save_entry> IOManager.cxx::L416 Saving sparse3d_reco_hit_charge0_tree entry 46
     [DEBUG]  <IOManager::save_entry> IOManager.cxx::L416 Saving sparse3d_reco_hit_charge1_tree entry 46
     [DEBUG]  <IOManager::save_entry> IOManager.cxx::L416 Saving sparse3d_reco_hit_charge2_tree entry 46
     [DEBUG]  <IOManager::save_entry> IOManager.cxx::L416 Saving sparse3d_reco_hit_key0_tree entry 46
     [DEBUG]  <IOManager::save_entry> IOManager.cxx::L416 Saving sparse3d_reco_hit_key1_tree entry 46
     [DEBUG]  <IOManager::save_entry> IOManager.cxx::L416 Saving sparse3d_reco_hit_key2_tree entry 46
     [DEBUG]  <IOManager::clear_entry> IOManager.cxx::L460 Set _last_event_id to externally set values: run = 39636 subrun = 1 event = 14514
27-Jan-2026 14:38:37 UTC  Closed output file "np02vd_raw_run039636_0075_df-s04-d2_dw_0_20250925T202201_reco_stage0_20260127T094921_offline_reco_stage1_20260127T143837_offline.root"
27-Jan-2026 14:38:37 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/usertests/c0/16/np02vd_raw_run039636_0075_df-s04-d2_dw_0_20250925T202201_reco_stage0_20260127T094921_offline.root"
     [DEBUG]  <ProcessDriver::finalize> ProcessDriver.cxx::L424 called
      [INFO]  <ProcessDriver::finalize::L427> Finalizing: SuperaBBoxInteraction
      [INFO]  <ProcessDriver::finalize::L427> Finalizing: SuperaSpacePoint
      [INFO]  <ProcessDriver::finalize::L434> Compiling time profile...
      [INFO]  <ProcessDriver::finalize::L457> Finalizing IO...
     [DEBUG]  <IOManager::finalize> IOManager.cxx::L611 start
    [NORMAL]  <IOManager::finalize> Writing sparse3d_reco_tree with 47 entries
    [NORMAL]  <IOManager::finalize> Writing sparse3d_reco_chi2_tree with 47 entries
    [NORMAL]  <IOManager::finalize> Writing sparse3d_reco_cryo_tree with 47 entries
    [NORMAL]  <IOManager::finalize> Writing sparse3d_reco_tpc_tree with 47 entries
    [NORMAL]  <IOManager::finalize> Writing sparse3d_reco_hit_charge0_tree with 47 entries
    [NORMAL]  <IOManager::finalize> Writing sparse3d_reco_hit_charge1_tree with 47 entries
    [NORMAL]  <IOManager::finalize> Writing sparse3d_reco_hit_charge2_tree with 47 entries
    [NORMAL]  <IOManager::finalize> Writing sparse3d_reco_hit_key0_tree with 47 entries
    [NORMAL]  <IOManager::finalize> Writing sparse3d_reco_hit_key1_tree with 47 entries
    [NORMAL]  <IOManager::finalize> Writing sparse3d_reco_hit_key2_tree with 47 entries
    [NORMAL]  <IOManager::finalize> Closing output file
      [INFO]  <IOManager::finalize::L639> Deleting data pointers
     [DEBUG]  <IOManager::reset> IOManager.cxx::L647 start
      [INFO]  <ProcessDriver::finalize::L459> Resetting...
     [DEBUG]  <ProcessDriver::reset> ProcessDriver.cxx::L25 Called
     [DEBUG]  <IOManager::reset> IOManager.cxx::L647 start

===========================================================================================================================================
TimeTracker printout (sec)                                   Min           Avg           Max         Median          RMS         nEvts   
===========================================================================================================================================
Full event                                                 8.47656       361.109       8127.34       33.8326       1479.49        47     
-------------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                   0.000568186    0.0073285     0.169587     0.00126738     0.0247819       47     
produce:opflash:OpFlashFinderVerticalDrift               0.000151886   0.00031333     0.0026418    0.000245076   0.000349845      47     
produce:pandora:StandardPandora                            7.2638        350.792       8016.07       28.9028       1455.28        47     
produce:pandoraWriter:StandardPandora                     0.0586166     0.129548       0.32482      0.116296      0.0554665       47     
produce:pandoraTrack:LArPandoraTrackCreation              0.0628654     0.831605       5.81754      0.494805       1.0247         47     
produce:pandoraShower:LArPandoraModularShowerCreation     0.111813       0.26754      0.553886      0.227968      0.113549        47     
produce:pandoraGnocalo:GnocchiCalorimetry                0.00983616     0.0274122     0.101871      0.0225686     0.0152913       47     
produce:pandorapid:Chi2ParticleID                        0.00071876     0.0015513    0.00431052    0.00125516    0.000749806      47     
[art]:TriggerResults:TriggerResultInserter               1.6834e-05    4.90861e-05   0.00018014    5.2555e-05    2.82909e-05      47     
end_path:out1:RootOutput                                  3.772e-06    1.00276e-05   5.9846e-05     8.071e-06    8.60389e-06      47     
end_path:supera:LArSoftSuperaDriver                       0.476086       8.77214       137.111       2.64058       24.3704        47     
end_path:out1:RootOutput(write)                           0.0866113     0.199568      0.527966      0.174891      0.0931786       47     
===========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5152.73 MB
  Peak resident set size usage (VmHWM): 4054.46 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039636_0075_df-s04-d2_dw_0_20250925T202201_reco_stage0_20260127T094921_offline_reco_stage1_20260127T143837_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run039636_0075_df-s04-d2_dw_0_20250925T202201_reco_stage0_20260127T094921_offline_reco_stage1_20260127T143837_offline.root
Ran successfully
{
  "name": "np02vd_raw_run039636_0075_df-s04-d2_dw_0_20250925T202201_reco_stage0_20260127T094921_offline_reco_stage1_20260127T143837_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage1_protodunevd_neutrino.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1769507703.0,
    "core.end_time": 1769524720.0,
    "core.events": [
      14330,
      14334,
      14338,
      14342,
      14346,
      14350,
      14354,
      14358,
      14362,
      14366,
      14370,
      14374,
      14378,
      14382,
      14386,
      14390,
      14394,
      14398,
      14402,
      14406,
      14410,
      14414,
      14418,
      14422,
      14426,
      14430,
      14434,
      14438,
      14442,
      14446,
      14450,
      14454,
      14458,
      14462,
      14466,
      14470,
      14474,
      14478,
      14482,
      14486,
      14490,
      14494,
      14498,
      14502,
      14506,
      14510,
      14514
    ],
    "core.event_count": 47,
    "core.first_event_number": 14330,
    "core.last_event_number": 14514,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39636
    ],
    "core.runs_subruns": [
      3963600001
    ]
  },
  "parents": [
    {
      "did": "usertests:np02vd_raw_run039636_0075_df-s04-d2_dw_0_20250925T202201_reco_stage0_20260127T094921_offline.root"
    }
  ]
}\tLArCV: np02vd_raw_run039636_0075_df-s04-d2_dw_0_20250925T202201_reco_stage0_20260127T094921_offline_reco_stage1_20260127T143837_offline_larcv.root
Forming larcv metadata
formed
{
  "name": "np02vd_raw_run039636_0075_df-s04-d2_dw_0_20250925T202201_reco_stage0_20260127T094921_offline_reco_stage1_20260127T143837_offline_larcv.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage1_protodunevd_neutrino.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1769507703.0,
    "core.end_time": 1769524720.0,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39636
    ],
    "core.runs_subruns": [
      3963600001
    ]
  },
  "parents": [
    {
      "did": "usertests:np02vd_raw_run039636_0075_df-s04-d2_dw_0_20250925T202201_reco_stage0_20260127T094921_offline.root"
    }
  ]
}
justIN time: 2026-02-04 08:45:39 UTC       justIN version: 01.06.00