justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 254250.17@dunegpschedd01.fnal.gov

Jobsub ID254250.17@dunegpschedd01.fnal.gov
Workflow ID10386
Stage ID1
User nameschellma@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-11-19 21:53:30
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce513
Last heartbeat2025-11-19 22:58:18
From worker nodeHostnameb9p12p1465.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job stateaborted
Started2025-11-19 22:24:53
Input filesmonte-carlo-010386-000004
Outputting started2025-11-19 22:35:25
Output files
Finished2025-11-19 22:58:18
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

dune/disk/fardet-hd/eb/24/prod_eminus_1-500MeV_dune10kt_1x2x6_20251002T183940Z_gen_000708_g4_detsim_20251115T185933Z_reco.root"
19-Nov-2025 23:35:13 CET  Initiating request to open input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/70/c5/prod_eminus_1-500MeV_dune10kt_1x2x6_20251002T184001Z_gen_000711_g4_detsim_20251115T190553Z_reco.root"
19-Nov-2025 23:35:13 CET  Opened input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/70/c5/prod_eminus_1-500MeV_dune10kt_1x2x6_20251002T184001Z_gen_000711_g4_detsim_20251115T190553Z_reco.root"
Begin processing the 951st record. run: 8428 subRun: 0 event: 7101 at 19-Nov-2025 23:35:13 CET
Begin processing the 952nd record. run: 8428 subRun: 0 event: 7102 at 19-Nov-2025 23:35:13 CET
Begin processing the 953rd record. run: 8428 subRun: 0 event: 7103 at 19-Nov-2025 23:35:13 CET
Begin processing the 954th record. run: 8428 subRun: 0 event: 7104 at 19-Nov-2025 23:35:14 CET
Begin processing the 955th record. run: 8428 subRun: 0 event: 7105 at 19-Nov-2025 23:35:14 CET
Begin processing the 956th record. run: 8428 subRun: 0 event: 7106 at 19-Nov-2025 23:35:14 CET
Begin processing the 957th record. run: 8428 subRun: 0 event: 7107 at 19-Nov-2025 23:35:14 CET
Begin processing the 958th record. run: 8428 subRun: 0 event: 7108 at 19-Nov-2025 23:35:14 CET
Begin processing the 959th record. run: 8428 subRun: 0 event: 7109 at 19-Nov-2025 23:35:14 CET
Begin processing the 960th record. run: 8428 subRun: 0 event: 7110 at 19-Nov-2025 23:35:15 CET
19-Nov-2025 23:35:15 CET  Closed input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/70/c5/prod_eminus_1-500MeV_dune10kt_1x2x6_20251002T184001Z_gen_000711_g4_detsim_20251115T190553Z_reco.root"
19-Nov-2025 23:35:15 CET  Initiating request to open input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/a7/36/prod_eminus_1-500MeV_dune10kt_1x2x6_20251002T184100Z_gen_000713_g4_detsim_20251115T193039Z_reco.root"
19-Nov-2025 23:35:15 CET  Opened input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/a7/36/prod_eminus_1-500MeV_dune10kt_1x2x6_20251002T184100Z_gen_000713_g4_detsim_20251115T193039Z_reco.root"
Begin processing the 961st record. run: 8428 subRun: 0 event: 7121 at 19-Nov-2025 23:35:15 CET
Begin processing the 962nd record. run: 8428 subRun: 0 event: 7122 at 19-Nov-2025 23:35:15 CET
Begin processing the 963rd record. run: 8428 subRun: 0 event: 7123 at 19-Nov-2025 23:35:15 CET
Begin processing the 964th record. run: 8428 subRun: 0 event: 7124 at 19-Nov-2025 23:35:16 CET
Begin processing the 965th record. run: 8428 subRun: 0 event: 7125 at 19-Nov-2025 23:35:16 CET
Begin processing the 966th record. run: 8428 subRun: 0 event: 7126 at 19-Nov-2025 23:35:16 CET
Begin processing the 967th record. run: 8428 subRun: 0 event: 7127 at 19-Nov-2025 23:35:16 CET
Begin processing the 968th record. run: 8428 subRun: 0 event: 7128 at 19-Nov-2025 23:35:16 CET
Begin processing the 969th record. run: 8428 subRun: 0 event: 7129 at 19-Nov-2025 23:35:16 CET
Begin processing the 970th record. run: 8428 subRun: 0 event: 7130 at 19-Nov-2025 23:35:16 CET
19-Nov-2025 23:35:17 CET  Closed input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/a7/36/prod_eminus_1-500MeV_dune10kt_1x2x6_20251002T184100Z_gen_000713_g4_detsim_20251115T193039Z_reco.root"
19-Nov-2025 23:35:17 CET  Initiating request to open input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/86/30/prod_eminus_1-500MeV_dune10kt_1x2x6_20251002T184130Z_gen_000716_g4_detsim_20251115T191342Z_reco.root"
19-Nov-2025 23:35:17 CET  Opened input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/86/30/prod_eminus_1-500MeV_dune10kt_1x2x6_20251002T184130Z_gen_000716_g4_detsim_20251115T191342Z_reco.root"
Begin processing the 971st record. run: 8428 subRun: 0 event: 7151 at 19-Nov-2025 23:35:17 CET
Begin processing the 972nd record. run: 8428 subRun: 0 event: 7152 at 19-Nov-2025 23:35:17 CET
Begin processing the 973rd record. run: 8428 subRun: 0 event: 7153 at 19-Nov-2025 23:35:17 CET
Begin processing the 974th record. run: 8428 subRun: 0 event: 7154 at 19-Nov-2025 23:35:17 CET
Begin processing the 975th record. run: 8428 subRun: 0 event: 7155 at 19-Nov-2025 23:35:17 CET
Begin processing the 976th record. run: 8428 subRun: 0 event: 7156 at 19-Nov-2025 23:35:18 CET
Begin processing the 977th record. run: 8428 subRun: 0 event: 7157 at 19-Nov-2025 23:35:18 CET
Begin processing the 978th record. run: 8428 subRun: 0 event: 7158 at 19-Nov-2025 23:35:18 CET
Begin processing the 979th record. run: 8428 subRun: 0 event: 7159 at 19-Nov-2025 23:35:18 CET
Begin processing the 980th record. run: 8428 subRun: 0 event: 7160 at 19-Nov-2025 23:35:18 CET
19-Nov-2025 23:35:18 CET  Closed input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/86/30/prod_eminus_1-500MeV_dune10kt_1x2x6_20251002T184130Z_gen_000716_g4_detsim_20251115T191342Z_reco.root"
19-Nov-2025 23:35:18 CET  Initiating request to open input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/eb/a9/prod_eminus_1-500MeV_dune10kt_1x2x6_20251002T184143Z_gen_000718_g4_detsim_20251115T185406Z_reco.root"
19-Nov-2025 23:35:18 CET  Opened input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/eb/a9/prod_eminus_1-500MeV_dune10kt_1x2x6_20251002T184143Z_gen_000718_g4_detsim_20251115T185406Z_reco.root"
Begin processing the 981st record. run: 8428 subRun: 0 event: 7171 at 19-Nov-2025 23:35:18 CET
Begin processing the 982nd record. run: 8428 subRun: 0 event: 7172 at 19-Nov-2025 23:35:19 CET
Begin processing the 983rd record. run: 8428 subRun: 0 event: 7173 at 19-Nov-2025 23:35:19 CET
Begin processing the 984th record. run: 8428 subRun: 0 event: 7174 at 19-Nov-2025 23:35:19 CET
Begin processing the 985th record. run: 8428 subRun: 0 event: 7175 at 19-Nov-2025 23:35:19 CET
Begin processing the 986th record. run: 8428 subRun: 0 event: 7176 at 19-Nov-2025 23:35:19 CET
Begin processing the 987th record. run: 8428 subRun: 0 event: 7177 at 19-Nov-2025 23:35:19 CET
Begin processing the 988th record. run: 8428 subRun: 0 event: 7178 at 19-Nov-2025 23:35:19 CET
Begin processing the 989th record. run: 8428 subRun: 0 event: 7179 at 19-Nov-2025 23:35:20 CET
Begin processing the 990th record. run: 8428 subRun: 0 event: 7180 at 19-Nov-2025 23:35:20 CET
19-Nov-2025 23:35:20 CET  Closed input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/eb/a9/prod_eminus_1-500MeV_dune10kt_1x2x6_20251002T184143Z_gen_000718_g4_detsim_20251115T185406Z_reco.root"
19-Nov-2025 23:35:20 CET  Initiating request to open input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/09/d1/prod_eminus_1-500MeV_dune10kt_1x2x6_20251002T184146Z_gen_000720_g4_detsim_20251115T184538Z_reco.root"
19-Nov-2025 23:35:20 CET  Opened input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/09/d1/prod_eminus_1-500MeV_dune10kt_1x2x6_20251002T184146Z_gen_000720_g4_detsim_20251115T184538Z_reco.root"
Begin processing the 991st record. run: 8428 subRun: 0 event: 7191 at 19-Nov-2025 23:35:20 CET
Begin processing the 992nd record. run: 8428 subRun: 0 event: 7192 at 19-Nov-2025 23:35:20 CET
Begin processing the 993rd record. run: 8428 subRun: 0 event: 7193 at 19-Nov-2025 23:35:20 CET
Begin processing the 994th record. run: 8428 subRun: 0 event: 7194 at 19-Nov-2025 23:35:20 CET
Begin processing the 995th record. run: 8428 subRun: 0 event: 7195 at 19-Nov-2025 23:35:21 CET
Begin processing the 996th record. run: 8428 subRun: 0 event: 7196 at 19-Nov-2025 23:35:21 CET
Begin processing the 997th record. run: 8428 subRun: 0 event: 7197 at 19-Nov-2025 23:35:21 CET
Begin processing the 998th record. run: 8428 subRun: 0 event: 7198 at 19-Nov-2025 23:35:22 CET
Begin processing the 999th record. run: 8428 subRun: 0 event: 7199 at 19-Nov-2025 23:35:22 CET
Begin processing the 1000th record. run: 8428 subRun: 0 event: 7200 at 19-Nov-2025 23:35:22 CET
19-Nov-2025 23:35:22 CET  Closed input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/09/d1/prod_eminus_1-500MeV_dune10kt_1x2x6_20251002T184146Z_gen_000720_g4_detsim_20251115T184538Z_reco.root"

========================================================================================================================
TimeTracker printout (sec)                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================
Full event                             0.0434973     0.0845549     0.834658      0.0724082     0.0816487      1000    
------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                0.00275958    0.00480682     0.0374257    0.00398921    0.00341551      1000    
end_path:triggerAna:TriggerAnaTree     0.0399951     0.0794145     0.830456      0.0672815     0.0816057      1000    
========================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1747.75 MB
  Peak resident set size usage (VmHWM): 968.54 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.

Processing output file prod_eminus_1-500MeV_dune10kt_1x2x6__trg_mc_2025a_tpg__tpg_dune10kt_1x2x6__reco__v10_12_01d01__triggerana_dune10kt_1x2x6__v10_12_01d01__s2000__l2000__ntuple__EMINUS_20251119T212954_c3.root
merged pass1_CERN_000004.json
justIN time: 2026-02-09 06:20:25 UTC       justIN version: 01.06.00