justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 254264.6@dunegpschedd01.fnal.gov

Jobsub ID254264.6@dunegpschedd01.fnal.gov
Workflow ID10385
Stage ID1
User nameschellma@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-11-19 22:36:11
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce513
Last heartbeat2025-11-19 22:52:29
From worker nodeHostnameb9p09p4833.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job stateaborted
Started2025-11-19 22:37:24
Input filesmonte-carlo-010385-000017
Outputting started2025-11-19 22:40:50
Output files
Finished2025-11-19 22:52:29
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ne10kt_1x2x6_20251003T020044Z_gen_004965_g4_detsim_20251115T193855Z_reco.root"
19-Nov-2025 23:40:40 CET  Initiating request to open input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/30/3d/prod_eminus_1-500MeV_dune10kt_1x2x6_20251003T020401Z_gen_004967_g4_detsim_20251115T185432Z_reco.root"
19-Nov-2025 23:40:40 CET  Opened input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/30/3d/prod_eminus_1-500MeV_dune10kt_1x2x6_20251003T020401Z_gen_004967_g4_detsim_20251115T185432Z_reco.root"
Begin processing the 951st record. run: 8428 subRun: 0 event: 49661 at 19-Nov-2025 23:40:40 CET
Begin processing the 952nd record. run: 8428 subRun: 0 event: 49662 at 19-Nov-2025 23:40:40 CET
Begin processing the 953rd record. run: 8428 subRun: 0 event: 49663 at 19-Nov-2025 23:40:40 CET
Begin processing the 954th record. run: 8428 subRun: 0 event: 49664 at 19-Nov-2025 23:40:40 CET
Begin processing the 955th record. run: 8428 subRun: 0 event: 49665 at 19-Nov-2025 23:40:41 CET
Begin processing the 956th record. run: 8428 subRun: 0 event: 49666 at 19-Nov-2025 23:40:41 CET
Begin processing the 957th record. run: 8428 subRun: 0 event: 49667 at 19-Nov-2025 23:40:41 CET
Begin processing the 958th record. run: 8428 subRun: 0 event: 49668 at 19-Nov-2025 23:40:41 CET
Begin processing the 959th record. run: 8428 subRun: 0 event: 49669 at 19-Nov-2025 23:40:41 CET
Begin processing the 960th record. run: 8428 subRun: 0 event: 49670 at 19-Nov-2025 23:40:41 CET
19-Nov-2025 23:40:41 CET  Closed input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/30/3d/prod_eminus_1-500MeV_dune10kt_1x2x6_20251003T020401Z_gen_004967_g4_detsim_20251115T185432Z_reco.root"
19-Nov-2025 23:40:41 CET  Initiating request to open input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/20/28/prod_eminus_1-500MeV_dune10kt_1x2x6_20251003T020409Z_gen_004968_g4_detsim_20251106T133018Z_reco.root"
19-Nov-2025 23:40:41 CET  Opened input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/20/28/prod_eminus_1-500MeV_dune10kt_1x2x6_20251003T020409Z_gen_004968_g4_detsim_20251106T133018Z_reco.root"
Begin processing the 961st record. run: 8428 subRun: 0 event: 49671 at 19-Nov-2025 23:40:42 CET
Begin processing the 962nd record. run: 8428 subRun: 0 event: 49672 at 19-Nov-2025 23:40:42 CET
Begin processing the 963rd record. run: 8428 subRun: 0 event: 49673 at 19-Nov-2025 23:40:42 CET
Begin processing the 964th record. run: 8428 subRun: 0 event: 49674 at 19-Nov-2025 23:40:42 CET
Begin processing the 965th record. run: 8428 subRun: 0 event: 49675 at 19-Nov-2025 23:40:42 CET
Begin processing the 966th record. run: 8428 subRun: 0 event: 49676 at 19-Nov-2025 23:40:42 CET
Begin processing the 967th record. run: 8428 subRun: 0 event: 49677 at 19-Nov-2025 23:40:42 CET
Begin processing the 968th record. run: 8428 subRun: 0 event: 49678 at 19-Nov-2025 23:40:42 CET
Begin processing the 969th record. run: 8428 subRun: 0 event: 49679 at 19-Nov-2025 23:40:42 CET
Begin processing the 970th record. run: 8428 subRun: 0 event: 49680 at 19-Nov-2025 23:40:43 CET
19-Nov-2025 23:40:43 CET  Closed input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/20/28/prod_eminus_1-500MeV_dune10kt_1x2x6_20251003T020409Z_gen_004968_g4_detsim_20251106T133018Z_reco.root"
19-Nov-2025 23:40:43 CET  Initiating request to open input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/d9/54/prod_eminus_1-500MeV_dune10kt_1x2x6_20251003T020446Z_gen_004974_g4_detsim_20251115T184543Z_reco.root"
19-Nov-2025 23:40:43 CET  Opened input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/d9/54/prod_eminus_1-500MeV_dune10kt_1x2x6_20251003T020446Z_gen_004974_g4_detsim_20251115T184543Z_reco.root"
Begin processing the 971st record. run: 8428 subRun: 0 event: 49731 at 19-Nov-2025 23:40:43 CET
Begin processing the 972nd record. run: 8428 subRun: 0 event: 49732 at 19-Nov-2025 23:40:43 CET
Begin processing the 973rd record. run: 8428 subRun: 0 event: 49733 at 19-Nov-2025 23:40:43 CET
Begin processing the 974th record. run: 8428 subRun: 0 event: 49734 at 19-Nov-2025 23:40:43 CET
Begin processing the 975th record. run: 8428 subRun: 0 event: 49735 at 19-Nov-2025 23:40:44 CET
Begin processing the 976th record. run: 8428 subRun: 0 event: 49736 at 19-Nov-2025 23:40:44 CET
Begin processing the 977th record. run: 8428 subRun: 0 event: 49737 at 19-Nov-2025 23:40:44 CET
Begin processing the 978th record. run: 8428 subRun: 0 event: 49738 at 19-Nov-2025 23:40:44 CET
Begin processing the 979th record. run: 8428 subRun: 0 event: 49739 at 19-Nov-2025 23:40:44 CET
Begin processing the 980th record. run: 8428 subRun: 0 event: 49740 at 19-Nov-2025 23:40:44 CET
19-Nov-2025 23:40:44 CET  Closed input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/d9/54/prod_eminus_1-500MeV_dune10kt_1x2x6_20251003T020446Z_gen_004974_g4_detsim_20251115T184543Z_reco.root"
19-Nov-2025 23:40:44 CET  Initiating request to open input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/2d/1d/prod_eminus_1-500MeV_dune10kt_1x2x6_20251003T020454Z_gen_004976_g4_detsim_20251115T194054Z_reco.root"
19-Nov-2025 23:40:44 CET  Opened input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/2d/1d/prod_eminus_1-500MeV_dune10kt_1x2x6_20251003T020454Z_gen_004976_g4_detsim_20251115T194054Z_reco.root"
Begin processing the 981st record. run: 8428 subRun: 0 event: 49751 at 19-Nov-2025 23:40:45 CET
Begin processing the 982nd record. run: 8428 subRun: 0 event: 49752 at 19-Nov-2025 23:40:45 CET
Begin processing the 983rd record. run: 8428 subRun: 0 event: 49753 at 19-Nov-2025 23:40:45 CET
Begin processing the 984th record. run: 8428 subRun: 0 event: 49754 at 19-Nov-2025 23:40:45 CET
Begin processing the 985th record. run: 8428 subRun: 0 event: 49755 at 19-Nov-2025 23:40:45 CET
Begin processing the 986th record. run: 8428 subRun: 0 event: 49756 at 19-Nov-2025 23:40:45 CET
Begin processing the 987th record. run: 8428 subRun: 0 event: 49757 at 19-Nov-2025 23:40:45 CET
Begin processing the 988th record. run: 8428 subRun: 0 event: 49758 at 19-Nov-2025 23:40:45 CET
Begin processing the 989th record. run: 8428 subRun: 0 event: 49759 at 19-Nov-2025 23:40:46 CET
Begin processing the 990th record. run: 8428 subRun: 0 event: 49760 at 19-Nov-2025 23:40:46 CET
19-Nov-2025 23:40:46 CET  Closed input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/2d/1d/prod_eminus_1-500MeV_dune10kt_1x2x6_20251003T020454Z_gen_004976_g4_detsim_20251115T194054Z_reco.root"
19-Nov-2025 23:40:46 CET  Initiating request to open input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/82/dd/prod_eminus_1-500MeV_dune10kt_1x2x6_20251003T020454Z_gen_004977_g4_detsim_20251115T185939Z_reco.root"
19-Nov-2025 23:40:46 CET  Opened input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/82/dd/prod_eminus_1-500MeV_dune10kt_1x2x6_20251003T020454Z_gen_004977_g4_detsim_20251115T185939Z_reco.root"
Begin processing the 991st record. run: 8428 subRun: 0 event: 49761 at 19-Nov-2025 23:40:46 CET
Begin processing the 992nd record. run: 8428 subRun: 0 event: 49762 at 19-Nov-2025 23:40:46 CET
Begin processing the 993rd record. run: 8428 subRun: 0 event: 49763 at 19-Nov-2025 23:40:46 CET
Begin processing the 994th record. run: 8428 subRun: 0 event: 49764 at 19-Nov-2025 23:40:47 CET
Begin processing the 995th record. run: 8428 subRun: 0 event: 49765 at 19-Nov-2025 23:40:47 CET
Begin processing the 996th record. run: 8428 subRun: 0 event: 49766 at 19-Nov-2025 23:40:47 CET
Begin processing the 997th record. run: 8428 subRun: 0 event: 49767 at 19-Nov-2025 23:40:47 CET
Begin processing the 998th record. run: 8428 subRun: 0 event: 49768 at 19-Nov-2025 23:40:47 CET
Begin processing the 999th record. run: 8428 subRun: 0 event: 49769 at 19-Nov-2025 23:40:47 CET
Begin processing the 1000th record. run: 8428 subRun: 0 event: 49770 at 19-Nov-2025 23:40:47 CET
19-Nov-2025 23:40:47 CET  Closed input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/82/dd/prod_eminus_1-500MeV_dune10kt_1x2x6_20251003T020454Z_gen_004977_g4_detsim_20251115T185939Z_reco.root"

========================================================================================================================
TimeTracker printout (sec)                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================
Full event                             0.0383773     0.0723136     0.660887      0.0637324     0.0577314      1000    
------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                0.00275481    0.00439851     0.0248729    0.00374307    0.00296783      1000    
end_path:triggerAna:TriggerAnaTree     0.0351284     0.0676718     0.654558      0.0590265     0.0576444      1000    
========================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1746.35 MB
  Peak resident set size usage (VmHWM): 963.109 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.

Processing output file prod_eminus_1-500MeV_dune10kt_1x2x6__trg_mc_2025a_tpg__tpg_dune10kt_1x2x6__reco__v10_12_01d01__triggerana_dune10kt_1x2x6__v10_12_01d01__s0__l2000__ntuple__EMINUS_20251119T212943_c16.root
merged pass1_CERN_000017.json
justIN time: 2026-02-09 06:19:13 UTC       justIN version: 01.06.00