justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 253245.36@dunegpschedd02.fnal.gov

Jobsub ID253245.36@dunegpschedd02.fnal.gov
Workflow ID10810
Stage ID1
User nameschellma@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-11-26 18:20:55
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce513
Last heartbeat2025-11-26 22:43:12
From worker nodeHostnameb9p27p6281.cern.ch
cpuinfoAMD EPYC 7302 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job stateoutputting_failed
Started2025-11-26 22:16:24
Input filesmonte-carlo-010810-000046
JobscriptExit code0
Real time22m (1329s)
CPU time17m (1040s = 78%)
Max RSS bytes1441349632 (1374 MiB)
Outputting started2025-11-26 22:38:34
Output files
Finished2025-11-26 22:43:12
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

en_002290_supernova_g4_detsim_20251113T194653Z_reco.root"
Begin processing the 451st record. run: 8188 subRun: 0 event: 22891 at 26-Nov-2025 23:36:14 CET
Begin processing the 452nd record. run: 8188 subRun: 0 event: 22892 at 26-Nov-2025 23:36:16 CET
Begin processing the 453rd record. run: 8188 subRun: 0 event: 22893 at 26-Nov-2025 23:36:18 CET
Begin processing the 454th record. run: 8188 subRun: 0 event: 22894 at 26-Nov-2025 23:36:20 CET
Begin processing the 455th record. run: 8188 subRun: 0 event: 22895 at 26-Nov-2025 23:36:23 CET
Begin processing the 456th record. run: 8188 subRun: 0 event: 22896 at 26-Nov-2025 23:36:27 CET
Begin processing the 457th record. run: 8188 subRun: 0 event: 22897 at 26-Nov-2025 23:36:29 CET
Begin processing the 458th record. run: 8188 subRun: 0 event: 22898 at 26-Nov-2025 23:36:33 CET
Begin processing the 459th record. run: 8188 subRun: 0 event: 22899 at 26-Nov-2025 23:36:35 CET
Begin processing the 460th record. run: 8188 subRun: 0 event: 22900 at 26-Nov-2025 23:36:37 CET
26-Nov-2025 23:36:40 CET  Closed input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/16/60/prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250928T122150Z_gen_002290_supernova_g4_detsim_20251113T194653Z_reco.root"
26-Nov-2025 23:36:40 CET  Initiating request to open input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/00/df/prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250928T122153Z_gen_002291_supernova_g4_detsim_20251114T111328Z_reco.root"
26-Nov-2025 23:36:40 CET  Opened input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/00/df/prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250928T122153Z_gen_002291_supernova_g4_detsim_20251114T111328Z_reco.root"
Begin processing the 461st record. run: 8188 subRun: 0 event: 22901 at 26-Nov-2025 23:36:41 CET
Begin processing the 462nd record. run: 8188 subRun: 0 event: 22902 at 26-Nov-2025 23:36:43 CET
Begin processing the 463rd record. run: 8188 subRun: 0 event: 22903 at 26-Nov-2025 23:36:46 CET
Begin processing the 464th record. run: 8188 subRun: 0 event: 22904 at 26-Nov-2025 23:36:48 CET
Begin processing the 465th record. run: 8188 subRun: 0 event: 22905 at 26-Nov-2025 23:36:50 CET
Begin processing the 466th record. run: 8188 subRun: 0 event: 22906 at 26-Nov-2025 23:36:53 CET
Begin processing the 467th record. run: 8188 subRun: 0 event: 22907 at 26-Nov-2025 23:36:56 CET
Begin processing the 468th record. run: 8188 subRun: 0 event: 22908 at 26-Nov-2025 23:36:59 CET
Begin processing the 469th record. run: 8188 subRun: 0 event: 22909 at 26-Nov-2025 23:37:02 CET
Begin processing the 470th record. run: 8188 subRun: 0 event: 22910 at 26-Nov-2025 23:37:05 CET
26-Nov-2025 23:37:07 CET  Closed input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/00/df/prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250928T122153Z_gen_002291_supernova_g4_detsim_20251114T111328Z_reco.root"
26-Nov-2025 23:37:07 CET  Initiating request to open input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/8f/ef/prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250928T122154Z_gen_002293_supernova_g4_detsim_20251113T152706Z_reco.root"
26-Nov-2025 23:37:07 CET  Opened input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/8f/ef/prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250928T122154Z_gen_002293_supernova_g4_detsim_20251113T152706Z_reco.root"
Begin processing the 471st record. run: 8188 subRun: 0 event: 22921 at 26-Nov-2025 23:37:07 CET
Begin processing the 472nd record. run: 8188 subRun: 0 event: 22922 at 26-Nov-2025 23:37:10 CET
Begin processing the 473rd record. run: 8188 subRun: 0 event: 22923 at 26-Nov-2025 23:37:13 CET
Begin processing the 474th record. run: 8188 subRun: 0 event: 22924 at 26-Nov-2025 23:37:14 CET
Begin processing the 475th record. run: 8188 subRun: 0 event: 22925 at 26-Nov-2025 23:37:17 CET
Begin processing the 476th record. run: 8188 subRun: 0 event: 22926 at 26-Nov-2025 23:37:19 CET
Begin processing the 477th record. run: 8188 subRun: 0 event: 22927 at 26-Nov-2025 23:37:23 CET
Begin processing the 478th record. run: 8188 subRun: 0 event: 22928 at 26-Nov-2025 23:37:25 CET
Begin processing the 479th record. run: 8188 subRun: 0 event: 22929 at 26-Nov-2025 23:37:27 CET
Begin processing the 480th record. run: 8188 subRun: 0 event: 22930 at 26-Nov-2025 23:37:29 CET
26-Nov-2025 23:37:31 CET  Closed input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/8f/ef/prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250928T122154Z_gen_002293_supernova_g4_detsim_20251113T152706Z_reco.root"
26-Nov-2025 23:37:31 CET  Initiating request to open input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/1c/2a/prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250928T122155Z_gen_002294_supernova_g4_detsim_20251114T110705Z_reco.root"
26-Nov-2025 23:37:31 CET  Opened input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/1c/2a/prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250928T122155Z_gen_002294_supernova_g4_detsim_20251114T110705Z_reco.root"
Begin processing the 481st record. run: 8188 subRun: 0 event: 22931 at 26-Nov-2025 23:37:32 CET
Begin processing the 482nd record. run: 8188 subRun: 0 event: 22932 at 26-Nov-2025 23:37:35 CET
Begin processing the 483rd record. run: 8188 subRun: 0 event: 22933 at 26-Nov-2025 23:37:38 CET
Begin processing the 484th record. run: 8188 subRun: 0 event: 22934 at 26-Nov-2025 23:37:41 CET
Begin processing the 485th record. run: 8188 subRun: 0 event: 22935 at 26-Nov-2025 23:37:43 CET
Begin processing the 486th record. run: 8188 subRun: 0 event: 22936 at 26-Nov-2025 23:37:47 CET
Begin processing the 487th record. run: 8188 subRun: 0 event: 22937 at 26-Nov-2025 23:37:49 CET
Begin processing the 488th record. run: 8188 subRun: 0 event: 22938 at 26-Nov-2025 23:37:51 CET
Begin processing the 489th record. run: 8188 subRun: 0 event: 22939 at 26-Nov-2025 23:37:54 CET
Begin processing the 490th record. run: 8188 subRun: 0 event: 22940 at 26-Nov-2025 23:37:56 CET
26-Nov-2025 23:37:58 CET  Closed input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/1c/2a/prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250928T122155Z_gen_002294_supernova_g4_detsim_20251114T110705Z_reco.root"
26-Nov-2025 23:37:58 CET  Initiating request to open input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/ca/06/prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250928T122157Z_gen_002298_supernova_g4_detsim_20251113T194323Z_reco.root"
26-Nov-2025 23:37:58 CET  Opened input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/ca/06/prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250928T122157Z_gen_002298_supernova_g4_detsim_20251113T194323Z_reco.root"
Begin processing the 491st record. run: 8188 subRun: 0 event: 22971 at 26-Nov-2025 23:37:59 CET
Begin processing the 492nd record. run: 8188 subRun: 0 event: 22972 at 26-Nov-2025 23:38:01 CET
Begin processing the 493rd record. run: 8188 subRun: 0 event: 22973 at 26-Nov-2025 23:38:04 CET
Begin processing the 494th record. run: 8188 subRun: 0 event: 22974 at 26-Nov-2025 23:38:06 CET
Begin processing the 495th record. run: 8188 subRun: 0 event: 22975 at 26-Nov-2025 23:38:10 CET
Begin processing the 496th record. run: 8188 subRun: 0 event: 22976 at 26-Nov-2025 23:38:13 CET
Begin processing the 497th record. run: 8188 subRun: 0 event: 22977 at 26-Nov-2025 23:38:16 CET
Begin processing the 498th record. run: 8188 subRun: 0 event: 22978 at 26-Nov-2025 23:38:18 CET
Begin processing the 499th record. run: 8188 subRun: 0 event: 22979 at 26-Nov-2025 23:38:20 CET
Begin processing the 500th record. run: 8188 subRun: 0 event: 22980 at 26-Nov-2025 23:38:23 CET
26-Nov-2025 23:38:25 CET  Closed input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/ca/06/prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250928T122157Z_gen_002298_supernova_g4_detsim_20251113T194323Z_reco.root"

========================================================================================================================
TimeTracker printout (sec)                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================
Full event                             0.991927       1.81275       3.96364       1.6852       0.542124        500    
------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                0.00311016    0.00648782     0.0843177    0.00526412    0.00534987       500    
end_path:triggerAna:TriggerAnaTree     0.987779       1.80592       3.95743       1.67968      0.542247        500    
========================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 2189.01 MB
  Peak resident set size usage (VmHWM): 1441.35 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.

Processing output file prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA__trg_mc_2025a_tpg__tpg_dune10kt_1x2x6__reco__v10_12_01d01__triggerana_dune10kt_1x2x6__v10_12_01d01__ntuple__CENTRAL_prod_v2_s040000_l005000_20251126T181234_c45.root
merged pass1_CERN_000046.json
justIN time: 2025-12-21 06:20:01 UTC       justIN version: 01.05.03