justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 258926.12@dunegpschedd01.fnal.gov

Jobsub ID258926.12@dunegpschedd01.fnal.gov
Workflow ID10811
Stage ID1
User nameschellma@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-11-26 18:20:56
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce505
Last heartbeat2025-11-26 22:11:58
From worker nodeHostnameb9p06p8389.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job stateoutputting_failed
Started2025-11-26 21:39:25
Input filesmonte-carlo-010811-000017
JobscriptExit code0
Real time25m (1555s)
CPU time20m (1224s = 78%)
Max RSS bytes1435353088 (1368 MiB)
Outputting started2025-11-26 22:05:21
Output files
Finished2025-11-26 22:11:58
List job events     (HTCondor job logs unavailable)

Jobscript log (last 10,000 characters)

en_003453_supernova_g4_detsim_20251113T185122Z_reco.root"
Begin processing the 451st record. run: 8146 subRun: 0 event: 34521 at 26-Nov-2025 23:02:44 CET
Begin processing the 452nd record. run: 8146 subRun: 0 event: 34522 at 26-Nov-2025 23:02:46 CET
Begin processing the 453rd record. run: 8146 subRun: 0 event: 34523 at 26-Nov-2025 23:02:49 CET
Begin processing the 454th record. run: 8146 subRun: 0 event: 34524 at 26-Nov-2025 23:02:52 CET
Begin processing the 455th record. run: 8146 subRun: 0 event: 34525 at 26-Nov-2025 23:02:55 CET
Begin processing the 456th record. run: 8146 subRun: 0 event: 34526 at 26-Nov-2025 23:02:58 CET
Begin processing the 457th record. run: 8146 subRun: 0 event: 34527 at 26-Nov-2025 23:03:01 CET
Begin processing the 458th record. run: 8146 subRun: 0 event: 34528 at 26-Nov-2025 23:03:05 CET
Begin processing the 459th record. run: 8146 subRun: 0 event: 34529 at 26-Nov-2025 23:03:08 CET
Begin processing the 460th record. run: 8146 subRun: 0 event: 34530 at 26-Nov-2025 23:03:10 CET
26-Nov-2025 23:03:13 CET  Closed input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/6a/47/prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250927T091954Z_gen_003453_supernova_g4_detsim_20251113T185122Z_reco.root"
26-Nov-2025 23:03:13 CET  Initiating request to open input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/f0/61/prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250927T091958Z_gen_003460_supernova_g4_detsim_20251113T172438Z_reco.root"
26-Nov-2025 23:03:13 CET  Opened input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/f0/61/prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250927T091958Z_gen_003460_supernova_g4_detsim_20251113T172438Z_reco.root"
Begin processing the 461st record. run: 8146 subRun: 0 event: 34591 at 26-Nov-2025 23:03:14 CET
Begin processing the 462nd record. run: 8146 subRun: 0 event: 34592 at 26-Nov-2025 23:03:18 CET
Begin processing the 463rd record. run: 8146 subRun: 0 event: 34593 at 26-Nov-2025 23:03:20 CET
Begin processing the 464th record. run: 8146 subRun: 0 event: 34594 at 26-Nov-2025 23:03:23 CET
Begin processing the 465th record. run: 8146 subRun: 0 event: 34595 at 26-Nov-2025 23:03:25 CET
Begin processing the 466th record. run: 8146 subRun: 0 event: 34596 at 26-Nov-2025 23:03:28 CET
Begin processing the 467th record. run: 8146 subRun: 0 event: 34597 at 26-Nov-2025 23:03:32 CET
Begin processing the 468th record. run: 8146 subRun: 0 event: 34598 at 26-Nov-2025 23:03:35 CET
Begin processing the 469th record. run: 8146 subRun: 0 event: 34599 at 26-Nov-2025 23:03:39 CET
Begin processing the 470th record. run: 8146 subRun: 0 event: 34600 at 26-Nov-2025 23:03:41 CET
26-Nov-2025 23:03:44 CET  Closed input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/f0/61/prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250927T091958Z_gen_003460_supernova_g4_detsim_20251113T172438Z_reco.root"
26-Nov-2025 23:03:44 CET  Initiating request to open input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/10/41/prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250927T092052Z_gen_003485_supernova_g4_detsim_20251113T175516Z_reco.root"
26-Nov-2025 23:03:44 CET  Opened input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/10/41/prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250927T092052Z_gen_003485_supernova_g4_detsim_20251113T175516Z_reco.root"
Begin processing the 471st record. run: 8146 subRun: 0 event: 34841 at 26-Nov-2025 23:03:44 CET
Begin processing the 472nd record. run: 8146 subRun: 0 event: 34842 at 26-Nov-2025 23:03:47 CET
Begin processing the 473rd record. run: 8146 subRun: 0 event: 34843 at 26-Nov-2025 23:03:50 CET
Begin processing the 474th record. run: 8146 subRun: 0 event: 34844 at 26-Nov-2025 23:03:52 CET
Begin processing the 475th record. run: 8146 subRun: 0 event: 34845 at 26-Nov-2025 23:03:55 CET
Begin processing the 476th record. run: 8146 subRun: 0 event: 34846 at 26-Nov-2025 23:03:59 CET
Begin processing the 477th record. run: 8146 subRun: 0 event: 34847 at 26-Nov-2025 23:04:01 CET
Begin processing the 478th record. run: 8146 subRun: 0 event: 34848 at 26-Nov-2025 23:04:04 CET
Begin processing the 479th record. run: 8146 subRun: 0 event: 34849 at 26-Nov-2025 23:04:07 CET
Begin processing the 480th record. run: 8146 subRun: 0 event: 34850 at 26-Nov-2025 23:04:09 CET
26-Nov-2025 23:04:11 CET  Closed input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/10/41/prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250927T092052Z_gen_003485_supernova_g4_detsim_20251113T175516Z_reco.root"
26-Nov-2025 23:04:11 CET  Initiating request to open input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/90/d2/prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250927T092148Z_gen_003501_supernova_g4_detsim_20251105T150507Z_reco.root"
26-Nov-2025 23:04:11 CET  Opened input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/90/d2/prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250927T092148Z_gen_003501_supernova_g4_detsim_20251105T150507Z_reco.root"
Begin processing the 481st record. run: 8146 subRun: 0 event: 35001 at 26-Nov-2025 23:04:12 CET
Begin processing the 482nd record. run: 8146 subRun: 0 event: 35002 at 26-Nov-2025 23:04:15 CET
Begin processing the 483rd record. run: 8146 subRun: 0 event: 35003 at 26-Nov-2025 23:04:20 CET
Begin processing the 484th record. run: 8146 subRun: 0 event: 35004 at 26-Nov-2025 23:04:22 CET
Begin processing the 485th record. run: 8146 subRun: 0 event: 35005 at 26-Nov-2025 23:04:25 CET
Begin processing the 486th record. run: 8146 subRun: 0 event: 35006 at 26-Nov-2025 23:04:29 CET
Begin processing the 487th record. run: 8146 subRun: 0 event: 35007 at 26-Nov-2025 23:04:31 CET
Begin processing the 488th record. run: 8146 subRun: 0 event: 35008 at 26-Nov-2025 23:04:34 CET
Begin processing the 489th record. run: 8146 subRun: 0 event: 35009 at 26-Nov-2025 23:04:37 CET
Begin processing the 490th record. run: 8146 subRun: 0 event: 35010 at 26-Nov-2025 23:04:39 CET
26-Nov-2025 23:04:42 CET  Closed input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/90/d2/prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250927T092148Z_gen_003501_supernova_g4_detsim_20251105T150507Z_reco.root"
26-Nov-2025 23:04:42 CET  Initiating request to open input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/5c/8c/prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250927T092157Z_gen_003508_supernova_g4_detsim_20251105T175150Z_reco.root"
26-Nov-2025 23:04:42 CET  Opened input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/5c/8c/prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250927T092157Z_gen_003508_supernova_g4_detsim_20251105T175150Z_reco.root"
Begin processing the 491st record. run: 8146 subRun: 0 event: 35071 at 26-Nov-2025 23:04:42 CET
Begin processing the 492nd record. run: 8146 subRun: 0 event: 35072 at 26-Nov-2025 23:04:45 CET
Begin processing the 493rd record. run: 8146 subRun: 0 event: 35073 at 26-Nov-2025 23:04:48 CET
Begin processing the 494th record. run: 8146 subRun: 0 event: 35074 at 26-Nov-2025 23:04:50 CET
Begin processing the 495th record. run: 8146 subRun: 0 event: 35075 at 26-Nov-2025 23:04:54 CET
Begin processing the 496th record. run: 8146 subRun: 0 event: 35076 at 26-Nov-2025 23:04:58 CET
Begin processing the 497th record. run: 8146 subRun: 0 event: 35077 at 26-Nov-2025 23:05:01 CET
Begin processing the 498th record. run: 8146 subRun: 0 event: 35078 at 26-Nov-2025 23:05:03 CET
Begin processing the 499th record. run: 8146 subRun: 0 event: 35079 at 26-Nov-2025 23:05:06 CET
Begin processing the 500th record. run: 8146 subRun: 0 event: 35080 at 26-Nov-2025 23:05:09 CET
26-Nov-2025 23:05:11 CET  Closed input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/5c/8c/prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250927T092157Z_gen_003508_supernova_g4_detsim_20251105T175150Z_reco.root"

========================================================================================================================
TimeTracker printout (sec)                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================
Full event                              1.03712       2.16631       5.17434       2.03911      0.705327        500    
------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                0.00331798    0.00691293     0.0314943    0.00570695    0.00395612       500    
end_path:triggerAna:TriggerAnaTree      1.03264        2.159        5.16913       2.03244       0.70506        500    
========================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 2202.17 MB
  Peak resident set size usage (VmHWM): 1435.35 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.

Processing output file prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA__trg_mc_2025a_tpg__tpg_dune10kt_1x2x6__reco__v10_12_01d01__triggerana_dune10kt_1x2x6__v10_12_01d01__ntuple__CENTRAL_prod_v2_s045000_l005000_20251126T181341_c16.root
merged pass1_CERN_000017.json
justIN time: 2025-12-21 06:36:31 UTC       justIN version: 01.05.03