Jobsub ID 268482.1@dunegpschedd01.fnal.gov

Jobsub ID	268482.1@dunegpschedd01.fnal.gov
Workflow ID	11221
Stage ID	1
User name	ykermaid@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	GPU	No
	RSS bytes	14680064000 (14000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-12-15 15:55:17
Site	US_UChicago
Entry	Engage_US_MWT2_iut2_gk02_condce_mcore
Last heartbeat	2025-12-15 19:42:26
From worker node	Hostname	uct2-c622.mwt2.org
	cpuinfo	AMD EPYC 7302 16-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	15728640000 (15000 MiB)
	Wall seconds limit	86400 (24 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-12-15 15:56:17
Input files	monte-carlo-011221-000001
Jobscript	Exit code	0
	Real time	3h (12662s)
	CPU time	3h (13346s = 105%)
	Max RSS bytes	12872400896 (12276 MiB)
Outputting started	2025-12-15 19:27:20
Output files	usertests:prod_cosmics_protodunevd_1_268482_1_1765814194_gen_g4_stage1_g4_stage2_detsim.err usertests:prod_cosmics_protodunevd_1_268482_1_1765814194_gen_g4_stage1_g4_stage2.err usertests:prod_cosmics_protodunevd_1_268482_1_1765814194_gen.root usertests:prod_cosmics_protodunevd_1_268482_1_1765814194_gen_g4_stage1_g4_stage2.root usertests:prod_cosmics_protodunevd_1_268482_1_1765814194_gen_g4_stage1.root usertests:prod_cosmics_protodunevd_1_268482_1_1765814194_gen_g4_stage1_g4_stage2.log usertests:prod_cosmics_protodunevd_1_268482_1_1765814194_gen_g4_stage1_g4_stage2_detsim.log usertests:prod_cosmics_protodunevd_1_268482_1_1765814194_gen_g4_stage1_g4_stage2.root.json usertests:prod_cosmics_protodunevd_1_268482_1_1765814194_gen.err usertests:prod_cosmics_protodunevd_1_268482_1_1765814194_gen_g4_stage1.err usertests:prod_cosmics_protodunevd_1_268482_1_1765814194_gen.log usertests:prod_cosmics_protodunevd_1_268482_1_1765814194_gen_g4_stage1_g4_stage2_detsim.root usertests:prod_cosmics_protodunevd_1_268482_1_1765814194_gen_g4_stage1.log
Finished	2025-12-15 19:42:26
Saved logs	justin-logs:268482.1-dunegpschedd01.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

00 00:00 0 
1538f846c000-1538f846d000 rw-p 00000000 00:00 0 
1538f846d000-1538f846e000 r--p 00000000 00:00 0 
1538f846e000-1538f846f000 r-xp 00000000 00:00 0 
1538f846f000-1538f8470000 rw-p 00000000 00:00 0 
1538f8470000-1538f8471000 r--p 00000000 00:00 0 
1538f8471000-1538f8472000 r-xp 00000000 00:00 0 
1538f8472000-1538f8473000 rw-p 00000000 00:00 0 
1538f8473000-1538f8474000 r--p 00000000 00:00 0 
1538f8474000-1538f8475000 r-xp 00000000 00:00 0 
1538f8475000-1538f8476000 rw-p 00000000 00:00 0 
1538f8476000-1538f8477000 r--p 00000000 00:00 0 
1538f8477000-1538f84c7000 rw-p 00000000 00:00 0 
1538f84c7000-1538f8600000 r--p 00000000 00:b6 191014                     /cvmfs/larsoft.opensciencegrid.org/products/root/v6_28_12/Linux64bit+3.10-2.17-e26-p3915-prof/lib/MultiProc.pcm
1538f8600000-1538f86e9000 r-xp 00000000 00:b6 2716492                    /cvmfs/larsoft.opensciencegrid.org/products/art/v3_14_04/slf7.x86_64.e26.prof/lib/libart_Framework_Art.so
1538f86e9000-1538f88e9000 ---p 000e9000 00:b6 2716492                    /cvmfs/larsoft.opensciencegrid.org/products/art/v3_14_04/slf7.x86_64.e26.prof/lib/libart_Framework_Art.so
1538f88e9000-1538f88ec000 r--p 000e9000 00:b6 2716492                    /cvmfs/larsoft.opensciencegrid.org/products/art/v3_14_04/slf7.x86_64.e26.prof/lib/libart_Framework_Art.so
1538f88ec000-1538f88ee000 rw-p 000ec000 00:b6 2716492                    /cvmfs/larsoft.opensciencegrid.org/products/art/v3_14_04/slf7.x86_64.e26.prof/lib/libart_Framework_Art.so
1538f88ee000-1538f88ef000 r-xp 00000000 00:00 0 
1538f88ef000-1538f8910000 rw-p 00000000 00:00 0 
1538f8910000-1538f8911000 r--p 00000000 00:00 0 
1538f8911000-1538f8912000 r-xp 00000000 00:00 0 
1538f8912000-1538f8913000 rw-p 00000000 00:00 0 
1538f8913000-1538f8914000 r--p 00000000 00:00 0 
1538f8914000-1538f8915000 r-xp 00000000 00:00 0 
1538f8915000-1538f8917000 rw-p 00000000 00:00 0 
1538f8917000-1538f8918000 r--p 00000000 00:00 0 
1538f8918000-1538f8919000 r-xp 00000000 00:00 0 
1538f8919000-1538f891a000 rw-p 00000000 00:00 0 
1538f891a000-1538f891b000 r--p 00000000 00:00 0 
1538f891b000-1538f891c000 r-xp 00000000 00:00 0 
1538f891c000-1538f891d000 rw-p 00000000 00:00 0 
1538f891d000-1538f891e000 r--p 00000000 00:00 0 
1538f891e000-1538f891f000 r-xp 00000000 00:00 0 
1538f891f000-1538f8920000 rw-p 00000000 00:00 0 
1538f8920000-1538f8921000 r--p 00000000 00:00 0 
1538f8921000-1538f8922000 r-xp 00000000 00:00 0 
1538f8922000-1538f8923000 rw-p 00000000 00:00 0 
1538f8923000-1538f898e000 r--p 00000000 00:b6 192143                     /cvmfs/larsoft.opensciencegrid.org/products/root/v6_28_12/Linux64bit+3.10-2.17-e26-p3915-prof/lib/RHTTPSniff.pcm
1538f898e000-1538f8a00000 r--p 00000000 00:b6 192403                     /cvmfs/larsoft.opensciencegrid.org/products/root/v6_28_12/Linux64bit+3.10-2.17-e26-p3915-prof/lib/XMLParser.pcm
1538f8a00000-1538f8a01000 r-xp 00000000 00:85 14569                      /usr/libexec/coreutils/libstdbuf.so
1538f8a01000-1538f8c00000 ---p 00001000 00:85 14569                      /usr/libexec/coreutils/libstdbuf.so
1538f8c00000-1538f8c01000 r--p 00000000 00:85 14569                      /usr/libexec/coreutils/libstdbuf.so
1538f8c01000-1538f8c02000 rw-p 00001000 00:85 14569                      /usr/libexec/coreutils/libstdbuf.so
1538f8c02000-1538f8c03000 r--p 00000000 00:00 0 
1538f8c03000-1538f8c04000 r-xp 00000000 00:00 0 
1538f8c04000-1538f8c05000 rw-p 00000000 00:00 0 
1538f8c05000-1538f8c06000 r--p 00000000 00:00 0 
1538f8c06000-1538f8c07000 r-xp 00000000 00:00 0 
1538f8c07000-1538f8c6c000 r--p 00000000 00:b6 192370                     /cvmfs/larsoft.opensciencegrid.org/products/root/v6_28_12/Linux64bit+3.10-2.17-e26-p3915-prof/lib/RCsg.pcm
1538f8c6c000-1538f8cf6000 r--p 00000000 00:b6 192174                     /cvmfs/larsoft.opensciencegrid.org/products/root/v6_28_12/Linux64bit+3.10-2.17-e26-p3915-prof/lib/Unfold.pcm
1538f8cf6000-1538f8d5b000 r--p 00000000 00:b6 190902                     /cvmfs/larsoft.opensciencegrid.org/products/root/v6_28_12/Linux64bit+3.10-2.17-e26-p3915-prof/lib/Rint.pcm
1538f8d5b000-1538f8de1000 r--p 00000000 00:b6 192547                     /cvmfs/larsoft.opensciencegrid.org/products/root/v6_28_12/Linux64bit+3.10-2.17-e26-p3915-prof/lib/ROOT_Foundation_Stage1_NoRTTI.pcm
1538f8de1000-1538f8e00000 r--p 00000000 00:b6 193915                     /cvmfs/larsoft.opensciencegrid.org/products/root/v6_28_12/Linux64bit+3.10-2.17-e26-p3915-prof/lib/ROOT_Config.pcm
1538f8e00000-1538f8e22000 r-xp 00000000 00:85 15188                      /usr/lib64/ld-2.17.so
1538f8e22000-1538f8e23000 rw-p 00000000 00:00 0 
1538f8e23000-1538f8e24000 r--p 00000000 00:00 0 
1538f8e24000-1538f8e25000 r-xp 00000000 00:00 0 
1538f8e25000-1538f8e26000 rw-p 00000000 00:00 0 
1538f8e26000-1538f8e27000 r--p 00000000 00:00 0 
1538f8e27000-1538f8e28000 r-xp 00000000 00:00 0 
1538f8e28000-1538f8e29000 rw-p 00000000 00:00 0 
1538f8e29000-1538f8e2a000 r--p 00000000 00:00 0 
1538f8e2a000-1538f8e2b000 r-xp 00000000 00:00 0 
1538f8e2b000-1538f8e2c000 rw-p 00000000 00:00 0 
1538f8e2c000-1538f8e2d000 r--p 00000000 00:00 0 
1538f8e2d000-1538f8e2e000 r-xp 00000000 00:00 0 
1538f8e2e000-1538f8e2f000 rw-p 00000000 00:00 0 
1538f8e2f000-1538f8e30000 r--p 00000000 00:00 0 
1538f8e30000-1538f8e31000 r-xp 00000000 00:00 0 
1538f8e31000-1538f8e32000 rw-p 00000000 00:00 0 
1538f8e32000-1538f8e57000 r--p 00000000 00:b6 194235                     /cvmfs/larsoft.opensciencegrid.org/products/root/v6_28_12/Linux64bit+3.10-2.17-e26-p3915-prof/lib/ROOT_Rtypes.pcm
1538f8e57000-1538f8eab000 r--p 00000000 00:b6 192281                     /cvmfs/larsoft.opensciencegrid.org/products/root/v6_28_12/Linux64bit+3.10-2.17-e26-p3915-prof/lib/ROOT_Foundation_C.pcm
1538f8eab000-1538f8ed8000 r--p 00000000 00:b6 192468                     /cvmfs/larsoft.opensciencegrid.org/products/root/v6_28_12/Linux64bit+3.10-2.17-e26-p3915-prof/lib/Cling_Runtime_Extra.pcm
1538f8ed8000-1538f9021000 r--p 00000000 00:b6 192290                     /cvmfs/larsoft.opensciencegrid.org/products/root/v6_28_12/Linux64bit+3.10-2.17-e26-p3915-prof/lib/libc.pcm
1538f9021000-1538f9022000 r--p 00021000 00:85 15188                      /usr/lib64/ld-2.17.so
1538f9022000-1538f9023000 rw-p 00022000 00:85 15188                      /usr/lib64/ld-2.17.so
1538f9023000-1538f9024000 rw-p 00000000 00:00 0 
1538f9024000-1538f9025000 r--p 00000000 00:00 0 
1538f9025000-1538f9026000 r-xp 00000000 00:00 0 
1538f9026000-1538f9027000 rw-p 00000000 00:00 0 
1538f9027000-1538f9028000 r--p 00000000 00:00 0 
1538f9028000-1538f9029000 r-xp 00000000 00:00 0 
1538f9029000-1538f902a000 rw-p 00000000 00:00 0 
1538f902a000-1538f902b000 r--p 00000000 00:00 0 
1538f902b000-1538f902c000 r-xp 00000000 00:00 0 
1538f902c000-1538f902d000 rw-p 00000000 00:00 0 
1538f902d000-1538f903e000 r--p 00000000 00:b6 158099                     /cvmfs/larsoft.opensciencegrid.org/products/gcc/v12_1_0/Linux64bit+3.10-2.17/include/c++/12.1.0/x86_64-pc-linux-gnu/bits/c++config.h
1538f903e000-1538f905d000 r--p 00000000 00:b6 193924                     /cvmfs/larsoft.opensciencegrid.org/products/root/v6_28_12/Linux64bit+3.10-2.17-e26-p3915-prof/lib/Cling_Runtime.pcm
1538f905d000-1538f90aa000 r--p 00000000 00:b6 192820                     /cvmfs/larsoft.opensciencegrid.org/products/root/v6_28_12/Linux64bit+3.10-2.17-e26-p3915-prof/lib/modules.idx
1538f90aa000-1538f90b8000 r--p 00000000 00:b6 192255                     /cvmfs/larsoft.opensciencegrid.org/products/root/v6_28_12/Linux64bit+3.10-2.17-e26-p3915-prof/lib/_Builtin_stddef_max_align_t.pcm
1538f90b8000-1538f90c6000 r--p 00000000 00:b6 193938                     /cvmfs/larsoft.opensciencegrid.org/products/root/v6_28_12/Linux64bit+3.10-2.17-e26-p3915-prof/lib/xlocale.h.pcm
1538f90c6000-1538f913a000 rw-p 00000000 00:00 0 
1538f913a000-1538f913b000 rw-p 00000000 00:00 0 
1538f913b000-1538f913c000 r--p 00000000 00:00 0 
1538f913c000-1538f913d000 r-xp 00000000 00:00 0 
1538f913d000-1538f913e000 rw-p 00000000 00:00 0 
1538f913e000-1538f913f000 r--p 00000000 00:00 0 
1538f913f000-1538f9140000 r-xp 00000000 00:00 0 
1538f9140000-1538f9141000 rw-p 00000000 00:00 0 
1538f9141000-1538f9142000 r--p 00000000 00:00 0 
1538f9142000-1538f9143000 r-xp 00000000 00:00 0 
1538f9143000-1538f9144000 rw-p 00000000 00:00 0 
1538f9144000-1538f9145000 r--p 00000000 00:00 0 
1538f9145000-1538f9146000 r-xp 00000000 00:00 0 
1538f9146000-1538f9151000 rw-p 00000000 00:00 0 
7ffe13772000-7ffe137b3000 rw-p 00000000 00:00 0                          [stack]
7ffe137f0000-7ffe137f4000 r--p 00000000 00:00 0                          [vvar]
7ffe137f4000-7ffe137f6000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 --xp 00000000 00:00 0                  [vsyscall]

ls: cannot access *_reco.root: No such file or directory
Forming g4 metadata
  G4 file: prod_cosmics_protodunevd_1_268482_1_1765814194_gen_g4_stage1_g4_stage2.root
    KEY: g4_stage2
    FCL:  protodunevd_g4_stage2.fcl
TClass::Init:0: RuntimeWarning: no dictionary for class sim::IDE is available
Successfully opened file prod_cosmics_protodunevd_1_268482_1_1765814194_gen_g4_stage1_g4_stage2.root
  Ran successfully
{
  "name": "prod_cosmics_protodunevd_1_268482_1_1765814194_gen_g4_stage1_g4_stage2.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_12_01d00",
    "core.data_tier": "detector-simulated",
    "core.runs": "1",
    "core.runs_subruns": "1",
    "dune.config_file": "protodunevd_g4_stage2.fcl",
    "dune.campaign": "pdvd-cosmics-test",
    "core.start_time": 1765814193.0,
    "core.end_time": 1765826833.0,
    "core.subruns": "1",
    "core.events": [
      1,
      2,
      3,
      4,
      5,
      6,
      7,
      8,
      9,
      10
    ],
    "core.event_count": 10,
    "core.first_event_number": 1,
    "core.last_event_number": 10
  }
}Forming reco metadata