justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 231896.168@dunegpschedd01.fnal.gov

Jobsub ID	231896.168@dunegpschedd01.fnal.gov
Workflow ID	8766
Stage ID	1
User name	mwrobel@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	GPU	No
	RSS bytes	5241831424 (4999 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-10-08 21:20:36
Site	US_PuertoRico
Entry	CMSHTPC_T3_US_PuertoRico_UPRM
Last heartbeat	2025-10-08 21:56:41
From worker node	Hostname	compute-0-5.local
	cpuinfo	Intel(R) Xeon(R) CPU W3550 @ 3.07GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	5242880000 (5000 MiB)
	Wall seconds limit	171000 (47 hours)
	GPU
	Inner Apptainer?	True
Job state	jobscript_error
Started	2025-10-08 21:23:57
Input files	ehn1-beam-np04:H4_v34b_-5GeV_-27.7_008266_231505_37_20250627T174314.root
Jobscript	Exit code	66
	Real time	0m (0s)
	CPU time	0m (0s = 0%)
	Max RSS bytes	0 (0 MiB)
Outputting started
Output files
Finished	2025-10-08 21:56:41
Saved logs	justin-logs:231896.168-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

with seed 649370880.
%MSG
%MSG-i BaseRadioGen:  Decay0Gen:co60apa@Construction 08-Oct-2025 17:24:48 AST  ModuleConstruction BaseRadioGen.h:311
Calculating the volume of STEEL_STAINLESS_Fe7Cr2Ni and the volume .*APA.* in the geometry.

%MSG
%MSG-i BaseRadioGen:  Decay0Gen:co60apa@Construction 08-Oct-2025 17:24:48 AST  ModuleConstruction BaseRadioGen.h:324
  Volume volAPAFrameYSide is an accepted volume of 3651cm^3.

%MSG
%MSG-i BaseRadioGen:  Decay0Gen:co60apa@Construction 08-Oct-2025 17:24:48 AST  ModuleConstruction BaseRadioGen.h:330
  It is a box of size 7.62 x 585.68 x 10.16 cm^3

%MSG
%MSG-i BaseRadioGen:  Decay0Gen:co60apa@Construction 08-Oct-2025 17:24:48 AST  ModuleConstruction BaseRadioGen.h:324
  Volume volAPAFrameYCenter is an accepted volume of 3593.56cm^3.

%MSG
%MSG-i BaseRadioGen:  Decay0Gen:co60apa@Construction 08-Oct-2025 17:24:48 AST  ModuleConstruction BaseRadioGen.h:330
  It is a box of size 7.62 x 585.68 x 10.16 cm^3

%MSG
%MSG-i BaseRadioGen:  Decay0Gen:co60apa@Construction 08-Oct-2025 17:24:48 AST  ModuleConstruction BaseRadioGen.h:324
  Volume volAPAFrameZSide is an accepted volume of 2395.2cm^3.

%MSG
%MSG-i BaseRadioGen:  Decay0Gen:co60apa@Construction 08-Oct-2025 17:24:48 AST  ModuleConstruction BaseRadioGen.h:330
  It is a box of size 7.62 x 10.16 x 230.002 cm^3

%MSG
%MSG-i BaseRadioGen:  Decay0Gen:co60apa@Construction 08-Oct-2025 17:24:48 AST  ModuleConstruction BaseRadioGen.h:324
  Volume volAPAFrameYOuterSupport is an accepted volume of 1813.72cm^3.

%MSG
%MSG-i BaseRadioGen:  Decay0Gen:co60apa@Construction 08-Oct-2025 17:24:48 AST  ModuleConstruction BaseRadioGen.h:330
  It is a box of size 0.3048 x 585.68 x 10.16 cm^3

%MSG
%MSG-i BaseRadioGen:  Decay0Gen:co60apa@Construction 08-Oct-2025 17:24:48 AST  ModuleConstruction BaseRadioGen.h:347
4 volumes correspond to the regex ".*APA.*".

%MSG
%MSG-i BaseRadioGen:  Decay0Gen:co60apa@Construction 08-Oct-2025 17:24:48 AST  ModuleConstruction BaseRadioGen.h:349
1 materials correspond to the regex "STEEL_STAINLESS_Fe7Cr2Ni".

%MSG
%MSG-i BaseRadioGen:  Decay0Gen:co60apa@Construction 08-Oct-2025 17:24:48 AST  ModuleConstruction BaseRadioGen.h:415
20 nodes (i.e. instance of the volumes) satisfy both the regexes.

%MSG
%MSG-i RANDOM:  CORSIKAGen:cosmicgenerator@Construction  08-Oct-2025 17:24:48 AST ModuleConstruction
Instantiated HepJamesRandom engine "cosmicgenerator:0:gen" with seed 0.
%MSG
%MSG-i NuRandomService:  CORSIKAGen:cosmicgenerator@Construction  08-Oct-2025 17:24:48 AST ModuleConstruction
Seeding HepJamesRandom engine "cosmicgenerator:gen" with seed 464486106.
%MSG
%MSG-i RANDOM:  CORSIKAGen:cosmicgenerator@Construction  08-Oct-2025 17:24:48 AST ModuleConstruction
Instantiated HepJamesRandom engine "cosmicgenerator:0:pois" with seed 0.
%MSG
%MSG-i NuRandomService:  CORSIKAGen:cosmicgenerator@Construction  08-Oct-2025 17:24:48 AST ModuleConstruction
Seeding HepJamesRandom engine "cosmicgenerator:pois" with seed 514926697.
%MSG
%MSG-i CorsikaGen:  CORSIKAGen:cosmicgenerator@Construction  08-Oct-2025 17:24:48 AST ModuleConstruction
For /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/p_showers_*db:p_showers_*db
Found 73 candidate files
Choosing file number 45

Selected /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/p_showers_210146.db

%MSG
%MSG-i CorsikaGen:  CORSIKAGen:cosmicgenerator@Construction  08-Oct-2025 17:24:48 AST ModuleConstruction
For /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/He_showers_*db:He_showers_*db
Found 76 candidate files
Choosing file number 60

Selected /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/He_showers_310156.db

%MSG
%MSG-i CorsikaGen:  CORSIKAGen:cosmicgenerator@Construction  08-Oct-2025 17:24:49 AST ModuleConstruction
For /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/N_showers_*db:N_showers_*db
Found 81 candidate files
Choosing file number 1

Selected /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/N_showers_410006.db

%MSG
%MSG-i CorsikaGen:  CORSIKAGen:cosmicgenerator@Construction  08-Oct-2025 17:24:49 AST ModuleConstruction
For /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/Mg_showers_*db:Mg_showers_*db
Found 81 candidate files
Choosing file number 50

Selected /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/Mg_showers_510126.db

%MSG
%MSG-i CorsikaGen:  CORSIKAGen:cosmicgenerator@Construction  08-Oct-2025 17:24:49 AST ModuleConstruction
For /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/Fe_showers_*db:Fe_showers_*db
Found 79 candidate files
Choosing file number 35

Selected /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/Fe_showers_610087.db

%MSG
%MSG-i CorsikaGen:  CORSIKAGen:cosmicgenerator@Construction  08-Oct-2025 17:24:49 AST ModuleConstruction
Fetching: /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/p_showers_210146.db 31227904

%MSG
dd: error reading '/cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/p_showers_210146.db': Input/output error
0+0 records in
0+0 records out
0 bytes (0 B) copied, 107.048 s, 0.0 kB/s
Wed Oct  8 17:26:36 2025
 program: dd bs=512k  if=/cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/p_showers_210146.db of=/var/tmp/ifdh_20104_0/p_showers_210146.dbexited status 1
delaying 6 ...
Wed Oct  8 17:26:42 2025
 retrying...
dd: error reading '/cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/p_showers_210146.db': Input/output error
0+0 records in
0+0 records out
0 bytes (0 B) copied, 50.356 s, 0.0 kB/s
Wed Oct  8 17:27:33 2025
 program: dd bs=512k  if=/cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/p_showers_210146.db of=/var/tmp/ifdh_20104_0/p_showers_210146.dbexited status 1
delaying 52 ...
Wed Oct  8 17:28:25 2025
 retrying...
dd: error reading '/cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/p_showers_210146.db': Input/output error
0+0 records in
0+0 records out
0 bytes (0 B) copied, 41.89 s, 0.0 kB/s
Wed Oct  8 17:29:07 2025
 program: dd bs=512k  if=/cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/p_showers_210146.db of=/var/tmp/ifdh_20104_0/p_showers_210146.dbexited status 1
delaying 26 ...
Wed Oct  8 17:29:33 2025
 retrying...
dd: error reading '/cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/p_showers_210146.db': Input/output error
0+0 records in
0+0 records out
0 bytes (0 B) copied, 42.8603 s, 0.0 kB/s
Wed Oct  8 17:30:16 2025
 program: dd bs=512k  if=/cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/p_showers_210146.db of=/var/tmp/ifdh_20104_0/p_showers_210146.dbexited status 1
delaying 49 ...
Wed Oct  8 17:31:05 2025
 retrying...
dd: error reading '/cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/p_showers_210146.db': Input/output error
0+0 records in
0+0 records out
0 bytes (0 B) copied, 33.0885 s, 0.0 kB/s
Wed Oct  8 17:31:38 2025
 program: dd bs=512k  if=/cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/p_showers_210146.db of=/var/tmp/ifdh_20104_0/p_showers_210146.dbexited status 1
delaying 120 ...
Wed Oct  8 17:33:38 2025
 retrying...
dd: error reading '/cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/p_showers_210146.db': Input/output error
0+0 records in
0+0 records out
0 bytes (0 B) copied, 42.2991 s, 0.0 kB/s
Wed Oct  8 17:34:21 2025
 program: dd bs=512k  if=/cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/p_showers_210146.db of=/var/tmp/ifdh_20104_0/p_showers_210146.dbexited status 1
delaying 347 ...
Wed Oct  8 17:40:08 2025
 retrying...
dd: error reading '/cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/p_showers_210146.db': Input/output error
0+0 records in
0+0 records out
0 bytes (0 B) copied, 127.592 s, 0.0 kB/s
Wed Oct  8 17:42:16 2025
 program: dd bs=512k  if=/cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/p_showers_210146.db of=/var/tmp/ifdh_20104_0/p_showers_210146.dbexited status 1
delaying 67 ...
Wed Oct  8 17:43:23 2025
 retrying...
dd: error reading '/cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/p_showers_210146.db': Input/output error
0+0 records in
0+0 records out
0 bytes (0 B) copied, 33.1186 s, 0.0 kB/s
Wed Oct  8 17:43:57 2025
 program: dd bs=512k  if=/cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/p_showers_210146.db of=/var/tmp/ifdh_20104_0/p_showers_210146.dbexited status 1
delaying 409 ...
Wed Oct  8 17:50:46 2025
 retrying...
dd: error reading '/cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/p_showers_210146.db': Input/output error
0+0 records in
0+0 records out
0 bytes (0 B) copied, 42.854 s, 0.0 kB/s
%MSG-s StdLibException:  CORSIKAGen:cosmicgenerator@Construction  08-Oct-2025 17:51:29 AST ModuleConstruction
Standard library exception caught in art
Copy Failed for: /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/p_showers_210146.db: exception: see error output
%MSG
Art has completed and will exit with status 66.
gen lar returns 66

justIN time: 2025-11-05 09:07:05 UTC       justIN version: 01.05.01