justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 226470.95@dunegpschedd02.fnal.gov

Jobsub ID	226470.95@dunegpschedd02.fnal.gov
Workflow ID	8766
Stage ID	1
User name	mwrobel@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	GPU	No
	RSS bytes	5241831424 (4999 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-10-08 21:24:00
Site	US_PuertoRico
Entry	CMSHTPC_T3_US_PuertoRico_UPRM
Last heartbeat	2025-10-08 22:59:55
From worker node	Hostname	compute-0-5.local
	cpuinfo	Intel(R) Xeon(R) CPU W3550 @ 3.07GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	5242880000 (5000 MiB)
	Wall seconds limit	171000 (47 hours)
	GPU
	Inner Apptainer?	True
Job state	jobscript_error
Started	2025-10-08 21:25:13
Input files	ehn1-beam-np04:H4_v34b_-5GeV_-27.7_004384_231504_77_20250627T174320.root
Jobscript	Exit code	66
	Real time	0m (0s)
	CPU time	0m (0s = 0%)
	Max RSS bytes	0 (0 MiB)
Outputting started
Output files
Finished	2025-10-08 22:59:55
Saved logs	justin-logs:226470.95-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

eYSide is an accepted volume of 3651cm^3.

%MSG
%MSG-i BaseRadioGen:  Decay0Gen:co60apa@Construction 08-Oct-2025 17:26:01 AST  ModuleConstruction BaseRadioGen.h:330
  It is a box of size 7.62 x 585.68 x 10.16 cm^3

%MSG
%MSG-i BaseRadioGen:  Decay0Gen:co60apa@Construction 08-Oct-2025 17:26:01 AST  ModuleConstruction BaseRadioGen.h:324
  Volume volAPAFrameYCenter is an accepted volume of 3593.56cm^3.

%MSG
%MSG-i BaseRadioGen:  Decay0Gen:co60apa@Construction 08-Oct-2025 17:26:01 AST  ModuleConstruction BaseRadioGen.h:330
  It is a box of size 7.62 x 585.68 x 10.16 cm^3

%MSG
%MSG-i BaseRadioGen:  Decay0Gen:co60apa@Construction 08-Oct-2025 17:26:01 AST  ModuleConstruction BaseRadioGen.h:324
  Volume volAPAFrameZSide is an accepted volume of 2395.2cm^3.

%MSG
%MSG-i BaseRadioGen:  Decay0Gen:co60apa@Construction 08-Oct-2025 17:26:01 AST  ModuleConstruction BaseRadioGen.h:330
  It is a box of size 7.62 x 10.16 x 230.002 cm^3

%MSG
%MSG-i BaseRadioGen:  Decay0Gen:co60apa@Construction 08-Oct-2025 17:26:01 AST  ModuleConstruction BaseRadioGen.h:324
  Volume volAPAFrameYOuterSupport is an accepted volume of 1813.72cm^3.

%MSG
%MSG-i BaseRadioGen:  Decay0Gen:co60apa@Construction 08-Oct-2025 17:26:01 AST  ModuleConstruction BaseRadioGen.h:330
  It is a box of size 0.3048 x 585.68 x 10.16 cm^3

%MSG
%MSG-i BaseRadioGen:  Decay0Gen:co60apa@Construction 08-Oct-2025 17:26:01 AST  ModuleConstruction BaseRadioGen.h:347
4 volumes correspond to the regex ".*APA.*".

%MSG
%MSG-i BaseRadioGen:  Decay0Gen:co60apa@Construction 08-Oct-2025 17:26:01 AST  ModuleConstruction BaseRadioGen.h:349
1 materials correspond to the regex "STEEL_STAINLESS_Fe7Cr2Ni".

%MSG
%MSG-i BaseRadioGen:  Decay0Gen:co60apa@Construction 08-Oct-2025 17:26:01 AST  ModuleConstruction BaseRadioGen.h:415
20 nodes (i.e. instance of the volumes) satisfy both the regexes.

%MSG
%MSG-i RANDOM:  CORSIKAGen:cosmicgenerator@Construction  08-Oct-2025 17:26:01 AST ModuleConstruction
Instantiated HepJamesRandom engine "cosmicgenerator:0:gen" with seed 0.
%MSG
%MSG-i NuRandomService:  CORSIKAGen:cosmicgenerator@Construction  08-Oct-2025 17:26:01 AST ModuleConstruction
Seeding HepJamesRandom engine "cosmicgenerator:gen" with seed 836597977.
%MSG
%MSG-i RANDOM:  CORSIKAGen:cosmicgenerator@Construction  08-Oct-2025 17:26:01 AST ModuleConstruction
Instantiated HepJamesRandom engine "cosmicgenerator:0:pois" with seed 0.
%MSG
%MSG-i NuRandomService:  CORSIKAGen:cosmicgenerator@Construction  08-Oct-2025 17:26:01 AST ModuleConstruction
Seeding HepJamesRandom engine "cosmicgenerator:pois" with seed 53020707.
%MSG
%MSG-i CorsikaGen:  CORSIKAGen:cosmicgenerator@Construction  08-Oct-2025 17:26:01 AST ModuleConstruction
For /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/p_showers_*db:p_showers_*db
Found 73 candidate files
Choosing file number 58

Selected /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/p_showers_210167.db

%MSG
%MSG-i CorsikaGen:  CORSIKAGen:cosmicgenerator@Construction  08-Oct-2025 17:26:01 AST ModuleConstruction
For /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/He_showers_*db:He_showers_*db
Found 76 candidate files
Choosing file number 7

Selected /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/He_showers_310018.db

%MSG
%MSG-i CorsikaGen:  CORSIKAGen:cosmicgenerator@Construction  08-Oct-2025 17:26:02 AST ModuleConstruction
For /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/N_showers_*db:N_showers_*db
Found 81 candidate files
Choosing file number 60

Selected /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/N_showers_410151.db

%MSG
%MSG-i CorsikaGen:  CORSIKAGen:cosmicgenerator@Construction  08-Oct-2025 17:26:02 AST ModuleConstruction
For /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/Mg_showers_*db:Mg_showers_*db
Found 81 candidate files
Choosing file number 76

Selected /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/Mg_showers_510195.db

%MSG
%MSG-i CorsikaGen:  CORSIKAGen:cosmicgenerator@Construction  08-Oct-2025 17:26:02 AST ModuleConstruction
For /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/Fe_showers_*db:Fe_showers_*db
Found 79 candidate files
Choosing file number 63

Selected /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/Fe_showers_610154.db

%MSG
%MSG-i CorsikaGen:  CORSIKAGen:cosmicgenerator@Construction  08-Oct-2025 17:26:02 AST ModuleConstruction
Fetching: /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/p_showers_210167.db 31211520

%MSG
59+1 records in
59+1 records out
31211520 bytes (31 MB) copied, 51.61 s, 605 kB/s
%MSG-i CorsikaGen:  CORSIKAGen:cosmicgenerator@Construction  08-Oct-2025 17:26:54 AST ModuleConstruction
Fetching: /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/He_showers_310018.db 1622016

%MSG
3+1 records in
3+1 records out
1622016 bytes (1.6 MB) copied, 0.00371077 s, 437 MB/s
%MSG-i CorsikaGen:  CORSIKAGen:cosmicgenerator@Construction  08-Oct-2025 17:26:54 AST ModuleConstruction
Fetching: /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/N_showers_410151.db 145408

%MSG
0+1 records in
0+1 records out
145408 bytes (145 kB) copied, 0.000556296 s, 261 MB/s
%MSG-i CorsikaGen:  CORSIKAGen:cosmicgenerator@Construction  08-Oct-2025 17:26:54 AST ModuleConstruction
Fetching: /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/Mg_showers_510195.db 146432

%MSG
dd: failed to open '/cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/Mg_showers_510195.db': Input/output error
Wed Oct  8 17:27:37 2025
 program: dd bs=512k  if=/cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/Mg_showers_510195.db of=/var/tmp/ifdh_20104_0/Mg_showers_510195.dbexited status 1
delaying 10 ...
Wed Oct  8 17:27:47 2025
 retrying...
dd: failed to open '/cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/Mg_showers_510195.db': Input/output error
Wed Oct  8 17:28:29 2025
 program: dd bs=512k  if=/cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/Mg_showers_510195.db of=/var/tmp/ifdh_20104_0/Mg_showers_510195.dbexited status 1
delaying 31 ...
Wed Oct  8 17:29:00 2025
 retrying...
dd: failed to open '/cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/Mg_showers_510195.db': Input/output error
Wed Oct  8 17:29:41 2025
 program: dd bs=512k  if=/cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/Mg_showers_510195.db of=/var/tmp/ifdh_20104_0/Mg_showers_510195.dbexited status 1
delaying 106 ...
Wed Oct  8 17:31:27 2025
 retrying...
dd: failed to open '/cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/Mg_showers_510195.db': Input/output error
Wed Oct  8 17:32:08 2025
 program: dd bs=512k  if=/cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/Mg_showers_510195.db of=/var/tmp/ifdh_20104_0/Mg_showers_510195.dbexited status 1
delaying 101 ...
Wed Oct  8 17:33:49 2025
 retrying...
dd: failed to open '/cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/Mg_showers_510195.db': Input/output error
Wed Oct  8 17:34:43 2025
 program: dd bs=512k  if=/cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/Mg_showers_510195.db of=/var/tmp/ifdh_20104_0/Mg_showers_510195.dbexited status 1
delaying 99 ...
Wed Oct  8 17:36:22 2025
 retrying...
dd: failed to open '/cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/Mg_showers_510195.db': Input/output error
Wed Oct  8 17:37:07 2025
 program: dd bs=512k  if=/cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/Mg_showers_510195.db of=/var/tmp/ifdh_20104_0/Mg_showers_510195.dbexited status 1
delaying 581 ...
Wed Oct  8 17:46:48 2025
 retrying...
dd: failed to open '/cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/Mg_showers_510195.db': Input/output error
Wed Oct  8 17:47:30 2025
 program: dd bs=512k  if=/cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/Mg_showers_510195.db of=/var/tmp/ifdh_20104_0/Mg_showers_510195.dbexited status 1
delaying 1510 ...
Wed Oct  8 18:12:40 2025
 retrying...
dd: failed to open '/cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/Mg_showers_510195.db': Input/output error
Wed Oct  8 18:13:29 2025
 program: dd bs=512k  if=/cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/Mg_showers_510195.db of=/var/tmp/ifdh_20104_0/Mg_showers_510195.dbexited status 1
delaying 2045 ...
Wed Oct  8 18:47:34 2025
 retrying...
dd: failed to open '/cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/Mg_showers_510195.db': Input/output error
%MSG-s StdLibException:  CORSIKAGen:cosmicgenerator@Construction  08-Oct-2025 18:48:22 AST ModuleConstruction
Standard library exception caught in art
Copy Failed for: /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/Cosmics/CERN/CORSIKA/standard/Mg_showers_510195.db: exception: see error output
%MSG
Art has completed and will exit with status 66.
gen lar returns 66

justIN time: 2025-11-05 09:07:02 UTC       justIN version: 01.05.01