justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 14353.0@dunegpschedd02.fnal.gov

Jobsub ID14353.0@dunegpschedd02.fnal.gov
Workflow ID325
Stage ID1
User namemsiden@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8387559424 (7999 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-08-04 15:52:01
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc02
Last heartbeat2025-08-04 15:56:41
From worker nodeHostnamewn-da-25.gina.surf.nl
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statejobscript_error
Started2025-08-04 15:55:18
Input fileshd-protodune:np04hd_raw_run027087_0008_dataflow3_datawriter_0_20240614T144941.hdf5
JobscriptExit code1
Real time0m (0s)
CPU time0m (0s = 0%)
Max RSS bytes0 (0 MiB)
Outputting started 
Output files
Finished2025-08-04 15:56:41
Saved logsjustin-logs:14353.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

_0/He_showers_310015.db

%MSG
%MSG-i CORSIKAGen:  CORSIKAGen:cosmicgenerator@Construction  04-Aug-2025 17:56:07 CEST ModuleConstruction
Attached db /var/tmp/ifdh_49759_0/N_showers_410052.db

%MSG
%MSG-i CORSIKAGen:  CORSIKAGen:cosmicgenerator@Construction  04-Aug-2025 17:56:07 CEST ModuleConstruction
Attached db /var/tmp/ifdh_49759_0/Mg_showers_510032.db

%MSG
%MSG-i CORSIKAGen:  CORSIKAGen:cosmicgenerator@Construction  04-Aug-2025 17:56:07 CEST ModuleConstruction
Attached db /var/tmp/ifdh_49759_0/Fe_showers_610184.db

%MSG
Cryo Boundary: 0=-391.52 ( + Buffer=-300)

Cryo Boundary: 1=463.52 ( + Buffer=300)

Cryo Boundary: 2=-72.3706 ( + Buffer=-300)

Cryo Boundary: 3=717.869 ( + Buffer=300)

Cryo Boundary: 4=-48.9237 ( + Buffer=-300)

Cryo Boundary: 5=806.116 ( + Buffer=300)

%MSG-i CORSIKAGen:  CORSIKAGen:cosmicgenerator@Construction  04-Aug-2025 17:56:07 CEST ModuleConstruction
Area extended by : 2000
Showers to be distributed betweeen: x=-2391.52,2463.52 & z=-2048.92,2806.12
Showers to be distributed with random XZ shift: 1000 cm
Showers to be distributed over area: 2357.14 m^2
Showers to be distributed over time: 0.00645 s
Showers to be distributed with time offset: -0.003325 s
Showers to be distributed at y: 717.869 cm
%MSG
For showers input 0 found e_hi=100000, e_lo=1.3, slope=-2.7, k=17200

For showers input 0 the number of showers per event is 309364

For showers input 1 found e_hi=100000, e_lo=5.2, slope=-2.7, k=9200

For showers input 1 the number of showers per event is 15675

For showers input 2 found e_hi=100000, e_lo=18.2, slope=-2.7, k=6200

For showers input 2 the number of showers per event is 1255

For showers input 3 found e_hi=100000, e_lo=31.2, slope=-2.7, k=9200

For showers input 3 the number of showers per event is 745

For showers input 4 found e_hi=100000, e_lo=72.8, slope=-2.7, k=6200

For showers input 4 the number of showers per event is 118

%MSG-i CORSIKAGen:  CORSIKAGen:cosmicgenerator@Construction  04-Aug-2025 17:56:07 CEST ModuleConstruction
For showers input 0 found particles.min(t)=375013

%MSG
%MSG-i CORSIKAGen:  CORSIKAGen:cosmicgenerator@Construction  04-Aug-2025 17:56:07 CEST ModuleConstruction
For showers input 1 found particles.min(t)=375706

%MSG
%MSG-i CORSIKAGen:  CORSIKAGen:cosmicgenerator@Construction  04-Aug-2025 17:56:07 CEST ModuleConstruction
For showers input 2 found particles.min(t)=377507

%MSG
%MSG-i CORSIKAGen:  CORSIKAGen:cosmicgenerator@Construction  04-Aug-2025 17:56:07 CEST ModuleConstruction
For showers input 3 found particles.min(t)=379677

%MSG
%MSG-i CORSIKAGen:  CORSIKAGen:cosmicgenerator@Construction  04-Aug-2025 17:56:07 CEST ModuleConstruction
For showers input 4 found particles.min(t)=377718

%MSG
%MSG-i CORSIKAGen:  CORSIKAGen:cosmicgenerator@Construction  04-Aug-2025 17:56:07 CEST ModuleConstruction
Found corsika timeoffset [ns]: 375013

%MSG
%MSG-i RANDOM:  ProtoDUNETriggeredBeam:generator@Construction  04-Aug-2025 17:56:07 CEST ModuleConstruction
Instantiated HepJamesRandom engine "generator:0:protoDUNEBeam" with seed 385643917.
%MSG
%MSG-i ProtoDUNETriggeredBeam:  ProtoDUNETriggeredBeam:generator@Construction  04-Aug-2025 17:56:07 CEST ModuleConstruction
Skip 0 first events from the input file.
%MSG
%MSG-i RANDOM:  RadioGen:kr85@Construction 04-Aug-2025 17:56:07 CEST  ModuleConstruction
Instantiated HepJamesRandom engine "kr85:0:" with seed 0.
%MSG
%MSG-i NuRandomService:  RadioGen:kr85@Construction 04-Aug-2025 17:56:07 CEST  ModuleConstruction
Seeding  engine "kr85:" with seed 411666373.
%MSG
%MSG-i RadioGen:  RadioGen:kr85@Construction 04-Aug-2025 17:56:07 CEST  ModuleConstruction
Configuring activity:
%MSG
[#0]  85Kr (0.00016 Bq/cm^3) in LAr from -2.5e+06 to 3e+06 ns in ( -360, 0, 0) to ( 360, 600, 700)
Number of overlays = 1249
%MSG-i H4Beam:  OpenFile 04-Aug-2025 17:56:08 CEST BeforeEvents H4BeamInput_source.cc:15
Opening Beam Input File root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/hd-protodune/75/d5/np04hd_raw_run027087_0008_dataflow3_datawriter_0_20240614T144941.hdf5
%MSG
Opening root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/hd-protodune/75/d5/np04hd_raw_run027087_0008_dataflow3_datawriter_0_20240614T144941.hdf5

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
[ No processed events ]
====================================================================================================================
%MSG-i NuRandomService:  RootOutput:out1@EndJob 04-Aug-2025 17:56:08 CEST  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'random'
  master seed: 676784315
  seed within: [ 1 ; 900000000 ]
   Configured value          Last value   ModuleLabel.InstanceName
          820293847              (same)   ar39
          421035493              (same)   ar42
          384905583              (same)   cosmicgenerator.gen
          887380314              (same)   cosmicgenerator.pois
          385643917              (same)   generator.protoDUNEBeam
          411666373              (same)   kr85

%MSG

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1622.63 MB
  Peak resident set size usage (VmHWM): 526.332 MB
  Details saved in: 'mem.db'
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 0 passed = 0 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport          0          0          0 out1

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 0.459209 Real = 0.784452

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 1622.63 VmHWM = 526.332

%MSG-s ArtException:  PostEndJob 04-Aug-2025 17:56:09 CEST ModuleEndJob
---- FatalRootError BEGIN
  Fatal Root Error: TNetXNGFile::Init
  root://dune.dcache.nikhef.nl/pnfs/nikhef.nl/data/dune/generic/rucio/hd-protodune/75/d5/np04hd_raw_run027087_0008_dataflow3_datawriter_0_20240614T144941.hdf5 not a ROOT file
  ROOT severity: 3000
---- FatalRootError END
%MSG
Art has completed and will exit with status 1.

Stderr:
Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/fifeuser3.opensciencegrid.org/sw/dune/a28fc6de1ef9b9a7111f96e906e774aacd9c0ad9/dunecore/v10_08_02d00/gdml/protodune_v7.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 1148
Info in <TGeoManager::CloseGeometry>: 36176 nodes/ 5435 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
59+1 records in
59+1 records out
31207424 bytes (31 MB) copied, 0.237209 s, 132 MB/s
3+1 records in
3+1 records out
1609728 bytes (1.6 MB) copied, 0.00320742 s, 502 MB/s
0+1 records in
0+1 records out
150528 bytes (151 kB) copied, 0.00122579 s, 123 MB/s
0+1 records in
0+1 records out
97280 bytes (97 kB) copied, 0.00103156 s, 94.3 MB/s
0+1 records in
0+1 records out
33792 bytes (34 kB) copied, 0.000331982 s, 102 MB/s

Error in processing gen
Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/fifeuser3.opensciencegrid.org/sw/dune/a28fc6de1ef9b9a7111f96e906e774aacd9c0ad9/dunecore/v10_08_02d00/gdml/protodune_v7.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 1148
Info in <TGeoManager::CloseGeometry>: 36176 nodes/ 5435 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
59+1 records in
59+1 records out
31207424 bytes (31 MB) copied, 0.237209 s, 132 MB/s
3+1 records in
3+1 records out
1609728 bytes (1.6 MB) copied, 0.00320742 s, 502 MB/s
0+1 records in
0+1 records out
150528 bytes (151 kB) copied, 0.00122579 s, 123 MB/s
0+1 records in
0+1 records out
97280 bytes (97 kB) copied, 0.00103156 s, 94.3 MB/s
0+1 records in
0+1 records out
33792 bytes (34 kB) copied, 0.000331982 s, 102 MB/s

Error running. Exiting with 1
justIN time: 2025-08-04 18:13:49 UTC       justIN version: 01.04.00