justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 26466.190@dunegpschedd02.fnal.gov

Jobsub ID26466.190@dunegpschedd02.fnal.gov
Workflow ID1854
Stage ID1
User nameepennacc@fnal.gov
HTCondor Groupgroup_dune.prod.mcsim
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-08-26 12:40:53
SiteUK_Imperial
EntryDUNE_T2_UK_London_IC_ceprod03
Last heartbeat2025-08-26 13:23:28
From worker nodeHostnamewh15.grid.hep.ph.ic.ac.uk
cpuinfoIntel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-08-26 13:09:06
Input filesmonte-carlo-001854-004389
JobscriptExit code0
Real time14m (842s)
CPU time13m (805s = 95%)
Max RSS bytes918962176 (876 MiB)
Outputting started2025-08-26 13:23:10
Output files
Finished2025-08-26 13:23:28
Saved logsjustin-logs:26466.190-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

imEnergyDeposit to Volume:  volTPCPlaneZOuter
%MSG-i AuxUnit:  larg4Main:largeant@Construction 26-Aug-2025 14:22:15 BST  ModuleConstruction
 Unit parsed = mm from unit category: Length
%MSG
%MSG-i AuxUnit:  larg4Main:largeant@Construction 26-Aug-2025 14:22:15 BST  ModuleConstruction
Valid StepLimit unit category obtained: Length
%MSG
%MSG-i fStepLimit:  larg4Main:largeant@Construction 26-Aug-2025 14:22:15 BST  ModuleConstruction
fStepLimit:  0.4  0.04 cm

%MSG
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%MSG-w LArG4DetectorService::setStepLimits:  larg4Main:largeant@Construction  26-Aug-2025 14:22:15 BST ModuleConstruction LArG4Detector.cc:336
Setting step limits from configuration file. This will OVERRIDE redundant stepLimit(s) set in the GDML file. Note that stepLimits are only active if enabled in the physicsListService via the appropriate parameter.
%MSG
%MSG-w LArG4DetectorService::setStepLimits:  larg4Main:largeant@Construction  26-Aug-2025 14:22:15 BST ModuleConstruction LArG4Detector.cc:372
New stepLimit matches previously set stepLimit from the GDML file for volume : volTPCActiveInner0 stepLimit : 0.4 mm. Nothing will be changed.
%MSG
%MSG-w LArG4DetectorService::setStepLimits:  larg4Main:largeant@Construction  26-Aug-2025 14:22:15 BST ModuleConstruction LArG4Detector.cc:372
New stepLimit matches previously set stepLimit from the GDML file for volume : volTPCActiveInner1 stepLimit : 0.4 mm. Nothing will be changed.
%MSG
%MSG-w LArG4DetectorService::setStepLimits:  larg4Main:largeant@Construction  26-Aug-2025 14:22:15 BST ModuleConstruction LArG4Detector.cc:372
New stepLimit matches previously set stepLimit from the GDML file for volume : volTPCActiveOuter stepLimit : 0.4 mm. Nothing will be changed.
%MSG
List SD Tree: 
Volume volTPCActiveInner0 has the following list of auxiliary information: 
--> Type: SensDet Value: SimEnergyDeposit
--> Type: StepLimit Value: 0.4
Volume volTPCPlaneUInner0 has the following list of auxiliary information: 
--> Type: SensDet Value: SimEnergyDeposit
--> Type: StepLimit Value: 0.4
Volume volTPCPlaneUInner1 has the following list of auxiliary information: 
--> Type: SensDet Value: SimEnergyDeposit
--> Type: StepLimit Value: 0.4
Volume volTPCPlaneVInner0 has the following list of auxiliary information: 
--> Type: SensDet Value: SimEnergyDeposit
--> Type: StepLimit Value: 0.4
Volume volTPCInner0 has the following list of auxiliary information: 
--> Type: SensDet Value: SimEnergyDeposit
--> Type: StepLimit Value: 0.4
Volume volTPCActiveInner1 has the following list of auxiliary information: 
--> Type: SensDet Value: SimEnergyDeposit
--> Type: StepLimit Value: 0.4
Volume volTPCPlaneZInner0 has the following list of auxiliary information: 
--> Type: SensDet Value: SimEnergyDeposit
--> Type: StepLimit Value: 0.4
Volume volTPCActiveOuter has the following list of auxiliary information: 
--> Type: SensDet Value: SimEnergyDeposit
--> Type: StepLimit Value: 0.4
Volume volTPCPlaneVInner1 has the following list of auxiliary information: 
--> Type: SensDet Value: SimEnergyDeposit
--> Type: StepLimit Value: 0.4
Volume volTPCPlaneZInner1 has the following list of auxiliary information: 
--> Type: SensDet Value: SimEnergyDeposit
--> Type: StepLimit Value: 0.4
Volume volTPCPlaneVOuter has the following list of auxiliary information: 
--> Type: SensDet Value: SimEnergyDeposit
--> Type: StepLimit Value: 0.4
Volume volTPCInner1 has the following list of auxiliary information: 
--> Type: SensDet Value: SimEnergyDeposit
--> Type: StepLimit Value: 0.4
Volume volTPCPlaneUOuter has the following list of auxiliary information: 
--> Type: SensDet Value: SimEnergyDeposit
--> Type: StepLimit Value: 0.4
Volume volTPCOuter has the following list of auxiliary information: 
--> Type: SensDet Value: SimEnergyDeposit
--> Type: StepLimit Value: 0.4
Volume volTPCPlaneZOuter has the following list of auxiliary information: 
--> Type: SensDet Value: SimEnergyDeposit
--> Type: StepLimit Value: 0.4
/
/volTPCActiveInner0_SimEnergyDeposit   *** Active 
/volTPCPlaneUInner0_SimEnergyDeposit   *** Active 
/volTPCPlaneUInner1_SimEnergyDeposit   *** Active 
/volTPCPlaneVInner0_SimEnergyDeposit   *** Active 
/volTPCInner0_SimEnergyDeposit   *** Active 
/volTPCActiveInner1_SimEnergyDeposit   *** Active 
/volTPCPlaneZInner0_SimEnergyDeposit   *** Active 
/volTPCActiveOuter_SimEnergyDeposit   *** Active 
/volTPCPlaneVInner1_SimEnergyDeposit   *** Active 
/volTPCPlaneZInner1_SimEnergyDeposit   *** Active 
/volTPCPlaneVOuter_SimEnergyDeposit   *** Active 
/volTPCInner1_SimEnergyDeposit   *** Active 
/volTPCPlaneUOuter_SimEnergyDeposit   *** Active 
/volTPCOuter_SimEnergyDeposit   *** Active 
/volTPCPlaneZOuter_SimEnergyDeposit   *** Active 
 Collection Capacity:  0
==================================================
nr of LV ======================:  1
%MSG-i RANDOM:  larg4Main:largeant@Construction 26-Aug-2025 14:22:15 BST  ModuleConstruction
Instantiated G4Engine engine "largeant:0:" with seed 0.
%MSG
%MSG-i NuRandomService:  larg4Main:largeant@Construction  26-Aug-2025 14:22:15 BST ModuleConstruction
Seeding G4Engine engine "largeant:" with seed 0.
%MSG
%MSG-s ArtException:  TriggerResultInserter:TriggerResults@Construction  26-Aug-2025 14:22:15 BST ModuleConstruction
cet::exception caught in art
---- CatalogServiceError BEGIN
  Input file not found: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250826T130910Z_gen_004389.root.
---- CatalogServiceError END
%MSG
Art has completed and will exit with status 24.
ls: cannot access *_20250826T130910Z_*: No such file or directory
ERROR: lar (geant4 step1) exit code: 21024 
output files size: 
../justin-jobscript: line 123: return: can only `return' from a function or sourced script
============detsim=========================
Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/dune.opensciencegrid.org/products/dune/dunecore/v10_06_00d01/gdml/dune10kt_v6_refactored_1x2x6.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Error: Unsupported GDML Tag Used :gdml_simple_extension. Please Check Geometry/Schema.
Error: Unsupported GDML Tag Used :extension. Please Check Geometry/Schema.
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 1149
Info in <TGeoManager::CloseGeometry>: 67545 nodes/ 5439 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
registering to  primaryGeneratorActionsMap_
registering to   eventActionsMap_
registering to   trackingActionsMap_
registering to  steppingActionsMap_
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
Using TESTbench spe!!! 
With: SPE_DAPHNE2_FBK_2022.dat
[14:22:50.597] I [  wct   ] loading compressed json file: /cvmfs/dune.opensciencegrid.org/products/dune/dunereco/v10_06_00d01/wire-cell-cfg/dune10kt-1x2x6-wires-larsoft-v1.json.bz2
[14:23:03.992] I [  wct   ] loading compressed json file: /cvmfs/dune.opensciencegrid.org/products/dune/dunereco/v10_06_00d01/wire-cell-cfg/dune-garfield-1d60563.json.bz2
[14:23:07.610] I [  wct   ] loading compressed json file: /cvmfs/dune.opensciencegrid.org/products/dune/dune_pardata/v01_84_00/WireCellData/protodune-noise-spectra-v1.json.bz2
Inputer: "wclsSimDepoSource:"
Outputer: "wclsSimChannelSink:postdrift"
Outputer: "wclsFrameSaver:simdigits"
wclsFrameSaver: promising to produce raw::RawDigits named "daq"
%MSG-s ArtException:  TriggerResultInserter:TriggerResults@Construction  26-Aug-2025 14:23:07 BST ModuleConstruction
cet::exception caught in art
---- CatalogServiceError BEGIN
  Input file not found: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250826T130910Z_gen_004389_supernova_g4.root.
---- CatalogServiceError END
%MSG
Art has completed and will exit with status 24.
ls: cannot access *_20250826T130910Z_*: No such file or directory
ERROR: lar (detsim) exit code: 41024 
output files size: 
../justin-jobscript: line 154: return: can only `return' from a function or sourced script
Traceback (most recent call last):
  File "/cvmfs/dune.opensciencegrid.org/products/dune/duneutil/v10_06_00d01/bin/extractor_prod.py", line 434, in <module>
    main()
  File "/cvmfs/dune.opensciencegrid.org/products/dune/duneutil/v10_06_00d01/bin/extractor_prod.py", line 373, in main
    mddict = expSpecificMetadata.getmetadata()
  File "/cvmfs/dune.opensciencegrid.org/products/dune/duneutil/v10_06_00d01/bin/extractor_prod.py", line 344, in getmetadata
    jobt = self.get_job(proc)
  File "/cvmfs/dune.opensciencegrid.org/products/dune/duneutil/v10_06_00d01/bin/extractor_prod.py", line 69, in get_job
    raise RuntimeError('sam_metadata_dumper returned nonzero exit status {}.'.format(rc))
RuntimeError: sam_metadata_dumper returned nonzero exit status 1.
Error reading metadata from file: Expecting value: line 1 column 1 (char 0)
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250826T130910Z_gen_004389_supernova_g4_detsim.root.ext.json 0
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250826T130910Z_gen_004389_supernova_g4_detsim.root.json 0
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250826T130910Z_gen_004389_supernova_g4_detsim.root.temp.json 0
justIN time: 2025-09-18 19:19:06 UTC       justIN version: 01.05.00