Jobsub ID 299911.0@dunegpschedd01.fnal.gov
| Jobsub ID | 299911.0@dunegpschedd01.fnal.gov |
| Workflow ID | 12285 |
| Stage ID | 1 |
| User name | nivedpm@fnal.gov |
| Requested | Processors | 4 |
| GPU | No |
| RSS bytes | 6815744000 (6500 MiB) |
| Wall seconds limit | 80000 (22 hours) |
| Submitted time | 2026-01-23 12:02:16 |
| Site | UK_Imperial |
| Entry | DUNE_T2_UK_London_IC_ceprod02 |
| Last heartbeat | 2026-01-23 14:50:08 |
| From worker node | Hostname | wn02.grid.hep.ph.ic.ac.uk |
| cpuinfo | AMD EPYC 7742 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 4 |
| RSS bytes | 7864320000 (7500 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2026-01-23 14:48:23 |
| Input files | monte-carlo-012285-000085
|
| Jobscript | Exit code | 0 |
| Real time | 1m (70s) |
| CPU time | 0m (42s = 59%) |
| Max RSS bytes | 730595328 (696 MiB) |
| Outputting started | 2026-01-23 14:49:33 |
| Output files | |
| Finished | 2026-01-23 14:50:08 |
| Saved logs | justin-logs:299911.0-dunegpschedd01.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
is is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory: ProcessEvent 23-Jan-2026 14:49:05 GMT run: 1 subRun: 0 event: 1
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory: ProcessEvent 23-Jan-2026 14:49:05 GMT run: 1 subRun: 0 event: 1
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w PhotonBackTrackerService: ProcessEvent 23-Jan-2026 14:49:05 GMT run: 1 subRun: 0 event: 1
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
%MSG-i generatePrimaries: larg4Main:largeant@BeginModule 23-Jan-2026 14:49:05 GMT run: 1 subRun: 0 event: 1 MCTruthEventAction.cc:112
Generating 5 particles
%MSG
%MSG-i generatePrimaries: larg4Main:largeant@BeginModule 23-Jan-2026 14:49:05 GMT run: 1 subRun: 0 event: 1 MCTruthEventAction.cc:112
Generating 2 particles
%MSG
%MSG-i generatePrimaries: larg4Main:largeant@BeginModule 23-Jan-2026 14:49:05 GMT run: 1 subRun: 0 event: 1 MCTruthEventAction.cc:112
Generating 4 particles
%MSG
%MSG-i ParticleListActionService: larg4Main:largeant@BeginModule 23-Jan-2026 14:49:07 GMT run: 1 subRun: 0 event: 1
Not Stored Process summary:
Pair : 4
conv : 362
compt : 10523
phot : 3695
annihil : 372
Brem : 3317
Ion : 5607
%MSG
%MSG-i endOfEventAction: larg4Main:largeant@BeginModule 23-Jan-2026 14:49:07 GMT run: 1 subRun: 0 event: 1 ParticleListAction.cc:817
MCTruth Handles Size: 3
%MSG
%MSG-i endOfEventAction: larg4Main:largeant@BeginModule 23-Jan-2026 14:49:07 GMT run: 1 subRun: 0 event: 1 ParticleListAction.cc:827
mclistHandle Size: 1
%MSG
%MSG-i endOfEventAction: larg4Main:largeant@BeginModule 23-Jan-2026 14:49:07 GMT run: 1 subRun: 0 event: 1 ParticleListAction.cc:830
Found 5 particles
%MSG
%MSG-i endOfEventAction: larg4Main:largeant@BeginModule 23-Jan-2026 14:49:07 GMT run: 1 subRun: 0 event: 1 ParticleListAction.cc:827
mclistHandle Size: 1
%MSG
%MSG-i endOfEventAction: larg4Main:largeant@BeginModule 23-Jan-2026 14:49:07 GMT run: 1 subRun: 0 event: 1 ParticleListAction.cc:830
Found 2 particles
%MSG
%MSG-i endOfEventAction: larg4Main:largeant@BeginModule 23-Jan-2026 14:49:07 GMT run: 1 subRun: 0 event: 1 ParticleListAction.cc:827
mclistHandle Size: 1
%MSG
%MSG-i endOfEventAction: larg4Main:largeant@BeginModule 23-Jan-2026 14:49:07 GMT run: 1 subRun: 0 event: 1 ParticleListAction.cc:830
Found 4 particles
%MSG
IonAndScint Module Producer
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActive
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveOuter
%MSG-i PhotonVisibilityService: PDFastSimPVS:PDFastSim@BeginModule 23-Jan-2026 14:49:07 GMT run: 1 subRun: 0 event: 1
PhotonVisibilityService Loading photon library from file /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/PhotonPropagation/LibraryData/lib_Protodunev7_merged_avg.root for 2352000 voxels and 90 optical detectors.
%MSG
%MSG-i PhotonLibrary: PDFastSimPVS:PDFastSim@BeginModule 23-Jan-2026 14:49:07 GMT run: 1 subRun: 0 event: 1
Reading photon library from input file: /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/PhotonPropagation/LibraryData/lib_Protodunev7_merged_avg.root
%MSG
23-Jan-2026 14:49:28 GMT Opened output file with pattern "prod_mpvmpr_protodunesp_1_299911_0_1769179718_gen_g4_stage1.root"
%MSG-i Root_Information: PDFastSimPVS:PDFastSim@BeginModule TClass::Init() 23-Jan-2026 14:49:28 GMT run: 1 subRun: 0 event: 1
no dictionary for class sim::IDE is available
ROOT severity: 2000
%MSG
%MSG-i FastCloning: PDFastSimPVS:PDFastSim@BeginModule 23-Jan-2026 14:49:28 GMT run: 1 subRun: 0 event: 1
Fast cloning event data products from input file.
%MSG
IonAndScint endJob.
23-Jan-2026 14:49:29 GMT Closed input file "prod_mpvmpr_protodunesp_1_299911_0_1769179718_gen.root"
Malformed TimeTracker database. The TimeEvent table is empty, but
the TimeModule table is not. This can happen if an exception has
been thrown from a module while processing the first event. Any
saved database file is suspect and should not be used.
%MSG-i NuRandomService: RootOutput:out1@EndJob 23-Jan-2026 14:49:29 GMT ModuleEndJob
Summary of seeds computed by the NuRandomService
Random policy: 'random'
master seed: 75021396
seed within: [ 1 ; 900000000 ]
Configured value Last value ModuleLabel.InstanceName
155850892 (same) IonAndScint.ISCalcAlg
529625924 (same) PDFastSim.photon
103187853 (same) PDFastSim.scinttime
185200419 (same) largeant
%MSG
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 1651.35 MB
Peak resident set size usage (VmHWM): 730.595 MB
====================================================================================================
TrigReport ---------- Event summary -------------
TrigReport Events total = 1 passed = 0 failed = 1
TrigReport ---------- Modules in End-path ----------
TrigReport Run Success Error Name
TrigReport 0 0 0 out1
TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 4.668465 Real = 26.114844
MemReport ---------- Memory summary [base-10 MB] ------
MemReport VmPeak = 1651.35 VmHWM = 730.595
%MSG-s ArtException: PostEndJob 23-Jan-2026 14:49:29 GMT ModuleEndJob
---- EventProcessorFailure BEGIN
EventProcessor: an exception occurred during current event processing
---- ScheduleExecutionFailure BEGIN
Path: ProcessingStopped.
---- OtherArt BEGIN
---- PhotonLibrary BEGIN
Error in ttree load, reading photon library: /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/PhotonPropagation/LibraryData/lib_Protodunev7_merged_avg.root
The above exception was thrown while processing module PDFastSimPVS/PDFastSim run: 1 subRun: 0 event: 1
---- PhotonLibrary END
---- OtherArt END
Exception going through path simulate
---- ScheduleExecutionFailure END
---- EventProcessorFailure END
---- FatalRootError BEGIN
Fatal Root Error: TTree::SetEntries
Tree branches have different numbers of entries, eg EventAuxiliary has 0 entries while simb::MCTruths_rain__MPVMPR. has 20 entries.
ROOT severity: 2000
---- FatalRootError END
%MSG
Art has completed and will exit with status 1.
Stderr:
Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/dune.opensciencegrid.org/products/dune/dunecore/v10_16_00d00/gdml/protodune_v7.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 1148
Info in <TGeoManager::CloseGeometry>: 36176 nodes/ 5435 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
registering to primaryGeneratorActionsMap_
registering to eventActionsMap_
registering to trackingActionsMap_
registering to steppingActionsMap_
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
Error in processing g4_stage1
Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/dune.opensciencegrid.org/products/dune/dunecore/v10_16_00d00/gdml/protodune_v7.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 1148
Info in <TGeoManager::CloseGeometry>: 36176 nodes/ 5435 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
registering to primaryGeneratorActionsMap_
registering to eventActionsMap_
registering to trackingActionsMap_
registering to steppingActionsMap_
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
ls: cannot access *larcvpdsp*.root: No such file or directory
Output files:
larcv files:
formed
{
"name": "",
"namespace": "usertests",
"metadata": {
"core.file_format": "root",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_16_00d00",
"core.data_tier": "root-tuple-virtual",
"core.runs": null,
"core.runs_subruns": null,
"dune.config_file": "run_supera_pdunesp_mpvmpr_d.fcl",
"dune.campaign": "sp-protodune-spine-reco",
"core.start_time": 1769179717.0,
"core.end_time": 1769179769.0
}
} 