Jobsub ID 274350.0@dunegpschedd02.fnal.gov
| Jobsub ID | 274350.0@dunegpschedd02.fnal.gov |
| Workflow ID | 12026 |
| Stage ID | 1 |
| User name | nivedpm@fnal.gov |
| Requested | Processors | 4 |
| GPU | No |
| RSS bytes | 6815744000 (6500 MiB) |
| Wall seconds limit | 80000 (22 hours) |
| Submitted time | 2026-01-17 22:00:55 |
| Site | UK_Imperial |
| Entry | DUNE_T2_UK_London_IC_ceprod00 |
| Last heartbeat | 2026-01-17 22:05:54 |
| From worker node | Hostname | wj09.grid.hep.ph.ic.ac.uk |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 4 |
| RSS bytes | 7864320000 (7500 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2026-01-17 22:03:04 |
| Input files | monte-carlo-012026-000001
|
| Jobscript | Exit code | 0 |
| Real time | 2m (143s) |
| CPU time | 0m (54s = 37%) |
| Max RSS bytes | 619319296 (590 MiB) |
| Outputting started | 2026-01-17 22:05:28 |
| Output files | |
| Finished | 2026-01-17 22:05:54 |
| Saved logs | justin-logs:274350.0-dunegpschedd02.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
ected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory: ProcessEvent 17-Jan-2026 22:04:27 GMT run: 1 subRun: 0 event: 1
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory: ProcessEvent 17-Jan-2026 22:04:27 GMT run: 1 subRun: 0 event: 1
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w PhotonBackTrackerService: ProcessEvent 17-Jan-2026 22:04:27 GMT run: 1 subRun: 0 event: 1
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
%MSG-i generatePrimaries: larg4Main:largeant@BeginModule 17-Jan-2026 22:04:27 GMT run: 1 subRun: 0 event: 1 MCTruthEventAction.cc:112
Generating 3 particles
%MSG
%MSG-i generatePrimaries: larg4Main:largeant@BeginModule 17-Jan-2026 22:04:27 GMT run: 1 subRun: 0 event: 1 MCTruthEventAction.cc:112
Generating 2 particles
%MSG
%MSG-i generatePrimaries: larg4Main:largeant@BeginModule 17-Jan-2026 22:04:27 GMT run: 1 subRun: 0 event: 1 MCTruthEventAction.cc:112
Generating 3 particles
%MSG
%MSG-i ParticleListActionService: larg4Main:largeant@BeginModule 17-Jan-2026 22:04:29 GMT run: 1 subRun: 0 event: 1
Not Stored Process summary:
Pair : 4
conv : 140
Ion : 2950
annihil : 152
Brem : 1127
compt : 5152
phot : 1549
%MSG
%MSG-i endOfEventAction: larg4Main:largeant@BeginModule 17-Jan-2026 22:04:29 GMT run: 1 subRun: 0 event: 1 ParticleListAction.cc:817
MCTruth Handles Size: 3
%MSG
%MSG-i endOfEventAction: larg4Main:largeant@BeginModule 17-Jan-2026 22:04:29 GMT run: 1 subRun: 0 event: 1 ParticleListAction.cc:827
mclistHandle Size: 1
%MSG
%MSG-i endOfEventAction: larg4Main:largeant@BeginModule 17-Jan-2026 22:04:29 GMT run: 1 subRun: 0 event: 1 ParticleListAction.cc:830
Found 3 particles
%MSG
%MSG-i endOfEventAction: larg4Main:largeant@BeginModule 17-Jan-2026 22:04:29 GMT run: 1 subRun: 0 event: 1 ParticleListAction.cc:827
mclistHandle Size: 1
%MSG
%MSG-i endOfEventAction: larg4Main:largeant@BeginModule 17-Jan-2026 22:04:29 GMT run: 1 subRun: 0 event: 1 ParticleListAction.cc:830
Found 2 particles
%MSG
%MSG-i endOfEventAction: larg4Main:largeant@BeginModule 17-Jan-2026 22:04:29 GMT run: 1 subRun: 0 event: 1 ParticleListAction.cc:827
mclistHandle Size: 1
%MSG
%MSG-i endOfEventAction: larg4Main:largeant@BeginModule 17-Jan-2026 22:04:29 GMT run: 1 subRun: 0 event: 1 ParticleListAction.cc:830
Found 3 particles
%MSG
IonAndScint Module Producer
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActive
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveOuter
%MSG-i PhotonVisibilityService: PDFastSimPVS:PDFastSim@BeginModule 17-Jan-2026 22:04:29 GMT run: 1 subRun: 0 event: 1
PhotonVisibilityService Loading photon library from file /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/PhotonPropagation/LibraryData/lib_Protodunev7_merged_avg.root for 2352000 voxels and 90 optical detectors.
%MSG
%MSG-i PhotonLibrary: PDFastSimPVS:PDFastSim@BeginModule 17-Jan-2026 22:04:29 GMT run: 1 subRun: 0 event: 1
Reading photon library from input file: /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/PhotonPropagation/LibraryData/lib_Protodunev7_merged_avg.root
%MSG
17-Jan-2026 22:05:22 GMT Opened output file with pattern "prod_muon_400MeV_protodunevd_1_274350_0_1768687407_gen_g4_stage1.root"
%MSG-i Root_Information: PDFastSimPVS:PDFastSim@BeginModule TClass::Init() 17-Jan-2026 22:05:23 GMT run: 1 subRun: 0 event: 1
no dictionary for class sim::IDE is available
ROOT severity: 2000
%MSG
%MSG-i FastCloning: PDFastSimPVS:PDFastSim@BeginModule 17-Jan-2026 22:05:23 GMT run: 1 subRun: 0 event: 1
Fast cloning event data products from input file.
%MSG
IonAndScint endJob.
17-Jan-2026 22:05:23 GMT Closed input file "prod_muon_400MeV_protodunevd_1_274350_0_1768687407_gen.root"
Malformed TimeTracker database. The TimeEvent table is empty, but
the TimeModule table is not. This can happen if an exception has
been thrown from a module while processing the first event. Any
saved database file is suspect and should not be used.
%MSG-i NuRandomService: RootOutput:out1@EndJob 17-Jan-2026 22:05:23 GMT ModuleEndJob
Summary of seeds computed by the NuRandomService
Random policy: 'random'
master seed: 210178627
seed within: [ 1 ; 900000000 ]
Configured value Last value ModuleLabel.InstanceName
762944633 (same) IonAndScint.ISCalcAlg
185590594 (same) PDFastSim.photon
1116091 (same) PDFastSim.scinttime
504522034 (same) largeant
%MSG
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 1497.74 MB
Peak resident set size usage (VmHWM): 620.347 MB
====================================================================================================
TrigReport ---------- Event summary -------------
TrigReport Events total = 1 passed = 0 failed = 1
TrigReport ---------- Modules in End-path ----------
TrigReport Run Success Error Name
TrigReport 0 0 0 out1
TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 5.443643 Real = 60.511906
MemReport ---------- Memory summary [base-10 MB] ------
MemReport VmPeak = 1497.74 VmHWM = 620.347
%MSG-s ArtException: PostEndJob 17-Jan-2026 22:05:23 GMT ModuleEndJob
---- EventProcessorFailure BEGIN
EventProcessor: an exception occurred during current event processing
---- ScheduleExecutionFailure BEGIN
Path: ProcessingStopped.
---- OtherArt BEGIN
---- PhotonLibrary BEGIN
Error in ttree load, reading photon library: /cvmfs/dune.osgstorage.org/pnfs/fnal.gov/usr/dune/persistent/stash/PhotonPropagation/LibraryData/lib_Protodunev7_merged_avg.root
The above exception was thrown while processing module PDFastSimPVS/PDFastSim run: 1 subRun: 0 event: 1
---- PhotonLibrary END
---- OtherArt END
Exception going through path simulate
---- ScheduleExecutionFailure END
---- EventProcessorFailure END
---- FatalRootError BEGIN
Fatal Root Error: TTree::SetEntries
Tree branches have different numbers of entries, eg EventAuxiliary has 0 entries while simb::MCTruths_rain__MPVMPR. has 1 entries.
ROOT severity: 2000
---- FatalRootError END
%MSG
Art has completed and will exit with status 1.
Stderr:
Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/dune.opensciencegrid.org/products/dune/dunecore/v10_16_00d00/gdml/protodune_v7.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 1148
Info in <TGeoManager::CloseGeometry>: 36176 nodes/ 5435 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
registering to primaryGeneratorActionsMap_
registering to eventActionsMap_
registering to trackingActionsMap_
registering to steppingActionsMap_
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
Error in processing g4_stage1
Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/dune.opensciencegrid.org/products/dune/dunecore/v10_16_00d00/gdml/protodune_v7.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 1148
Info in <TGeoManager::CloseGeometry>: 36176 nodes/ 5435 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
registering to primaryGeneratorActionsMap_
registering to eventActionsMap_
registering to trackingActionsMap_
registering to steppingActionsMap_
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
ls: cannot access *larcvpdsp*.root: No such file or directory
Output files:
larcv files:
formed
{
"name": "",
"namespace": "usertests",
"metadata": {
"core.file_format": "root",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_16_00d00",
"core.data_tier": "root-tuple-virtual",
"core.runs": null,
"core.runs_subruns": null,
"dune.config_file": "run_supera_pdunesp_mpvmpr_d.fcl",
"dune.campaign": "sp-protodune-spine-reco",
"core.start_time": 1768687405.0,
"core.end_time": 1768687524.0
}
} 