justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 278790.45@dunegpschedd01.fnal.gov

Jobsub ID278790.45@dunegpschedd01.fnal.gov
Workflow ID11951
Stage ID1
User nameusher@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit14400 (4 hours)
Submitted time2026-01-15 05:47:30
SiteFR_CCIN2P3
EntryDUNE_FR_CCIN2P3_cccondorce02
Last heartbeat2026-01-15 05:49:27
From worker nodeHostnameccwcondor0648
cpuinfoAMD EPYC 7302 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit106200 (29 hours)
GPU
Inner Apptainer?True
Job statestalled
Started2026-01-15 05:49:01
Input filesmonte-carlo-011951-000015
Outputting started2026-01-15 05:49:27
Output files
Finished2026-01-15 06:18:13
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

:  ProcessEvent 15-Jan-2026 06:49:24 CET  run: 200015 subRun: 0 event: 18
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 06:49:24 CET  run: 200015 subRun: 0 event: 18
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w BackTrackerService:  ProcessEvent 15-Jan-2026 06:49:24 CET  run: 200015 subRun: 0 event: 18
Rebuild failed to get the SimChannels. This is expected when running on a generation or simulation step.
%MSG
%MSG-w PhotonBackTrackerService:  ProcessEvent 15-Jan-2026 06:49:24 CET  run: 200015 subRun: 0 event: 18
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
%MSG-i NuRandomService:  MultiPartVertex:generator@BeginModule  15-Jan-2026 06:49:24 CET run: 200015 subRun: 0 event: 18
Random seed for this event, engine 'generator.GenVertex': 831295371
%MSG
Processing a new event...
Generating a rain particle at (276.476,-457.307,750.502)
Generated boost parameter gamma beta = 0.376555, cos(theta) = -0.379246, phi = 0.127456
Generating a boost (0.323429,0.0414476,-0.133646)
Event Vertex @ (276.476,-457.307,750.502) ... 10 particles...
  0th instance PDG -13
  1th instance PDG 111
  2th instance PDG 2112
  3th instance PDG 211
  4th instance PDG 2212
  5th instance PDG -211
  6th instance PDG 2212
  7th instance PDG 111
  8th instance PDG 2112
  9th instance PDG 2212
E: 0.904055 , m: 0.105658 , KE: 0.798397
E: 0.655718 , m: 0.134977 , KE: 0.520741
E: 1.46415 , m: 0.939565 , KE: 0.524586
E: 0.739229 , m: 0.13957 , KE: 0.599659
E: 1.14186 , m: 0.938272 , KE: 0.203586
E: 0.50613 , m: 0.13957 , KE: 0.36656
E: 0.988096 , m: 0.938272 , KE: 0.049824
E: 0.426082 , m: 0.134977 , KE: 0.291105
E: 1.69158 , m: 0.939565 , KE: 0.752018
E: 1.73005 , m: 0.938272 , KE: 0.791778
total KE: 4.89825
KE range: 15 , pdg :  -13
particle KE sf: 15, p : 0.897859 , mass :  0.105658 , mom_sf: 13.4555
particle E : 0.904055 , scaled E : 12.0816
KE range: 2.5 , pdg :  111
particle KE sf: 2.5, p : 0.641675 , mass :  0.134977 , mom_sf: 2.22928
particle E : 0.655718 , scaled E : 1.43683
KE range: 2.5 , pdg :  211
particle KE sf: 2.5, p : 0.725933 , mass :  0.13957 , mom_sf: 2.24919
particle E : 0.739229 , scaled E : 1.63872
KE range: 2.5 , pdg :  -211
particle KE sf: 2.5, p : 0.486506 , mass :  0.13957 , mom_sf: 2.15148
particle E : 0.50613 , scaled E : 1.05597
KE range: 2.5 , pdg :  111
particle KE sf: 2.5, p : 0.404138 , mass :  0.134977 , mom_sf: 2.10848
particle E : 0.426082 , scaled E : 0.86274
Total number particles: 0
Begin processing the 19th record. run: 200015 subRun: 0 event: 19 at 15-Jan-2026 06:49:24 CET
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 06:49:24 CET  run: 200015 subRun: 0 event: 19
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 06:49:24 CET  run: 200015 subRun: 0 event: 19
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 06:49:24 CET  run: 200015 subRun: 0 event: 19
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w BackTrackerService:  ProcessEvent 15-Jan-2026 06:49:24 CET  run: 200015 subRun: 0 event: 19
Rebuild failed to get the SimChannels. This is expected when running on a generation or simulation step.
%MSG
%MSG-w PhotonBackTrackerService:  ProcessEvent 15-Jan-2026 06:49:24 CET  run: 200015 subRun: 0 event: 19
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
%MSG-i NuRandomService:  MultiPartVertex:generator@BeginModule  15-Jan-2026 06:49:24 CET run: 200015 subRun: 0 event: 19
Random seed for this event, engine 'generator.GenVertex': 370772754
%MSG
Processing a new event...
Generating a rain particle at (-316.412,472.618,784.144)
Generated boost parameter gamma beta = 0.374921, cos(theta) = 0.367995, phi = 0.757297
Generating a boost (0.237211,0.22424,0.129188)
Event Vertex @ (-316.412,472.618,784.144) ... 3 particles...
  0th instance PDG -11
  1th instance PDG 2212
  2th instance PDG 2212
E: 0.217052 , m: 0.000510999 , KE: 0.216541
E: 1.42881 , m: 0.938272 , KE: 0.490539
E: 1.21658 , m: 0.938272 , KE: 0.278307
total KE: 0.985386
gen_total_KE: 0.382533 , KE scale factor :  0.388206
KE range: 15 , pdg :  -11
particle KE sf: 5.82309, p : 0.217051 , mass :  0.000510999 , mom_sf: 5.81176
particle E : 0.217052 , scaled E : 1.26145
KE range: 1 , pdg :  2212
particle KE sf: 0.388206, p : 1.07756 , mass :  0.938272 , mom_sf: 0.582227
particle E : 1.42881 , scaled E : 1.1287
KE range: 1 , pdg :  2212
particle KE sf: 0.388206, p : 0.774409 , mass :  0.938272 , mom_sf: 0.59794
particle E : 1.21658 , scaled E : 1.04631
Total number particles: 0
Begin processing the 20th record. run: 200015 subRun: 0 event: 20 at 15-Jan-2026 06:49:24 CET
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 06:49:24 CET  run: 200015 subRun: 0 event: 20
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 06:49:24 CET  run: 200015 subRun: 0 event: 20
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 06:49:24 CET  run: 200015 subRun: 0 event: 20
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w BackTrackerService:  ProcessEvent 15-Jan-2026 06:49:24 CET  run: 200015 subRun: 0 event: 20
Rebuild failed to get the SimChannels. This is expected when running on a generation or simulation step.
%MSG
%MSG-w PhotonBackTrackerService:  ProcessEvent 15-Jan-2026 06:49:24 CET  run: 200015 subRun: 0 event: 20
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
%MSG-i NuRandomService:  MultiPartVertex:generator@BeginModule  15-Jan-2026 06:49:24 CET run: 200015 subRun: 0 event: 20
Random seed for this event, engine 'generator.GenVertex': 354208898
%MSG
Processing a new event...
Generating a rain particle at (-259.406,-257.247,1161.93)
Generated boost parameter gamma beta = 1.65733, cos(theta) = -0.806187, phi = 3.65783
Generating a boost (-0.44057,-0.250059,-0.690269)
Event Vertex @ (-259.406,-257.247,1161.93) ... 1 particles...
  0th instance PDG 11
E: 0.071861 , m: 0.000510999 , KE: 0.07135
total KE: 0.07135
gen_total_KE: 0.0436146 , KE scale factor :  0.611276
KE range: 15 , pdg :  11
particle KE sf: 9.16914, p : 0.0718592 , mass :  0.000510999 , mom_sf: 9.11128
particle E : 0.071861 , scaled E : 0.654729
Total number particles: 0
15-Jan-2026 06:49:26 CET  Closed output file "mpvmpr_monte-carlo-011951-000015_gen.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                    0.000286561   0.00081758    0.00686054    0.000401646    0.0014051       20     
--------------------------------------------------------------------------------------------------------------------------------
source:EmptyEvent(read)                       3.4204e-05    7.24941e-05   0.00046191    4.0301e-05    9.55043e-05      20     
simulate:rns:RandomNumberSaver                1.0039e-05    3.17575e-05   0.000367653   1.0515e-05    7.79016e-05      20     
simulate:generator:MultiPartVertex            6.6876e-05    0.000203434   0.00139844    0.000146321   0.00027817       20     
[art]:TriggerResults:TriggerResultInserter     5.059e-06    9.0017e-06    5.9362e-05     6.277e-06    1.15697e-05      20     
end_path:out1:RootOutput                       8.01e-07     1.88395e-06    1.583e-05     9.915e-07    3.23438e-06      20     
end_path:out1:RootOutput(write)               3.9705e-05    0.000327222    0.0036397    6.85245e-05   0.000785007      20     
================================================================================================================================
%MSG-i NuRandomService:  RootOutput:out1@EndJob 15-Jan-2026 06:49:26 CET  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1
   Configured value          Last value   ModuleLabel.InstanceName
        (per event)           354208898   generator.GenVertex

%MSG

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1334.44 MB
  Peak resident set size usage (VmHWM): 566.78 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 20 passed = 20 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport         20         20          0 out1

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 1.747753 Real = 2.948121

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 1334.44 VmHWM = 566.78

Art has completed and will exit with status 0.
-rw-r--r-- 1 dune001 lbno 108K Jan 15 06:49 mpvmpr_monte-carlo-011951-000015_gen.root
Input fcl file: mpvmpr_gen_1x2x6.fcl
genType: mpvmpr
Wrote processed DID to justin-processed-dids.txt: monte-carlo-011951-000015
=== GEN job completed successfully ===
justIN time: 2026-02-04 11:49:33 UTC       justIN version: 01.06.00