justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 278790.28@dunegpschedd01.fnal.gov

Jobsub ID278790.28@dunegpschedd01.fnal.gov
Workflow ID11951
Stage ID1
User nameusher@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit14400 (4 hours)
Submitted time2026-01-15 05:47:29
SiteFR_CCIN2P3
EntryDUNE_FR_CCIN2P3_cccondorce01
Last heartbeat2026-01-15 05:49:56
From worker nodeHostnameccwcondor0646
cpuinfoAMD EPYC 7302 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit106200 (29 hours)
GPU
Inner Apptainer?True
Job statestalled
Started2026-01-15 05:48:56
Input filesmonte-carlo-011951-000011
Outputting started2026-01-15 05:49:56
Output files
Finished2026-01-15 06:18:13
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

t: 18
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 06:49:53 CET  run: 200011 subRun: 0 event: 18
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w BackTrackerService:  ProcessEvent 15-Jan-2026 06:49:53 CET  run: 200011 subRun: 0 event: 18
Rebuild failed to get the SimChannels. This is expected when running on a generation or simulation step.
%MSG
%MSG-w PhotonBackTrackerService:  ProcessEvent 15-Jan-2026 06:49:53 CET  run: 200011 subRun: 0 event: 18
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
%MSG-i NuRandomService:  MultiPartVertex:generator@BeginModule  15-Jan-2026 06:49:53 CET run: 200011 subRun: 0 event: 18
Random seed for this event, engine 'generator.GenVertex': 75687120
%MSG
Processing a new event...
Generating a rain particle at (213.6,-277.502,181.277)
Generated boost parameter gamma beta = 1.23124, cos(theta) = -0.117464, phi = 3.05449
Generating a boost (-0.767936,0.0670588,-0.0911796)
Event Vertex @ (213.6,-277.502,181.277) ... 7 particles...
  0th instance PDG -11
  1th instance PDG 111
  2th instance PDG -211
  3th instance PDG 2112
  4th instance PDG 2212
  5th instance PDG 211
  6th instance PDG 2212
E: 0.0720311 , m: 0.000510999 , KE: 0.0715201
E: 0.682749 , m: 0.134977 , KE: 0.547772
E: 0.969981 , m: 0.13957 , KE: 0.830411
E: 1.74366 , m: 0.939565 , KE: 0.804092
E: 1.51961 , m: 0.938272 , KE: 0.581342
E: 0.361924 , m: 0.13957 , KE: 0.222354
E: 1.35174 , m: 0.938272 , KE: 0.41347
total KE: 3.47096
KE range: 15 , pdg :  -11
particle KE sf: 15, p : 0.0720293 , mass :  0.000510999 , mom_sf: 14.9011
particle E : 0.0720311 , scaled E : 1.07331
KE range: 2.5 , pdg :  111
particle KE sf: 2.5, p : 0.669274 , mass :  0.134977 , mom_sf: 2.23875
particle E : 0.682749 , scaled E : 1.50441
KE range: 2.5 , pdg :  -211
particle KE sf: 2.5, p : 0.959887 , mass :  0.13957 , mom_sf: 2.3036
particle E : 0.969981 , scaled E : 2.2156
KE range: 2.5 , pdg :  211
particle KE sf: 2.5, p : 0.333929 , mass :  0.13957 , mom_sf: 2.04027
particle E : 0.361924 , scaled E : 0.695454
Total number particles: 0
Begin processing the 19th record. run: 200011 subRun: 0 event: 19 at 15-Jan-2026 06:49:53 CET
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 06:49:53 CET  run: 200011 subRun: 0 event: 19
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 06:49:53 CET  run: 200011 subRun: 0 event: 19
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 06:49:53 CET  run: 200011 subRun: 0 event: 19
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w BackTrackerService:  ProcessEvent 15-Jan-2026 06:49:53 CET  run: 200011 subRun: 0 event: 19
Rebuild failed to get the SimChannels. This is expected when running on a generation or simulation step.
%MSG
%MSG-w PhotonBackTrackerService:  ProcessEvent 15-Jan-2026 06:49:53 CET  run: 200011 subRun: 0 event: 19
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
%MSG-i NuRandomService:  MultiPartVertex:generator@BeginModule  15-Jan-2026 06:49:53 CET run: 200011 subRun: 0 event: 19
Random seed for this event, engine 'generator.GenVertex': 255295176
%MSG
Processing a new event...
Generating a rain particle at (99.6898,446.671,1124.34)
Generated boost parameter gamma beta = 2.81254, cos(theta) = 0.290415, phi = 1.24572
Generating a boost (0.287957,0.854387,0.273634)
Event Vertex @ (99.6898,446.671,1124.34) ... 1 particles...
  0th instance PDG -13
E: 0.595733 , m: 0.105658 , KE: 0.490075
total KE: 0.490075
gen_total_KE: 0.00987345 , KE scale factor :  0.0201468
KE range: 15 , pdg :  -13
particle KE sf: 0.302202, p : 0.586288 , mass :  0.105658 , mom_sf: 0.393522
particle E : 0.595733 , scaled E : 0.25376
Total number particles: 0
Begin processing the 20th record. run: 200011 subRun: 0 event: 20 at 15-Jan-2026 06:49:53 CET
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 06:49:53 CET  run: 200011 subRun: 0 event: 20
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 06:49:53 CET  run: 200011 subRun: 0 event: 20
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 06:49:53 CET  run: 200011 subRun: 0 event: 20
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w BackTrackerService:  ProcessEvent 15-Jan-2026 06:49:53 CET  run: 200011 subRun: 0 event: 20
Rebuild failed to get the SimChannels. This is expected when running on a generation or simulation step.
%MSG
%MSG-w PhotonBackTrackerService:  ProcessEvent 15-Jan-2026 06:49:53 CET  run: 200011 subRun: 0 event: 20
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
%MSG-i NuRandomService:  MultiPartVertex:generator@BeginModule  15-Jan-2026 06:49:53 CET run: 200011 subRun: 0 event: 20
Random seed for this event, engine 'generator.GenVertex': 898897704
%MSG
Processing a new event...
Generating a rain particle at (-195.631,-544.661,566.03)
Generated boost parameter gamma beta = 2.86346, cos(theta) = 0.695148, phi = 0.703443
Generating a boost (0.517568,0.438996,0.656279)
Event Vertex @ (-195.631,-544.661,566.03) ... 6 particles...
  0th instance PDG -11
  1th instance PDG 111
  2th instance PDG 111
  3th instance PDG -211
  4th instance PDG 2112
  5th instance PDG 211
E: 0.573689 , m: 0.000510999 , KE: 0.573178
E: 0.528286 , m: 0.134977 , KE: 0.393309
E: 1.06355 , m: 0.134977 , KE: 0.928576
E: 0.874139 , m: 0.13957 , KE: 0.734569
E: 1.80044 , m: 0.939565 , KE: 0.860874
E: 0.77231 , m: 0.13957 , KE: 0.63274
total KE: 4.12325
KE range: 15 , pdg :  -11
particle KE sf: 15, p : 0.573689 , mass :  0.000510999 , mom_sf: 14.9875
particle E : 0.573689 , scaled E : 8.59818
KE range: 2.5 , pdg :  111
particle KE sf: 2.5, p : 0.510751 , mass :  0.134977 , mom_sf: 2.17341
particle E : 0.528286 , scaled E : 1.11825
KE range: 2.5 , pdg :  111
particle KE sf: 2.5, p : 1.05495 , mass :  0.134977 , mom_sf: 2.32494
particle E : 1.06355 , scaled E : 2.45642
KE range: 2.5 , pdg :  -211
particle KE sf: 2.5, p : 0.862925 , mass :  0.13957 , mom_sf: 2.28416
particle E : 0.874139 , scaled E : 1.97599
KE range: 2.5 , pdg :  211
particle KE sf: 2.5, p : 0.759594 , mass :  0.13957 , mom_sf: 2.25878
particle E : 0.77231 , scaled E : 1.72142
Total number particles: 0
15-Jan-2026 06:49:56 CET  Closed output file "mpvmpr_monte-carlo-011951-000011_gen.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                    0.000347725   0.000838366   0.00599242    0.000517085   0.00119949       20     
--------------------------------------------------------------------------------------------------------------------------------
source:EmptyEvent(read)                       4.3743e-05    7.52042e-05   0.000452714   4.8085e-05    8.89532e-05      20     
simulate:rns:RandomNumberSaver                1.3155e-05    2.97933e-05   0.000292842   1.38565e-05   6.06709e-05      20     
simulate:generator:MultiPartVertex            8.0542e-05    0.000226134   0.00111105    0.000174534   0.000236863      20     
[art]:TriggerResults:TriggerResultInserter     7.714e-06    1.02359e-05   3.9685e-05     8.576e-06    6.78694e-06      20     
end_path:out1:RootOutput                       1.172e-06    2.03435e-06   1.3736e-05     1.357e-06    2.69864e-06      20     
end_path:out1:RootOutput(write)               4.8311e-05    0.000299012   0.00327856    6.38955e-05   0.000699592      20     
================================================================================================================================
%MSG-i NuRandomService:  RootOutput:out1@EndJob 15-Jan-2026 06:49:56 CET  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1
   Configured value          Last value   ModuleLabel.InstanceName
        (per event)           898897704   generator.GenVertex

%MSG

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1334.44 MB
  Peak resident set size usage (VmHWM): 563.192 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 20 passed = 20 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport         20         20          0 out1

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 1.823703 Real = 4.010628

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 1334.44 VmHWM = 563.192

Art has completed and will exit with status 0.
-rw-r--r-- 1 dune001 lbno 110K Jan 15 06:49 mpvmpr_monte-carlo-011951-000011_gen.root
Input fcl file: mpvmpr_gen_1x2x6.fcl
genType: mpvmpr
Wrote processed DID to justin-processed-dids.txt: monte-carlo-011951-000011
=== GEN job completed successfully ===
justIN time: 2026-02-04 11:32:24 UTC       justIN version: 01.06.00