justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 278790.46@dunegpschedd01.fnal.gov

Jobsub ID278790.46@dunegpschedd01.fnal.gov
Workflow ID11951
Stage ID1
User nameusher@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit14400 (4 hours)
Submitted time2026-01-15 05:47:30
SiteFR_CCIN2P3
EntryDUNE_FR_CCIN2P3_cccondorce01
Last heartbeat2026-01-15 05:49:56
From worker nodeHostnameccwcondor0646
cpuinfoAMD EPYC 7302 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit106200 (29 hours)
GPU
Inner Apptainer?True
Job statestalled
Started2026-01-15 05:48:44
Input filesmonte-carlo-011951-000002
Outputting started2026-01-15 05:49:56
Output files
Finished2026-01-15 06:18:13
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

0457
Generating a boost (-0.370494,-0.0609238,-0.149208)
Event Vertex @ (-61.2034,38.2872,361.839) ... 2 particles...
  0th instance PDG 13
  1th instance PDG 211
E: 0.964653 , m: 0.105658 , KE: 0.858995
E: 0.585847 , m: 0.13957 , KE: 0.446277
total KE: 1.30527
KE range: 15 , pdg :  13
particle KE sf: 15, p : 0.958849 , mass :  0.105658 , mom_sf: 13.5476
particle E : 0.964653 , scaled E : 12.9906
KE range: 2.5 , pdg :  211
particle KE sf: 2.5, p : 0.568979 , mass :  0.13957 , mom_sf: 2.19249
particle E : 0.585847 , scaled E : 1.25526
Total number particles: 0
Begin processing the 18th record. run: 200002 subRun: 0 event: 18 at 15-Jan-2026 06:49:53 CET
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 06:49:53 CET  run: 200002 subRun: 0 event: 18
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 06:49:53 CET  run: 200002 subRun: 0 event: 18
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 06:49:53 CET  run: 200002 subRun: 0 event: 18
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w BackTrackerService:  ProcessEvent 15-Jan-2026 06:49:53 CET  run: 200002 subRun: 0 event: 18
Rebuild failed to get the SimChannels. This is expected when running on a generation or simulation step.
%MSG
%MSG-w PhotonBackTrackerService:  ProcessEvent 15-Jan-2026 06:49:53 CET  run: 200002 subRun: 0 event: 18
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
%MSG-i NuRandomService:  MultiPartVertex:generator@BeginModule  15-Jan-2026 06:49:53 CET run: 200002 subRun: 0 event: 18
Random seed for this event, engine 'generator.GenVertex': 175745176
%MSG
Processing a new event...
Generating a rain particle at (173.17,213.708,609.538)
Generated boost parameter gamma beta = 2.75862, cos(theta) = 0.0385987, phi = 3.37887
Generating a boost (-0.913113,-0.220824,0.036288)
Event Vertex @ (173.17,213.708,609.538) ... 7 particles...
  0th instance PDG 11
  1th instance PDG -211
  2th instance PDG 2112
  3th instance PDG 211
  4th instance PDG 2212
  5th instance PDG 2212
  6th instance PDG 2112
E: 0.854455 , m: 0.000510999 , KE: 0.853944
E: 0.370384 , m: 0.13957 , KE: 0.230814
E: 1.2023 , m: 0.939565 , KE: 0.262739
E: 0.871256 , m: 0.13957 , KE: 0.731686
E: 1.8199 , m: 0.938272 , KE: 0.881631
E: 1.02966 , m: 0.938272 , KE: 0.0913892
E: 1.4508 , m: 0.939565 , KE: 0.511238
total KE: 3.56344
KE range: 15 , pdg :  11
particle KE sf: 15, p : 0.854455 , mass :  0.000510999 , mom_sf: 14.9916
particle E : 0.854455 , scaled E : 12.8097
KE range: 2.5 , pdg :  -211
particle KE sf: 2.5, p : 0.343081 , mass :  0.13957 , mom_sf: 2.04873
particle E : 0.370384 , scaled E : 0.716605
KE range: 2.5 , pdg :  211
particle KE sf: 2.5, p : 0.860004 , mass :  0.13957 , mom_sf: 2.28351
particle E : 0.871256 , scaled E : 1.96879
Total number particles: 0
Begin processing the 19th record. run: 200002 subRun: 0 event: 19 at 15-Jan-2026 06:49:53 CET
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 06:49:53 CET  run: 200002 subRun: 0 event: 19
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 06:49:53 CET  run: 200002 subRun: 0 event: 19
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 06:49:53 CET  run: 200002 subRun: 0 event: 19
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w BackTrackerService:  ProcessEvent 15-Jan-2026 06:49:53 CET  run: 200002 subRun: 0 event: 19
Rebuild failed to get the SimChannels. This is expected when running on a generation or simulation step.
%MSG
%MSG-w PhotonBackTrackerService:  ProcessEvent 15-Jan-2026 06:49:53 CET  run: 200002 subRun: 0 event: 19
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
%MSG-i NuRandomService:  MultiPartVertex:generator@BeginModule  15-Jan-2026 06:49:53 CET run: 200002 subRun: 0 event: 19
Random seed for this event, engine 'generator.GenVertex': 630700987
%MSG
Processing a new event...
Generating a rain particle at (-260.641,90.7351,123.838)
Generated boost parameter gamma beta = 0.270184, cos(theta) = -0.0218234, phi = 0.362055
Generating a boost (0.243864,0.0923637,-0.00569222)
Event Vertex @ (-260.641,90.7351,123.838) ... 1 particles...
  0th instance PDG 11
E: 0.916431 , m: 0.000510999 , KE: 0.91592
total KE: 0.91592
gen_total_KE: 0.697307 , KE scale factor :  0.761319
KE range: 15 , pdg :  11
particle KE sf: 11.4198, p : 0.916431 , mass :  0.000510999 , mom_sf: 11.414
particle E : 0.916431 , scaled E : 10.4601
Total number particles: 0
Begin processing the 20th record. run: 200002 subRun: 0 event: 20 at 15-Jan-2026 06:49:53 CET
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 06:49:53 CET  run: 200002 subRun: 0 event: 20
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 06:49:53 CET  run: 200002 subRun: 0 event: 20
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 06:49:53 CET  run: 200002 subRun: 0 event: 20
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w BackTrackerService:  ProcessEvent 15-Jan-2026 06:49:53 CET  run: 200002 subRun: 0 event: 20
Rebuild failed to get the SimChannels. This is expected when running on a generation or simulation step.
%MSG
%MSG-w PhotonBackTrackerService:  ProcessEvent 15-Jan-2026 06:49:53 CET  run: 200002 subRun: 0 event: 20
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
%MSG-i NuRandomService:  MultiPartVertex:generator@BeginModule  15-Jan-2026 06:49:53 CET run: 200002 subRun: 0 event: 20
Random seed for this event, engine 'generator.GenVertex': 788357890
%MSG
Processing a new event...
Generating a rain particle at (83.3878,-498.848,73.851)
Generated boost parameter gamma beta = 0.991461, cos(theta) = -0.982576, phi = 5.99948
Generating a boost (0.125628,-0.0366298,-0.6918)
Event Vertex @ (83.3878,-498.848,73.851) ... 1 particles...
  0th instance PDG 11
E: 0.314267 , m: 0.000510999 , KE: 0.313756
total KE: 0.313756
gen_total_KE: 0.166701 , KE scale factor :  0.531307
KE range: 15 , pdg :  11
particle KE sf: 7.96961, p : 0.314267 , mass :  0.000510999 , mom_sf: 7.95829
particle E : 0.314267 , scaled E : 2.50103
Total number particles: 0
15-Jan-2026 06:49:56 CET  Closed output file "mpvmpr_monte-carlo-011951-000002_gen.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                    0.000295188   0.000734775   0.00601823    0.000390932   0.00122544       20     
--------------------------------------------------------------------------------------------------------------------------------
source:EmptyEvent(read)                       3.3773e-05    7.06762e-05   0.000576517   4.0722e-05    0.000116813      20     
simulate:rns:RandomNumberSaver                 1.055e-05    2.9808e-05    0.000366872   1.16015e-05   7.73447e-05      20     
simulate:generator:MultiPartVertex            7.3408e-05    0.000181282   0.00108913    0.000142654   0.000212366      20     
[art]:TriggerResults:TriggerResultInserter     5.751e-06    8.52265e-06   4.0186e-05     6.783e-06    7.28129e-06      20     
end_path:out1:RootOutput                       9.52e-07     1.71125e-06   1.3255e-05     1.107e-06    2.65025e-06      20     
end_path:out1:RootOutput(write)               4.1078e-05    0.000273803   0.00313729    5.6702e-05    0.000676017      20     
================================================================================================================================
%MSG-i NuRandomService:  RootOutput:out1@EndJob 15-Jan-2026 06:49:56 CET  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1
   Configured value          Last value   ModuleLabel.InstanceName
        (per event)           788357890   generator.GenVertex

%MSG

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1334.44 MB
  Peak resident set size usage (VmHWM): 562.749 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 20 passed = 20 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport         20         20          0 out1

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 1.772394 Real = 3.983709

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 1334.44 VmHWM = 562.749

Art has completed and will exit with status 0.
-rw-r--r-- 1 dune001 lbno 107K Jan 15 06:49 mpvmpr_monte-carlo-011951-000002_gen.root
Input fcl file: mpvmpr_gen_1x2x6.fcl
genType: mpvmpr
Wrote processed DID to justin-processed-dids.txt: monte-carlo-011951-000002
=== GEN job completed successfully ===
justIN time: 2026-02-04 11:25:17 UTC       justIN version: 01.06.00