justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 278790.29@dunegpschedd01.fnal.gov

Jobsub ID278790.29@dunegpschedd01.fnal.gov
Workflow ID11951
Stage ID1
User nameusher@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit14400 (4 hours)
Submitted time2026-01-15 05:47:29
SiteFR_CCIN2P3
EntryDUNE_FR_CCIN2P3_cccondorce01
Last heartbeat2026-01-15 05:49:47
From worker nodeHostnameccwcondor0724
cpuinfoAMD EPYC 7302 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit106200 (29 hours)
GPU
Inner Apptainer?True
Job statestalled
Started2026-01-15 05:49:01
Input filesmonte-carlo-011951-000016
Outputting started2026-01-15 05:49:47
Output files
Finished2026-01-15 06:18:13
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

un: 0 event: 18
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
%MSG-i NuRandomService:  MultiPartVertex:generator@BeginModule  15-Jan-2026 06:49:43 CET run: 200016 subRun: 0 event: 18
Random seed for this event, engine 'generator.GenVertex': 822323972
%MSG
Processing a new event...
Generating a rain particle at (42.6074,-481.884,497.987)
Generated boost parameter gamma beta = 2.83026, cos(theta) = -0.220166, phi = 2.2938
Generating a boost (-0.608541,0.689639,-0.20759)
Event Vertex @ (42.6074,-481.884,497.987) ... 7 particles...
  0th instance PDG -13
  1th instance PDG 3122
  2th instance PDG 22
  3th instance PDG 2212
  4th instance PDG 111
  5th instance PDG 2212
  6th instance PDG -211
E: 0.944045 , m: 0.105658 , KE: 0.838387
E: 2.09783 , m: 1.11568 , KE: 0.982147
E: 0.884418 , m: 0 , KE: 0.884418
E: 1.31379 , m: 0.938272 , KE: 0.375515
E: 0.794663 , m: 0.134977 , KE: 0.659686
E: 1.58555 , m: 0.938272 , KE: 0.647276
E: 0.339615 , m: 0.13957 , KE: 0.200045
total KE: 4.58747
KE range: 15 , pdg :  -13
particle KE sf: 15, p : 0.938114 , mass :  0.105658 , mom_sf: 13.5176
particle E : 0.944045 , scaled E : 12.6815
KE range: 3 , pdg :  3122
particle KE sf: 3, p : 1.77655 , mass :  1.11568 , mom_sf: 2.19859
particle E : 2.09783 , scaled E : 4.06212
KE range: 2.5 , pdg :  22
particle KE sf: 2.5, p : 0.884418 , mass :  0 , mom_sf: 2.5
particle E : 0.884418 , scaled E : 2.21105
KE range: 2.5 , pdg :  111
particle KE sf: 2.5, p : 0.783116 , mass :  0.134977 , mom_sf: 2.2718
particle E : 0.794663 , scaled E : 1.78419
KE range: 2.5 , pdg :  -211
particle KE sf: 2.5, p : 0.30961 , mass :  0.13957 , mom_sf: 2.01631
particle E : 0.339615 , scaled E : 0.639682
Total number particles: 0
Begin processing the 19th record. run: 200016 subRun: 0 event: 19 at 15-Jan-2026 06:49:43 CET
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 06:49:43 CET  run: 200016 subRun: 0 event: 19
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 06:49:43 CET  run: 200016 subRun: 0 event: 19
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 06:49:43 CET  run: 200016 subRun: 0 event: 19
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w BackTrackerService:  ProcessEvent 15-Jan-2026 06:49:43 CET  run: 200016 subRun: 0 event: 19
Rebuild failed to get the SimChannels. This is expected when running on a generation or simulation step.
%MSG
%MSG-w PhotonBackTrackerService:  ProcessEvent 15-Jan-2026 06:49:43 CET  run: 200016 subRun: 0 event: 19
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
%MSG-i NuRandomService:  MultiPartVertex:generator@BeginModule  15-Jan-2026 06:49:43 CET run: 200016 subRun: 0 event: 19
Random seed for this event, engine 'generator.GenVertex': 741370771
%MSG
Processing a new event...
Generating a rain particle at (-156.521,461.143,1070.86)
Generated boost parameter gamma beta = 1.19783, cos(theta) = -0.555267, phi = 0.403002
Generating a boost (0.587287,0.250382,-0.426251)
Event Vertex @ (-156.521,461.143,1070.86) ... 3 particles...
  0th instance PDG -13
  1th instance PDG 2112
  2th instance PDG 111
E: 0.154386 , m: 0.105658 , KE: 0.0487282
E: 1.88598 , m: 0.939565 , KE: 0.946415
E: 0.577324 , m: 0.134977 , KE: 0.442347
total KE: 1.43749
KE range: 15 , pdg :  -13
particle KE sf: 15, p : 0.112568 , mass :  0.105658 , mom_sf: 7.37229
particle E : 0.154386 , scaled E : 0.836581
KE range: 2.5 , pdg :  111
particle KE sf: 2.5, p : 0.561323 , mass :  0.134977 , mom_sf: 2.19745
particle E : 0.577324 , scaled E : 1.24084
Total number particles: 0
Begin processing the 20th record. run: 200016 subRun: 0 event: 20 at 15-Jan-2026 06:49:43 CET
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 06:49:43 CET  run: 200016 subRun: 0 event: 20
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 06:49:43 CET  run: 200016 subRun: 0 event: 20
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 06:49:43 CET  run: 200016 subRun: 0 event: 20
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w BackTrackerService:  ProcessEvent 15-Jan-2026 06:49:43 CET  run: 200016 subRun: 0 event: 20
Rebuild failed to get the SimChannels. This is expected when running on a generation or simulation step.
%MSG
%MSG-w PhotonBackTrackerService:  ProcessEvent 15-Jan-2026 06:49:43 CET  run: 200016 subRun: 0 event: 20
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
%MSG-i NuRandomService:  MultiPartVertex:generator@BeginModule  15-Jan-2026 06:49:43 CET run: 200016 subRun: 0 event: 20
Random seed for this event, engine 'generator.GenVertex': 486425150
%MSG
Processing a new event...
Generating a rain particle at (-126.495,559.376,1081.52)
Generated boost parameter gamma beta = 2.26468, cos(theta) = -0.259848, phi = 5.0715
Generating a boost (0.310455,-0.827012,-0.237706)
Event Vertex @ (-126.495,559.376,1081.52) ... 9 particles...
  0th instance PDG -13
  1th instance PDG 211
  2th instance PDG 211
  3th instance PDG 22
  4th instance PDG 111
  5th instance PDG 2112
  6th instance PDG 2212
  7th instance PDG 3122
  8th instance PDG 2212
E: 0.530502 , m: 0.105658 , KE: 0.424844
E: 1.00202 , m: 0.13957 , KE: 0.862452
E: 0.410829 , m: 0.13957 , KE: 0.271259
E: 0.828221 , m: 0 , KE: 0.828221
E: 0.947535 , m: 0.134977 , KE: 0.812558
E: 1.07824 , m: 0.939565 , KE: 0.138671
E: 1.35505 , m: 0.938272 , KE: 0.416779
E: 1.69769 , m: 1.11568 , KE: 0.582006
E: 1.58865 , m: 0.938272 , KE: 0.650377
total KE: 4.98717
KE range: 15 , pdg :  -13
particle KE sf: 15, p : 0.519874 , mass :  0.105658 , mom_sf: 12.4597
particle E : 0.530502 , scaled E : 6.47832
KE range: 2.5 , pdg :  211
particle KE sf: 2.5, p : 0.992254 , mass :  0.13957 , mom_sf: 2.30934
particle E : 1.00202 , scaled E : 2.2957
KE range: 2.5 , pdg :  211
particle KE sf: 2.5, p : 0.386394 , mass :  0.13957 , mom_sf: 2.08522
particle E : 0.410829 , scaled E : 0.817716
KE range: 2.5 , pdg :  22
particle KE sf: 2.5, p : 0.828221 , mass :  0 , mom_sf: 2.5
particle E : 0.828221 , scaled E : 2.07055
KE range: 2.5 , pdg :  111
particle KE sf: 2.5, p : 0.937872 , mass :  0.134977 , mom_sf: 2.30539
particle E : 0.947535 , scaled E : 2.16637
KE range: 3 , pdg :  3122
particle KE sf: 3, p : 1.27961 , mass :  1.11568 , mom_sf: 2.05942
particle E : 1.69769 , scaled E : 2.8617
Total number particles: 0
15-Jan-2026 06:49:46 CET  Closed output file "mpvmpr_monte-carlo-011951-000016_gen.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                    0.000324831   0.00124242     0.0154807    0.000403629   0.00327179       20     
--------------------------------------------------------------------------------------------------------------------------------
source:EmptyEvent(read)                       3.4314e-05    7.83107e-05   0.000661345   4.1002e-05    0.000135987      20     
simulate:rns:RandomNumberSaver                1.1151e-05    3.11591e-05   0.000369676   1.2398e-05    7.7783e-05       20     
simulate:generator:MultiPartVertex            7.6354e-05    0.000197442   0.00123319    0.000142684   0.000240958      20     
[art]:TriggerResults:TriggerResultInserter     6.091e-06    9.1063e-06    4.4915e-05     7.354e-06    8.23134e-06      20     
end_path:out1:RootOutput                       8.52e-07      2.113e-06    1.9206e-05     1.147e-06    3.94259e-06      20     
end_path:out1:RootOutput(write)               4.7289e-05    0.000737026    0.0121741    5.6467e-05    0.00262875       20     
================================================================================================================================
%MSG-i NuRandomService:  RootOutput:out1@EndJob 15-Jan-2026 06:49:46 CET  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1
   Configured value          Last value   ModuleLabel.InstanceName
        (per event)           486425150   generator.GenVertex

%MSG

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1334.44 MB
  Peak resident set size usage (VmHWM): 547.197 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 20 passed = 20 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport         20         20          0 out1

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 1.919138 Real = 4.077197

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 1334.44 VmHWM = 547.197

Art has completed and will exit with status 0.
-rw-r--r-- 1 dune001 lbno 111K Jan 15 06:49 mpvmpr_monte-carlo-011951-000016_gen.root
Input fcl file: mpvmpr_gen_1x2x6.fcl
genType: mpvmpr
Wrote processed DID to justin-processed-dids.txt: monte-carlo-011951-000016
=== GEN job completed successfully ===
justIN time: 2026-02-04 11:32:24 UTC       justIN version: 01.06.00