justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 279313.58@dunegpschedd01.fnal.gov

Jobsub ID279313.58@dunegpschedd01.fnal.gov
Workflow ID11977
Stage ID1
User nameusher@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit14400 (4 hours)
Submitted time2026-01-15 21:20:19
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc02
Last heartbeat2026-01-15 22:21:01
From worker nodeHostnamewn-da-12.gina.surf.nl
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job stateoutputting_failed
Started2026-01-15 21:21:24
Input filesmonte-carlo-011977-000026
JobscriptExit code0
Real time0m (15s)
CPU time0m (13s = 86%)
Max RSS bytes590725120 (563 MiB)
Outputting started2026-01-15 21:21:40
Output files
Finished2026-01-15 22:21:01
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

22
  4th instance PDG 211
  5th instance PDG 2112
E: 0.539035 , m: 0.000510999 , KE: 0.538524
E: 1.47283 , m: 0.939565 , KE: 0.533266
E: 0.393763 , m: 0.13957 , KE: 0.254193
E: 1.3074 , m: 1.11568 , KE: 0.191723
E: 0.901553 , m: 0.13957 , KE: 0.761983
E: 1.56155 , m: 0.939565 , KE: 0.621989
total KE: 2.90168
KE range: 15 , pdg :  11
particle KE sf: 15, p : 0.539035 , mass :  0.000510999 , mom_sf: 14.9867
particle E : 0.539035 , scaled E : 8.07837
KE range: 2.5 , pdg :  -211
particle KE sf: 2.5, p : 0.368198 , mass :  0.13957 , mom_sf: 2.07058
particle E : 0.393763 , scaled E : 0.775052
KE range: 3 , pdg :  3122
particle KE sf: 3, p : 0.681586 , mass :  1.11568 , mom_sf: 1.86407
particle E : 1.3074 , scaled E : 1.69085
KE range: 2.5 , pdg :  211
particle KE sf: 2.5, p : 0.890684 , mass :  0.13957 , mom_sf: 2.2901
particle E : 0.901553 , scaled E : 2.04453
Total number particles: 0
Begin processing the 19th record. run: 200026 subRun: 0 event: 19 at 15-Jan-2026 22:21:38 CET
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 22:21:38 CET  run: 200026 subRun: 0 event: 19
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 22:21:38 CET  run: 200026 subRun: 0 event: 19
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 22:21:38 CET  run: 200026 subRun: 0 event: 19
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w BackTrackerService:  ProcessEvent 15-Jan-2026 22:21:38 CET  run: 200026 subRun: 0 event: 19
Rebuild failed to get the SimChannels. This is expected when running on a generation or simulation step.
%MSG
%MSG-w PhotonBackTrackerService:  ProcessEvent 15-Jan-2026 22:21:38 CET  run: 200026 subRun: 0 event: 19
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
%MSG-i NuRandomService:  MultiPartVertex:generator@BeginModule  15-Jan-2026 22:21:38 CET run: 200026 subRun: 0 event: 19
Random seed for this event, engine 'generator.GenVertex': 84878575
%MSG
Processing a new event...
Generating a rain particle at (332.401,-317.714,941.287)
Generated boost parameter gamma beta = 1.87665, cos(theta) = -0.95024, phi = 3.19375
Generating a boost (-0.27455,-0.0143338,-0.83861)
Event Vertex @ (332.401,-317.714,941.287) ... 10 particles...
  0th instance PDG -11
  1th instance PDG 111
  2th instance PDG 2212
  3th instance PDG -211
  4th instance PDG 211
  5th instance PDG 2112
  6th instance PDG 2112
  7th instance PDG 3122
  8th instance PDG 2212
  9th instance PDG 2112
E: 0.757289 , m: 0.000510999 , KE: 0.756778
E: 0.779055 , m: 0.134977 , KE: 0.644078
E: 1.62661 , m: 0.938272 , KE: 0.688334
E: 0.605126 , m: 0.13957 , KE: 0.465556
E: 0.345503 , m: 0.13957 , KE: 0.205933
E: 1.35259 , m: 0.939565 , KE: 0.413023
E: 1.65156 , m: 0.939565 , KE: 0.711995
E: 1.3533 , m: 1.11568 , KE: 0.237618
E: 1.79713 , m: 0.938272 , KE: 0.858861
E: 1.06242 , m: 0.939565 , KE: 0.122859
total KE: 5.10503
KE range: 15 , pdg :  -11
particle KE sf: 15, p : 0.757289 , mass :  0.000510999 , mom_sf: 14.9906
particle E : 0.757289 , scaled E : 11.3522
KE range: 2.5 , pdg :  111
particle KE sf: 2.5, p : 0.767273 , mass :  0.134977 , mom_sf: 2.2677
particle E : 0.779055 , scaled E : 1.74517
KE range: 2.5 , pdg :  -211
particle KE sf: 2.5, p : 0.58881 , mass :  0.13957 , mom_sf: 2.20099
particle E : 0.605126 , scaled E : 1.30346
KE range: 2.5 , pdg :  211
particle KE sf: 2.5, p : 0.316057 , mass :  0.13957 , mom_sf: 2.02288
particle E : 0.345503 , scaled E : 0.654401
KE range: 3 , pdg :  3122
particle KE sf: 3, p : 0.765946 , mass :  1.11568 , mom_sf: 1.89141
particle E : 1.3533 , scaled E : 1.82853
Total number particles: 0
Begin processing the 20th record. run: 200026 subRun: 0 event: 20 at 15-Jan-2026 22:21:38 CET
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 22:21:38 CET  run: 200026 subRun: 0 event: 20
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 22:21:38 CET  run: 200026 subRun: 0 event: 20
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 22:21:38 CET  run: 200026 subRun: 0 event: 20
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w BackTrackerService:  ProcessEvent 15-Jan-2026 22:21:38 CET  run: 200026 subRun: 0 event: 20
Rebuild failed to get the SimChannels. This is expected when running on a generation or simulation step.
%MSG
%MSG-w PhotonBackTrackerService:  ProcessEvent 15-Jan-2026 22:21:38 CET  run: 200026 subRun: 0 event: 20
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
%MSG-i NuRandomService:  MultiPartVertex:generator@BeginModule  15-Jan-2026 22:21:38 CET run: 200026 subRun: 0 event: 20
Random seed for this event, engine 'generator.GenVertex': 180778534
%MSG
Processing a new event...
Generating a rain particle at (265.787,154.64,947.045)
Generated boost parameter gamma beta = 1.74827, cos(theta) = 0.288304, phi = 0.716702
Generating a boost (0.626686,0.545999,0.250257)
Event Vertex @ (265.787,154.64,947.045) ... 9 particles...
  0th instance PDG 13
  1th instance PDG 3122
  2th instance PDG 111
  3th instance PDG 211
  4th instance PDG 2112
  5th instance PDG 111
  6th instance PDG 2212
  7th instance PDG 2112
  8th instance PDG -211
E: 1.033 , m: 0.105658 , KE: 0.927341
E: 2.0333 , m: 1.11568 , KE: 0.917624
E: 0.73593 , m: 0.134977 , KE: 0.600953
E: 0.863926 , m: 0.13957 , KE: 0.724356
E: 1.4998 , m: 0.939565 , KE: 0.560239
E: 0.806909 , m: 0.134977 , KE: 0.671932
E: 1.47652 , m: 0.938272 , KE: 0.538251
E: 1.25396 , m: 0.939565 , KE: 0.314393
E: 0.993133 , m: 0.13957 , KE: 0.853563
total KE: 6.10865
KE range: 15 , pdg :  13
particle KE sf: 15, p : 1.02758 , mass :  0.105658 , mom_sf: 13.6392
particle E : 1.033 , scaled E : 14.0158
KE range: 3 , pdg :  3122
particle KE sf: 3, p : 1.69988 , mass :  1.11568 , mom_sf: 2.17909
particle E : 2.0333 , scaled E : 3.86855
KE range: 2.5 , pdg :  111
particle KE sf: 2.5, p : 0.723446 , mass :  0.134977 , mom_sf: 2.25557
particle E : 0.73593 , scaled E : 1.63736
KE range: 2.5 , pdg :  211
particle KE sf: 2.5, p : 0.852578 , mass :  0.13957 , mom_sf: 2.28186
particle E : 0.863926 , scaled E : 1.95046
KE range: 2.5 , pdg :  111
particle KE sf: 2.5, p : 0.795539 , mass :  0.134977 , mom_sf: 2.27491
particle E : 0.806909 , scaled E : 1.81481
KE range: 2.5 , pdg :  -211
particle KE sf: 2.5, p : 0.983277 , mass :  0.13957 , mom_sf: 2.30778
particle E : 0.993133 , scaled E : 2.27348
Total number particles: 0
15-Jan-2026 22:21:39 CET  Closed output file "mpvmpr_monte-carlo-011977-000026_gen.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                    0.000271418   0.000746845   0.00591618    0.000418328   0.00120336       20     
--------------------------------------------------------------------------------------------------------------------------------
source:EmptyEvent(read)                       3.2692e-05    6.51728e-05   0.000430832   4.1168e-05    8.79779e-05      20     
simulate:rns:RandomNumberSaver                1.0189e-05    2.58706e-05    0.0002833    1.1317e-05    5.91672e-05      20     
simulate:generator:MultiPartVertex            6.0075e-05    0.000187719    0.001152     0.000137147   0.000228967      20     
[art]:TriggerResults:TriggerResultInserter     5.111e-06    1.15073e-05   7.3762e-05     5.605e-06    1.74074e-05      20     
end_path:out1:RootOutput                       8.62e-07     1.9462e-06    1.5709e-05     1.107e-06    3.17934e-06      20     
end_path:out1:RootOutput(write)               3.7893e-05    0.000274421   0.00300356    5.65275e-05   0.000642515      20     
================================================================================================================================
%MSG-i NuRandomService:  RootOutput:out1@EndJob 15-Jan-2026 22:21:39 CET  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1
   Configured value          Last value   ModuleLabel.InstanceName
        (per event)           180778534   generator.GenVertex

%MSG

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1336.57 MB
  Peak resident set size usage (VmHWM): 584.782 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 20 passed = 20 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport         20         20          0 out1

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 1.398067 Real = 1.465633

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 1336.57 VmHWM = 584.782

Art has completed and will exit with status 0.
-rw-r--r--. 1 dune009 dune 108K Jan 15 22:21 mpvmpr_monte-carlo-011977-000026_gen.root
Input fcl file: mpvmpr_gen_1x2x6.fcl
genType: mpvmpr
Wrote processed DID to justin-processed-dids.txt: monte-carlo-011977-000026
=== GEN job completed successfully ===
justIN time: 2026-02-04 04:49:26 UTC       justIN version: 01.06.00