justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 273436.23@dunegpschedd02.fnal.gov

Jobsub ID273436.23@dunegpschedd02.fnal.gov
Workflow ID11970
Stage ID1
User nameusher@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit14400 (4 hours)
Submitted time2026-01-15 18:50:13
SiteUK_RAL-Tier1
EntryLIGO_UK_RAL_arc_ce04
Last heartbeat2026-01-15 19:52:50
From worker nodeHostnamedune001-1249023.0-lcg2705.gridpp.rl.ac.uk
cpuinfoAMD EPYC 9654 96-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job stateoutputting_failed
Started2026-01-15 18:51:16
Input filesmonte-carlo-011970-000036
JobscriptExit code0
Real time0m (24s)
CPU time0m (14s = 58%)
Max RSS bytes533266432 (508 MiB)
Outputting started2026-01-15 18:51:41
Output files
Finished2026-01-15 19:52:50
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

30463
KE range: 2.5 , pdg :  -211
particle KE sf: 2.5, p : 1.05942 , mass :  0.13957 , mom_sf: 2.32025
particle E : 1.06857 , scaled E : 2.46208
Total number particles: 0
Begin processing the 18th record. run: 200036 subRun: 0 event: 18 at 15-Jan-2026 18:51:39 UTC
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 18:51:39 UTC  run: 200036 subRun: 0 event: 18
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 18:51:39 UTC  run: 200036 subRun: 0 event: 18
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 18:51:39 UTC  run: 200036 subRun: 0 event: 18
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w BackTrackerService:  ProcessEvent 15-Jan-2026 18:51:39 UTC  run: 200036 subRun: 0 event: 18
Rebuild failed to get the SimChannels. This is expected when running on a generation or simulation step.
%MSG
%MSG-w PhotonBackTrackerService:  ProcessEvent 15-Jan-2026 18:51:39 UTC  run: 200036 subRun: 0 event: 18
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
%MSG-i NuRandomService:  MultiPartVertex:generator@BeginModule  15-Jan-2026 18:51:39 UTC run: 200036 subRun: 0 event: 18
Random seed for this event, engine 'generator.GenVertex': 212531009
%MSG
Processing a new event...
Generating a rain particle at (-319.196,-479.71,896.541)
Generated boost parameter gamma beta = 1.44893, cos(theta) = -0.147308, phi = 4.20767
Generating a boost (-0.393641,-0.712535,-0.121237)
Event Vertex @ (-319.196,-479.71,896.541) ... 3 particles...
  0th instance PDG -13
  1th instance PDG 2112
  2th instance PDG 111
E: 0.192033 , m: 0.105658 , KE: 0.0863755
E: 0.966424 , m: 0.939565 , KE: 0.0268588
E: 0.751821 , m: 0.134977 , KE: 0.616844
total KE: 0.730078
gen_total_KE: 0.194955 , KE scale factor :  0.267033
KE range: 15 , pdg :  -13
particle KE sf: 4.0055, p : 0.160353 , mass :  0.105658 , mom_sf: 2.73834
particle E : 0.192033 , scaled E : 0.451635
KE range: 1 , pdg :  2112
particle KE sf: 0.267033, p : 0.226258 , mass :  0.939565 , mom_sf: 0.514077
particle E : 0.966424 , scaled E : 0.946737
KE range: 2.5 , pdg :  111
particle KE sf: 0.667583, p : 0.739605 , mass :  0.134977 , mom_sf: 0.716395
particle E : 0.751821 , scaled E : 0.546772
Total number particles: 0
Begin processing the 19th record. run: 200036 subRun: 0 event: 19 at 15-Jan-2026 18:51:39 UTC
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 18:51:39 UTC  run: 200036 subRun: 0 event: 19
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 18:51:39 UTC  run: 200036 subRun: 0 event: 19
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 18:51:39 UTC  run: 200036 subRun: 0 event: 19
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w BackTrackerService:  ProcessEvent 15-Jan-2026 18:51:39 UTC  run: 200036 subRun: 0 event: 19
Rebuild failed to get the SimChannels. This is expected when running on a generation or simulation step.
%MSG
%MSG-w PhotonBackTrackerService:  ProcessEvent 15-Jan-2026 18:51:39 UTC  run: 200036 subRun: 0 event: 19
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
%MSG-i NuRandomService:  MultiPartVertex:generator@BeginModule  15-Jan-2026 18:51:39 UTC run: 200036 subRun: 0 event: 19
Random seed for this event, engine 'generator.GenVertex': 245038460
%MSG
Processing a new event...
Generating a rain particle at (332.57,108.453,110.825)
Generated boost parameter gamma beta = 1.50038, cos(theta) = 0.0751432, phi = 3.52726
Generating a boost (-0.768815,-0.312138,0.0625279)
Event Vertex @ (332.57,108.453,110.825) ... 2 particles...
  0th instance PDG 13
  1th instance PDG -211
E: 0.314487 , m: 0.105658 , KE: 0.208829
E: 0.650529 , m: 0.13957 , KE: 0.510959
total KE: 0.719787
gen_total_KE: 0.222775 , KE scale factor :  0.309501
KE range: 15 , pdg :  13
particle KE sf: 4.64252, p : 0.296206 , mass :  0.105658 , mom_sf: 3.61216
particle E : 0.314487 , scaled E : 1.07515
KE range: 2.5 , pdg :  -211
particle KE sf: 0.773753, p : 0.63538 , mass :  0.13957 , mom_sf: 0.812737
particle E : 0.650529 , scaled E : 0.534926
Total number particles: 0
Begin processing the 20th record. run: 200036 subRun: 0 event: 20 at 15-Jan-2026 18:51:39 UTC
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 18:51:39 UTC  run: 200036 subRun: 0 event: 20
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 18:51:39 UTC  run: 200036 subRun: 0 event: 20
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 18:51:39 UTC  run: 200036 subRun: 0 event: 20
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w BackTrackerService:  ProcessEvent 15-Jan-2026 18:51:39 UTC  run: 200036 subRun: 0 event: 20
Rebuild failed to get the SimChannels. This is expected when running on a generation or simulation step.
%MSG
%MSG-w PhotonBackTrackerService:  ProcessEvent 15-Jan-2026 18:51:39 UTC  run: 200036 subRun: 0 event: 20
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
%MSG-i NuRandomService:  MultiPartVertex:generator@BeginModule  15-Jan-2026 18:51:39 UTC run: 200036 subRun: 0 event: 20
Random seed for this event, engine 'generator.GenVertex': 114897669
%MSG
Processing a new event...
Generating a rain particle at (47.391,-430.074,1125.68)
Generated boost parameter gamma beta = 1.23573, cos(theta) = 0.491658, phi = 4.9018
Generating a boost (0.127452,-0.664803,0.382192)
Event Vertex @ (47.391,-430.074,1125.68) ... 6 particles...
  0th instance PDG 13
  1th instance PDG 22
  2th instance PDG 2212
  3th instance PDG 2112
  4th instance PDG 2112
  5th instance PDG 2112
E: 1.09838 , m: 0.105658 , KE: 0.992721
E: 0.129377 , m: 0 , KE: 0.129377
E: 1.80014 , m: 0.938272 , KE: 0.861863
E: 1.01215 , m: 0.939565 , KE: 0.0725896
E: 1.9321 , m: 0.939565 , KE: 0.992536
E: 1.00771 , m: 0.939565 , KE: 0.0681414
total KE: 3.11723
KE range: 15 , pdg :  13
particle KE sf: 15, p : 1.09329 , mass :  0.105658 , mom_sf: 13.7165
particle E : 1.09838 , scaled E : 14.9965
KE range: 2.5 , pdg :  22
particle KE sf: 2.5, p : 0.129377 , mass :  0 , mom_sf: 2.5
particle E : 0.129377 , scaled E : 0.323442
Total number particles: 0
15-Jan-2026 18:51:40 UTC  Closed output file "mpvmpr_monte-carlo-011970-000036_gen.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                    0.000328395   0.000812068   0.00611419    0.000494956   0.00122862       20     
--------------------------------------------------------------------------------------------------------------------------------
source:EmptyEvent(read)                       4.1402e-05    7.51279e-05   0.000422417   5.1763e-05    8.22179e-05      20     
simulate:rns:RandomNumberSaver                1.1077e-05    2.63783e-05   0.000248655   1.31145e-05   5.11746e-05      20     
simulate:generator:MultiPartVertex            8.1843e-05    0.000208712   0.00100734    0.000164909   0.000191243      20     
[art]:TriggerResults:TriggerResultInserter     5.999e-06    8.66505e-06   4.0712e-05     6.865e-06    7.40462e-06      20     
end_path:out1:RootOutput                       9.21e-07     1.81715e-06   1.3871e-05    1.0865e-06    2.78839e-06      20     
end_path:out1:RootOutput(write)               4.7231e-05    0.000313511    0.0036449    6.3936e-05    0.000778109      20     
================================================================================================================================
%MSG-i NuRandomService:  RootOutput:out1@EndJob 15-Jan-2026 18:51:40 UTC  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1
   Configured value          Last value   ModuleLabel.InstanceName
        (per event)           114897669   generator.GenVertex

%MSG

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1334.45 MB
  Peak resident set size usage (VmHWM): 574.366 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 20 passed = 20 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport         20         20          0 out1

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 1.596918 Real = 2.226250

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 1334.45 VmHWM = 574.366

Art has completed and will exit with status 0.
-rw-r--r-- 1 dune001 dune 108K Jan 15 18:51 mpvmpr_monte-carlo-011970-000036_gen.root
Input fcl file: mpvmpr_gen_1x2x6.fcl
genType: mpvmpr
Wrote processed DID to justin-processed-dids.txt: monte-carlo-011970-000036
=== GEN job completed successfully ===
justIN time: 2026-02-04 09:27:51 UTC       justIN version: 01.06.00