justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 279313.47@dunegpschedd01.fnal.gov

Jobsub ID279313.47@dunegpschedd01.fnal.gov
Workflow ID11977
Stage ID1
User nameusher@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit14400 (4 hours)
Submitted time2026-01-15 21:20:19
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc02
Last heartbeat2026-01-15 22:21:05
From worker nodeHostnamewn-da-12.gina.surf.nl
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job stateoutputting_failed
Started2026-01-15 21:21:26
Input filesmonte-carlo-011977-000029
JobscriptExit code0
Real time0m (13s)
CPU time0m (12s = 92%)
Max RSS bytes590045184 (562 MiB)
Outputting started2026-01-15 21:21:40
Output files
Finished2026-01-15 22:21:05
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

029 subRun: 0 event: 18 at 15-Jan-2026 22:21:39 CET
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 22:21:39 CET  run: 200029 subRun: 0 event: 18
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 22:21:39 CET  run: 200029 subRun: 0 event: 18
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 22:21:39 CET  run: 200029 subRun: 0 event: 18
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w BackTrackerService:  ProcessEvent 15-Jan-2026 22:21:39 CET  run: 200029 subRun: 0 event: 18
Rebuild failed to get the SimChannels. This is expected when running on a generation or simulation step.
%MSG
%MSG-w PhotonBackTrackerService:  ProcessEvent 15-Jan-2026 22:21:39 CET  run: 200029 subRun: 0 event: 18
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
%MSG-i NuRandomService:  MultiPartVertex:generator@BeginModule  15-Jan-2026 22:21:39 CET run: 200029 subRun: 0 event: 18
Random seed for this event, engine 'generator.GenVertex': 338754446
%MSG
Processing a new event...
Generating a rain particle at (13.5067,-121.171,669.469)
Generated boost parameter gamma beta = 2.53179, cos(theta) = -0.566148, phi = 3.36919
Generating a boost (-0.746896,-0.172987,-0.526563)
Event Vertex @ (13.5067,-121.171,669.469) ... 2 particles...
  0th instance PDG 11
  1th instance PDG 2112
E: 0.394444 , m: 0.000510999 , KE: 0.393933
E: 1.02818 , m: 0.939565 , KE: 0.0886126
total KE: 0.482546
gen_total_KE: 0.971577 , KE scale factor :  2.01344
KE range: 15 , pdg :  11
particle KE sf: 30.2016, p : 0.394444 , mass :  0.000510999 , mom_sf: 30.1638
particle E : 0.394444 , scaled E : 11.8979
KE range: 1 , pdg :  2112
particle KE sf: 2.01344, p : 0.417573 , mass :  0.939565 , mom_sf: 1.45097
particle E : 1.02818 , scaled E : 1.11798
Total number particles: 0
Begin processing the 19th record. run: 200029 subRun: 0 event: 19 at 15-Jan-2026 22:21:39 CET
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 22:21:39 CET  run: 200029 subRun: 0 event: 19
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 22:21:39 CET  run: 200029 subRun: 0 event: 19
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 22:21:39 CET  run: 200029 subRun: 0 event: 19
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w BackTrackerService:  ProcessEvent 15-Jan-2026 22:21:39 CET  run: 200029 subRun: 0 event: 19
Rebuild failed to get the SimChannels. This is expected when running on a generation or simulation step.
%MSG
%MSG-w PhotonBackTrackerService:  ProcessEvent 15-Jan-2026 22:21:39 CET  run: 200029 subRun: 0 event: 19
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
%MSG-i NuRandomService:  MultiPartVertex:generator@BeginModule  15-Jan-2026 22:21:39 CET run: 200029 subRun: 0 event: 19
Random seed for this event, engine 'generator.GenVertex': 265207891
%MSG
Processing a new event...
Generating a rain particle at (120.123,-208.772,192.026)
Generated boost parameter gamma beta = 0.148924, cos(theta) = -0.481566, phi = 3.34001
Generating a boost (-0.126562,-0.0254475,-0.0709345)
Event Vertex @ (120.123,-208.772,192.026) ... 2 particles...
  0th instance PDG 11
  1th instance PDG 22
E: 0.45656 , m: 0.000510999 , KE: 0.456049
E: 0.0479829 , m: 0 , KE: 0.0479829
total KE: 0.504032
gen_total_KE: 0.200085 , KE scale factor :  0.396969
KE range: 15 , pdg :  11
particle KE sf: 5.95454, p : 0.456559 , mass :  0.000510999 , mom_sf: 5.949
particle E : 0.45656 , scaled E : 2.71607
KE range: 2.5 , pdg :  22
particle KE sf: 0.992423, p : 0.0479829 , mass :  0 , mom_sf: 0.992423
particle E : 0.0479829 , scaled E : 0.0476193
Total number particles: 0
Begin processing the 20th record. run: 200029 subRun: 0 event: 20 at 15-Jan-2026 22:21:39 CET
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 22:21:39 CET  run: 200029 subRun: 0 event: 20
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 22:21:39 CET  run: 200029 subRun: 0 event: 20
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 22:21:39 CET  run: 200029 subRun: 0 event: 20
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w BackTrackerService:  ProcessEvent 15-Jan-2026 22:21:39 CET  run: 200029 subRun: 0 event: 20
Rebuild failed to get the SimChannels. This is expected when running on a generation or simulation step.
%MSG
%MSG-w PhotonBackTrackerService:  ProcessEvent 15-Jan-2026 22:21:39 CET  run: 200029 subRun: 0 event: 20
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
%MSG-i NuRandomService:  MultiPartVertex:generator@BeginModule  15-Jan-2026 22:21:39 CET run: 200029 subRun: 0 event: 20
Random seed for this event, engine 'generator.GenVertex': 676049909
%MSG
Processing a new event...
Generating a rain particle at (-121.102,-246.477,326.779)
Generated boost parameter gamma beta = 0.818473, cos(theta) = 0.405783, phi = 3.45417
Generating a boost (-0.550832,-0.178015,0.257012)
Event Vertex @ (-121.102,-246.477,326.779) ... 6 particles...
  0th instance PDG 13
  1th instance PDG 211
  2th instance PDG -211
  3th instance PDG 22
  4th instance PDG 2212
  5th instance PDG 111
E: 0.241418 , m: 0.105658 , KE: 0.13576
E: 0.529029 , m: 0.13957 , KE: 0.389459
E: 0.489108 , m: 0.13957 , KE: 0.349538
E: 0.446562 , m: 0 , KE: 0.446562
E: 1.15071 , m: 0.938272 , KE: 0.212439
E: 0.745159 , m: 0.134977 , KE: 0.610182
total KE: 2.14394
KE range: 15 , pdg :  13
particle KE sf: 15, p : 0.217069 , mass :  0.105658 , mom_sf: 9.85608
particle E : 0.241418 , scaled E : 2.14206
KE range: 2.5 , pdg :  211
particle KE sf: 2.5, p : 0.510286 , mass :  0.13957 , mom_sf: 2.16434
particle E : 0.529029 , scaled E : 1.11322
KE range: 2.5 , pdg :  -211
particle KE sf: 2.5, p : 0.468772 , mass :  0.13957 , mom_sf: 2.14125
particle E : 0.489108 , scaled E : 1.01342
KE range: 2.5 , pdg :  22
particle KE sf: 2.5, p : 0.446562 , mass :  0 , mom_sf: 2.5
particle E : 0.446562 , scaled E : 1.11641
KE range: 2.5 , pdg :  111
particle KE sf: 2.5, p : 0.732832 , mass :  0.134977 , mom_sf: 2.25827
particle E : 0.745159 , scaled E : 1.66043
Total number particles: 0
15-Jan-2026 22:21:39 CET  Closed output file "mpvmpr_monte-carlo-011977-000029_gen.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                    0.000263561   0.000552237   0.00364391    0.000342609   0.000718597      20     
--------------------------------------------------------------------------------------------------------------------------------
source:EmptyEvent(read)                       3.2312e-05    4.45774e-05   0.000168301   3.41145e-05   3.04092e-05      20     
simulate:rns:RandomNumberSaver                 9.328e-06    1.89582e-05   0.000171848    1.027e-05    3.51684e-05      20     
simulate:generator:MultiPartVertex            5.9935e-05    0.00014658    0.000816667   0.000109935   0.000156699      20     
[art]:TriggerResults:TriggerResultInserter     5.129e-06    6.7619e-06    2.9086e-05     5.545e-06    5.13726e-06      20     
end_path:out1:RootOutput                       7.72e-07     1.39765e-06    8.486e-06     1.007e-06    1.63638e-06      20     
end_path:out1:RootOutput(write)                3.698e-05    0.000188305   0.00189701    4.72695e-05   0.000407479      20     
================================================================================================================================
%MSG-i NuRandomService:  RootOutput:out1@EndJob 15-Jan-2026 22:21:39 CET  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1
   Configured value          Last value   ModuleLabel.InstanceName
        (per event)           676049909   generator.GenVertex

%MSG

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1336.57 MB
  Peak resident set size usage (VmHWM): 584.327 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 20 passed = 20 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport         20         20          0 out1

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 1.059191 Real = 1.109269

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 1336.57 VmHWM = 584.327

Art has completed and will exit with status 0.
-rw-r--r--. 1 dune009 dune 111K Jan 15 22:21 mpvmpr_monte-carlo-011977-000029_gen.root
Input fcl file: mpvmpr_gen_1x2x6.fcl
genType: mpvmpr
Wrote processed DID to justin-processed-dids.txt: monte-carlo-011977-000029
=== GEN job completed successfully ===
justIN time: 2026-02-04 06:07:40 UTC       justIN version: 01.06.00