justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 273436.1@dunegpschedd02.fnal.gov

Jobsub ID273436.1@dunegpschedd02.fnal.gov
Workflow ID11970
Stage ID1
User nameusher@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit14400 (4 hours)
Submitted time2026-01-15 18:50:13
SiteES_PIC
EntryDUNE_T1_ES_PIC_ce16-multicore
Last heartbeat2026-01-15 19:57:22
From worker nodeHostnamehnode51.pic.es
cpuinfoAMD EPYC 7402P 24-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job stateoutputting_failed
Started2026-01-15 18:51:06
Input filesmonte-carlo-011970-000023
JobscriptExit code0
Real time0m (17s)
CPU time0m (12s = 70%)
Max RSS bytes594513920 (566 MiB)
Outputting started2026-01-15 18:51:23
Output files
Finished2026-01-15 19:57:22
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

phi = 0.140028
Generating a boost (0.44348,0.0625088,0.824348)
Event Vertex @ (-146.085,-246.974,857.61) ... 2 particles...
  0th instance PDG 13
  1th instance PDG 2112
E: 0.451589 , m: 0.105658 , KE: 0.345931
E: 1.81885 , m: 0.939565 , KE: 0.879288
total KE: 1.22522
KE range: 15 , pdg :  13
particle KE sf: 15, p : 0.439055 , mass :  0.105658 , mom_sf: 12.0567
particle E : 0.451589 , scaled E : 5.29462
Total number particles: 0
Begin processing the 18th record. run: 200023 subRun: 0 event: 18 at 15-Jan-2026 19:51:21 CET
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 19:51:21 CET  run: 200023 subRun: 0 event: 18
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 19:51:21 CET  run: 200023 subRun: 0 event: 18
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 19:51:21 CET  run: 200023 subRun: 0 event: 18
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w BackTrackerService:  ProcessEvent 15-Jan-2026 19:51:21 CET  run: 200023 subRun: 0 event: 18
Rebuild failed to get the SimChannels. This is expected when running on a generation or simulation step.
%MSG
%MSG-w PhotonBackTrackerService:  ProcessEvent 15-Jan-2026 19:51:21 CET  run: 200023 subRun: 0 event: 18
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
%MSG-i NuRandomService:  MultiPartVertex:generator@BeginModule  15-Jan-2026 19:51:21 CET run: 200023 subRun: 0 event: 18
Random seed for this event, engine 'generator.GenVertex': 647543363
%MSG
Processing a new event...
Generating a rain particle at (-65.9675,273.059,562.095)
Generated boost parameter gamma beta = 1.29869, cos(theta) = -0.274259, phi = 3.76911
Generating a boost (-0.616786,-0.447365,-0.217303)
Event Vertex @ (-65.9675,273.059,562.095) ... 3 particles...
  0th instance PDG -13
  1th instance PDG 22
  2th instance PDG 2212
E: 0.451313 , m: 0.105658 , KE: 0.345655
E: 0.723618 , m: 0 , KE: 0.723618
E: 1.14688 , m: 0.938272 , KE: 0.208611
total KE: 1.27788
KE range: 15 , pdg :  -13
particle KE sf: 15, p : 0.438771 , mass :  0.105658 , mom_sf: 12.0551
particle E : 0.451313 , scaled E : 5.29049
KE range: 2.5 , pdg :  22
particle KE sf: 2.5, p : 0.723618 , mass :  0 , mom_sf: 2.5
particle E : 0.723618 , scaled E : 1.80905
Total number particles: 0
Begin processing the 19th record. run: 200023 subRun: 0 event: 19 at 15-Jan-2026 19:51:21 CET
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 19:51:21 CET  run: 200023 subRun: 0 event: 19
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 19:51:21 CET  run: 200023 subRun: 0 event: 19
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 19:51:21 CET  run: 200023 subRun: 0 event: 19
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w BackTrackerService:  ProcessEvent 15-Jan-2026 19:51:21 CET  run: 200023 subRun: 0 event: 19
Rebuild failed to get the SimChannels. This is expected when running on a generation or simulation step.
%MSG
%MSG-w PhotonBackTrackerService:  ProcessEvent 15-Jan-2026 19:51:21 CET  run: 200023 subRun: 0 event: 19
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
%MSG-i NuRandomService:  MultiPartVertex:generator@BeginModule  15-Jan-2026 19:51:21 CET run: 200023 subRun: 0 event: 19
Random seed for this event, engine 'generator.GenVertex': 312023591
%MSG
Processing a new event...
Generating a rain particle at (205.757,-512.564,732.346)
Generated boost parameter gamma beta = 2.587, cos(theta) = 0.317319, phi = 3.94876
Generating a boost (-0.611698,-0.638927,0.295976)
Event Vertex @ (205.757,-512.564,732.346) ... 3 particles...
  0th instance PDG 11
  1th instance PDG 3122
  2th instance PDG -211
E: 0.357419 , m: 0.000510999 , KE: 0.356908
E: 1.75793 , m: 1.11568 , KE: 0.642247
E: 0.301192 , m: 0.13957 , KE: 0.161622
total KE: 1.16078
KE range: 15 , pdg :  11
particle KE sf: 15, p : 0.357419 , mass :  0.000510999 , mom_sf: 14.98
particle E : 0.357419 , scaled E : 5.35414
KE range: 3 , pdg :  3122
particle KE sf: 3, p : 1.35852 , mass :  1.11568 , mom_sf: 2.0835
particle E : 1.75793 , scaled E : 3.04242
KE range: 2.5 , pdg :  -211
particle KE sf: 2.5, p : 0.266902 , mass :  0.13957 , mom_sf: 1.96852
particle E : 0.301192 , scaled E : 0.543625
Total number particles: 0
Begin processing the 20th record. run: 200023 subRun: 0 event: 20 at 15-Jan-2026 19:51:21 CET
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 19:51:21 CET  run: 200023 subRun: 0 event: 20
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 19:51:21 CET  run: 200023 subRun: 0 event: 20
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 19:51:21 CET  run: 200023 subRun: 0 event: 20
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w BackTrackerService:  ProcessEvent 15-Jan-2026 19:51:21 CET  run: 200023 subRun: 0 event: 20
Rebuild failed to get the SimChannels. This is expected when running on a generation or simulation step.
%MSG
%MSG-w PhotonBackTrackerService:  ProcessEvent 15-Jan-2026 19:51:21 CET  run: 200023 subRun: 0 event: 20
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
%MSG-i NuRandomService:  MultiPartVertex:generator@BeginModule  15-Jan-2026 19:51:21 CET run: 200023 subRun: 0 event: 20
Random seed for this event, engine 'generator.GenVertex': 234517290
%MSG
Processing a new event...
Generating a rain particle at (-149.772,340.485,56.9495)
Generated boost parameter gamma beta = 2.15912, cos(theta) = 0.993395, phi = 0.851288
Generating a boost (0.0686153,0.0783102,0.901408)
Event Vertex @ (-149.772,340.485,56.9495) ... 2 particles...
  0th instance PDG -13
  1th instance PDG -211
E: 0.148848 , m: 0.105658 , KE: 0.0431901
E: 0.885364 , m: 0.13957 , KE: 0.745794
total KE: 0.788984
gen_total_KE: 0.706775 , KE scale factor :  0.895804
KE range: 15 , pdg :  -13
particle KE sf: 13.4371, p : 0.104843 , mass :  0.105658 , mom_sf: 6.46507
particle E : 0.148848 , scaled E : 0.686006
KE range: 2.5 , pdg :  -211
particle KE sf: 2.23951, p : 0.874294 , mass :  0.13957 , mom_sf: 2.06383
particle E : 0.885364 , scaled E : 1.80978
Total number particles: 0
15-Jan-2026 19:51:23 CET  Closed output file "mpvmpr_monte-carlo-011970-000023_gen.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                    0.000274609   0.000741152    0.0072944    0.00032831    0.00151052       20     
--------------------------------------------------------------------------------------------------------------------------------
source:EmptyEvent(read)                       3.0141e-05    6.83662e-05   0.000643621   3.2286e-05    0.000134159      20     
simulate:rns:RandomNumberSaver                 8.561e-06    3.40583e-05   0.000486566    9.075e-06    0.000103876      20     
simulate:generator:MultiPartVertex            8.1013e-05    0.000209831   0.00192612    0.000108528   0.000395402      20     
[art]:TriggerResults:TriggerResultInserter     4.79e-06     9.7697e-06    9.8183e-05    4.9705e-06    2.02908e-05      20     
end_path:out1:RootOutput                         7e-07      1.92955e-06   2.1731e-05     7.75e-07     4.55492e-06      20     
end_path:out1:RootOutput(write)               3.4231e-05    0.000244354   0.00283917    4.3977e-05    0.000608559      20     
================================================================================================================================
%MSG-i NuRandomService:  RootOutput:out1@EndJob 15-Jan-2026 19:51:23 CET  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1
   Configured value          Last value   ModuleLabel.InstanceName
        (per event)           234517290   generator.GenVertex

%MSG

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1336.57 MB
  Peak resident set size usage (VmHWM): 575.586 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 20 passed = 20 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport         20         20          0 out1

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 1.516562 Real = 1.778026

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 1336.57 VmHWM = 575.586

Art has completed and will exit with status 0.
-rw-r--r-- 1 dune001 dune 107K Jan 15 19:51 mpvmpr_monte-carlo-011970-000023_gen.root
Input fcl file: mpvmpr_gen_1x2x6.fcl
genType: mpvmpr
Wrote processed DID to justin-processed-dids.txt: monte-carlo-011970-000023
=== GEN job completed successfully ===
justIN time: 2026-02-04 06:06:06 UTC       justIN version: 01.06.00