justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 273436.13@dunegpschedd02.fnal.gov

Jobsub ID273436.13@dunegpschedd02.fnal.gov
Workflow ID11970
Stage ID1
User nameusher@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit14400 (4 hours)
Submitted time2026-01-15 18:50:13
SiteES_PIC
EntryDUNE_T1_ES_PIC_ce15-multicore
Last heartbeat2026-01-15 19:53:43
From worker nodeHostnamehnode56.pic.es
cpuinfoAMD EPYC 7402P 24-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job stateoutputting_failed
Started2026-01-15 18:51:06
Input filesmonte-carlo-011970-000025
JobscriptExit code0
Real time0m (18s)
CPU time0m (14s = 77%)
Max RSS bytes593592320 (566 MiB)
Outputting started2026-01-15 18:51:25
Output files
Finished2026-01-15 19:53:43
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

Run: 0 event: 18 at 15-Jan-2026 19:51:23 CET
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 19:51:23 CET  run: 200025 subRun: 0 event: 18
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 19:51:23 CET  run: 200025 subRun: 0 event: 18
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 19:51:23 CET  run: 200025 subRun: 0 event: 18
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w BackTrackerService:  ProcessEvent 15-Jan-2026 19:51:23 CET  run: 200025 subRun: 0 event: 18
Rebuild failed to get the SimChannels. This is expected when running on a generation or simulation step.
%MSG
%MSG-w PhotonBackTrackerService:  ProcessEvent 15-Jan-2026 19:51:23 CET  run: 200025 subRun: 0 event: 18
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
%MSG-i NuRandomService:  MultiPartVertex:generator@BeginModule  15-Jan-2026 19:51:23 CET run: 200025 subRun: 0 event: 18
Random seed for this event, engine 'generator.GenVertex': 772419176
%MSG
Processing a new event...
Generating a rain particle at (-239.885,405.639,830.037)
Generated boost parameter gamma beta = 2.53528, cos(theta) = -0.720939, phi = 4.54577
Generating a boost (-0.106916,-0.635735,-0.670655)
Event Vertex @ (-239.885,405.639,830.037) ... 4 particles...
  0th instance PDG -13
  1th instance PDG -211
  2th instance PDG 211
  3th instance PDG 2212
E: 0.796909 , m: 0.105658 , KE: 0.691251
E: 0.392935 , m: 0.13957 , KE: 0.253365
E: 0.229426 , m: 0.13957 , KE: 0.0898557
E: 1.51683 , m: 0.938272 , KE: 0.578558
total KE: 1.61303
KE range: 15 , pdg :  -13
particle KE sf: 15, p : 0.789874 , mass :  0.105658 , mom_sf: 13.2602
particle E : 0.796909 , scaled E : 10.4744
KE range: 2.5 , pdg :  -211
particle KE sf: 2.5, p : 0.367312 , mass :  0.13957 , mom_sf: 2.06984
particle E : 0.392935 , scaled E : 0.772982
KE range: 2.5 , pdg :  211
particle KE sf: 2.5, p : 0.182089 , mass :  0.13957 , mom_sf: 1.84748
particle E : 0.229426 , scaled E : 0.364209
Total number particles: 0
Begin processing the 19th record. run: 200025 subRun: 0 event: 19 at 15-Jan-2026 19:51:23 CET
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 19:51:23 CET  run: 200025 subRun: 0 event: 19
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 19:51:23 CET  run: 200025 subRun: 0 event: 19
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 19:51:23 CET  run: 200025 subRun: 0 event: 19
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w BackTrackerService:  ProcessEvent 15-Jan-2026 19:51:23 CET  run: 200025 subRun: 0 event: 19
Rebuild failed to get the SimChannels. This is expected when running on a generation or simulation step.
%MSG
%MSG-w PhotonBackTrackerService:  ProcessEvent 15-Jan-2026 19:51:23 CET  run: 200025 subRun: 0 event: 19
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
%MSG-i NuRandomService:  MultiPartVertex:generator@BeginModule  15-Jan-2026 19:51:23 CET run: 200025 subRun: 0 event: 19
Random seed for this event, engine 'generator.GenVertex': 753113770
%MSG
Processing a new event...
Generating a rain particle at (114.022,-415.652,823.812)
Generated boost parameter gamma beta = 2.51807, cos(theta) = 0.912531, phi = 1.05854
Generating a boost (0.186319,0.331336,0.8481)
Event Vertex @ (114.022,-415.652,823.812) ... 2 particles...
  0th instance PDG 11
  1th instance PDG 2212
E: 0.530982 , m: 0.000510999 , KE: 0.530471
E: 1.29222 , m: 0.938272 , KE: 0.353944
total KE: 0.884415
gen_total_KE: 0.993366 , KE scale factor :  1.12319
KE range: 15 , pdg :  11
particle KE sf: 16.8478, p : 0.530982 , mass :  0.000510999 , mom_sf: 16.8326
particle E : 0.530982 , scaled E : 8.93781
KE range: 1 , pdg :  2212
particle KE sf: 1.12319, p : 0.88852 , mass :  0.938272 , mom_sf: 1.07012
particle E : 1.29222 , scaled E : 1.33582
Total number particles: 0
Begin processing the 20th record. run: 200025 subRun: 0 event: 20 at 15-Jan-2026 19:51:23 CET
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 19:51:23 CET  run: 200025 subRun: 0 event: 20
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 19:51:23 CET  run: 200025 subRun: 0 event: 20
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w ParticleInventory:  ProcessEvent 15-Jan-2026 19:51:23 CET  run: 200025 subRun: 0 event: 20
Rebuild failed to get the MCParticles. This is expected when running on a generation or simulation step.
%MSG
%MSG-w BackTrackerService:  ProcessEvent 15-Jan-2026 19:51:23 CET  run: 200025 subRun: 0 event: 20
Rebuild failed to get the SimChannels. This is expected when running on a generation or simulation step.
%MSG
%MSG-w PhotonBackTrackerService:  ProcessEvent 15-Jan-2026 19:51:23 CET  run: 200025 subRun: 0 event: 20
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
%MSG-i NuRandomService:  MultiPartVertex:generator@BeginModule  15-Jan-2026 19:51:23 CET run: 200025 subRun: 0 event: 20
Random seed for this event, engine 'generator.GenVertex': 69694718
%MSG
Processing a new event...
Generating a rain particle at (-141.186,-529.363,422.801)
Generated boost parameter gamma beta = 1.8765, cos(theta) = -0.571811, phi = 1.78474
Generating a boost (-0.153716,0.707491,-0.504628)
Event Vertex @ (-141.186,-529.363,422.801) ... 7 particles...
  0th instance PDG 11
  1th instance PDG 2212
  2th instance PDG -211
  3th instance PDG 2112
  4th instance PDG 2112
  5th instance PDG 211
  6th instance PDG 2212
E: 0.869603 , m: 0.000510999 , KE: 0.869092
E: 1.72598 , m: 0.938272 , KE: 0.787709
E: 0.209568 , m: 0.13957 , KE: 0.0699983
E: 1.82251 , m: 0.939565 , KE: 0.882942
E: 1.62325 , m: 0.939565 , KE: 0.683689
E: 1.04928 , m: 0.13957 , KE: 0.90971
E: 1.60734 , m: 0.938272 , KE: 0.669071
total KE: 4.87221
KE range: 15 , pdg :  11
particle KE sf: 15, p : 0.869603 , mass :  0.000510999 , mom_sf: 14.9918
particle E : 0.869603 , scaled E : 13.0369
KE range: 2.5 , pdg :  -211
particle KE sf: 2.5, p : 0.15633 , mass :  0.13957 , mom_sf: 1.80328
particle E : 0.209568 , scaled E : 0.314566
KE range: 2.5 , pdg :  211
particle KE sf: 2.5, p : 1.03996 , mass :  0.13957 , mom_sf: 2.31722
particle E : 1.04928 , scaled E : 2.41384
Total number particles: 0
15-Jan-2026 19:51:24 CET  Closed output file "mpvmpr_monte-carlo-011970-000025_gen.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                    0.000463123   0.000994704   0.00647415    0.000692279   0.00127168       20     
--------------------------------------------------------------------------------------------------------------------------------
source:EmptyEvent(read)                       5.3851e-05    8.75596e-05   0.00035164    6.7637e-05    6.35493e-05      20     
simulate:rns:RandomNumberSaver                 1.368e-05    3.49754e-05   0.000274907    2.187e-05    5.55174e-05      20     
simulate:generator:MultiPartVertex            0.000124953   0.000258824   0.00131046    0.000193636   0.000246233      20     
[art]:TriggerResults:TriggerResultInserter     7.331e-06    1.08559e-05   3.9751e-05     9.105e-06    6.8177e-06       20     
end_path:out1:RootOutput                       1.31e-06     3.0296e-06     1.323e-05    2.5405e-06    2.41186e-06      20     
end_path:out1:RootOutput(write)               9.0302e-05    0.000394244   0.00370115    0.000151834   0.00077853       20     
================================================================================================================================
%MSG-i NuRandomService:  RootOutput:out1@EndJob 15-Jan-2026 19:51:24 CET  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1
   Configured value          Last value   ModuleLabel.InstanceName
        (per event)            69694718   generator.GenVertex

%MSG

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1336.57 MB
  Peak resident set size usage (VmHWM): 577.012 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 20 passed = 20 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport         20         20          0 out1

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 1.560711 Real = 1.778868

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 1336.57 VmHWM = 577.012

Art has completed and will exit with status 0.
-rw-r--r-- 1 dune001 dune 113K Jan 15 19:51 mpvmpr_monte-carlo-011970-000025_gen.root
Input fcl file: mpvmpr_gen_1x2x6.fcl
genType: mpvmpr
Wrote processed DID to justin-processed-dids.txt: monte-carlo-011970-000025
=== GEN job completed successfully ===
justIN time: 2026-02-04 04:48:38 UTC       justIN version: 01.06.00