justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 242542.1@dunegpschedd02.fnal.gov

Jobsub ID242542.1@dunegpschedd02.fnal.gov
Workflow ID9996
Stage ID1
User namemdias@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-11-12 23:57:42
SiteUS_UChicago
EntryEngage_US_MWT2_iut2_condce_mcore
Last heartbeat2025-11-13 01:40:58
From worker nodeHostnameuct2-c614.mwt2.org
cpuinfoAMD EPYC 7302 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
GPU
Inner Apptainer?True
Job statestalled
Started2025-11-13 01:00:16
Input filesehn1-beam-np04:H4_v34b_5GeV_-27.7_005573_423807_65_20250627T163831.root
Outputting started2025-11-13 01:12:23
Output files
Finished2025-11-13 02:02:29
List job events     (HTCondor job logs unavailable)

Jobscript log (last 10,000 characters)

1 event: 22
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
IonAndScint Module Producer
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveOuter
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveInner
IonAndScint Module Producer
Skip SimEnergyDeposit in: 
Skip SimEnergyDeposit in: priorSCE
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveOuter
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveInner
Begin processing the 4th record. run: 20250627 subRun: 1 event: 23 at 12-Nov-2025 19:11:02 CST
%MSG-w PhotonBackTrackerService:  ProcessEvent 12-Nov-2025 19:11:06 CST  run: 20250627 subRun: 1 event: 23
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
IonAndScint Module Producer
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveOuter
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveInner
IonAndScint Module Producer
Skip SimEnergyDeposit in: 
Skip SimEnergyDeposit in: priorSCE
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveOuter
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveInner
Begin processing the 5th record. run: 20250627 subRun: 1 event: 24 at 12-Nov-2025 19:11:14 CST
%MSG-w PhotonBackTrackerService:  ProcessEvent 12-Nov-2025 19:11:17 CST  run: 20250627 subRun: 1 event: 24
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
IonAndScint Module Producer
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveOuter
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveInner
IonAndScint Module Producer
Skip SimEnergyDeposit in: 
Skip SimEnergyDeposit in: priorSCE
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveOuter
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveInner
Begin processing the 6th record. run: 20250627 subRun: 1 event: 25 at 12-Nov-2025 19:11:24 CST
%MSG-w PhotonBackTrackerService:  ProcessEvent 12-Nov-2025 19:11:27 CST  run: 20250627 subRun: 1 event: 25
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
IonAndScint Module Producer
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveOuter
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveInner
IonAndScint Module Producer
Skip SimEnergyDeposit in: 
Skip SimEnergyDeposit in: priorSCE
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveOuter
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveInner
Begin processing the 7th record. run: 20250627 subRun: 1 event: 26 at 12-Nov-2025 19:11:33 CST
%MSG-w PhotonBackTrackerService:  ProcessEvent 12-Nov-2025 19:11:37 CST  run: 20250627 subRun: 1 event: 26
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
IonAndScint Module Producer
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveOuter
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveInner
IonAndScint Module Producer
Skip SimEnergyDeposit in: 
Skip SimEnergyDeposit in: priorSCE
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveOuter
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveInner
Begin processing the 8th record. run: 20250627 subRun: 1 event: 27 at 12-Nov-2025 19:11:45 CST
%MSG-w PhotonBackTrackerService:  ProcessEvent 12-Nov-2025 19:11:48 CST  run: 20250627 subRun: 1 event: 27
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
IonAndScint Module Producer
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveOuter
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveInner
IonAndScint Module Producer
Skip SimEnergyDeposit in: 
Skip SimEnergyDeposit in: priorSCE
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveOuter
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveInner
Begin processing the 9th record. run: 20250627 subRun: 1 event: 28 at 12-Nov-2025 19:11:55 CST
%MSG-w PhotonBackTrackerService:  ProcessEvent 12-Nov-2025 19:11:59 CST  run: 20250627 subRun: 1 event: 28
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
IonAndScint Module Producer
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveOuter
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveInner
IonAndScint Module Producer
Skip SimEnergyDeposit in: 
Skip SimEnergyDeposit in: priorSCE
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveOuter
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveInner
Begin processing the 10th record. run: 20250627 subRun: 1 event: 29 at 12-Nov-2025 19:12:08 CST
%MSG-w PhotonBackTrackerService:  ProcessEvent 12-Nov-2025 19:12:11 CST  run: 20250627 subRun: 1 event: 29
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
IonAndScint Module Producer
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveOuter
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveInner
IonAndScint Module Producer
Skip SimEnergyDeposit in: 
Skip SimEnergyDeposit in: priorSCE
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveOuter
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveInner
12-Nov-2025 19:12:22 CST  Closed output file "pdhd_prod_beam__242542_1_1_20251113T010020Z_gen_g4_IonScintPDExt.root"
12-Nov-2025 19:12:22 CST  Closed input file "temp_largeant.root"
IonAndScint endJob.
IonAndScint endJob.

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                      6.00119       8.41239       15.2541       7.94263       2.50264        10     
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.000309341   0.000442186   0.00121562    0.00035657    0.000262577      10     
simulate:rns:RandomNumberSaver                 3.742e-05    8.19739e-05   0.000408136   4.45085e-05   0.000108878      10     
simulate:IonAndScint:IonAndScint                1.13738       1.48677       1.82938       1.50895      0.217587        10     
simulate:IonAndScintExternal:IonAndScint      0.00186959    0.00267461    0.00420021    0.00226169    0.000851047      10     
simulate:PDFastSimExternal:PDFastSimPVS        0.130019      0.863984       5.64929      0.291213       1.60108        10     
[art]:TriggerResults:TriggerResultInserter    1.3605e-05    1.80097e-05   4.3241e-05    1.42065e-05   8.79917e-06      10     
end_path:out1:RootOutput                       3.657e-06    8.1504e-06    2.4496e-05     4.183e-06    7.86107e-06      10     
end_path:out1:RootOutput(write)                 4.58836       6.05795       7.76852       6.14559      0.948792        10     
================================================================================================================================
%MSG-i NuRandomService:  RootOutput:out1@EndJob 12-Nov-2025 19:12:22 CST  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'random'
  master seed: 725405936
  seed within: [ 1 ; 900000000 ]
   Configured value          Last value   ModuleLabel.InstanceName
          316451167              (same)   IonAndScint.ISCalcAlg
          351503570              (same)   IonAndScintExternal.ISCalcAlg
          262479636              (same)   PDFastSimExternal.photon
          563831314              (same)   PDFastSimExternal.scinttime

%MSG

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5321.8 MB
  Peak resident set size usage (VmHWM): 4342.38 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 10 passed = 10 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport         10         10          0 out1

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 122.462212 Real = 125.875744

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 5321.8 VmHWM = 4342.38

Art has completed and will exit with status 0.
ionscint pdext lar returns 0
==== Start *.json ====
cat: *.json: No such file or directory
==== End *.json ====
.:
total 2106288
-rw-r--r--. 1 dune osgvo     258182 Nov 12 19:12 jobscript.log
-rw-r--r--. 1 dune osgvo        151 Nov 12 19:12 justin-processed-pfns.txt
-rw-r--r--. 1 dune osgvo        519 Nov 12 19:12 g4_protodunehd_hist.root
-rw-r--r--. 1 dune osgvo 2155683219 Nov 12 19:12 pdhd_prod_beam__242542_1_1_20251113T010020Z_gen_g4_IonScintPDExt.root
-rw-r--r--. 1 dune osgvo        519 Nov 12 19:08 g4_protodunehd_stage1_hist.root
-rw-r--r--. 1 dune osgvo     716493 Nov 12 19:04 prod_cosmics_beam_protodunehd_hist.root
-rw-r--r--. 1 dune osgvo     118784 Nov 12 19:04 mem.db
-rw-r--r--. 1 dune osgvo      28672 Nov 12 19:04 time.db
-rw-r--r--. 1 dune osgvo        144 Nov 12 19:00 all-input-dids.txt
justIN time: 2026-02-10 01:13:45 UTC       justIN version: 01.06.00