justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 249094.20@dunegpschedd01.fnal.gov

Jobsub ID249094.20@dunegpschedd01.fnal.gov
Workflow ID10113
Stage ID1
User namemdias@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-11-13 19:25:55
SiteUS_UChicago
EntryEngage_US_MWT2_uiuc_condce_mcore
Last heartbeat2025-11-13 23:08:59
From worker nodeHostnamemwt2-c168.campuscluster.illinois.edu
cpuinfoIntel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
GPU
Inner Apptainer?True
Job stateaborted
Started2025-11-13 22:35:26
Input filesehn1-beam-np04:H4_v34b_1GeV_-27.7_007947_220069_12_20250614T022952.root
Outputting started2025-11-13 22:46:41
Output files
Finished2025-11-13 23:08:59
List job events     (HTCondor job logs unavailable)

Jobscript log (last 10,000 characters)

subRun: 1 event: 72
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
IonAndScint Module Producer
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveOuter
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveInner
IonAndScint Module Producer
Skip SimEnergyDeposit in: 
Skip SimEnergyDeposit in: priorSCE
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveOuter
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveInner
Begin processing the 4th record. run: 20250614 subRun: 1 event: 73 at 13-Nov-2025 16:45:04 CST
%MSG-w PhotonBackTrackerService:  ProcessEvent 13-Nov-2025 16:45:08 CST  run: 20250614 subRun: 1 event: 73
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
IonAndScint Module Producer
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveOuter
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveInner
IonAndScint Module Producer
Skip SimEnergyDeposit in: 
Skip SimEnergyDeposit in: priorSCE
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveOuter
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveInner
Begin processing the 5th record. run: 20250614 subRun: 1 event: 74 at 13-Nov-2025 16:45:14 CST
%MSG-w PhotonBackTrackerService:  ProcessEvent 13-Nov-2025 16:45:19 CST  run: 20250614 subRun: 1 event: 74
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
IonAndScint Module Producer
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveOuter
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveInner
IonAndScint Module Producer
Skip SimEnergyDeposit in: 
Skip SimEnergyDeposit in: priorSCE
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveOuter
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveInner
Begin processing the 6th record. run: 20250614 subRun: 1 event: 75 at 13-Nov-2025 16:45:29 CST
%MSG-w PhotonBackTrackerService:  ProcessEvent 13-Nov-2025 16:45:32 CST  run: 20250614 subRun: 1 event: 75
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
IonAndScint Module Producer
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveOuter
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveInner
IonAndScint Module Producer
Skip SimEnergyDeposit in: 
Skip SimEnergyDeposit in: priorSCE
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveOuter
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveInner
Begin processing the 7th record. run: 20250614 subRun: 1 event: 76 at 13-Nov-2025 16:45:40 CST
%MSG-w PhotonBackTrackerService:  ProcessEvent 13-Nov-2025 16:45:45 CST  run: 20250614 subRun: 1 event: 76
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
IonAndScint Module Producer
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveOuter
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveInner
IonAndScint Module Producer
Skip SimEnergyDeposit in: 
Skip SimEnergyDeposit in: priorSCE
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveOuter
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveInner
Begin processing the 8th record. run: 20250614 subRun: 1 event: 77 at 13-Nov-2025 16:45:55 CST
%MSG-w PhotonBackTrackerService:  ProcessEvent 13-Nov-2025 16:46:00 CST  run: 20250614 subRun: 1 event: 77
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
IonAndScint Module Producer
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveOuter
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveInner
IonAndScint Module Producer
Skip SimEnergyDeposit in: 
Skip SimEnergyDeposit in: priorSCE
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveOuter
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveInner
Begin processing the 9th record. run: 20250614 subRun: 1 event: 78 at 13-Nov-2025 16:46:08 CST
%MSG-w PhotonBackTrackerService:  ProcessEvent 13-Nov-2025 16:46:13 CST  run: 20250614 subRun: 1 event: 78
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
IonAndScint Module Producer
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveOuter
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveInner
IonAndScint Module Producer
Skip SimEnergyDeposit in: 
Skip SimEnergyDeposit in: priorSCE
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveOuter
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveInner
Begin processing the 10th record. run: 20250614 subRun: 1 event: 79 at 13-Nov-2025 16:46:24 CST
%MSG-w PhotonBackTrackerService:  ProcessEvent 13-Nov-2025 16:46:30 CST  run: 20250614 subRun: 1 event: 79
Rebuild failed to get the OpDetBTRs. This is expected when running on a generation or simulation step.
%MSG
IonAndScint Module Producer
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveOuter
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveInner
IonAndScint Module Producer
Skip SimEnergyDeposit in: 
Skip SimEnergyDeposit in: priorSCE
SimEnergyDeposit input module: largeant, instance name: LArG4DetectorServicevolTPCActiveOuter
Skip SimEnergyDeposit in: LArG4DetectorServicevolTPCActiveInner
13-Nov-2025 16:46:40 CST  Closed output file "pdhd_prod_beam__249094_20_1_20251113T223529Z_gen_g4_IonScintPDExt.root"
13-Nov-2025 16:46:40 CST  Closed input file "temp_largeant.root"
IonAndScint endJob.
IonAndScint endJob.

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                      6.30862       9.62069       15.1144       9.57332       2.40503        10     
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.000201212   0.00197949     0.017166     0.000234311   0.00506396       10     
simulate:rns:RandomNumberSaver                3.4815e-05    5.72211e-05   0.000234488   3.68185e-05   5.91464e-05      10     
simulate:IonAndScint:IonAndScint                1.09692       1.6197        2.13093       1.57598      0.297302        10     
simulate:IonAndScintExternal:IonAndScint      0.00174274    0.00300305    0.00510858    0.00250804    0.00117022       10     
simulate:PDFastSimExternal:PDFastSimPVS        0.176287       1.05618       7.3256       0.290909       2.09898        10     
[art]:TriggerResults:TriggerResultInserter     1.187e-05    1.61873e-05   4.1887e-05    1.31005e-05   8.72487e-06      10     
end_path:out1:RootOutput                       2.743e-06    4.3425e-06    1.5221e-05     3.069e-06    3.6422e-06       10     
end_path:out1:RootOutput(write)                 4.67274       6.9394        9.02654       6.64026       1.31954        10     
================================================================================================================================
%MSG-i NuRandomService:  RootOutput:out1@EndJob 13-Nov-2025 16:46:40 CST  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'random'
  master seed: 80883104
  seed within: [ 1 ; 900000000 ]
   Configured value          Last value   ModuleLabel.InstanceName
           22590189              (same)   IonAndScint.ISCalcAlg
          642434424              (same)   IonAndScintExternal.ISCalcAlg
          343573835              (same)   PDFastSimExternal.photon
          686690441              (same)   PDFastSimExternal.scinttime

%MSG

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5901.87 MB
  Peak resident set size usage (VmHWM): 4908.4 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 10 passed = 10 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport         10         10          0 out1

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 140.582432 Real = 145.123607

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 5901.87 VmHWM = 4908.4

Art has completed and will exit with status 0.
ionscint pdext lar returns 0
==== Start *.json ====
cat: *.json: No such file or directory
==== End *.json ====
.:
total 2161704
-rw-r--r-- 1 dune osgvo     249054 Nov 13 16:46 jobscript.log
-rw-r--r-- 1 dune osgvo        151 Nov 13 16:46 justin-processed-pfns.txt
-rw-r--r-- 1 dune osgvo        519 Nov 13 16:46 g4_protodunehd_hist.root
-rw-r--r-- 1 dune osgvo 2212438190 Nov 13 16:46 pdhd_prod_beam__249094_20_1_20251113T223529Z_gen_g4_IonScintPDExt.root
-rw-r--r-- 1 dune osgvo        519 Nov 13 16:42 g4_protodunehd_stage1_hist.root
-rw-r--r-- 1 dune osgvo     716146 Nov 13 16:37 prod_cosmics_beam_protodunehd_hist.root
-rw-r--r-- 1 dune osgvo     118784 Nov 13 16:37 mem.db
-rw-r--r-- 1 dune osgvo      28672 Nov 13 16:37 time.db
-rw-r--r-- 1 dune osgvo        144 Nov 13 16:35 all-input-dids.txt
justIN time: 2026-02-09 18:54:38 UTC       justIN version: 01.06.00