justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 243272.2@dunegpschedd02.fnal.gov

Jobsub ID243272.2@dunegpschedd02.fnal.gov
Workflow ID9986
Stage ID1
User nameemvilla@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-11-13 18:15:38
SiteUS_Wisconsin
EntryHCCHTPC_US_Wisconsin_osg01_rhel7
Last heartbeat2025-11-13 21:15:20
From worker nodeHostnamee2615
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit82800 (23 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-13 19:20:34
Input filesfardet-hd:prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T185233Z_gen_000883_supernova_g4_detsim.root
JobscriptExit code0
Real time1h (6868s)
CPU time4m (285s = 4%)
Max RSS bytes800366592 (763 MiB)
Outputting started2025-11-13 21:15:03
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/emvilla/production/fnal/09986/1/001/prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T185233Z_gen_000883_supernova_g4_detsim_ana_2025-11-13T_192042Z_tpstream.root
Finished2025-11-13 21:15:20
Saved logsjustin-logs:243272.2-dunegpschedd02.fnal.gov.logs.tgz
List job events     (HTCondor job logs unavailable)

Jobscript log (last 10,000 characters)

=== Start last 100 lines of lar log file ===
StandardRawDigitPrepService::ctor: Fetching ROI building service.
StandardRawDigitPrepService::ctor:   ROI building service: @0x3e07ca0
StandardRawDigitPrepService::ctor: Fetching wire building service.
StandardRawDigitPrepService::ctor:   Wire building service: @0x24e04b0
StandardRawDigitPrepService::ctor: StandardRawDigitPrepService:
StandardRawDigitPrepService::ctor:              LogLevel: 1
StandardRawDigitPrepService::ctor:               SkipBad: 1
StandardRawDigitPrepService::ctor:             SkipNoisy: 0
StandardRawDigitPrepService::ctor:   ChannelStatusOnline: 0
StandardRawDigitPrepService::ctor:          DoMitigation: 0
StandardRawDigitPrepService::ctor:  DoEarlySignalFinding: 0
StandardRawDigitPrepService::ctor:        DoNoiseRemoval: 0
StandardRawDigitPrepService::ctor:       DoDeconvolution: 1
StandardRawDigitPrepService::ctor:  DoPedestalAdjustment: 0
StandardRawDigitPrepService::ctor:                 DoROI: 1
StandardRawDigitPrepService::ctor:               DoWires: 1
StandardRawDigitPrepService::ctor:                DoDump: 0
StandardRawDigitPrepService::ctor:  DoIntermediateStates: 0
StandardRawDigitPrepService::ctor: No display tools.
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
13-Nov-2025 13:20:59 CST  Initiating request to open input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/d8/01/prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T185233Z_gen_000883_supernova_g4_detsim.root"
13-Nov-2025 13:21:01 CST  Opened input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/d8/01/prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T185233Z_gen_000883_supernova_g4_detsim.root"
Begin processing the 1st record. run: 8111 subRun: 0 event: 44101 at 13-Nov-2025 13:21:03 CST
Creating new TTree for tpmakerTPC::TriggerAnaTree1x2x2
13-Nov-2025 13:22:12 CST  Opened output file with pattern "triggerAnaTree.root"
Begin processing the 2nd record. run: 8111 subRun: 0 event: 44102 at 13-Nov-2025 14:18:15 CST
Begin processing the 3rd record. run: 8111 subRun: 0 event: 44103 at 13-Nov-2025 14:19:28 CST
Begin processing the 4th record. run: 8111 subRun: 0 event: 44104 at 13-Nov-2025 14:20:36 CST
Begin processing the 5th record. run: 8111 subRun: 0 event: 44105 at 13-Nov-2025 14:21:50 CST
Begin processing the 6th record. run: 8111 subRun: 0 event: 44106 at 13-Nov-2025 14:23:00 CST
Begin processing the 7th record. run: 8111 subRun: 0 event: 44107 at 13-Nov-2025 14:24:08 CST
Begin processing the 8th record. run: 8111 subRun: 0 event: 44108 at 13-Nov-2025 14:25:18 CST
Begin processing the 9th record. run: 8111 subRun: 0 event: 44109 at 13-Nov-2025 14:26:26 CST
Begin processing the 10th record. run: 8111 subRun: 0 event: 44110 at 13-Nov-2025 14:27:32 CST
Begin processing the 11th record. run: 8111 subRun: 0 event: 44111 at 13-Nov-2025 14:28:42 CST
Begin processing the 12th record. run: 8111 subRun: 0 event: 44112 at 13-Nov-2025 14:29:53 CST
Begin processing the 13th record. run: 8111 subRun: 0 event: 44113 at 13-Nov-2025 14:31:03 CST
Begin processing the 14th record. run: 8111 subRun: 0 event: 44114 at 13-Nov-2025 14:32:14 CST
Begin processing the 15th record. run: 8111 subRun: 0 event: 44115 at 13-Nov-2025 14:33:25 CST
Begin processing the 16th record. run: 8111 subRun: 0 event: 44116 at 13-Nov-2025 14:34:30 CST
Begin processing the 17th record. run: 8111 subRun: 0 event: 44117 at 13-Nov-2025 14:35:39 CST
Begin processing the 18th record. run: 8111 subRun: 0 event: 44118 at 13-Nov-2025 14:36:50 CST
Begin processing the 19th record. run: 8111 subRun: 0 event: 44119 at 13-Nov-2025 14:38:00 CST
Begin processing the 20th record. run: 8111 subRun: 0 event: 44120 at 13-Nov-2025 14:39:10 CST
Begin processing the 21st record. run: 8111 subRun: 0 event: 44121 at 13-Nov-2025 14:40:20 CST
Begin processing the 22nd record. run: 8111 subRun: 0 event: 44122 at 13-Nov-2025 14:41:32 CST
Begin processing the 23rd record. run: 8111 subRun: 0 event: 44123 at 13-Nov-2025 14:42:43 CST
Begin processing the 24th record. run: 8111 subRun: 0 event: 44124 at 13-Nov-2025 14:43:54 CST
Begin processing the 25th record. run: 8111 subRun: 0 event: 44125 at 13-Nov-2025 14:45:05 CST
Begin processing the 26th record. run: 8111 subRun: 0 event: 44126 at 13-Nov-2025 14:46:15 CST
Begin processing the 27th record. run: 8111 subRun: 0 event: 44127 at 13-Nov-2025 14:47:19 CST
Begin processing the 28th record. run: 8111 subRun: 0 event: 44128 at 13-Nov-2025 14:48:28 CST
Begin processing the 29th record. run: 8111 subRun: 0 event: 44129 at 13-Nov-2025 14:49:39 CST
Begin processing the 30th record. run: 8111 subRun: 0 event: 44130 at 13-Nov-2025 14:50:49 CST
Begin processing the 31st record. run: 8111 subRun: 0 event: 44131 at 13-Nov-2025 14:52:01 CST
Begin processing the 32nd record. run: 8111 subRun: 0 event: 44132 at 13-Nov-2025 14:53:05 CST
Begin processing the 33rd record. run: 8111 subRun: 0 event: 44133 at 13-Nov-2025 14:54:14 CST
Begin processing the 34th record. run: 8111 subRun: 0 event: 44134 at 13-Nov-2025 14:55:20 CST
Begin processing the 35th record. run: 8111 subRun: 0 event: 44135 at 13-Nov-2025 14:56:28 CST
Begin processing the 36th record. run: 8111 subRun: 0 event: 44136 at 13-Nov-2025 14:57:37 CST
Begin processing the 37th record. run: 8111 subRun: 0 event: 44137 at 13-Nov-2025 14:58:49 CST
Begin processing the 38th record. run: 8111 subRun: 0 event: 44138 at 13-Nov-2025 15:00:02 CST
Begin processing the 39th record. run: 8111 subRun: 0 event: 44139 at 13-Nov-2025 15:01:10 CST
Begin processing the 40th record. run: 8111 subRun: 0 event: 44140 at 13-Nov-2025 15:02:21 CST
Begin processing the 41st record. run: 8111 subRun: 0 event: 44141 at 13-Nov-2025 15:03:31 CST
Begin processing the 42nd record. run: 8111 subRun: 0 event: 44142 at 13-Nov-2025 15:04:39 CST
Begin processing the 43rd record. run: 8111 subRun: 0 event: 44143 at 13-Nov-2025 15:05:50 CST
Begin processing the 44th record. run: 8111 subRun: 0 event: 44144 at 13-Nov-2025 15:06:56 CST
Begin processing the 45th record. run: 8111 subRun: 0 event: 44145 at 13-Nov-2025 15:08:06 CST
Begin processing the 46th record. run: 8111 subRun: 0 event: 44146 at 13-Nov-2025 15:09:15 CST
Begin processing the 47th record. run: 8111 subRun: 0 event: 44147 at 13-Nov-2025 15:10:27 CST
Begin processing the 48th record. run: 8111 subRun: 0 event: 44148 at 13-Nov-2025 15:11:36 CST
Begin processing the 49th record. run: 8111 subRun: 0 event: 44149 at 13-Nov-2025 15:12:46 CST
Begin processing the 50th record. run: 8111 subRun: 0 event: 44150 at 13-Nov-2025 15:13:55 CST
13-Nov-2025 15:15:02 CST  Closed output file "triggerAnaTree.root"
13-Nov-2025 15:15:02 CST  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/d8/01/prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T185233Z_gen_000883_supernova_g4_detsim.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                      63.4029       69.1447       74.5245       69.5465       2.15744        50     
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.000440502   0.00420847     0.0352099    0.000896562   0.00913026       50     
makers:tpmakerTPC:TriggerPrimitiveMakerTPC      63.3028       69.0636       74.452        69.4675       2.1578         50     
[art]:TriggerResults:TriggerResultInserter    2.2502e-05    4.97003e-05   0.000133893   4.21345e-05   2.44625e-05      50     
end_path:out1:RootOutput                       5.441e-06    1.17988e-05    3.18e-05     1.01545e-05   5.75161e-06      50     
end_path:triggerAnaDumpTPs:TriggerAnaTree      0.0631962     0.0730054     0.0970947     0.0695047    0.00905041       50     
end_path:out1:RootOutput(write)               0.000992203   0.00287973     0.0114097    0.00169294    0.00240552       50     
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 1540.71 MB
  Peak resident set size usage (VmHWM): 800.367 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
.:
total 2965324
-rw-r--r-- 1 slot1_24 slot1_24 3033722984 Nov 13 15:15 triggerAnaTree.root
-rw-r--r-- 1 slot1_24 slot1_24    2541594 Nov 13 15:15 prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T185233Z_gen_000883_supernova_g4_detsim_ana_2025-11-13T_192042Z_tpstream.root
-rw-r--r-- 1 slot1_24 slot1_24     139264 Nov 13 15:15 mem.db
-rw-r--r-- 1 slot1_24 slot1_24      32768 Nov 13 15:15 time.db
-rw-r--r-- 1 slot1_24 slot1_24      12518 Nov 13 15:15 prodmarley_nue_flat_cc_dune10kt_1x2x2_20250926T185233Z_gen_000883_supernova_g4_detsim_reco_2025-11-13T_192042Z.log
-rw-r--r-- 1 slot1_24 slot1_24       9613 Nov 13 15:15 jobscript.log
-rw-r--r-- 1 slot1_24 slot1_24        180 Nov 13 15:15 justin-processed-pfns.txt
-rw-r--r-- 1 slot1_24 slot1_24        101 Nov 13 13:20 all-input-dids.txt
-rw-r--r-- 1 slot1_24 slot1_24          0 Nov 13 13:20 debugprod.log
justIN time: 2026-02-10 04:35:36 UTC       justIN version: 01.06.00