justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 267198.0@dunegpschedd01.fnal.gov

Jobsub ID267198.0@dunegpschedd01.fnal.gov
Workflow ID11158
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-12 02:52:54
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-12-13 12:08:00
From worker nodeHostnamedunegli-8047102-0-fnpc17011.fnal.gov
cpuinfoIntel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-13 00:50:48
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T171216Z_gen_000884_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time11h (40616s)
CPU time10h (39483s = 97%)
Max RSS bytes2791292928 (2661 MiB)
Outputting started2025-12-13 12:07:45
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11158/1/001/analysisLowE_20251213T005050Z.root
Finished2025-12-13 12:08:00
Saved logsjustin-logs:267198.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

der
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  13-Dec-2025 12:06:13 UTC run: 5026 subRun: 0 event: 8840
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  13-Dec-2025 12:06:13 UTC run: 5026 subRun: 0 event: 8840
Trying to add data product: ShowerPCA. This element does not exist in the element holder
%MSG
Boundary wire vector sizes: 6724, 6655, 6627
minwire 0: 35
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.145538, 
Output 1: 0.113448, 0.0663149, 0.12878, 0.691458, 
Output 2: 0.0961182, 0.240302, 0.628365, 0.0352151, 
Output 3: 0.981617, 0.0105369, 0.00124541, 0.00660038, 
Output 4: 0.385923, 0.570641, 0.0400203, 0.00341532, 
Output 5: 0.525479, 0.405341, 0.0560446, 0.0131353, 
Output 6: 0.0245012, 0.0604395, 0.0739984, 0.841061, 

13-Dec-2025 12:06:17 UTC  Closed output file "matt_reco2.root"
13-Dec-2025 12:06:17 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/d3/51/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T171216Z_gen_000884_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2127.08       4024.54       4642.4        4408.64       825.276        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00101314    0.00356657     0.0125847    0.00190768    0.00362009       10     
reco:pandora:StandardPandora                            2124.39       3967.29       4575.92       4336.66       799.247        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000724777     0.13836      0.221827      0.157733      0.0701667       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000629488    0.0150696     0.0868766    0.00530493     0.0244636       10     
reco:pandoracalo:Calorimetry                          0.000490253   0.00296192     0.0189065    0.00132765    0.00533314       10     
reco:pandorapid:Chi2ParticleID                        0.000131824   0.00110458    0.00731737    0.000315688   0.00210662       10     
reco:cvnmap:CVNMapper                                  0.0759798      0.17353      0.306303      0.159949      0.0688145       10     
reco:cvneva:CVNEvaluator                                1.34863       2.96962       10.2929       2.31248       2.46053        10     
reco:energyrecnumu:EnergyReco                          0.113542       53.3007       71.3603       64.8568       26.7935        10     
reco:energyrecnue:EnergyReco                          0.00219628    0.00536242     0.0142762    0.00392887    0.00373243       10     
reco:energyrecnc:EnergyReco                            0.0108319     0.0167338     0.0331607     0.0152041    0.00602029       10     
[art]:TriggerResults:TriggerResultInserter            3.0825e-05    6.98707e-05   0.000257638    5.285e-05    6.34115e-05      10     
end_path:out1:RootOutput                              1.0348e-05    2.20249e-05    8.566e-05    1.44465e-05   2.14434e-05      10     
end_path:out1:RootOutput(write)                        0.496564      0.616763      0.985585      0.565486      0.144897        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 11901.2 MB
  Peak resident set size usage (VmHWM): 2791.29 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 82.5763 | hit correction : 1.2809
hit initial : 229.623 | hit correction : 1.28015
hit initial : 219.532 | hit correction : 1.27994
hit initial : 174.709 | hit correction : 1.27985
 *** Total Charge : 1368.93
 Looking at view: 1 | Number of hits in view: 7
hit initial : 288.323 | hit correction : 1.28066
hit initial : 197.133 | hit correction : 1.28093
hit initial : 52.651 | hit correction : 1.28004
hit initial : 13.7144 | hit correction : 1.28
hit initial : 488.707 | hit correction : 1.27989
 *** Total Charge : 1332.2
 Looking at view: 2 | Number of hits in view: 5
Begin processing the 10th record. run: 5026 subRun: 0 event: 8840 at 13-Dec-2025 12:07:37 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  6
This pfp has size:  51
Added hit list length: 51 | Max was set to: -3.40282e+38
This pfp has size:  9
This pfp has size:  6
True Neutron Energy: 939.569
True Electron Energy: 27.9063
Completed MCParticle Loop
True Neutrino Energy: 44.474
 Size of all hits: 51
hit initial : 37.5622 | hit correction : 1.21975
hit initial : 58.15 | hit correction : 1.21967
hit initial : 90.3887 | hit correction : 1.21752
hit initial : 69.0318 | hit correction : 1.21683
hit initial : 122.849 | hit correction : 1.21678
hit initial : 17.2607 | hit correction : 1.21674
hit initial : 60.6293 | hit correction : 1.21762
hit initial : 310.211 | hit correction : 1.21835
hit initial : 167.348 | hit correction : 1.21872
hit initial : 91.2445 | hit correction : 1.21926
hit initial : 479.702 | hit correction : 1.2193
hit initial : 389.441 | hit correction : 1.21942
hit initial : 256.121 | hit correction : 1.21947
hit initial : 79.9752 | hit correction : 1.21948
 *** Total Charge : 2717.89
 Looking at view: 0 | Number of hits in view: 14
hit initial : 15.6716 | hit correction : 1.21652
hit initial : 21.8925 | hit correction : 1.21967
hit initial : 60.0481 | hit correction : 1.21968
hit initial : 27.3075 | hit correction : 1.21969
hit initial : 59.0706 | hit correction : 1.21735
hit initial : 180.125 | hit correction : 1.21743
hit initial : 57.6706 | hit correction : 1.21675
hit initial : 144.042 | hit correction : 1.21678
hit initial : 37.008 | hit correction : 1.21676
hit initial : 109.569 | hit correction : 1.21768
hit initial : 288.663 | hit correction : 1.21814
hit initial : 394.814 | hit correction : 1.21857
hit initial : 352.051 | hit correction : 1.21893
hit initial : 282.443 | hit correction : 1.21925
hit initial : 299.018 | hit correction : 1.21937
hit initial : 319.235 | hit correction : 1.21944
hit initial : 263.542 | hit correction : 1.21946
 *** Total Charge : 3548.92
 Looking at view: 1 | Number of hits in view: 17
hit initial : 80.4738 | hit correction : 1.21967
hit initial : 22.0129 | hit correction : 1.21967
hit initial : 34.6148 | hit correction : 1.21742
hit initial : 387.596 | hit correction : 1.2173
hit initial : 159.613 | hit correction : 1.21674
hit initial : 179.991 | hit correction : 1.21774
hit initial : 242.838 | hit correction : 1.21811
hit initial : 189.304 | hit correction : 1.2184
hit initial : 227.089 | hit correction : 1.2187
hit initial : 219.04 | hit correction : 1.21885
hit initial : 120.294 | hit correction : 1.21911
hit initial : 134.25 | hit correction : 1.21921
hit initial : 132.66 | hit correction : 1.21928
hit initial : 119.894 | hit correction : 1.21933
hit initial : 110.032 | hit correction : 1.21937
hit initial : 138.735 | hit correction : 1.21941
hit initial : 119.932 | hit correction : 1.21945
hit initial : 115.211 | hit correction : 1.21946
hit initial : 119.684 | hit correction : 1.21948
hit initial : 134.172 | hit correction : 1.21949
 *** Total Charge : 3640.47
 Looking at view: 2 | Number of hits in view: 20
13-Dec-2025 12:07:42 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.110993      0.213215       0.45089      0.174803      0.0964791       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000619028   0.000774001   0.00147882    0.000692831   0.000246523      10     
end_path:ana:AnalyzeEvents         0.110152      0.212165      0.448652      0.173933      0.0961486       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 9811.84 MB
  Peak resident set size usage (VmHWM): 1539.42 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 13:29:59 UTC       justIN version: 01.06.00