justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 268035.0@dunegpschedd01.fnal.gov

Jobsub ID268035.0@dunegpschedd01.fnal.gov
Workflow ID11175
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-13 12:46:38
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-12-13 20:54:22
From worker nodeHostnamedunegli-7890507-0-fnpc17010.fnal.gov
cpuinfoIntel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-13 12:47:50
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T012900Z_gen_002987_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time8h (29178s)
CPU time8h (28846s = 98%)
Max RSS bytes2805141504 (2675 MiB)
Outputting started2025-12-13 20:54:08
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11175/1/001/analysisLowE_20251213T124751Z.root
Finished2025-12-13 20:54:22
Saved logsjustin-logs:268035.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

older
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  13-Dec-2025 20:52:40 UTC run: 5027 subRun: 0 event: 29870
Trying to add data product: ShowerPCA. This element does not exist in the element holder
%MSG
Boundary wire vector sizes: 6663, 6588, 6660
minwire 0: 61
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 2185, 2684
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.1037, 
Output 1: 0.057179, 0.102364, 0.152338, 0.688119, 
Output 2: 0.0460454, 0.227851, 0.703582, 0.0225212, 
Output 3: 0.938813, 0.0392597, 0.00808342, 0.0138444, 
Output 4: 0.38827, 0.499968, 0.0997915, 0.0119706, 
Output 5: 0.559323, 0.373266, 0.0583327, 0.00907799, 
Output 6: 0.0232839, 0.0531116, 0.0675128, 0.856092, 

13-Dec-2025 20:52:47 UTC  Closed output file "matt_reco2.root"
13-Dec-2025 20:52:47 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/03/7f/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T012900Z_gen_002987_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2396.62       2884.04       3522.07       2796.82       371.949        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00130614    0.00665197     0.0261924    0.00166493    0.00950383       10     
reco:pandora:StandardPandora                            2390.09       2880.38       3518.97       2792.87       372.322        10     
reco:pandoraTrack:LArPandoraTrackCreation              0.0023608     0.0942852     0.171406      0.0980631     0.0560169       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.00271542     0.0100901     0.0318711    0.00849378    0.00763491       10     
reco:pandoracalo:Calorimetry                          0.000439903   0.00207007    0.00869293    0.00115945    0.00250544       10     
reco:pandorapid:Chi2ParticleID                        0.000101509   0.000525065   0.00357201    0.000178509   0.00101705       10     
reco:cvnmap:CVNMapper                                  0.0706181     0.121463      0.277796      0.108475      0.0543614       10     
reco:cvneva:CVNEvaluator                                1.20989       2.7306        5.31853       2.6168        1.11099        10     
reco:energyrecnumu:EnergyReco                          0.108622      0.126812      0.143141      0.127459      0.0115487       10     
reco:energyrecnue:EnergyReco                          0.00196747    0.00373048     0.0130505    0.00286983    0.00314147       10     
reco:energyrecnc:EnergyReco                            0.0112225     0.0124171     0.0131076     0.0125848     0.0006374       10     
[art]:TriggerResults:TriggerResultInserter            2.7046e-05    4.40851e-05   0.000136231   3.1459e-05    3.20395e-05      10     
end_path:out1:RootOutput                               9.153e-06    1.5487e-05    4.2941e-05    1.14075e-05   9.65647e-06      10     
end_path:out1:RootOutput(write)                        0.416098      0.544642      0.779256      0.544927      0.0975748       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 11913.9 MB
  Peak resident set size usage (VmHWM): 2805.14 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 167.968 | hit correction : 1.07294
hit initial : 120.547 | hit correction : 1.07124
hit initial : 213.518 | hit correction : 1.07293
hit initial : 96.0317 | hit correction : 1.0729
hit initial : 24.1485 | hit correction : 1.07195
hit initial : 55.4391 | hit correction : 1.07142
hit initial : 138.306 | hit correction : 1.07144
hit initial : 67.2813 | hit correction : 1.07143
hit initial : 126.686 | hit correction : 1.07061
hit initial : 246.092 | hit correction : 1.07035
hit initial : 287.709 | hit correction : 1.06996
hit initial : 266.353 | hit correction : 1.06976
hit initial : 278.595 | hit correction : 1.0695
hit initial : 317.144 | hit correction : 1.06922
hit initial : 328.732 | hit correction : 1.06906
hit initial : 300.358 | hit correction : 1.06878
hit initial : 280.228 | hit correction : 1.06858
 *** Total Charge : 6600.95
 Looking at view: 0 | Number of hits in view: 33
hit initial : 43.6322 | hit correction : 1.09417
hit initial : 19.7681 | hit correction : 1.09417
hit initial : 154.465 | hit correction : 1.09431
hit initial : 320.044 | hit correction : 1.09388
hit initial : 180.033 | hit correction : 1.09344
hit initial : 47.0746 | hit correction : 1.07619
hit initial : 21.8685 | hit correction : 1.07618
hit initial : 66.5548 | hit correction : 1.07122
hit initial : 211.25 | hit correction : 1.07119
hit initial : 388.244 | hit correction : 1.07121
hit initial : 633.955 | hit correction : 1.07124
hit initial : 371.767 | hit correction : 1.07124
hit initial : 309.111 | hit correction : 1.07292
hit initial : 171.185 | hit correction : 1.07292
hit initial : 259.347 | hit correction : 1.07293
hit initial : 49.6227 | hit correction : 1.07142
hit initial : 138.265 | hit correction : 1.07142
hit initial : 180.942 | hit correction : 1.07058
hit initial : 263.87 | hit correction : 1.06985
hit initial : 58.4819 | hit correction : 1.06983
hit initial : 127.27 | hit correction : 1.06912
hit initial : 449.595 | hit correction : 1.06916
hit initial : 145.107 | hit correction : 1.06911
hit initial : 199.743 | hit correction : 1.06859
 *** Total Charge : 5169.19
 Looking at view: 1 | Number of hits in view: 24
hit initial : 30.9525 | hit correction : 1.07084
hit initial : 23.7136 | hit correction : 1.0852
hit initial : 66.4167 | hit correction : 1.09416
hit initial : 189.008 | hit correction : 1.09416
hit initial : 151.399 | hit correction : 1.09441
hit initial : 171.434 | hit correction : 1.0938
hit initial : 159.462 | hit correction : 1.09348
hit initial : 123.935 | hit correction : 1.09327
hit initial : 85.1211 | hit correction : 1.07619
hit initial : 35.1271 | hit correction : 1.07277
hit initial : 267.329 | hit correction : 1.0729
hit initial : 55.9268 | hit correction : 1.07309
hit initial : 439.32 | hit correction : 1.07115
hit initial : 470.546 | hit correction : 1.07119
hit initial : 534.775 | hit correction : 1.0713
hit initial : 180.934 | hit correction : 1.07128
hit initial : 154.858 | hit correction : 1.07123
hit initial : 178.337 | hit correction : 1.07297
hit initial : 164.859 | hit correction : 1.07292
hit initial : 149.711 | hit correction : 1.07288
hit initial : 25.7697 | hit correction : 1.07199
hit initial : 118.172 | hit correction : 1.07143
hit initial : 165.222 | hit correction : 1.07141
hit initial : 336.534 | hit correction : 1.07059
hit initial : 227.899 | hit correction : 1.07016
hit initial : 276.224 | hit correction : 1.06987
hit initial : 213.064 | hit correction : 1.06968
hit initial : 166.755 | hit correction : 1.06949
hit initial : 182.324 | hit correction : 1.06932
hit initial : 300.283 | hit correction : 1.06914
hit initial : 201.178 | hit correction : 1.06902
hit initial : 156.024 | hit correction : 1.06884
hit initial : 202.878 | hit correction : 1.06868
hit initial : 202.022 | hit correction : 1.06854
hit initial : 154.201 | hit correction : 1.06835
 *** Total Charge : 7046.11
 Looking at view: 2 | Number of hits in view: 35
13-Dec-2025 20:54:01 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.101739       0.1303       0.159272      0.129729      0.0175997       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000515847   0.000696466   0.00146424    0.000606846   0.000268275      10     
end_path:ana:AnalyzeEvents         0.100966      0.129378      0.158313      0.129035      0.0174027       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 9810.8 MB
  Peak resident set size usage (VmHWM): 1537.36 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 12:22:55 UTC       justIN version: 01.06.00