justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 267927.0@dunegpschedd01.fnal.gov

Jobsub ID267927.0@dunegpschedd01.fnal.gov
Workflow ID11173
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-13 08:50:26
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-12-13 17:07:08
From worker nodeHostnamedunegli-7890418-0-fnpc18002.fnal.gov
cpuinfoAMD EPYC 7551 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-13 08:51:56
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T230255Z_gen_001106_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time8h (29684s)
CPU time8h (29449s = 99%)
Max RSS bytes2801520640 (2671 MiB)
Outputting started2025-12-13 17:06:42
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11173/1/001/analysisLowE_20251213T085222Z.root
Finished2025-12-13 17:07:08
Saved logsjustin-logs:267927.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

05:09 UTC run: 5027 subRun: 0 event: 11060
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  13-Dec-2025 17:05:09 UTC run: 5027 subRun: 0 event: 11060
Trying to add data product: ShowerPCA. This element does not exist in the element holder
%MSG
Boundary wire vector sizes: 6571, 6561, 6601
minwire 0: 41
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 155, 654
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.12347, 
Output 1: 0.0542681, 0.0993712, 0.124556, 0.721805, 
Output 2: 0.0398013, 0.227075, 0.717437, 0.0156869, 
Output 3: 0.959325, 0.031813, 0.00339638, 0.00546611, 
Output 4: 0.336674, 0.541539, 0.11007, 0.0117173, 
Output 5: 0.372204, 0.520509, 0.0954906, 0.0117964, 
Output 6: 0.0198638, 0.0632743, 0.0839996, 0.832862, 

13-Dec-2025 17:05:14 UTC  Closed output file "matt_reco2.root"
13-Dec-2025 17:05:14 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/3e/95/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T230255Z_gen_001106_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2168.29       2933.62       3905.35       2851.07       536.225        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000927716   0.00480936     0.0196605     0.0016509    0.00662164       10     
reco:pandora:StandardPandora                            2165.66       2923.4        3841.19       2843.45       525.638        10     
reco:pandoraTrack:LArPandoraTrackCreation              0.0135393     0.0777677     0.158354      0.0728462     0.0354141       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.00241054     0.0102818     0.0504063    0.00326859     0.0140731       10     
reco:pandoracalo:Calorimetry                          0.000540999   0.00223015     0.0090543    0.000661024   0.00314968       10     
reco:pandorapid:Chi2ParticleID                        0.000139335   0.00100581    0.00464906    0.000170695   0.00166003       10     
reco:cvnmap:CVNMapper                                  0.0596835     0.101373      0.230752      0.0843975     0.0475683       10     
reco:cvneva:CVNEvaluator                                1.95202       3.27134       9.71343       2.6512        2.21098        10     
reco:energyrecnumu:EnergyReco                          0.0961296      6.24288       61.3167      0.115021       18.358         10     
reco:energyrecnue:EnergyReco                           0.0017082    0.00473494     0.0198864    0.00224299    0.00533563       10     
reco:energyrecnc:EnergyReco                           0.00952427     0.0139165     0.0234883     0.0133295    0.00395447       10     
[art]:TriggerResults:TriggerResultInserter            2.3677e-05    5.1418e-05    0.000215953   2.81815e-05   5.64212e-05      10     
end_path:out1:RootOutput                              1.0825e-05    2.40676e-05   0.000107348   1.30605e-05   2.81513e-05      10     
end_path:out1:RootOutput(write)                        0.371389      0.487668      0.713469      0.432669       0.12577        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21572.2 MB
  Peak resident set size usage (VmHWM): 2801.52 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 23.8467 | hit correction : 1.29833
hit initial : 120.392 | hit correction : 1.2984
hit initial : 38.5654 | hit correction : 1.29854
hit initial : 217.761 | hit correction : 1.29681
hit initial : 200.545 | hit correction : 1.29588
hit initial : 26.7179 | hit correction : 1.29671
hit initial : 32.7161 | hit correction : 1.29613
hit initial : 323.797 | hit correction : 1.29746
hit initial : 106.172 | hit correction : 1.2957
hit initial : 357.591 | hit correction : 1.29778
hit initial : 226.587 | hit correction : 1.29789
hit initial : 88.2246 | hit correction : 1.29786
 *** Total Charge : 2287.03
 Looking at view: 2 | Number of hits in view: 12
Begin processing the 10th record. run: 5027 subRun: 0 event: 11060 at 13-Dec-2025 17:06:35 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  6
This pfp has size:  50
Added hit list length: 50 | Max was set to: -3.40282e+38
This pfp has size:  7
This pfp has size:  4
True Neutron Energy: 939.571
True Electron Energy: 23.042
Completed MCParticle Loop
True Neutrino Energy: 26.3423
 Size of all hits: 50
hit initial : 15.8485 | hit correction : 1.11753
hit initial : 41.5279 | hit correction : 1.11756
hit initial : 20.3007 | hit correction : 1.11755
hit initial : 48.3425 | hit correction : 1.11731
hit initial : 40.6723 | hit correction : 1.1173
hit initial : 158.107 | hit correction : 1.11435
hit initial : 75.215 | hit correction : 1.11436
hit initial : 446.892 | hit correction : 1.11514
hit initial : 207.48 | hit correction : 1.11503
hit initial : 173.143 | hit correction : 1.11549
hit initial : 77.6106 | hit correction : 1.1149
hit initial : 237.064 | hit correction : 1.11229
hit initial : 212.472 | hit correction : 1.11238
hit initial : 53.6776 | hit correction : 1.11242
hit initial : 281.585 | hit correction : 1.11194
hit initial : 203.132 | hit correction : 1.11167
hit initial : 74.7742 | hit correction : 1.11163
 *** Total Charge : 2637.42
 Looking at view: 0 | Number of hits in view: 17
hit initial : 84.3767 | hit correction : 1.11743
hit initial : 30.7882 | hit correction : 1.11748
hit initial : 56.2656 | hit correction : 1.11738
hit initial : 144.165 | hit correction : 1.11434
hit initial : 55.4119 | hit correction : 1.11434
hit initial : 72.3743 | hit correction : 1.11434
hit initial : 250.408 | hit correction : 1.11516
hit initial : 281.747 | hit correction : 1.11494
hit initial : 99.8078 | hit correction : 1.11492
hit initial : 207.201 | hit correction : 1.11546
hit initial : 97.2656 | hit correction : 1.11552
hit initial : 136.074 | hit correction : 1.11241
hit initial : 189.87 | hit correction : 1.11235
hit initial : 236.196 | hit correction : 1.11216
hit initial : 294.645 | hit correction : 1.11187
hit initial : 189.128 | hit correction : 1.11167
 *** Total Charge : 2702
 Looking at view: 1 | Number of hits in view: 16
hit initial : 24.2654 | hit correction : 1.11753
hit initial : 55.4346 | hit correction : 1.11754
hit initial : 92.4716 | hit correction : 1.11732
hit initial : 65.4124 | hit correction : 1.1143
hit initial : 182.828 | hit correction : 1.11437
hit initial : 37.1344 | hit correction : 1.1144
hit initial : 54.0006 | hit correction : 1.11504
hit initial : 561.101 | hit correction : 1.115
hit initial : 145.443 | hit correction : 1.11541
hit initial : 185.065 | hit correction : 1.11558
hit initial : 24.5263 | hit correction : 1.11478
hit initial : 16.2003 | hit correction : 1.11243
hit initial : 377.373 | hit correction : 1.11237
hit initial : 337.225 | hit correction : 1.11213
hit initial : 154.635 | hit correction : 1.11182
hit initial : 142.079 | hit correction : 1.11164
hit initial : 106.352 | hit correction : 1.11151
 *** Total Charge : 2853.18
 Looking at view: 2 | Number of hits in view: 17
13-Dec-2025 17:06:40 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                          0.11313      0.389717      0.813293      0.388006       0.19198        10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000498256   0.000689345   0.00127602    0.000621494   0.000204062      10     
end_path:ana:AnalyzeEvents         0.112221      0.388786       0.8126       0.387205      0.191928        10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19476.5 MB
  Peak resident set size usage (VmHWM): 1540.71 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 12:06:20 UTC       justIN version: 01.06.00