justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 262198.0@dunegpschedd02.fnal.gov

Jobsub ID262198.0@dunegpschedd02.fnal.gov
Workflow ID11175
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-13 12:34:37
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-12-13 21:10:12
From worker nodeHostnamedunegli-7890507-0-fnpc17010.fnal.gov
cpuinfoIntel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-13 12:35:31
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T030029Z_gen_004572_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time8h (30870s)
CPU time8h (30157s = 97%)
Max RSS bytes2796052480 (2666 MiB)
Outputting started2025-12-13 21:10:01
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11175/1/001/analysisLowE_20251213T123532Z.root
Finished2025-12-13 21:10:12
Saved logsjustin-logs:262198.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

PandoraModularShowerCreation:pandoraShower@BeginModule  13-Dec-2025 21:08:10 UTC run: 5027 subRun: 0 event: 45720
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  13-Dec-2025 21:08:10 UTC run: 5027 subRun: 0 event: 45720
Trying to add data product: ShowerPCA. This element does not exist in the element holder
%MSG
Boundary wire vector sizes: 6741, 6704, 6620
minwire 0: 34
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.143798, 
Output 1: 0.114887, 0.0909245, 0.143861, 0.650328, 
Output 2: 0.0924074, 0.218622, 0.65472, 0.034251, 
Output 3: 0.970641, 0.018523, 0.00242003, 0.00841583, 
Output 4: 0.459607, 0.487395, 0.0470097, 0.00598859, 
Output 5: 0.402046, 0.483831, 0.0923418, 0.0217809, 
Output 6: 0.050774, 0.108372, 0.105824, 0.735029, 

13-Dec-2025 21:08:20 UTC  Closed output file "matt_reco2.root"
13-Dec-2025 21:08:20 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/ee/68/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T030029Z_gen_004572_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2369.62       3014.03       4104.73       2894.81       532.98         10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00109667    0.00681709     0.0414727    0.00128097     0.012404        10     
reco:pandora:StandardPandora                            2367.14       3010.91       4101.86       2892.25       533.33         10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000488783    0.0653398     0.115462      0.0953184     0.0511914       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000459388    0.0113269     0.0376818    0.00785106     0.011994        10     
reco:pandoracalo:Calorimetry                          0.000321134   0.00187792    0.00724716    0.00116636    0.00199443       10     
reco:pandorapid:Chi2ParticleID                        0.000103891   0.000602692   0.00261592    0.00020099    0.000863999      10     
reco:cvnmap:CVNMapper                                  0.0741157     0.108738      0.170798      0.103861      0.0300458       10     
reco:cvneva:CVNEvaluator                                1.41025       2.22545       5.45053       1.88052       1.12569        10     
reco:energyrecnumu:EnergyReco                          0.104471      0.124734      0.149433      0.122893      0.0143816       10     
reco:energyrecnue:EnergyReco                          0.00223434    0.00680791     0.0196295     0.0030829    0.00597084       10     
reco:energyrecnc:EnergyReco                            0.0117898     0.0142251     0.0193217     0.0137308    0.00209989       10     
[art]:TriggerResults:TriggerResultInserter            2.8165e-05    4.38536e-05   0.000139493   3.2928e-05    3.21502e-05      10     
end_path:out1:RootOutput                               9.539e-06    1.55985e-05   4.5441e-05    1.14665e-05   1.04186e-05      10     
end_path:out1:RootOutput(write)                        0.467355      0.558612      0.660535      0.579192      0.0656835       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 11908.6 MB
  Peak resident set size usage (VmHWM): 2796.05 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
 Looking at view: 2 | Number of hits in view: 9
Begin processing the 10th record. run: 5027 subRun: 0 event: 45720 at 13-Dec-2025 21:09:49 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  7
This pfp has size:  63
Added hit list length: 63 | Max was set to: -3.40282e+38
This pfp has size:  11
This pfp has size:  8
True Neutron Energy: 939.567
True Electron Energy: 29.0373
Completed MCParticle Loop
True Neutrino Energy: 37.5571
 Size of all hits: 63
hit initial : 20.753 | hit correction : 1.29016
hit initial : 50.5118 | hit correction : 1.29008
hit initial : 33.1218 | hit correction : 1.29013
hit initial : 28.7762 | hit correction : 1.28127
hit initial : 14.0694 | hit correction : 1.28131
hit initial : 104.633 | hit correction : 1.27931
hit initial : 171.813 | hit correction : 1.27921
hit initial : 58.6081 | hit correction : 1.27903
hit initial : 16.5523 | hit correction : 1.27674
hit initial : 51.9395 | hit correction : 1.27671
hit initial : 20.5286 | hit correction : 1.27671
hit initial : 197.733 | hit correction : 1.27806
hit initial : 148.356 | hit correction : 1.27808
hit initial : 131.504 | hit correction : 1.26877
hit initial : 117.29 | hit correction : 1.26869
hit initial : 100.609 | hit correction : 1.26794
hit initial : 43.4607 | hit correction : 1.26694
hit initial : 714.361 | hit correction : 1.26661
hit initial : 201.503 | hit correction : 1.26559
hit initial : 65.3377 | hit correction : 1.2655
hit initial : 77.1317 | hit correction : 1.26465
hit initial : 39.0396 | hit correction : 1.2646
 *** Total Charge : 3061.87
 Looking at view: 0 | Number of hits in view: 22
hit initial : 35.8162 | hit correction : 1.2901
hit initial : 33.7041 | hit correction : 1.29012
hit initial : 34.2665 | hit correction : 1.28125
hit initial : 28.0516 | hit correction : 1.28125
hit initial : 53.9339 | hit correction : 1.27938
hit initial : 97.8349 | hit correction : 1.27926
hit initial : 66.6778 | hit correction : 1.27922
hit initial : 15.3126 | hit correction : 1.27671
hit initial : 41.1632 | hit correction : 1.27672
hit initial : 18.1025 | hit correction : 1.27673
hit initial : 88.0061 | hit correction : 1.27831
hit initial : 89.0481 | hit correction : 1.27805
hit initial : 110.493 | hit correction : 1.2779
hit initial : 16.5968 | hit correction : 1.27772
hit initial : 153.063 | hit correction : 1.26873
hit initial : 155.404 | hit correction : 1.2686
hit initial : 23.4873 | hit correction : 1.26756
hit initial : 52.8973 | hit correction : 1.2672
hit initial : 53.0738 | hit correction : 1.26688
hit initial : 679.374 | hit correction : 1.26661
hit initial : 177.996 | hit correction : 1.26562
hit initial : 130.459 | hit correction : 1.26564
hit initial : 2.0477 | hit correction : 1.26496
hit initial : 87.1678 | hit correction : 1.26465
hit initial : 10.5946 | hit correction : 1.26458
 *** Total Charge : 2865.41
 Looking at view: 1 | Number of hits in view: 25
hit initial : 62.0138 | hit correction : 1.29011
hit initial : 29.4379 | hit correction : 1.29013
hit initial : 24.7448 | hit correction : 1.28123
hit initial : 23.9643 | hit correction : 1.28123
hit initial : 172.52 | hit correction : 1.27926
hit initial : 185.474 | hit correction : 1.27917
hit initial : 40.9683 | hit correction : 1.27669
hit initial : 53.8428 | hit correction : 1.27668
hit initial : 248.502 | hit correction : 1.27824
hit initial : 119.966 | hit correction : 1.27782
hit initial : 32.7517 | hit correction : 1.26884
hit initial : 139.924 | hit correction : 1.26845
hit initial : 350.305 | hit correction : 1.2668
hit initial : 1112.6 | hit correction : 1.26649
hit initial : 54.4622 | hit correction : 1.26553
hit initial : 13.1687 | hit correction : 1.26477
 *** Total Charge : 3387.81
 Looking at view: 2 | Number of hits in view: 16
13-Dec-2025 21:09:55 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.140826      0.192367      0.306537      0.166967      0.0545036       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000679665    0.0090095     0.0825778    0.000794301    0.0245237       10     
end_path:ana:AnalyzeEvents         0.139779      0.183095       0.30431      0.166051      0.0504119       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 9816.05 MB
  Peak resident set size usage (VmHWM): 1543.43 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 12:23:21 UTC       justIN version: 01.06.00