justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 267119.0@dunegpschedd01.fnal.gov

Jobsub ID267119.0@dunegpschedd01.fnal.gov
Workflow ID11114
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-11 23:46:43
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-12-12 01:38:42
From worker nodeHostnamedunegli-7878631-0-fnpc23006.fnal.gov
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-11 23:48:09
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T022935Z_gen_004033_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time1h (6623s)
CPU time1h (6579s = 99%)
Max RSS bytes2920284160 (2785 MiB)
Outputting started2025-12-12 01:38:32
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11114/1/001/analysisLowE_20251211T234810Z.root
Finished2025-12-12 01:38:42
Saved logsjustin-logs:267119.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

1905, 
Output 1: 0.0610286, 0.0657612, 0.176238, 0.696973, 
Output 2: 0.0790813, 0.16579, 0.72526, 0.0298683, 
Output 3: 0.963639, 0.0289547, 0.00285373, 0.00455262, 
Output 4: 0.416696, 0.525389, 0.0521235, 0.00579152, 
Output 5: 0.435022, 0.432011, 0.1068, 0.0261675, 
Output 6: 0.0364982, 0.0884331, 0.0861165, 0.788952, 

Begin processing the 10th record. run: 5027 subRun: 0 event: 40330 at 12-Dec-2025 01:26:58 UTC
Error: A CaloHitList is empty
Error: A CaloHitList is empty
Boundary wire vector sizes: 6800, 6797, 6688
minwire 0: 28
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0875122, 
Output 1: 0.145402, 0.0564663, 0.170034, 0.628097, 
Output 2: 0.0620149, 0.11327, 0.811302, 0.0134135, 
Output 3: 0.941935, 0.0527481, 0.0025328, 0.00278439, 
Output 4: 0.492438, 0.449208, 0.0495349, 0.00881861, 
Output 5: 0.277742, 0.542998, 0.145081, 0.0341788, 
Output 6: 0.0991032, 0.218564, 0.136508, 0.545825, 

12-Dec-2025 01:38:04 UTC  Closed output file "matt_reco2.root"
12-Dec-2025 01:38:04 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/7e/4d/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T022935Z_gen_004033_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              645.64        653.621       669.028       651.962       7.11143        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00233804     0.0059863     0.0151462    0.00357366    0.00449988       10     
reco:pandora:StandardPandora                            644.483       652.464       668.035       650.957       7.25843        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.00123534    0.00803295     0.0502681    0.00370382      0.01417        10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.00131527    0.00450632     0.0172576    0.00329372    0.00432797       10     
reco:pandoracalo:Calorimetry                          0.000213639   0.00255845     0.0163437    0.000259436   0.00480247       10     
reco:pandorapid:Chi2ParticleID                        5.4344e-05    0.000196723   0.00114794    6.1703e-05    0.000321921      10     
reco:cvnmap:CVNMapper                                  0.0212912     0.027996      0.0511703     0.0259245    0.00799367       10     
reco:cvneva:CVNEvaluator                               0.678921      0.813308       1.95907      0.685187      0.381963        10     
reco:energyrecnumu:EnergyReco                         0.00279989     0.0345093     0.0858606    0.00572706     0.0374721       10     
reco:energyrecnue:EnergyReco                          0.000745346   0.00168601     0.0047473    0.000786405   0.00154106       10     
reco:energyrecnc:EnergyReco                           0.00459898     0.0500229     0.104818      0.0632168     0.0395091       10     
[art]:TriggerResults:TriggerResultInserter             7.594e-06    1.34026e-05   3.5848e-05    1.02045e-05   7.83689e-06      10     
end_path:out1:RootOutput                               5.18e-06     1.0948e-05    1.6983e-05    9.8535e-06    4.23013e-06      10     
end_path:out1:RootOutput(write)                        0.198114       0.20722      0.221287      0.200552     0.00945935       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21683.1 MB
  Peak resident set size usage (VmHWM): 2920.28 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 77.9408 | hit correction : 1.17879
hit initial : 405.098 | hit correction : 1.17917
hit initial : 396.148 | hit correction : 1.17961
 *** Total Charge : 1699.35
 Looking at view: 0 | Number of hits in view: 5
hit initial : 67.7177 | hit correction : 1.17869
hit initial : 174.607 | hit correction : 1.17878
hit initial : 196.519 | hit correction : 1.17895
hit initial : 64.7326 | hit correction : 1.18007
hit initial : 187.746 | hit correction : 1.17913
hit initial : 190.788 | hit correction : 1.17929
hit initial : 194.595 | hit correction : 1.17956
hit initial : 209.049 | hit correction : 1.17987
 *** Total Charge : 1516.28
 Looking at view: 1 | Number of hits in view: 8
hit initial : 36.5602 | hit correction : 1.1786
hit initial : 220.189 | hit correction : 1.17875
hit initial : 410.15 | hit correction : 1.17993
hit initial : 147.434 | hit correction : 1.18035
hit initial : 234.664 | hit correction : 1.17898
hit initial : 213.194 | hit correction : 1.17917
hit initial : 276.027 | hit correction : 1.17945
 *** Total Charge : 1814.23
 Looking at view: 2 | Number of hits in view: 7
Begin processing the 9th record. run: 5027 subRun: 0 event: 40329 at 12-Dec-2025 01:38:28 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
This pfp has size:  35
Added hit list length: 35 | Max was set to: -3.40282e+38
True Electron Energy: 27.3687
Completed MCParticle Loop
True Neutrino Energy: 31.1225
 Size of all hits: 35
hit initial : 32.9246 | hit correction : 1.25809
hit initial : 33.574 | hit correction : 1.25807
hit initial : 75.2151 | hit correction : 1.26092
hit initial : 54.0678 | hit correction : 1.26043
hit initial : 157.94 | hit correction : 1.26047
hit initial : 276.601 | hit correction : 1.26048
hit initial : 308.199 | hit correction : 1.26057
hit initial : 256.46 | hit correction : 1.26084
hit initial : 289.91 | hit correction : 1.2611
hit initial : 438.244 | hit correction : 1.26107
hit initial : 322.921 | hit correction : 1.26099
 *** Total Charge : 2831.7
 Looking at view: 0 | Number of hits in view: 11
hit initial : 127.203 | hit correction : 1.26084
hit initial : 580.375 | hit correction : 1.26086
hit initial : 95.5744 | hit correction : 1.26111
hit initial : 82.8776 | hit correction : 1.26051
hit initial : 238.083 | hit correction : 1.26109
hit initial : 75.3627 | hit correction : 1.25691
hit initial : 241.412 | hit correction : 1.25689
hit initial : 184.706 | hit correction : 1.25686
hit initial : 36.1579 | hit correction : 1.2609
hit initial : 226.526 | hit correction : 1.26053
 *** Total Charge : 2378.83
 Looking at view: 1 | Number of hits in view: 10
hit initial : 38.477 | hit correction : 1.26079
hit initial : 23.1079 | hit correction : 1.26029
hit initial : 82.7908 | hit correction : 1.26044
hit initial : 141.943 | hit correction : 1.26047
hit initial : 195.246 | hit correction : 1.26047
hit initial : 291.175 | hit correction : 1.26052
hit initial : 183.127 | hit correction : 1.2607
hit initial : 159.76 | hit correction : 1.26093
hit initial : 120.186 | hit correction : 1.26112
hit initial : 228.022 | hit correction : 1.26115
hit initial : 197.926 | hit correction : 1.26084
hit initial : 165.849 | hit correction : 1.26095
hit initial : 240.858 | hit correction : 1.26115
hit initial : 127.922 | hit correction : 1.2611
 *** Total Charge : 2769.25
 Looking at view: 2 | Number of hits in view: 14
Begin processing the 10th record. run: 5027 subRun: 0 event: 40330 at 12-Dec-2025 01:38:30 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.567
True Electron Energy: 10.9258
Completed MCParticle Loop
True Neutrino Energy: 14.6554
 Size of all hits: 0
12-Dec-2025 01:38:32 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0469945     0.0613244     0.0829529     0.0589125    0.00875321       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000291027   0.000373293   0.000592905   0.000358712   7.98603e-05      10     
end_path:ana:AnalyzeEvents         0.0465529     0.0608327     0.0820187     0.0584535     0.0086303       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19477.6 MB
  Peak resident set size usage (VmHWM): 1542.48 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 14:58:56 UTC       justIN version: 01.06.00