justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 262313.0@dunegpschedd02.fnal.gov

Jobsub ID262313.0@dunegpschedd02.fnal.gov
Workflow ID11170
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-13 22:07:05
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-12-14 09:13:40
From worker nodeHostnamedunegli-8066212-0-fnpc17126.fnal.gov
cpuinfoIntel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-14 03:31:19
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T012950Z_gen_003044_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time5h (20528s)
CPU time5h (19844s = 96%)
Max RSS bytes2843238400 (2711 MiB)
Outputting started2025-12-14 09:13:27
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11170/1/001/analysisLowE_20251214T033119Z.root
Finished2025-12-14 09:13:40
Saved logsjustin-logs:262313.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

STATUS_CODE_NOT_INITIALIZED
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  14-Dec-2025 09:11:50 UTC run: 5027 subRun: 0 event: 30440
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  14-Dec-2025 09:11:50 UTC run: 5027 subRun: 0 event: 30440
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
Boundary wire vector sizes: 6817, 6827, 6802
minwire 0: 48
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.112694, 
Output 1: 0.0454631, 0.0606104, 0.188155, 0.705771, 
Output 2: 0.0341432, 0.122506, 0.823075, 0.0202754, 
Output 3: 0.955735, 0.0281336, 0.0069194, 0.00921215, 
Output 4: 0.572744, 0.346062, 0.0673168, 0.0138774, 
Output 5: 0.338559, 0.45658, 0.171734, 0.0331266, 
Output 6: 0.018851, 0.0620899, 0.0694937, 0.849566, 

14-Dec-2025 09:11:54 UTC  Closed output file "matt_reco2.root"
14-Dec-2025 09:11:54 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/fa/e6/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T012950Z_gen_003044_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1853.58       1981.53       2285.63       1896.15       150.78         10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000773029    0.0103599     0.0549504    0.000991315    0.0169306       10     
reco:pandora:StandardPandora                            1852.09       1979.38       2283.48       1892.56       150.778        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000387175    0.0429569     0.0818179     0.0543372     0.0324413       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000348586   0.00353795     0.0127618    0.00209822    0.00371354       10     
reco:pandoracalo:Calorimetry                          0.00023747    0.00644128     0.0560111    0.000596543    0.016547        10     
reco:pandorapid:Chi2ParticleID                        6.1635e-05    0.000260203   0.00135821    0.000140349   0.000368568      10     
reco:cvnmap:CVNMapper                                  0.0841844     0.125684      0.153172      0.120776      0.0196933       10     
reco:cvneva:CVNEvaluator                               0.909388       1.30215       3.69506      0.981675      0.811638        10     
reco:energyrecnumu:EnergyReco                          0.0849174     0.105772      0.154441      0.0969797     0.0209742       10     
reco:energyrecnue:EnergyReco                          0.00166834    0.00369062     0.0107186    0.00223124     0.0033322       10     
reco:energyrecnc:EnergyReco                           0.00848531     0.0110015     0.0143566    0.00995053    0.00204016       10     
[art]:TriggerResults:TriggerResultInserter            1.6162e-05    2.50218e-05   6.7719e-05    2.0151e-05    1.45095e-05      10     
end_path:out1:RootOutput                               7.308e-06    9.5159e-06    2.1539e-05     7.946e-06    4.14525e-06      10     
end_path:out1:RootOutput(write)                        0.353547      0.539861       1.42436      0.377845      0.312237        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 14056.6 MB
  Peak resident set size usage (VmHWM): 2843.24 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 256.215 | hit correction : 1.03454
hit initial : 289.762 | hit correction : 1.03478
hit initial : 265.966 | hit correction : 1.03488
hit initial : 145.328 | hit correction : 1.03494
 *** Total Charge : 3337.96
 Looking at view: 0 | Number of hits in view: 16
hit initial : 211.194 | hit correction : 1.02373
hit initial : 96.1869 | hit correction : 1.02373
hit initial : 17.62 | hit correction : 1.02054
hit initial : 102.383 | hit correction : 1.03389
hit initial : 102.871 | hit correction : 1.03463
hit initial : 244.557 | hit correction : 1.03393
hit initial : 201.571 | hit correction : 1.03462
hit initial : 303.119 | hit correction : 1.03402
hit initial : 139.056 | hit correction : 1.03464
hit initial : 471.812 | hit correction : 1.03412
hit initial : 161.202 | hit correction : 1.03462
hit initial : 281.77 | hit correction : 1.03424
hit initial : 284.971 | hit correction : 1.03473
hit initial : 419.227 | hit correction : 1.03483
hit initial : 177.839 | hit correction : 1.03491
 *** Total Charge : 3322.5
 Looking at view: 1 | Number of hits in view: 15
hit initial : 17.004 | hit correction : 1.03557
hit initial : 257.195 | hit correction : 1.02374
hit initial : 133.575 | hit correction : 1.02369
hit initial : 31.7161 | hit correction : 1.02055
hit initial : 39.1912 | hit correction : 1.03462
hit initial : 645.906 | hit correction : 1.03462
hit initial : 64.9298 | hit correction : 1.03398
hit initial : 1416.18 | hit correction : 1.03402
hit initial : 431.175 | hit correction : 1.03442
hit initial : 368.417 | hit correction : 1.03472
hit initial : 452.502 | hit correction : 1.03489
hit initial : 37.57 | hit correction : 1.035
 *** Total Charge : 4024.74
 Looking at view: 2 | Number of hits in view: 12
Begin processing the 10th record. run: 5027 subRun: 0 event: 30440 at 14-Dec-2025 09:13:20 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  5
This pfp has size:  27
Added hit list length: 27 | Max was set to: -3.40282e+38
This pfp has size:  8
This pfp has size:  4
True Neutron Energy: 939.569
True Electron Energy: 13.979
Completed MCParticle Loop
True Neutrino Energy: 17.7067
 Size of all hits: 27
hit initial : 104.915 | hit correction : 1.44722
hit initial : 358.676 | hit correction : 1.44724
hit initial : 362.455 | hit correction : 1.44721
hit initial : 131.055 | hit correction : 1.4472
hit initial : 80.1863 | hit correction : 1.44864
hit initial : 195.875 | hit correction : 1.44851
hit initial : 65.3793 | hit correction : 1.44865
 *** Total Charge : 1879.74
 Looking at view: 0 | Number of hits in view: 7
hit initial : 33.399 | hit correction : 1.44721
hit initial : 168.316 | hit correction : 1.4472
hit initial : 274.706 | hit correction : 1.44718
hit initial : 223.697 | hit correction : 1.44719
hit initial : 146.041 | hit correction : 1.44724
hit initial : 95.0996 | hit correction : 1.44802
hit initial : 59.2232 | hit correction : 1.44879
hit initial : 114.926 | hit correction : 1.44765
hit initial : 119.373 | hit correction : 1.44873
hit initial : 47.753 | hit correction : 1.44863
 *** Total Charge : 1856.56
 Looking at view: 1 | Number of hits in view: 10
hit initial : 134.592 | hit correction : 1.4472
hit initial : 90.1 | hit correction : 1.44721
hit initial : 78.9536 | hit correction : 1.44717
hit initial : 99.7251 | hit correction : 1.44716
hit initial : 142.005 | hit correction : 1.4472
hit initial : 248.695 | hit correction : 1.44717
hit initial : 166.34 | hit correction : 1.44744
hit initial : 74.3962 | hit correction : 1.44801
hit initial : 48.2913 | hit correction : 1.44874
hit initial : 194.428 | hit correction : 1.44861
 *** Total Charge : 1849.27
 Looking at view: 2 | Number of hits in view: 10
14-Dec-2025 09:13:26 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.105308      0.195006      0.421111      0.163235      0.083268        10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000527278   0.000748564   0.00130477    0.000701749   0.000197442      10     
end_path:ana:AnalyzeEvents         0.104664      0.194039      0.420277      0.162349      0.0831914       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 11959.1 MB
  Peak resident set size usage (VmHWM): 1536.53 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 12:09:27 UTC       justIN version: 01.06.00