justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 249102.64@dunegpschedd02.fnal.gov

Jobsub ID249102.64@dunegpschedd02.fnal.gov
Workflow ID10433
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-11-21 08:44:12
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-11-21 10:56:33
From worker nodeHostnamedunegli-7833571-0-fnpc22019.fnal.gov
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-21 08:45:23
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T193728Z_gen_000846_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time2h (7859s)
CPU time2h (7788s = 99%)
Max RSS bytes2917928960 (2782 MiB)
Outputting started2025-11-21 10:56:23
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/10433/1/001/analysisLowE_20251121T084524Z.root
Finished2025-11-21 10:56:33
Saved logsjustin-logs:249102.64-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

, 
Output 1: 0.0933934, 0.102974, 0.216626, 0.587007, 
Output 2: 0.0279227, 0.114692, 0.840428, 0.0169575, 
Output 3: 0.956315, 0.0278272, 0.00570255, 0.0101556, 
Output 4: 0.568241, 0.372965, 0.0504114, 0.00838298, 
Output 5: 0.447855, 0.348686, 0.15505, 0.0484088, 
Output 6: 0.0346684, 0.0828618, 0.0781892, 0.804281, 

Begin processing the 10th record. run: 3799 subRun: 0 event: 8460 at 21-Nov-2025 10:44:37 UTC
Error: A CaloHitList is empty
Error: A CaloHitList is empty
Boundary wire vector sizes: 6895, 6982, 6938
minwire 0: 15
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 1473, 1972
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0713181, 
Output 1: 0.0588071, 0.0428862, 0.0739608, 0.824346, 
Output 2: 0.143831, 0.312621, 0.513989, 0.0295587, 
Output 3: 0.956713, 0.036897, 0.00230517, 0.00408453, 
Output 4: 0.462926, 0.468648, 0.0611441, 0.00728169, 
Output 5: 0.800683, 0.190731, 0.007383, 0.00120338, 
Output 6: 0.0168607, 0.0675025, 0.0811441, 0.834493, 

21-Nov-2025 10:55:51 UTC  Closed output file "matt_reco2.root"
21-Nov-2025 10:55:51 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/16/53/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T193728Z_gen_000846_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              671.576       776.603       876.76         784.3        52.1381        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00172891    0.00673224     0.0141744    0.00713241    0.00324006       10     
reco:pandora:StandardPandora                            670.416       775.34        875.715       783.277       52.2283        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000414726    0.0146456      0.10395     0.00444073     0.0299144       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000295279   0.00247649    0.00901373    0.00134782    0.00246864       10     
reco:pandoracalo:Calorimetry                          0.000189309    0.0150076     0.0413886     0.0119202     0.015084        10     
reco:pandorapid:Chi2ParticleID                        5.6517e-05    0.000291988   0.00101229    8.3348e-05    0.000359304      10     
reco:cvnmap:CVNMapper                                  0.0336658     0.0450125     0.0539683     0.0469701     0.0066239       10     
reco:cvneva:CVNEvaluator                               0.688371      0.894701       2.42258      0.708684      0.510886        10     
reco:energyrecnumu:EnergyReco                         0.00525734     0.0459882     0.0661037     0.0573037     0.0220509       10     
reco:energyrecnue:EnergyReco                          0.000822708   0.00374406    0.00555742    0.00475066    0.00189949       10     
reco:energyrecnc:EnergyReco                           0.00445813     0.0205478     0.0597352    0.00499779     0.0240648       10     
[art]:TriggerResults:TriggerResultInserter            1.3135e-05    2.23584e-05   7.7206e-05    1.5705e-05    1.85683e-05      10     
end_path:out1:RootOutput                               8.726e-06    1.30365e-05   2.9987e-05    9.9135e-06    6.33855e-06      10     
end_path:out1:RootOutput(write)                        0.198507      0.213627      0.237289      0.209766      0.0139183       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21677.1 MB
  Peak resident set size usage (VmHWM): 2917.93 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.713
True Electron Energy: 12.5055
Completed MCParticle Loop
True Neutrino Energy: 21.455
 Size of all hits: 0
Begin processing the 6th record. run: 3799 subRun: 0 event: 8456 at 21-Nov-2025 10:56:13 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.566
True Electron Energy: 7.9525
Completed MCParticle Loop
True Neutrino Energy: 20.5445
 Size of all hits: 0
Begin processing the 7th record. run: 3799 subRun: 0 event: 8457 at 21-Nov-2025 10:56:15 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Electron Energy: 11.354
Completed MCParticle Loop
True Neutrino Energy: 16.865
 Size of all hits: 0
Begin processing the 8th record. run: 3799 subRun: 0 event: 8458 at 21-Nov-2025 10:56:17 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
This pfp has size:  29
Added hit list length: 29 | Max was set to: -3.40282e+38
True Electron Energy: 13.6066
Completed MCParticle Loop
True Neutrino Energy: 30.8515
 Size of all hits: 29
hit initial : 27.5524 | hit correction : 1.08936
hit initial : 127.262 | hit correction : 1.0918
hit initial : 35.7935 | hit correction : 1.09184
hit initial : 96.6569 | hit correction : 1.09099
hit initial : 456.193 | hit correction : 1.09099
hit initial : 276.978 | hit correction : 1.09079
hit initial : 152.178 | hit correction : 1.09017
hit initial : 266.771 | hit correction : 1.08997
hit initial : 232.626 | hit correction : 1.08959
hit initial : 120.855 | hit correction : 1.08946
 *** Total Charge : 1955.12
 Looking at view: 0 | Number of hits in view: 10
hit initial : 66.21 | hit correction : 1.09186
hit initial : 40.6428 | hit correction : 1.09143
hit initial : 69.6195 | hit correction : 1.09167
hit initial : 57.3707 | hit correction : 1.08954
hit initial : 100.05 | hit correction : 1.08949
hit initial : 151.259 | hit correction : 1.08922
hit initial : 89.2418 | hit correction : 1.08922
hit initial : 149.722 | hit correction : 1.0908
hit initial : 504.46 | hit correction : 1.09078
hit initial : 42.5427 | hit correction : 1.09081
hit initial : 101.481 | hit correction : 1.09006
hit initial : 276.168 | hit correction : 1.08956
 *** Total Charge : 1797.62
 Looking at view: 1 | Number of hits in view: 12
hit initial : 33.7062 | hit correction : 1.08929
hit initial : 282.459 | hit correction : 1.09167
hit initial : 587.245 | hit correction : 1.09079
hit initial : 138.164 | hit correction : 1.09077
hit initial : 326.326 | hit correction : 1.09006
hit initial : 294.271 | hit correction : 1.08955
hit initial : 102.49 | hit correction : 1.08939
 *** Total Charge : 1924.32
 Looking at view: 2 | Number of hits in view: 7
Begin processing the 9th record. run: 3799 subRun: 0 event: 8459 at 21-Nov-2025 10:56:19 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 940.884
True Electron Energy: 17.0571
Completed MCParticle Loop
True Neutrino Energy: 48.7321
 Size of all hits: 0
Begin processing the 10th record. run: 3799 subRun: 0 event: 8460 at 21-Nov-2025 10:56:21 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.579
True Electron Energy: 30.303
Completed MCParticle Loop
True Neutrino Energy: 39.1512
 Size of all hits: 0
21-Nov-2025 10:56:23 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0525999     0.059505      0.081908      0.0544971    0.00890814       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000443139   0.000601764   0.00110897    0.00052819    0.000187775      10     
end_path:ana:AnalyzeEvents         0.0518908     0.0586672     0.0800301     0.0537721    0.00862922       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19470.1 MB
  Peak resident set size usage (VmHWM): 1530.14 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-09 01:58:44 UTC       justIN version: 01.06.00