justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 267078.1@dunegpschedd01.fnal.gov

Jobsub ID267078.1@dunegpschedd01.fnal.gov
Workflow ID11112
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-11 22:30:49
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-12-12 02:33:26
From worker nodeHostnamedunegli-7878614-0-fnpc19122.fnal.gov
cpuinfoAMD EPYC 7502 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-11 22:31:28
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T160715Z_gen_000266_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time4h (14505s)
CPU time3h (14395s = 99%)
Max RSS bytes2764058624 (2636 MiB)
Outputting started2025-12-12 02:33:14
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11112/1/001/analysisLowE_20251211T223129Z.root
Finished2025-12-12 02:33:26
Saved logsjustin-logs:267078.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

: 
Output 0: 0.0999674, 
Output 1: 0.0469696, 0.0602255, 0.138762, 0.754043, 
Output 2: 0.0745547, 0.304381, 0.578195, 0.0428693, 
Output 3: 0.90393, 0.0728696, 0.0123822, 0.0108184, 
Output 4: 0.242401, 0.63138, 0.117698, 0.00852083, 
Output 5: 0.822656, 0.152741, 0.0200906, 0.00451314, 
Output 6: 0.0161695, 0.0547993, 0.0739569, 0.855074, 

Begin processing the 10th record. run: 5026 subRun: 0 event: 2660 at 12-Dec-2025 02:08:13 UTC
Error: A CaloHitList is empty
Error: A CaloHitList is empty
Boundary wire vector sizes: 6697, 6587, 6573
minwire 0: 8
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.159918, 
Output 1: 0.115373, 0.102882, 0.146402, 0.635342, 
Output 2: 0.0301446, 0.179861, 0.752272, 0.0377223, 
Output 3: 0.969804, 0.0125842, 0.00417119, 0.0134408, 
Output 4: 0.562425, 0.387818, 0.0447993, 0.004958, 
Output 5: 0.236813, 0.289833, 0.341663, 0.131691, 
Output 6: 0.0609002, 0.107295, 0.101778, 0.730027, 

12-Dec-2025 02:32:34 UTC  Closed output file "matt_reco2.root"
12-Dec-2025 02:32:34 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/53/e2/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T160715Z_gen_000266_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1377.18       1438.25       1474.24       1441.11       30.9214        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00106381    0.00618164     0.018648     0.00373434    0.00591417       10     
reco:pandora:StandardPandora                            1374.31       1436.87       1473.01       1439.85       31.2496        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000982816    0.0179446     0.0665932    0.00820433     0.0193406       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000298062   0.00353134    0.00768048    0.00294838    0.00243703       10     
reco:pandoracalo:Calorimetry                          0.000143101   0.00703425     0.0229425    0.000426453   0.00986273       10     
reco:pandorapid:Chi2ParticleID                         4.851e-05    0.000188192   0.000887547   8.3085e-05    0.000243862      10     
reco:cvnmap:CVNMapper                                  0.0232687     0.0264224     0.0351406     0.0258492    0.00325489       10     
reco:cvneva:CVNEvaluator                               0.780694      0.941738       2.26905      0.797934      0.442496        10     
reco:energyrecnumu:EnergyReco                         0.00471489     0.0456532     0.0998265     0.0484243     0.0367732       10     
reco:energyrecnue:EnergyReco                          0.000829065   0.00325877    0.00700764    0.00116213    0.00280918       10     
reco:energyrecnc:EnergyReco                           0.00643361     0.0511063     0.144081      0.0375064     0.0502295       10     
[art]:TriggerResults:TriggerResultInserter             1.281e-05    2.09472e-05   7.1941e-05     1.544e-05    1.70524e-05      10     
end_path:out1:RootOutput                               5.04e-06     1.0422e-05     3.12e-05      8.66e-06     7.01192e-06      10     
end_path:out1:RootOutput(write)                        0.267036      0.275907      0.311291      0.271717      0.0123744       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21683.7 MB
  Peak resident set size usage (VmHWM): 2764.06 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 238.615 | hit correction : 1.05708
hit initial : 101.074 | hit correction : 1.05643
hit initial : 252.114 | hit correction : 1.05628
hit initial : 266.756 | hit correction : 1.05613
hit initial : 382.959 | hit correction : 1.05614
hit initial : 556.047 | hit correction : 1.05611
hit initial : 646.181 | hit correction : 1.05606
hit initial : 593.117 | hit correction : 1.05603
hit initial : 369.992 | hit correction : 1.05594
hit initial : 319.184 | hit correction : 1.05585
hit initial : 303.999 | hit correction : 1.05574
hit initial : 261.823 | hit correction : 1.05558
hit initial : 280.496 | hit correction : 1.05544
 *** Total Charge : 7065.39
 Looking at view: 1 | Number of hits in view: 28
Begin processing the 8th record. run: 5026 subRun: 0 event: 2658 at 12-Dec-2025 02:33:05 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Electron Energy: 0.942217
Completed MCParticle Loop
True Neutrino Energy: 4.22574
 Size of all hits: 0
Begin processing the 9th record. run: 5026 subRun: 0 event: 2659 at 12-Dec-2025 02:33:08 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  3
This pfp has size:  35
Added hit list length: 35 | Max was set to: -3.40282e+38
This pfp has size:  17
True Neutron Energy: 939.699
True Electron Energy: 31.6265
Completed MCParticle Loop
True Neutrino Energy: 40.6053
 Size of all hits: 35
hit initial : 222.77 | hit correction : 1.45865
hit initial : 232.899 | hit correction : 1.45879
hit initial : 114.784 | hit correction : 1.45888
hit initial : 20.8166 | hit correction : 1.4589
hit initial : 194.143 | hit correction : 1.45852
hit initial : 230.337 | hit correction : 1.4585
hit initial : 206.838 | hit correction : 1.45848
 *** Total Charge : 1783.3
 Looking at view: 0 | Number of hits in view: 7
hit initial : 53.989 | hit correction : 1.45584
hit initial : 75.7235 | hit correction : 1.45585
hit initial : 271.448 | hit correction : 1.45834
hit initial : 25.7271 | hit correction : 1.45732
hit initial : 209.082 | hit correction : 1.45865
hit initial : 637.834 | hit correction : 1.4586
hit initial : 631.789 | hit correction : 1.45853
hit initial : 103.986 | hit correction : 1.45855
hit initial : 228.702 | hit correction : 1.45847
hit initial : 378.621 | hit correction : 1.4584
 *** Total Charge : 3816.41
 Looking at view: 1 | Number of hits in view: 10
hit initial : 24.7127 | hit correction : 1.45929
hit initial : 19.4166 | hit correction : 1.45935
hit initial : 136.775 | hit correction : 1.45825
hit initial : 175.091 | hit correction : 1.45831
hit initial : 122.306 | hit correction : 1.45841
hit initial : 125.992 | hit correction : 1.45844
hit initial : 131.27 | hit correction : 1.45864
hit initial : 145.889 | hit correction : 1.4588
hit initial : 192.132 | hit correction : 1.45889
hit initial : 39.3648 | hit correction : 1.45894
hit initial : 142.029 | hit correction : 1.45854
hit initial : 110.076 | hit correction : 1.45851
hit initial : 92.9993 | hit correction : 1.45856
hit initial : 96.8881 | hit correction : 1.45857
hit initial : 102.245 | hit correction : 1.45847
hit initial : 109.764 | hit correction : 1.45855
hit initial : 130.095 | hit correction : 1.45848
hit initial : 109.172 | hit correction : 1.45851
 *** Total Charge : 2926.19
 Looking at view: 2 | Number of hits in view: 18
Begin processing the 10th record. run: 5026 subRun: 0 event: 2660 at 12-Dec-2025 02:33:11 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.569
True Electron Energy: 2.15105
Completed MCParticle Loop
True Neutrino Energy: 5.87558
 Size of all hits: 0
12-Dec-2025 02:33:13 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0637634     0.0756907     0.0946303     0.0790542    0.00992239       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000349172   0.00059461    0.00108346    0.000575199   0.000231727      10     
end_path:ana:AnalyzeEvents         0.0632621     0.074916      0.0931172     0.0782875    0.00972856       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19472.6 MB
  Peak resident set size usage (VmHWM): 1531.45 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 15:00:10 UTC       justIN version: 01.06.00