justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 261125.1@dunegpschedd02.fnal.gov

Jobsub ID261125.1@dunegpschedd02.fnal.gov
Workflow ID11113
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-11 17:20:22
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-12-12 04:09:34
From worker nodeHostnamedunegli-8029107-0-fnpc19104.fnal.gov
cpuinfoAMD EPYC 7502 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-11 17:21:18
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T230848Z_gen_001224_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time10h (38877s)
CPU time10h (38508s = 99%)
Max RSS bytes2688335872 (2563 MiB)
Outputting started2025-12-12 04:09:16
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11113/1/001/analysisLowE_20251211T172120Z.root
Finished2025-12-12 04:09:34
Saved logsjustin-logs:261125.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

, 0.0137308, 
Output 3: 0.989429, 0.00804399, 0.000541885, 0.00198471, 
Output 4: 0.628354, 0.343764, 0.0231171, 0.00476454, 
Output 5: 0.149889, 0.334423, 0.321687, 0.194001, 
Output 6: 0.0907094, 0.190424, 0.119608, 0.599259, 

Begin processing the 10th record. run: 5027 subRun: 0 event: 12240 at 12-Dec-2025 03:42:53 UTC
Error: A CaloHitList is empty
Error: A CaloHitList is empty
Boundary wire vector sizes: 6702, 6638, 6704
minwire 0: 12
minwire 1: 1
minwire 2: 1
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 613, 1112
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.12132, 
Output 1: 0.0823912, 0.100961, 0.177044, 0.639604, 
Output 2: 0.0420773, 0.206592, 0.724902, 0.0264285, 
Output 3: 0.959053, 0.026936, 0.00513896, 0.00887239, 
Output 4: 0.490804, 0.446854, 0.0565894, 0.00575242, 
Output 5: 0.463589, 0.403704, 0.111351, 0.0213564, 
Output 6: 0.0258462, 0.0601809, 0.0697797, 0.844193, 

12-Dec-2025 04:08:38 UTC  Closed output file "matt_reco2.root"
12-Dec-2025 04:08:38 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/6d/7d/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T230848Z_gen_001224_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1541.56       3865.97       5273.79       4408.83       1235.69        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00305025     0.0165008     0.045646      0.014489       0.01197        10     
reco:pandora:StandardPandora                            1540.37       3862.96       5262.78       4406.29       1234.16        10     
reco:pandoraTrack:LArPandoraTrackCreation              0.001014      0.0677456     0.497344     0.00890696     0.146088        10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000717077   0.00727624     0.0337975     0.003086      0.0100504       10     
reco:pandoracalo:Calorimetry                          0.000209551    0.0248538     0.082089     0.00358385     0.0344878       10     
reco:pandorapid:Chi2ParticleID                        0.000100161   0.000747459   0.00331711    0.000270793   0.00106915       10     
reco:cvnmap:CVNMapper                                  0.038953      0.0976942     0.172314      0.0947943     0.0446471       10     
reco:cvneva:CVNEvaluator                               0.776573       2.2251        9.40155       1.42406       2.44065        10     
reco:energyrecnumu:EnergyReco                         0.00922273     0.124802      0.197862       0.12607      0.051669        10     
reco:energyrecnue:EnergyReco                          0.00129229    0.00907478     0.0167897    0.00931487    0.00530388       10     
reco:energyrecnc:EnergyReco                           0.00640662     0.0186948     0.076357      0.0138397     0.0196068       10     
[art]:TriggerResults:TriggerResultInserter             1.048e-05    6.25925e-05   0.000170941   5.41305e-05   4.73114e-05      10     
end_path:out1:RootOutput                               6.51e-06     2.56852e-05   9.7321e-05    1.84705e-05   2.52223e-05      10     
end_path:out1:RootOutput(write)                        0.284917      0.415335      0.661663      0.361207      0.121807        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21686.8 MB
  Peak resident set size usage (VmHWM): 2688.34 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 49.2308 | hit correction : 1.46571
hit initial : 130.516 | hit correction : 1.46347
hit initial : 166.756 | hit correction : 1.46421
hit initial : 183.092 | hit correction : 1.46428
hit initial : 168.848 | hit correction : 1.46426
hit initial : 180.582 | hit correction : 1.46394
 *** Total Charge : 1542.9
 Looking at view: 1 | Number of hits in view: 9
hit initial : 107.239 | hit correction : 1.46572
hit initial : 44.6894 | hit correction : 1.46577
hit initial : 25.0575 | hit correction : 1.46436
hit initial : 122.91 | hit correction : 1.46336
hit initial : 349.003 | hit correction : 1.46424
hit initial : 400.547 | hit correction : 1.46412
 *** Total Charge : 1536.72
 Looking at view: 2 | Number of hits in view: 6
Begin processing the 9th record. run: 5027 subRun: 0 event: 12239 at 12-Dec-2025 04:09:10 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  3
This pfp has size:  42
Added hit list length: 42 | Max was set to: -3.40282e+38
This pfp has size:  32
True Electron Energy: 63.8606
Completed MCParticle Loop
True Neutrino Energy: 68.7678
 Size of all hits: 42
hit initial : 38.8698 | hit correction : 1.21169
hit initial : 37.0834 | hit correction : 1.21257
hit initial : 95.9439 | hit correction : 1.20892
hit initial : 241.359 | hit correction : 1.20897
hit initial : 238.124 | hit correction : 1.20923
hit initial : 133.338 | hit correction : 1.20994
hit initial : 273.948 | hit correction : 1.21022
hit initial : 449.494 | hit correction : 1.21023
hit initial : 339.762 | hit correction : 1.21076
hit initial : 303.294 | hit correction : 1.21119
hit initial : 332.804 | hit correction : 1.21161
hit initial : 390.124 | hit correction : 1.21205
hit initial : 396.921 | hit correction : 1.21226
hit initial : 190.982 | hit correction : 1.21242
 *** Total Charge : 4192.17
 Looking at view: 0 | Number of hits in view: 14
hit initial : 100.328 | hit correction : 1.21255
hit initial : 73.7042 | hit correction : 1.21039
hit initial : 171.771 | hit correction : 1.2104
hit initial : 63.3433 | hit correction : 1.21043
hit initial : 29.8463 | hit correction : 1.20879
hit initial : 151.449 | hit correction : 1.20889
hit initial : 273.785 | hit correction : 1.20906
hit initial : 281.701 | hit correction : 1.20949
hit initial : 243.422 | hit correction : 1.20999
hit initial : 237.901 | hit correction : 1.21046
hit initial : 239.771 | hit correction : 1.21087
hit initial : 215.611 | hit correction : 1.21116
hit initial : 198.273 | hit correction : 1.21142
hit initial : 192.551 | hit correction : 1.21166
hit initial : 189.617 | hit correction : 1.2119
hit initial : 229.885 | hit correction : 1.21212
hit initial : 271.742 | hit correction : 1.21224
hit initial : 199.655 | hit correction : 1.21239
 *** Total Charge : 4073.75
 Looking at view: 1 | Number of hits in view: 18
hit initial : 365.851 | hit correction : 1.21244
hit initial : 68.1899 | hit correction : 1.20885
hit initial : 91.393 | hit correction : 1.21022
hit initial : 220.155 | hit correction : 1.21045
hit initial : 495.648 | hit correction : 1.20905
hit initial : 653.749 | hit correction : 1.21012
hit initial : 469.846 | hit correction : 1.21108
hit initial : 306.45 | hit correction : 1.21157
hit initial : 283.167 | hit correction : 1.21194
hit initial : 382.791 | hit correction : 1.2122
 *** Total Charge : 4040.99
 Looking at view: 2 | Number of hits in view: 10
Begin processing the 10th record. run: 5027 subRun: 0 event: 12240 at 12-Dec-2025 04:09:13 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.577
True Electron Energy: 19.6198
Completed MCParticle Loop
True Neutrino Energy: 25.008
 Size of all hits: 0
12-Dec-2025 04:09:15 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0632187     0.0728975     0.0935511     0.0713414    0.00924857       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000325513   0.00038651    0.000487793   0.000386293   4.11561e-05      10     
end_path:ana:AnalyzeEvents         0.0628014     0.0723757     0.0930333     0.0706612    0.00921952       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19472.6 MB
  Peak resident set size usage (VmHWM): 1531.98 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 14:57:49 UTC       justIN version: 01.06.00