justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 261346.2@dunegpschedd02.fnal.gov

Jobsub ID261346.2@dunegpschedd02.fnal.gov
Workflow ID11109
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-11 23:16:43
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-12-12 03:18:14
From worker nodeHostnamedunegli-7875976-0-fnpc19105.fnal.gov
cpuinfoAMD EPYC 7502 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-11 23:17:23
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T012945Z_gen_003272_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time4h (14433s)
CPU time3h (14298s = 99%)
Max RSS bytes2916405248 (2781 MiB)
Outputting started2025-12-12 03:17:57
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11109/1/001/analysisLowE_20251211T231724Z.root
Finished2025-12-12 03:18:14
Saved logsjustin-logs:261346.2-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

o add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  12-Dec-2025 03:17:00 UTC run: 5026 subRun: 0 event: 32720
Trying to add data product: ShowerPCA. This element does not exist in the element holder
%MSG
Boundary wire vector sizes: 6950, 6929, 6732
minwire 0: 2
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 666, 1165
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.111303, 
Output 1: 0.101563, 0.0782163, 0.128195, 0.692026, 
Output 2: 0.384474, 0.240488, 0.300951, 0.074087, 
Output 3: 0.980029, 0.00979535, 0.000779228, 0.00939656, 
Output 4: 0.395128, 0.591276, 0.012707, 0.000888534, 
Output 5: 0.9067, 0.0896695, 0.00240513, 0.00122551, 
Output 6: 0.0107548, 0.0338107, 0.0488731, 0.906561, 

12-Dec-2025 03:17:02 UTC  Closed output file "matt_reco2.root"
12-Dec-2025 03:17:02 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/24/1d/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T012945Z_gen_003272_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1368.14       1428.51       1539.02       1405.72       53.2177        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00124756    0.00504269     0.0169168    0.00322453    0.00479943       10     
reco:pandora:StandardPandora                            1366.87       1426.98       1537.74       1404.24       53.3078        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000354552    0.0140239     0.0625943    0.00383734     0.0188611       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000314392   0.00459208    0.00990263    0.00469995     0.003188        10     
reco:pandoracalo:Calorimetry                          0.000158171   0.00471757     0.0222898    0.000455194   0.00831921       10     
reco:pandorapid:Chi2ParticleID                         4.757e-05    0.000171849   0.000583934   8.47955e-05   0.000195453      10     
reco:cvnmap:CVNMapper                                  0.0219457     0.0269173     0.0422143     0.0232596    0.00687312       10     
reco:cvneva:CVNEvaluator                               0.848721       1.0749        2.36623      0.878326      0.448101        10     
reco:energyrecnumu:EnergyReco                         0.00520628     0.056508      0.104941      0.0716412     0.0411612       10     
reco:energyrecnue:EnergyReco                          0.000839196   0.00400686     0.0117984    0.00162225    0.00363996       10     
reco:energyrecnc:EnergyReco                           0.00677923     0.0314507     0.0837753    0.00925267     0.0300937       10     
[art]:TriggerResults:TriggerResultInserter            1.2931e-05    2.28563e-05    6.975e-05     1.882e-05    1.60094e-05      10     
end_path:out1:RootOutput                               6.91e-06     9.3671e-06    2.4181e-05     7.625e-06    4.9663e-06       10     
end_path:out1:RootOutput(write)                        0.268013      0.309994      0.441167      0.287276      0.0543492       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21681.4 MB
  Peak resident set size usage (VmHWM): 2916.41 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 198.107 | hit correction : 1.24291
hit initial : 52.8247 | hit correction : 1.24302
hit initial : 300.264 | hit correction : 1.24297
hit initial : 247.064 | hit correction : 1.24353
hit initial : 274.796 | hit correction : 1.24411
hit initial : 185.823 | hit correction : 1.24466
hit initial : 184.423 | hit correction : 1.24517
hit initial : 144.316 | hit correction : 1.24553
hit initial : 136.038 | hit correction : 1.24585
hit initial : 145.493 | hit correction : 1.24611
hit initial : 178.323 | hit correction : 1.24635
hit initial : 155.633 | hit correction : 1.24667
hit initial : 163.662 | hit correction : 1.24701
hit initial : 167.121 | hit correction : 1.24733
hit initial : 163.486 | hit correction : 1.24765
hit initial : 56.7487 | hit correction : 1.24781
 *** Total Charge : 4861.5
 Looking at view: 2 | Number of hits in view: 22
Begin processing the 9th record. run: 5026 subRun: 0 event: 32719 at 12-Dec-2025 03:17:48 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Electron Energy: 35.2126
Completed MCParticle Loop
True Neutrino Energy: 50.7361
 Size of all hits: 0
Begin processing the 10th record. run: 5026 subRun: 0 event: 32720 at 12-Dec-2025 03:17:52 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  3
This pfp has size:  41
Added hit list length: 41 | Max was set to: -3.40282e+38
True Neutron Energy: 939.576
True Electron Energy: 24.2881
Completed MCParticle Loop
True Neutrino Energy: 27.5948
 Size of all hits: 41
hit initial : 161.976 | hit correction : 1.01879
hit initial : 17.0114 | hit correction : 1.02116
hit initial : 26.8694 | hit correction : 1.02118
hit initial : 213.457 | hit correction : 1.02287
hit initial : 232.582 | hit correction : 1.02193
hit initial : 92.2763 | hit correction : 1.02182
hit initial : 313.202 | hit correction : 1.02241
hit initial : 355.43 | hit correction : 1.02264
hit initial : 273.613 | hit correction : 1.02214
hit initial : 255.168 | hit correction : 1.0228
hit initial : 245.704 | hit correction : 1.02278
hit initial : 285.714 | hit correction : 1.02258
hit initial : 288.881 | hit correction : 1.02239
hit initial : 206.072 | hit correction : 1.02232
 *** Total Charge : 3033.98
 Looking at view: 0 | Number of hits in view: 14
hit initial : 69.141 | hit correction : 1.02283
hit initial : 118.035 | hit correction : 1.02195
hit initial : 24.6612 | hit correction : 1.02117
hit initial : 717.681 | hit correction : 1.02227
hit initial : 99.6997 | hit correction : 1.0225
 *** Total Charge : 1052.13
 Looking at view: 1 | Number of hits in view: 5
hit initial : 215.724 | hit correction : 1.02397
hit initial : 97.9376 | hit correction : 1.02392
hit initial : 13.7581 | hit correction : 1.02418
hit initial : 150.359 | hit correction : 1.02278
hit initial : 182.927 | hit correction : 1.0229
hit initial : 164.177 | hit correction : 1.02289
hit initial : 210.334 | hit correction : 1.02276
hit initial : 334.193 | hit correction : 1.02267
hit initial : 215.292 | hit correction : 1.02207
hit initial : 207.944 | hit correction : 1.02187
hit initial : 67.0825 | hit correction : 1.0217
hit initial : 35.0445 | hit correction : 1.02372
hit initial : 54.0711 | hit correction : 1.02005
hit initial : 32.6559 | hit correction : 1.02009
hit initial : 32.3401 | hit correction : 1.02286
hit initial : 197.85 | hit correction : 1.02362
hit initial : 42.3843 | hit correction : 1.0236
hit initial : 433.199 | hit correction : 1.02232
hit initial : 243.382 | hit correction : 1.02248
hit initial : 163.852 | hit correction : 1.02244
hit initial : 243.377 | hit correction : 1.02232
hit initial : 157.983 | hit correction : 1.02267
 *** Total Charge : 3575.02
 Looking at view: 2 | Number of hits in view: 22
12-Dec-2025 03:17:56 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0694793     0.0949781     0.125255      0.0928506     0.0181981       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000364733   0.00141257    0.00908664    0.000508109   0.00256333       10     
end_path:ana:AnalyzeEvents         0.0689665     0.0933778     0.124617      0.0917362     0.0180267       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19471.4 MB
  Peak resident set size usage (VmHWM): 1530.77 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 14:57:49 UTC       justIN version: 01.06.00