justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 267904.0@dunegpschedd01.fnal.gov

Jobsub ID267904.0@dunegpschedd01.fnal.gov
Workflow ID11168
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-13 08:02:23
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-12-13 20:58:12
From worker nodeHostnamedunegli-7890498-0-fnpc18005.fnal.gov
cpuinfoAMD EPYC 7551 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-13 08:04:39
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250116T143313Z_gen_001000_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time12h (46394s)
CPU time12h (44143s = 95%)
Max RSS bytes2791837696 (2662 MiB)
Outputting started2025-12-13 20:57:54
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11168/1/001/analysisLowE_20251213T080441Z.root
Finished2025-12-13 20:58:12
Saved logsjustin-logs:267904.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

eation:pandoraShower@BeginModule  13-Dec-2025 20:56:41 UTC run: 4696 subRun: 0 event: 10000
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  13-Dec-2025 20:56:41 UTC run: 4696 subRun: 0 event: 10000
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
Boundary wire vector sizes: 6799, 6693, 6750
minwire 0: 87
minwire 1: 1
minwire 2: 1
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 2379, 2878
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.112051, 
Output 1: 0.048139, 0.152394, 0.156359, 0.643109, 
Output 2: 0.0485364, 0.224865, 0.700168, 0.0264305, 
Output 3: 0.943134, 0.046099, 0.00519394, 0.00557264, 
Output 4: 0.456329, 0.472245, 0.0652284, 0.0061975, 
Output 5: 0.461647, 0.391869, 0.121182, 0.0253018, 
Output 6: 0.0320423, 0.0806473, 0.0880259, 0.799284, 

13-Dec-2025 20:56:45 UTC  Closed output file "matt_reco2.root"
13-Dec-2025 20:56:45 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/22/54/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250116T143313Z_gen_001000_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2677.64       4591.63       7925.29       4182.74       1635.14        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00117167     0.0073191     0.0268752     0.0024662    0.00849387       10     
reco:pandora:StandardPandora                            2674.67       4586.26       7904.75       4179.53       1630.95        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000630237    0.0985781     0.339024      0.0886024     0.0933045       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000582077    0.0212291     0.153514     0.00646084     0.0442829       10     
reco:pandoracalo:Calorimetry                          0.00041413     0.004931      0.0385819    0.000935076    0.0112416       10     
reco:pandorapid:Chi2ParticleID                        0.000109396   0.00191179     0.0169148    0.000196736   0.00500286       10     
reco:cvnmap:CVNMapper                                  0.097534      0.180186      0.346963      0.129809      0.0836726       10     
reco:cvneva:CVNEvaluator                                2.11149       4.21769       17.6435       2.44466       4.56476        10     
reco:energyrecnumu:EnergyReco                          0.098706      0.161381      0.353502      0.147576      0.0700443       10     
reco:energyrecnue:EnergyReco                           0.0021295    0.00630534     0.0187516    0.00308721    0.00544912       10     
reco:energyrecnc:EnergyReco                            0.0100482     0.0155998     0.0250988     0.0136924    0.00489583       10     
[art]:TriggerResults:TriggerResultInserter            3.0507e-05    0.000107365   0.000623734   4.12225e-05   0.000173329      10     
end_path:out1:RootOutput                              1.3034e-05    3.55228e-05   0.00018074    1.75075e-05   4.87939e-05      10     
end_path:out1:RootOutput(write)                        0.389009      0.645717       1.57414      0.431261      0.359895        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21575.2 MB
  Peak resident set size usage (VmHWM): 2791.84 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
Added hit list length: 67 | Max was set to: -3.40282e+38
This pfp has size:  10
This pfp has size:  5
This pfp has size:  4
True Neutron Energy: 939.57
True Electron Energy: 34.4378
Completed MCParticle Loop
True Neutrino Energy: 38.6097
 Size of all hits: 67
hit initial : 23.77 | hit correction : 1.09885
hit initial : 37.4985 | hit correction : 1.09882
hit initial : 62.7491 | hit correction : 1.09204
hit initial : 114.947 | hit correction : 1.09201
hit initial : 81.5319 | hit correction : 1.09146
hit initial : 33.3564 | hit correction : 1.09148
hit initial : 132.028 | hit correction : 1.08624
hit initial : 87.3049 | hit correction : 1.08624
hit initial : 28.1472 | hit correction : 1.08619
hit initial : 5.50765 | hit correction : 1.08584
hit initial : 30.7849 | hit correction : 1.08283
hit initial : 99.1456 | hit correction : 1.08298
hit initial : 285.168 | hit correction : 1.08293
hit initial : 348.494 | hit correction : 1.08281
hit initial : 310.831 | hit correction : 1.0827
hit initial : 264.902 | hit correction : 1.0826
hit initial : 304.642 | hit correction : 1.0825
hit initial : 316.389 | hit correction : 1.08239
hit initial : 329.077 | hit correction : 1.08227
hit initial : 308.872 | hit correction : 1.08214
hit initial : 324.911 | hit correction : 1.08203
hit initial : 299.691 | hit correction : 1.08196
hit initial : 20.7013 | hit correction : 1.08188
 *** Total Charge : 4172.61
 Looking at view: 0 | Number of hits in view: 23
hit initial : 15.6377 | hit correction : 1.0988
hit initial : 42.3185 | hit correction : 1.09882
hit initial : 54.5956 | hit correction : 1.09204
hit initial : 46.4892 | hit correction : 1.09201
hit initial : 35.4076 | hit correction : 1.09145
hit initial : 42.4833 | hit correction : 1.09146
hit initial : 148.33 | hit correction : 1.08626
hit initial : 71.8131 | hit correction : 1.08624
hit initial : 21.7202 | hit correction : 1.08276
hit initial : 219.907 | hit correction : 1.08282
hit initial : 738.88 | hit correction : 1.08276
hit initial : 921.386 | hit correction : 1.08251
hit initial : 851.318 | hit correction : 1.0822
 *** Total Charge : 3478.59
 Looking at view: 1 | Number of hits in view: 13
hit initial : 26.693 | hit correction : 1.07415
hit initial : 18.1628 | hit correction : 1.07408
hit initial : 39.6361 | hit correction : 1.08591
hit initial : 43.924 | hit correction : 1.09879
hit initial : 30.9055 | hit correction : 1.0988
hit initial : 168.915 | hit correction : 1.09201
hit initial : 58.5144 | hit correction : 1.09145
hit initial : 88.7852 | hit correction : 1.09146
hit initial : 177.761 | hit correction : 1.08623
hit initial : 75.1517 | hit correction : 1.08624
hit initial : 30.8423 | hit correction : 1.08278
hit initial : 162.028 | hit correction : 1.08308
hit initial : 288.253 | hit correction : 1.08289
hit initial : 223.886 | hit correction : 1.08281
hit initial : 180.744 | hit correction : 1.08278
hit initial : 141.092 | hit correction : 1.08265
hit initial : 127.808 | hit correction : 1.08261
hit initial : 122.279 | hit correction : 1.08263
hit initial : 129.358 | hit correction : 1.08257
hit initial : 125.015 | hit correction : 1.08249
hit initial : 175.775 | hit correction : 1.08245
hit initial : 146.435 | hit correction : 1.08242
hit initial : 141.042 | hit correction : 1.08234
hit initial : 205.912 | hit correction : 1.08229
hit initial : 156.317 | hit correction : 1.08221
hit initial : 141.968 | hit correction : 1.08213
hit initial : 150.439 | hit correction : 1.08207
hit initial : 163.682 | hit correction : 1.08202
hit initial : 136.843 | hit correction : 1.08197
hit initial : 156.132 | hit correction : 1.08195
hit initial : 132.284 | hit correction : 1.08193
 *** Total Charge : 4298.51
 Looking at view: 2 | Number of hits in view: 31
13-Dec-2025 20:57:53 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0972462     0.166966      0.236319      0.150347      0.0470551       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.00066379    0.00109027    0.00384692    0.000764315   0.00092444       10     
end_path:ana:AnalyzeEvents         0.0962245     0.165494      0.235284       0.14919      0.046552        10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19470.9 MB
  Peak resident set size usage (VmHWM): 1527.62 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 12:05:50 UTC       justIN version: 01.06.00