justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 267955.1@dunegpschedd01.fnal.gov

Jobsub ID267955.1@dunegpschedd01.fnal.gov
Workflow ID11176
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-13 09:46:28
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-12-13 20:03:25
From worker nodeHostnamedunegli-8047081-0-fnpc9021.fnal.gov
cpuinfoIntel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-13 09:46:51
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T015959Z_gen_003426_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time10h (36981s)
CPU time10h (36645s = 99%)
Max RSS bytes2796724224 (2667 MiB)
Outputting started2025-12-13 20:03:12
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11176/1/001/analysisLowE_20251213T094652Z.root
Finished2025-12-13 20:03:25
Saved logsjustin-logs:267955.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  13-Dec-2025 20:02:03 UTC run: 5027 subRun: 0 event: 34260
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
Boundary wire vector sizes: 6703, 6678, 6646
minwire 0: 74
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 2306, 2805
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0791553, 
Output 1: 0.0675305, 0.0582515, 0.140996, 0.733222, 
Output 2: 0.122532, 0.232443, 0.588233, 0.0567923, 
Output 3: 0.931509, 0.0416502, 0.00994153, 0.0168992, 
Output 4: 0.652746, 0.312338, 0.0306017, 0.00431447, 
Output 5: 0.64666, 0.304974, 0.0398566, 0.00850951, 
Output 6: 0.06792, 0.0878236, 0.0730942, 0.771162, 

13-Dec-2025 20:02:06 UTC  Closed output file "matt_reco2.root"
13-Dec-2025 20:02:06 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/f8/4d/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T015959Z_gen_003426_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2230.36       3662.22       5192.01       3886.82       966.135        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00113169    0.00444677     0.0168337    0.00176579    0.00542355       10     
reco:pandora:StandardPandora                            2228.08       3658.63       5188.78       3883.82       965.233        10     
reco:pandoraTrack:LArPandoraTrackCreation              0.0565233     0.142841      0.218816      0.140426      0.0594852       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.00215289     0.0127613     0.0708094    0.00637456     0.0196838       10     
reco:pandoracalo:Calorimetry                          0.000556329   0.00321186     0.0201347    0.00117283    0.00570313       10     
reco:pandorapid:Chi2ParticleID                        0.000156553   0.000959955   0.00749712    0.000233774   0.00218016       10     
reco:cvnmap:CVNMapper                                  0.0850923     0.170486      0.332463      0.154568      0.0740071       10     
reco:cvneva:CVNEvaluator                                1.57493       2.56538       8.90778       1.89189       2.12286        10     
reco:energyrecnumu:EnergyReco                          0.0922545     0.137749      0.184674      0.131201      0.0266883       10     
reco:energyrecnue:EnergyReco                          0.00141518    0.00328356     0.0062571    0.00291885    0.00140714       10     
reco:energyrecnc:EnergyReco                            0.0085811     0.0126055     0.0157526     0.0134533    0.00223557       10     
[art]:TriggerResults:TriggerResultInserter             1.661e-05    4.93217e-05   0.000210271   3.3817e-05    5.43244e-05      10     
end_path:out1:RootOutput                               6.102e-06    1.63692e-05    4.311e-05    1.3743e-05    1.00356e-05      10     
end_path:out1:RootOutput(write)                        0.443227      0.532534      0.776275      0.516554      0.088404        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 11905.9 MB
  Peak resident set size usage (VmHWM): 2796.72 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 73.4585 | hit correction : 1.29125
hit initial : 101.419 | hit correction : 1.29213
hit initial : 88.2919 | hit correction : 1.29194
hit initial : 81.1159 | hit correction : 1.29208
hit initial : 107.839 | hit correction : 1.29051
hit initial : 244.328 | hit correction : 1.29051
hit initial : 71.9139 | hit correction : 1.28972
hit initial : 127.825 | hit correction : 1.28968
hit initial : 47.7558 | hit correction : 1.28967
hit initial : 650.411 | hit correction : 1.28823
hit initial : 435.131 | hit correction : 1.28659
hit initial : 450.694 | hit correction : 1.28645
 *** Total Charge : 5188.42
 Looking at view: 0 | Number of hits in view: 27
hit initial : 30.9339 | hit correction : 1.31371
hit initial : 103.85 | hit correction : 1.3137
hit initial : 86.7758 | hit correction : 1.31369
hit initial : 47.8047 | hit correction : 1.30415
hit initial : 52.0261 | hit correction : 1.30417
hit initial : 213.29 | hit correction : 1.29874
hit initial : 408.995 | hit correction : 1.2984
hit initial : 317.681 | hit correction : 1.298
hit initial : 157.449 | hit correction : 1.29772
hit initial : 62.1295 | hit correction : 1.29776
hit initial : 113.354 | hit correction : 1.29115
hit initial : 49.8855 | hit correction : 1.29208
hit initial : 28.6643 | hit correction : 1.29138
hit initial : 351.46 | hit correction : 1.2911
hit initial : 136.217 | hit correction : 1.29202
hit initial : 153.739 | hit correction : 1.29211
hit initial : 38.2357 | hit correction : 1.29226
hit initial : 114.354 | hit correction : 1.29058
hit initial : 208.742 | hit correction : 1.2905
hit initial : 220.693 | hit correction : 1.29032
hit initial : 263.087 | hit correction : 1.29001
hit initial : 282.174 | hit correction : 1.28964
hit initial : 272.522 | hit correction : 1.28925
hit initial : 241.483 | hit correction : 1.28872
hit initial : 218.106 | hit correction : 1.28823
hit initial : 213.189 | hit correction : 1.28779
hit initial : 251.332 | hit correction : 1.28727
hit initial : 295.681 | hit correction : 1.28677
hit initial : 293.966 | hit correction : 1.28635
 *** Total Charge : 6757.49
 Looking at view: 1 | Number of hits in view: 29
hit initial : 193.673 | hit correction : 1.28582
hit initial : 132.759 | hit correction : 1.3137
hit initial : 66.908 | hit correction : 1.3137
hit initial : 75.7686 | hit correction : 1.30414
hit initial : 15.7225 | hit correction : 1.30415
hit initial : 128.714 | hit correction : 1.29901
hit initial : 92.9408 | hit correction : 1.29829
hit initial : 572.909 | hit correction : 1.29831
hit initial : 300.475 | hit correction : 1.29776
hit initial : 55.0938 | hit correction : 1.2976
hit initial : 47.0179 | hit correction : 1.29129
hit initial : 287.443 | hit correction : 1.29121
hit initial : 101.205 | hit correction : 1.29165
hit initial : 135.421 | hit correction : 1.29196
hit initial : 111.444 | hit correction : 1.29156
hit initial : 193.869 | hit correction : 1.29215
hit initial : 138.09 | hit correction : 1.29077
hit initial : 50.5183 | hit correction : 1.29226
hit initial : 157.061 | hit correction : 1.2906
hit initial : 184.058 | hit correction : 1.29053
hit initial : 139.72 | hit correction : 1.29047
hit initial : 122.599 | hit correction : 1.29033
hit initial : 187.457 | hit correction : 1.29014
hit initial : 186.479 | hit correction : 1.28988
hit initial : 185.552 | hit correction : 1.28962
hit initial : 167.087 | hit correction : 1.28942
hit initial : 169.489 | hit correction : 1.2891
hit initial : 166.394 | hit correction : 1.28872
hit initial : 172.839 | hit correction : 1.28838
hit initial : 194.344 | hit correction : 1.28802
hit initial : 205.381 | hit correction : 1.28761
hit initial : 198.75 | hit correction : 1.28716
hit initial : 227.596 | hit correction : 1.28675
hit initial : 202.269 | hit correction : 1.28651
hit initial : 190.718 | hit correction : 1.28617
 *** Total Charge : 7439.7
 Looking at view: 2 | Number of hits in view: 35
13-Dec-2025 20:03:10 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.114614      0.133163      0.149652      0.135779      0.0109199       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000482241   0.000740189   0.00184871    0.000589247   0.000382681      10     
end_path:ana:AnalyzeEvents          0.11367      0.132204      0.147755      0.134929      0.0106729       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 9812.57 MB
  Peak resident set size usage (VmHWM): 1538.53 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 12:22:23 UTC       justIN version: 01.06.00