justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 267782.1@dunegpschedd01.fnal.gov

Jobsub ID267782.1@dunegpschedd01.fnal.gov
Workflow ID11171
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-13 05:40:16
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-12-13 17:36:39
From worker nodeHostnamedunegli-7890610-0-fnpc9038.fnal.gov
cpuinfoIntel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-13 05:41:16
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T232730Z_gen_001708_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time11h (42900s)
CPU time11h (41750s = 97%)
Max RSS bytes2839162880 (2707 MiB)
Outputting started2025-12-13 17:36:17
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11171/1/001/analysisLowE_20251213T054118Z.root
Finished2025-12-13 17:36:39
Saved logsjustin-logs:267782.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  13-Dec-2025 17:33:21 UTC run: 5027 subRun: 0 event: 17080
Trying to add data product: ShowerPCA. This element does not exist in the element holder
%MSG
Boundary wire vector sizes: 6788, 6774, 6717
minwire 0: 32
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 1270, 1769
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.124688, 
Output 1: 0.324563, 0.114488, 0.252619, 0.308331, 
Output 2: 0.035901, 0.0620797, 0.894807, 0.00721207, 
Output 3: 0.96209, 0.0353476, 0.00105891, 0.00150358, 
Output 4: 0.46016, 0.476578, 0.0515118, 0.01175, 
Output 5: 0.212387, 0.414296, 0.244425, 0.128892, 
Output 6: 0.188669, 0.314806, 0.146608, 0.349917, 

13-Dec-2025 17:34:32 UTC  Closed output file "matt_reco2.root"
13-Dec-2025 17:34:32 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/e5/bb/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T232730Z_gen_001708_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              3922.52       4251.64       4477.18       4274.19       180.802        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00193344    0.00641238     0.0233136    0.00233551     0.0082708       10     
reco:pandora:StandardPandora                            3844.66       4181.19       4402.29       4206.92       182.288        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.00161823     0.134426      0.208097      0.183015      0.0845202       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.00311409     0.0281077     0.143891      0.0108602     0.0424589       10     
reco:pandoracalo:Calorimetry                          0.000446453   0.00345904     0.0136199    0.00136345     0.0043256       10     
reco:pandorapid:Chi2ParticleID                        0.000124643   0.00101708    0.00686043    0.000359243   0.00195242       10     
reco:cvnmap:CVNMapper                                  0.136714      0.189401      0.300106      0.180611      0.0420388       10     
reco:cvneva:CVNEvaluator                                1.58748       2.96127       8.98788       2.42933       2.0249         10     
reco:energyrecnumu:EnergyReco                           60.4275       66.4281       74.3517       65.3004       4.09981        10     
reco:energyrecnue:EnergyReco                           0.0038283    0.00582004     0.0167734    0.00411773    0.00382214       10     
reco:energyrecnc:EnergyReco                            0.015318      0.0171311     0.0216157     0.0166191    0.00180523       10     
[art]:TriggerResults:TriggerResultInserter            4.2933e-05    6.06931e-05   0.000179748   4.89145e-05   3.98138e-05      10     
end_path:out1:RootOutput                               1.304e-05    2.15223e-05   6.3082e-05    1.71825e-05   1.40185e-05      10     
end_path:out1:RootOutput(write)                        0.599958      0.681439      0.901932      0.656597      0.0790414       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 11905.1 MB
  Peak resident set size usage (VmHWM): 2839.16 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 123.407 | hit correction : 1.03013
hit initial : 77.6302 | hit correction : 1.03019
hit initial : 204.009 | hit correction : 1.02967
hit initial : 707.789 | hit correction : 1.02954
hit initial : 988.011 | hit correction : 1.02934
hit initial : 588.034 | hit correction : 1.02916
hit initial : 277.392 | hit correction : 1.02887
hit initial : 261.142 | hit correction : 1.02848
hit initial : 310.2 | hit correction : 1.02814
hit initial : 370.748 | hit correction : 1.02776
hit initial : 328.618 | hit correction : 1.02751
hit initial : 256.129 | hit correction : 1.027
hit initial : 114.891 | hit correction : 1.02681
hit initial : 69.0143 | hit correction : 1.02649
 *** Total Charge : 6050.47
 Looking at view: 0 | Number of hits in view: 29
hit initial : 47.3586 | hit correction : 1.09445
hit initial : 104.8 | hit correction : 1.09444
hit initial : 43.4735 | hit correction : 1.09444
hit initial : 76.3968 | hit correction : 1.09355
hit initial : 56.9013 | hit correction : 1.09353
hit initial : 84.2383 | hit correction : 1.0621
hit initial : 53.3811 | hit correction : 1.06208
hit initial : 31.6972 | hit correction : 1.04888
hit initial : 188.495 | hit correction : 1.05033
hit initial : 271.604 | hit correction : 1.05034
hit initial : 96.5458 | hit correction : 1.05034
hit initial : 45.7913 | hit correction : 1.04101
hit initial : 30.48 | hit correction : 1.041
hit initial : 25.038 | hit correction : 1.03194
hit initial : 24.1923 | hit correction : 1.03254
hit initial : 17.4175 | hit correction : 1.03194
hit initial : 16.2396 | hit correction : 1.03253
hit initial : 99.5144 | hit correction : 1.03015
hit initial : 56.9692 | hit correction : 1.03014
hit initial : 97.3752 | hit correction : 1.02963
hit initial : 556.581 | hit correction : 1.02952
hit initial : 1170.06 | hit correction : 1.02936
hit initial : 570.599 | hit correction : 1.02912
hit initial : 109.099 | hit correction : 1.02905
hit initial : 92.8364 | hit correction : 1.0285
hit initial : 168.867 | hit correction : 1.02795
hit initial : 714.79 | hit correction : 1.02772
hit initial : 100.358 | hit correction : 1.02683
hit initial : 83.0065 | hit correction : 1.02648
hit initial : 47.0574 | hit correction : 1.02646
 *** Total Charge : 5267.51
 Looking at view: 1 | Number of hits in view: 30
hit initial : 19.4124 | hit correction : 1.0632
hit initial : 18.0881 | hit correction : 1.04507
hit initial : 79.202 | hit correction : 1.09444
hit initial : 126.46 | hit correction : 1.09444
hit initial : 86.3559 | hit correction : 1.09353
hit initial : 93.1447 | hit correction : 1.09354
hit initial : 158.481 | hit correction : 1.06209
hit initial : 37.4895 | hit correction : 1.0489
hit initial : 330.072 | hit correction : 1.05034
hit initial : 94.7889 | hit correction : 1.05027
hit initial : 179.352 | hit correction : 1.05034
hit initial : 91.5262 | hit correction : 1.041
hit initial : 51.3102 | hit correction : 1.03194
hit initial : 43.0602 | hit correction : 1.03253
hit initial : 191.274 | hit correction : 1.0301
hit initial : 84.9294 | hit correction : 1.03017
hit initial : 20.2777 | hit correction : 1.02979
hit initial : 195.272 | hit correction : 1.02974
hit initial : 15.5186 | hit correction : 1.03012
hit initial : 25.3743 | hit correction : 1.02963
hit initial : 622.184 | hit correction : 1.02954
hit initial : 195.94 | hit correction : 1.02946
hit initial : 806.582 | hit correction : 1.02926
hit initial : 663.948 | hit correction : 1.02911
hit initial : 292.436 | hit correction : 1.02871
hit initial : 254.034 | hit correction : 1.02838
hit initial : 231.719 | hit correction : 1.02808
hit initial : 263.382 | hit correction : 1.02777
hit initial : 412.581 | hit correction : 1.02754
hit initial : 301.583 | hit correction : 1.02697
hit initial : 99.1402 | hit correction : 1.02667
hit initial : 246.837 | hit correction : 1.02652
 *** Total Charge : 6560.03
 Looking at view: 2 | Number of hits in view: 32
13-Dec-2025 17:36:13 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0831361     0.244391      0.502095      0.200857      0.116248        10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000507011   0.00478116     0.0224301    0.000806097   0.00804487       10     
end_path:ana:AnalyzeEvents         0.0821701     0.239331      0.501291      0.196012      0.117253        10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 9807.62 MB
  Peak resident set size usage (VmHWM): 1534.72 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 12:04:36 UTC       justIN version: 01.06.00