justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 267465.0@dunegpschedd01.fnal.gov

Jobsub ID267465.0@dunegpschedd01.fnal.gov
Workflow ID11169
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-12 20:43:49
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-12-13 13:14:41
From worker nodeHostnamedunegli-7890457-0-fnpc17005.fnal.gov
cpuinfoIntel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-13 00:51:56
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T031744Z_gen_004879_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time12h (44546s)
CPU time12h (43490s = 97%)
Max RSS bytes2786660352 (2657 MiB)
Outputting started2025-12-13 13:14:23
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11169/1/001/analysisLowE_20251213T005157Z.root
Finished2025-12-13 13:14:41
Saved logsjustin-logs:267465.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

event: 48790
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  13-Dec-2025 13:12:34 UTC run: 5027 subRun: 0 event: 48790
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
Boundary wire vector sizes: 7086, 6967, 6899
minwire 0: 21
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.131182, 
Output 1: 0.12212, 0.107732, 0.256659, 0.513489, 
Output 2: 0.053077, 0.0803049, 0.852982, 0.0136358, 
Output 3: 0.97381, 0.0224812, 0.00131377, 0.00239476, 
Output 4: 0.431708, 0.504947, 0.0531429, 0.0102016, 
Output 5: 0.251223, 0.461509, 0.20481, 0.0824573, 
Output 6: 0.115549, 0.225903, 0.132865, 0.525683, 

13-Dec-2025 13:12:39 UTC  Closed output file "matt_reco2.root"
13-Dec-2025 13:12:39 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/ae/f2/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T031744Z_gen_004879_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              3472.77       4405.89       5092.38       4605.46       524.964        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00169227    0.00613071     0.0225809    0.00231219    0.00777039       10     
reco:pandora:StandardPandora                            3470.11       4401.56       5088.86       4597.19       524.447        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000765301    0.167929      0.241112      0.199252      0.0684192       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000739517    0.0151441     0.0778995    0.00731848     0.0215471       10     
reco:pandoracalo:Calorimetry                          0.000512444   0.00359667     0.0188161    0.00166198    0.00518057       10     
reco:pandorapid:Chi2ParticleID                        0.000140694   0.00120416    0.00786828    0.000367419   0.00225667       10     
reco:cvnmap:CVNMapper                                  0.137685      0.203514       0.32425      0.187753      0.0526906       10     
reco:cvneva:CVNEvaluator                                1.64814       3.12638       11.2531       2.33067       2.7178         10     
reco:energyrecnumu:EnergyReco                          0.122897      0.167778      0.288949      0.146862      0.0530045       10     
reco:energyrecnue:EnergyReco                          0.00367409    0.00690914     0.0271631    0.00407572    0.00688443       10     
reco:energyrecnc:EnergyReco                            0.0117207     0.0162159     0.0204312     0.0159559    0.00279256       10     
[art]:TriggerResults:TriggerResultInserter            3.8491e-05    7.12122e-05   0.000253912   5.0138e-05    6.12671e-05      10     
end_path:out1:RootOutput                              1.3193e-05    2.30965e-05   7.3841e-05     1.777e-05    1.70844e-05      10     
end_path:out1:RootOutput(write)                        0.554903      0.614803      0.955368       0.58123      0.114385        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 11897.3 MB
  Peak resident set size usage (VmHWM): 2786.66 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 151.561 | hit correction : 1.03106
hit initial : 154.227 | hit correction : 1.03126
 *** Total Charge : 3947.28
 Looking at view: 0 | Number of hits in view: 23
hit initial : 28.2845 | hit correction : 1.02248
hit initial : 28.9862 | hit correction : 1.02247
hit initial : 31.6561 | hit correction : 1.01864
hit initial : 20.0733 | hit correction : 1.02583
hit initial : 16.9864 | hit correction : 1.02703
hit initial : 26.3066 | hit correction : 1.02684
hit initial : 22.4519 | hit correction : 1.02681
hit initial : 221.135 | hit correction : 1.02755
hit initial : 128.93 | hit correction : 1.02754
hit initial : 109.478 | hit correction : 1.02581
hit initial : 238.01 | hit correction : 1.02576
hit initial : 266.132 | hit correction : 1.02561
hit initial : 208.472 | hit correction : 1.02546
hit initial : 93.534 | hit correction : 1.02542
hit initial : 23.7818 | hit correction : 1.02007
hit initial : 29.2212 | hit correction : 1.02008
hit initial : 100.539 | hit correction : 1.02658
hit initial : 22.2103 | hit correction : 1.02778
hit initial : 44.5072 | hit correction : 1.0278
hit initial : 54.3747 | hit correction : 1.02779
hit initial : 135.746 | hit correction : 1.02849
hit initial : 152.541 | hit correction : 1.0278
hit initial : 179.787 | hit correction : 1.02791
hit initial : 297.973 | hit correction : 1.02864
hit initial : 308.09 | hit correction : 1.02896
hit initial : 309.371 | hit correction : 1.02911
hit initial : 268.58 | hit correction : 1.02921
hit initial : 234.454 | hit correction : 1.02937
hit initial : 235.363 | hit correction : 1.02954
hit initial : 354.595 | hit correction : 1.02978
hit initial : 487.288 | hit correction : 1.0299
hit initial : 318.558 | hit correction : 1.03006
hit initial : 251.207 | hit correction : 1.03043
hit initial : 283.508 | hit correction : 1.03088
hit initial : 165.301 | hit correction : 1.03124
 *** Total Charge : 5859.35
 Looking at view: 1 | Number of hits in view: 35
hit initial : 16.192 | hit correction : 1.02807
hit initial : 23.4212 | hit correction : 1.02749
hit initial : 25.6092 | hit correction : 1.02914
hit initial : 28.8685 | hit correction : 1.02863
hit initial : 23.8982 | hit correction : 1.02245
hit initial : 48.1593 | hit correction : 1.02247
hit initial : 57.6562 | hit correction : 1.01866
hit initial : 26.3731 | hit correction : 1.0256
hit initial : 68.9415 | hit correction : 1.02683
hit initial : 29.9439 | hit correction : 1.01953
hit initial : 494.342 | hit correction : 1.02754
hit initial : 15.4814 | hit correction : 1.02698
hit initial : 283.91 | hit correction : 1.02583
hit initial : 343.809 | hit correction : 1.02561
hit initial : 153.838 | hit correction : 1.0254
hit initial : 127.165 | hit correction : 1.02542
hit initial : 71.8734 | hit correction : 1.02008
hit initial : 165.615 | hit correction : 1.02657
hit initial : 21.1125 | hit correction : 1.02656
hit initial : 76.5562 | hit correction : 1.02778
hit initial : 15.6775 | hit correction : 1.02783
hit initial : 187.288 | hit correction : 1.02776
hit initial : 19.6334 | hit correction : 1.02847
hit initial : 318.32 | hit correction : 1.02804
hit initial : 192.335 | hit correction : 1.02848
hit initial : 160.213 | hit correction : 1.02874
hit initial : 181.2 | hit correction : 1.02892
hit initial : 159.074 | hit correction : 1.02905
hit initial : 163.845 | hit correction : 1.02912
hit initial : 199.963 | hit correction : 1.02915
hit initial : 187.653 | hit correction : 1.02926
hit initial : 175.912 | hit correction : 1.02937
hit initial : 164.629 | hit correction : 1.02949
hit initial : 210.155 | hit correction : 1.02963
hit initial : 253.283 | hit correction : 1.02981
hit initial : 591.914 | hit correction : 1.02991
hit initial : 184.133 | hit correction : 1.03013
hit initial : 207.574 | hit correction : 1.03034
hit initial : 576.635 | hit correction : 1.03098
 *** Total Charge : 6428.89
 Looking at view: 2 | Number of hits in view: 39
13-Dec-2025 13:14:22 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                          0.13179      0.214351      0.598813      0.168499      0.130006        10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000848265   0.00491345     0.0164252    0.00113268    0.00597999       10     
end_path:ana:AnalyzeEvents         0.130374      0.209044      0.581192      0.167093      0.125385        10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 9814.21 MB
  Peak resident set size usage (VmHWM): 1540.89 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 12:08:01 UTC       justIN version: 01.06.00