justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 268132.0@dunegpschedd01.fnal.gov

Jobsub ID268132.0@dunegpschedd01.fnal.gov
Workflow ID11170
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-13 21:09:03
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-12-14 09:14:49
From worker nodeHostnamedunegli-8066212-0-fnpc17126.fnal.gov
cpuinfoIntel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-14 03:32:12
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T003144Z_gen_002450_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time5h (20544s)
CPU time5h (19893s = 96%)
Max RSS bytes2841546752 (2709 MiB)
Outputting started2025-12-14 09:14:37
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11170/1/001/analysisLowE_20251214T033214Z.root
Finished2025-12-14 09:14:49
Saved logsjustin-logs:268132.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

hower@BeginModule  14-Dec-2025 09:13:21 UTC run: 5027 subRun: 0 event: 24500
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  14-Dec-2025 09:13:21 UTC run: 5027 subRun: 0 event: 24500
Trying to add data product: ShowerPCA. This element does not exist in the element holder
%MSG
Boundary wire vector sizes: 6609, 6537, 6607
minwire 0: 42
minwire 1: 1
minwire 2: 3
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 1481, 1980
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0875115, 
Output 1: 0.0707592, 0.0618678, 0.120105, 0.747268, 
Output 2: 0.0483377, 0.164145, 0.772621, 0.0148959, 
Output 3: 0.946824, 0.0440346, 0.00380597, 0.00533585, 
Output 4: 0.547747, 0.39089, 0.0531961, 0.00816688, 
Output 5: 0.238223, 0.602278, 0.140906, 0.0185927, 
Output 6: 0.0433095, 0.100409, 0.0960512, 0.76023, 

14-Dec-2025 09:13:25 UTC  Closed output file "matt_reco2.root"
14-Dec-2025 09:13:25 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/c3/eb/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T003144Z_gen_002450_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1815.88       1985.88       2357.6        1880.65       189.076        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000727655   0.00772793     0.0394781    0.00134582     0.0119653       10     
reco:pandora:StandardPandora                            1813.84       1983.56       2355.06       1878.83       189.013        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000343632    0.0547041     0.090324      0.0605813     0.0313234       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000357862   0.00390239     0.0115724    0.00223843    0.00361814       10     
reco:pandoracalo:Calorimetry                          0.000207619   0.00115239     0.0030958    0.000619146   0.00104423       10     
reco:pandorapid:Chi2ParticleID                        6.6603e-05    0.000272574   0.00135713    0.000151624   0.000367513      10     
reco:cvnmap:CVNMapper                                  0.107773      0.134736      0.184707       0.13108      0.0243443       10     
reco:cvneva:CVNEvaluator                                1.09875       1.53851       3.63982       1.29942      0.722534        10     
reco:energyrecnumu:EnergyReco                          0.0851695      0.10844      0.141122      0.100384      0.0192605       10     
reco:energyrecnue:EnergyReco                          0.00162564    0.00384875    0.00994546    0.00226741    0.00305727       10     
reco:energyrecnc:EnergyReco                           0.00782171     0.0110789     0.0165728    0.00984655    0.00254832       10     
[art]:TriggerResults:TriggerResultInserter            1.8635e-05    2.86333e-05   7.5692e-05    2.29645e-05   1.62704e-05      10     
end_path:out1:RootOutput                               7.785e-06    9.5705e-06     1.911e-05     8.246e-06    3.27296e-06      10     
end_path:out1:RootOutput(write)                        0.347697      0.453115      0.641489      0.412707      0.105905        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 14051.9 MB
  Peak resident set size usage (VmHWM): 2841.55 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
Number of reconstructed particles:  5
This pfp has size:  67
Added hit list length: 67 | Max was set to: -3.40282e+38
This pfp has size:  9
True Neutron Energy: 939.569
True Electron Energy: 42.5991
Completed MCParticle Loop
True Neutrino Energy: 64.6638
 Size of all hits: 67
hit initial : 61.6807 | hit correction : 1.3408
hit initial : 171.316 | hit correction : 1.34069
hit initial : 96.9568 | hit correction : 1.34059
hit initial : 69.3718 | hit correction : 1.33539
hit initial : 160.749 | hit correction : 1.3354
hit initial : 203.927 | hit correction : 1.33542
hit initial : 62.8641 | hit correction : 1.32812
hit initial : 84.2185 | hit correction : 1.32812
hit initial : 28.596 | hit correction : 1.32477
hit initial : 132.994 | hit correction : 1.32405
hit initial : 980.344 | hit correction : 1.32389
hit initial : 654.175 | hit correction : 1.32382
hit initial : 272.555 | hit correction : 1.32336
hit initial : 153.238 | hit correction : 1.32304
hit initial : 128.062 | hit correction : 1.32213
 *** Total Charge : 4327.94
 Looking at view: 0 | Number of hits in view: 15
hit initial : 96.4098 | hit correction : 1.34073
hit initial : 204.74 | hit correction : 1.34068
hit initial : 87.3125 | hit correction : 1.34062
hit initial : 96.8109 | hit correction : 1.33548
hit initial : 212.52 | hit correction : 1.33542
hit initial : 112.071 | hit correction : 1.33533
hit initial : 28.1744 | hit correction : 1.32807
hit initial : 84.0555 | hit correction : 1.3281
hit initial : 61.6979 | hit correction : 1.32814
hit initial : 22.6369 | hit correction : 1.32488
hit initial : 82.1588 | hit correction : 1.32483
hit initial : 75.8403 | hit correction : 1.3248
hit initial : 163.512 | hit correction : 1.32412
hit initial : 399.759 | hit correction : 1.32415
hit initial : 568.051 | hit correction : 1.32403
hit initial : 324.207 | hit correction : 1.32391
hit initial : 60.9949 | hit correction : 1.32305
hit initial : 213.652 | hit correction : 1.32363
hit initial : 181.896 | hit correction : 1.32344
hit initial : 179.436 | hit correction : 1.32324
hit initial : 190.393 | hit correction : 1.32306
hit initial : 214.871 | hit correction : 1.32288
hit initial : 234.582 | hit correction : 1.32263
hit initial : 234.704 | hit correction : 1.32225
 *** Total Charge : 5479.46
 Looking at view: 1 | Number of hits in view: 24
hit initial : 96.2866 | hit correction : 1.34081
hit initial : 30.8957 | hit correction : 1.34057
hit initial : 253.474 | hit correction : 1.34062
hit initial : 216.383 | hit correction : 1.33549
hit initial : 186 | hit correction : 1.33532
hit initial : 21.2745 | hit correction : 1.33518
hit initial : 80.2019 | hit correction : 1.32809
hit initial : 92.9481 | hit correction : 1.32814
hit initial : 132.668 | hit correction : 1.32483
hit initial : 60.424 | hit correction : 1.32473
hit initial : 114.141 | hit correction : 1.32459
hit initial : 97.9687 | hit correction : 1.32443
hit initial : 212.41 | hit correction : 1.32402
hit initial : 290.157 | hit correction : 1.32421
hit initial : 215.073 | hit correction : 1.32403
hit initial : 166.446 | hit correction : 1.32365
hit initial : 29.5692 | hit correction : 1.32301
hit initial : 327.711 | hit correction : 1.32342
hit initial : 172.296 | hit correction : 1.32374
hit initial : 143.054 | hit correction : 1.32354
hit initial : 117.475 | hit correction : 1.32341
hit initial : 128.209 | hit correction : 1.32329
hit initial : 133.001 | hit correction : 1.32316
hit initial : 131.609 | hit correction : 1.32303
hit initial : 147.329 | hit correction : 1.32291
hit initial : 149.183 | hit correction : 1.32276
hit initial : 204.847 | hit correction : 1.32259
hit initial : 173.884 | hit correction : 1.32227
 *** Total Charge : 5471.89
 Looking at view: 2 | Number of hits in view: 28
14-Dec-2025 09:14:35 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0815319      0.12758      0.195979      0.125258      0.0279082       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000433763   0.000595652   0.00107264    0.000534874   0.000175832      10     
end_path:ana:AnalyzeEvents         0.0808924      0.12678       0.19538      0.124568      0.0278721       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 11956.5 MB
  Peak resident set size usage (VmHWM): 1536.26 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 12:08:02 UTC       justIN version: 01.06.00