justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 261917.0@dunegpschedd02.fnal.gov

Jobsub ID261917.0@dunegpschedd02.fnal.gov
Workflow ID11173
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-13 04:56:13
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-12-13 17:08:13
From worker nodeHostnamedunegli-7890499-0-fnpc18005.fnal.gov
cpuinfoAMD EPYC 7551 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-13 04:56:47
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250116T125947Z_gen_000084_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time12h (43858s)
CPU time11h (41499s = 94%)
Max RSS bytes2842415104 (2710 MiB)
Outputting started2025-12-13 17:07:45
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11173/1/001/analysisLowE_20251213T045649Z.root
Finished2025-12-13 17:08:13
Saved logsjustin-logs:261917.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  13-Dec-2025 17:04:57 UTC run: 4696 subRun: 0 event: 840
Trying to add data product: ShowerPCA. This element does not exist in the element holder
%MSG
Boundary wire vector sizes: 6738, 6764, 6751
minwire 0: 47
minwire 1: 1
minwire 2: 1
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 862, 1361
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.13207, 
Output 1: 0.0674558, 0.0517246, 0.124824, 0.755996, 
Output 2: 0.0351902, 0.167681, 0.776027, 0.0211016, 
Output 3: 0.968437, 0.0209334, 0.00385098, 0.00677855, 
Output 4: 0.524835, 0.370785, 0.0884863, 0.015894, 
Output 5: 0.347367, 0.440116, 0.178376, 0.0341403, 
Output 6: 0.0257203, 0.0718876, 0.0835932, 0.818799, 

13-Dec-2025 17:06:11 UTC  Closed output file "matt_reco2.root"
13-Dec-2025 17:06:12 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/9c/93/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250116T125947Z_gen_000084_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              3815.56       4345.7        4806.27       4356.12       265.845        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00128638     0.0136239     0.102302     0.00174621     0.0299863       10     
reco:pandora:StandardPandora                            3751.12       4272.38       4729.22       4277.94       263.985        10     
reco:pandoraTrack:LArPandoraTrackCreation              0.0104748     0.112347      0.182295      0.118879      0.0421315       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.00228827     0.0274041     0.137778     0.00661229     0.0427423       10     
reco:pandoracalo:Calorimetry                          0.000572608   0.00325378     0.0177364     0.0015963    0.00490039       10     
reco:pandorapid:Chi2ParticleID                        0.000195979   0.00106456    0.00774837    0.000345175    0.0022304       10     
reco:cvnmap:CVNMapper                                  0.136519       0.1891       0.301771      0.169127      0.0539274       10     
reco:cvneva:CVNEvaluator                                2.2659        4.24346       11.8359       3.2422        2.63633        10     
reco:energyrecnumu:EnergyReco                           60.3166       67.9423       74.2502       69.8719       5.43006        10     
reco:energyrecnue:EnergyReco                          0.00284829    0.00553459     0.0163012    0.00385752    0.00388014       10     
reco:energyrecnc:EnergyReco                            0.0149153     0.0177236     0.0296462     0.016513     0.00410524       10     
[art]:TriggerResults:TriggerResultInserter            3.4115e-05    5.97718e-05   0.000234581    3.964e-05    5.84992e-05      10     
end_path:out1:RootOutput                              1.3485e-05    2.94703e-05   0.000120406   1.8109e-05    3.08459e-05      10     
end_path:out1:RootOutput(write)                        0.593373      0.767642       1.18709      0.697957      0.174723        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21576.7 MB
  Peak resident set size usage (VmHWM): 2842.42 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
 Looking at view: 0 | Number of hits in view: 40
hit initial : 18.2793 | hit correction : 1.20636
hit initial : 24.1301 | hit correction : 1.21794
hit initial : 108.425 | hit correction : 1.2179
hit initial : 108.5 | hit correction : 1.21788
hit initial : 84.4806 | hit correction : 1.2099
hit initial : 76.8848 | hit correction : 1.20989
hit initial : 17.0969 | hit correction : 1.20992
hit initial : 40.0809 | hit correction : 1.20857
hit initial : 105.258 | hit correction : 1.20857
hit initial : 49.5808 | hit correction : 1.20857
hit initial : 70.517 | hit correction : 1.20634
hit initial : 123.923 | hit correction : 1.20634
hit initial : 41.8631 | hit correction : 1.20635
hit initial : 25.9187 | hit correction : 1.20549
hit initial : 40.8987 | hit correction : 1.2055
hit initial : 62.3519 | hit correction : 1.20809
hit initial : 151.01 | hit correction : 1.2081
hit initial : 74.2544 | hit correction : 1.2081
hit initial : 142.447 | hit correction : 1.20165
hit initial : 106.45 | hit correction : 1.20166
hit initial : 35.609 | hit correction : 1.20169
hit initial : 219.634 | hit correction : 1.20106
hit initial : 321.387 | hit correction : 1.20084
hit initial : 380.586 | hit correction : 1.20004
hit initial : 21.6005 | hit correction : 1.19683
hit initial : 251.42 | hit correction : 1.19949
hit initial : 55.1805 | hit correction : 1.19682
hit initial : 28.7918 | hit correction : 1.19682
hit initial : 84.1149 | hit correction : 1.19804
hit initial : 94.4027 | hit correction : 1.19906
hit initial : 272.534 | hit correction : 1.19804
hit initial : 445.156 | hit correction : 1.19812
hit initial : 183.728 | hit correction : 1.18752
hit initial : 491.582 | hit correction : 1.18735
hit initial : 269.428 | hit correction : 1.18715
hit initial : 305.567 | hit correction : 1.1863
hit initial : 81.363 | hit correction : 1.18608
 *** Total Charge : 6010.99
 Looking at view: 1 | Number of hits in view: 37
hit initial : 29.9868 | hit correction : 1.21148
hit initial : 31.878 | hit correction : 1.20768
hit initial : 73.3699 | hit correction : 1.21792
hit initial : 163.186 | hit correction : 1.21787
hit initial : 41.4676 | hit correction : 1.20985
hit initial : 156.882 | hit correction : 1.20989
hit initial : 26.9707 | hit correction : 1.20865
hit initial : 151.52 | hit correction : 1.20857
hit initial : 17.3006 | hit correction : 1.20852
hit initial : 17.3696 | hit correction : 1.2064
hit initial : 212.942 | hit correction : 1.20633
hit initial : 52.8567 | hit correction : 1.20632
hit initial : 101.695 | hit correction : 1.20551
hit initial : 241.921 | hit correction : 1.20809
hit initial : 45.114 | hit correction : 1.2081
hit initial : 23.0928 | hit correction : 1.20809
hit initial : 218.865 | hit correction : 1.20147
hit initial : 189.771 | hit correction : 1.20167
hit initial : 84.7376 | hit correction : 1.20166
hit initial : 174.45 | hit correction : 1.20107
hit initial : 208.359 | hit correction : 1.20075
hit initial : 200.296 | hit correction : 1.20037
hit initial : 13.4581 | hit correction : 1.19975
hit initial : 295.955 | hit correction : 1.19993
hit initial : 64.7783 | hit correction : 1.19681
hit initial : 320.645 | hit correction : 1.19945
hit initial : 37.4412 | hit correction : 1.19683
hit initial : 238.86 | hit correction : 1.198
hit initial : 21.9932 | hit correction : 1.19802
hit initial : 339.975 | hit correction : 1.19904
hit initial : 198.386 | hit correction : 1.19797
hit initial : 21.9754 | hit correction : 1.19804
hit initial : 402.41 | hit correction : 1.19835
hit initial : 304.736 | hit correction : 1.19815
hit initial : 110.459 | hit correction : 1.18765
hit initial : 353.429 | hit correction : 1.18756
hit initial : 225.679 | hit correction : 1.18721
hit initial : 219.431 | hit correction : 1.18686
hit initial : 338.474 | hit correction : 1.18643
hit initial : 427.441 | hit correction : 1.18612
 *** Total Charge : 7668.89
 Looking at view: 2 | Number of hits in view: 40
13-Dec-2025 17:07:44 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.115276       0.17091      0.375182      0.141938      0.0752762       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000745575   0.00111115    0.00216322    0.000890112   0.000491343      10     
end_path:ana:AnalyzeEvents         0.112832      0.169423      0.373905      0.139814      0.075409        10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19475.2 MB
  Peak resident set size usage (VmHWM): 1535.52 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 12:04:52 UTC       justIN version: 01.06.00