justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 268017.1@dunegpschedd01.fnal.gov

Jobsub ID268017.1@dunegpschedd01.fnal.gov
Workflow ID11176
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-13 12:00:35
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-12-13 19:30:35
From worker nodeHostnamedunegli-8047088-0-fnpc17005.fnal.gov
cpuinfoIntel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-13 12:00:51
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T004412Z_gen_002762_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time7h (26970s)
CPU time7h (26688s = 98%)
Max RSS bytes2798768128 (2669 MiB)
Outputting started2025-12-13 19:30:22
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11176/1/001/analysisLowE_20251213T120052Z.root
Finished2025-12-13 19:30:35
Saved logsjustin-logs:268017.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

C run: 5027 subRun: 0 event: 27620
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  13-Dec-2025 19:28:51 UTC run: 5027 subRun: 0 event: 27620
Trying to add data product: ShowerPCA. This element does not exist in the element holder
%MSG
Boundary wire vector sizes: 6711, 6725, 6684
minwire 0: 67
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 651, 1150
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.062999, 
Output 1: 0.138129, 0.0873208, 0.091403, 0.683147, 
Output 2: 0.121432, 0.141068, 0.710747, 0.0267526, 
Output 3: 0.957936, 0.0396517, 0.00117951, 0.00123259, 
Output 4: 0.967654, 0.0303262, 0.00140669, 0.000613319, 
Output 5: 0.135069, 0.571531, 0.231215, 0.0621858, 
Output 6: 0.0796667, 0.302338, 0.155724, 0.462272, 

13-Dec-2025 19:28:56 UTC  Closed output file "matt_reco2.root"
13-Dec-2025 19:28:57 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/5e/6c/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T004412Z_gen_002762_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2191.2        2666.49       3550.85       2477.39       457.556        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000919397   0.00724499     0.039494     0.00160679     0.0122207       10     
reco:pandora:StandardPandora                            2188.69       2663.49       3547.9        2472.64       457.522        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.00306362     0.076244      0.122056      0.0842386     0.0409833       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.00122418    0.00955754     0.0484414    0.00645676     0.0131735       10     
reco:pandoracalo:Calorimetry                          0.00033126    0.00204268     0.0114707    0.000869519   0.00319801       10     
reco:pandorapid:Chi2ParticleID                        0.000100519   0.000639824   0.00489635    0.000174185   0.00141935       10     
reco:cvnmap:CVNMapper                                  0.0685082     0.0984228      0.18379      0.0895809     0.0337514       10     
reco:cvneva:CVNEvaluator                                1.21742       2.13253       5.70015       1.80288       1.21052        10     
reco:energyrecnumu:EnergyReco                          0.0987082     0.143336      0.186778       0.14146      0.0258322       10     
reco:energyrecnue:EnergyReco                          0.00161463    0.00273216    0.00442224    0.00247731    0.000930272      10     
reco:energyrecnc:EnergyReco                           0.00806081     0.0131589     0.0226079     0.0123027    0.00398707       10     
[art]:TriggerResults:TriggerResultInserter            2.3966e-05    4.91104e-05   0.000190508   3.16125e-05   4.77561e-05      10     
end_path:out1:RootOutput                               8.976e-06    1.62943e-05    5.384e-05    1.24065e-05   1.27608e-05      10     
end_path:out1:RootOutput(write)                        0.328697      0.517184      0.736957       0.50489      0.114419        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 11909.5 MB
  Peak resident set size usage (VmHWM): 2798.77 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 120.262 | hit correction : 1.23189
hit initial : 121.658 | hit correction : 1.23487
hit initial : 123.145 | hit correction : 1.23514
hit initial : 12.1982 | hit correction : 1.23512
hit initial : 86.4447 | hit correction : 1.23478
hit initial : 143.479 | hit correction : 1.23538
hit initial : 333.985 | hit correction : 1.2354
hit initial : 238.065 | hit correction : 1.23597
hit initial : 226.017 | hit correction : 1.23653
hit initial : 151.741 | hit correction : 1.23704
hit initial : 77.3988 | hit correction : 1.23833
hit initial : 552.872 | hit correction : 1.23816
hit initial : 61.2737 | hit correction : 1.23835
hit initial : 403.919 | hit correction : 1.23884
hit initial : 459.785 | hit correction : 1.23994
hit initial : 247.391 | hit correction : 1.24056
hit initial : 77.9341 | hit correction : 1.24182
hit initial : 112.17 | hit correction : 1.24192
hit initial : 248.441 | hit correction : 1.24214
hit initial : 36.0248 | hit correction : 1.24245
hit initial : 12.9746 | hit correction : 1.24271
hit initial : 65.4113 | hit correction : 1.24304
hit initial : 82.187 | hit correction : 1.24305
hit initial : 23.8409 | hit correction : 1.24312
 *** Total Charge : 5920.4
 Looking at view: 1 | Number of hits in view: 36
hit initial : 25.7322 | hit correction : 1.21797
hit initial : 28.5211 | hit correction : 1.21571
hit initial : 31.5288 | hit correction : 1.21571
hit initial : 20.6425 | hit correction : 1.22352
hit initial : 85.0297 | hit correction : 1.22354
hit initial : 42.243 | hit correction : 1.22653
hit initial : 86.86 | hit correction : 1.22821
hit initial : 41.1985 | hit correction : 1.22822
hit initial : 30.2471 | hit correction : 1.2317
hit initial : 63.5129 | hit correction : 1.23167
hit initial : 315.533 | hit correction : 1.23167
hit initial : 65.0459 | hit correction : 1.23194
hit initial : 165.952 | hit correction : 1.23194
hit initial : 43.4851 | hit correction : 1.23508
hit initial : 46.0447 | hit correction : 1.23513
hit initial : 316.869 | hit correction : 1.23492
hit initial : 69.0193 | hit correction : 1.23484
hit initial : 129.411 | hit correction : 1.23534
hit initial : 148.463 | hit correction : 1.23552
hit initial : 109.625 | hit correction : 1.23542
hit initial : 189.395 | hit correction : 1.23588
hit initial : 299.949 | hit correction : 1.23645
hit initial : 402.141 | hit correction : 1.23718
hit initial : 823.914 | hit correction : 1.23846
hit initial : 220.026 | hit correction : 1.23829
hit initial : 462.462 | hit correction : 1.24003
hit initial : 354.546 | hit correction : 1.24123
hit initial : 263.033 | hit correction : 1.24216
hit initial : 202.707 | hit correction : 1.24291
hit initial : 43.4896 | hit correction : 1.24317
 *** Total Charge : 6338.85
 Looking at view: 2 | Number of hits in view: 30
Begin processing the 10th record. run: 5027 subRun: 0 event: 27620 at 13-Dec-2025 19:30:15 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  5
This pfp has size:  7
Added hit list length: 7 | Max was set to: -3.40282e+38
This pfp has size:  6
True Neutron Energy: 939.577
True Electron Energy: 2.05056
Completed MCParticle Loop
True Neutrino Energy: 5.33411
 Size of all hits: 7
hit initial : 58.4177 | hit correction : 1.21594
hit initial : 103.691 | hit correction : 1.21594
hit initial : 43.8175 | hit correction : 1.2159
 *** Total Charge : 250.391
 Looking at view: 0 | Number of hits in view: 3
hit initial : 80.6912 | hit correction : 1.21593
hit initial : 77.8561 | hit correction : 1.21593
 *** Total Charge : 192.782
 Looking at view: 1 | Number of hits in view: 2
hit initial : 162.13 | hit correction : 1.21594
hit initial : 26.6618 | hit correction : 1.21595
 *** Total Charge : 229.56
 Looking at view: 2 | Number of hits in view: 2
13-Dec-2025 19:30:20 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.136567      0.174467      0.254524      0.171389      0.0311381       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000663693   0.000808489    0.0014706    0.000756501   0.000224567      10     
end_path:ana:AnalyzeEvents         0.135705      0.173411      0.252358      0.170442      0.030819        10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 9815.71 MB
  Peak resident set size usage (VmHWM): 1543.03 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 12:22:55 UTC       justIN version: 01.06.00