justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 267800.1@dunegpschedd01.fnal.gov

Jobsub ID267800.1@dunegpschedd01.fnal.gov
Workflow ID11171
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-13 06:00:17
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-12-13 18:38:03
From worker nodeHostnamedunegli-7890610-0-fnpc9038.fnal.gov
cpuinfoIntel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-13 06:02:06
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250116T130643Z_gen_000155_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time12h (45339s)
CPU time12h (43756s = 96%)
Max RSS bytes2813440000 (2683 MiB)
Outputting started2025-12-13 18:37:46
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11171/1/001/analysisLowE_20251213T060208Z.root
Finished2025-12-13 18:38:03
Saved logsjustin-logs:267800.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

r:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  13-Dec-2025 18:36:25 UTC run: 4696 subRun: 0 event: 1550
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  13-Dec-2025 18:36:25 UTC run: 4696 subRun: 0 event: 1550
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
Boundary wire vector sizes: 6931, 6860, 6772
minwire 0: 6
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 146, 645
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.107637, 
Output 1: 0.0571867, 0.0741541, 0.110325, 0.758335, 
Output 2: 0.0485512, 0.25085, 0.670977, 0.0296217, 
Output 3: 0.969478, 0.017531, 0.003398, 0.00959319, 
Output 4: 0.54496, 0.393386, 0.0553195, 0.00633496, 
Output 5: 0.659059, 0.277847, 0.0521581, 0.0109362, 
Output 6: 0.00872159, 0.0296611, 0.0478841, 0.913733, 

13-Dec-2025 18:36:29 UTC  Closed output file "matt_reco2.root"
13-Dec-2025 18:36:29 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/e0/0d/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250116T130643Z_gen_000155_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2454.73       4493.97       5970.4        5038.33       1179.01        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00160333    0.00623384     0.0218807    0.00274501    0.00742329       10     
reco:pandora:StandardPandora                            2452.16       4489.71       5957.13       5034.77       1177.37        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.00162194     0.133818      0.266233      0.152295      0.092617        10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.00304111     0.0142253     0.0728647    0.00729342     0.0199771       10     
reco:pandoracalo:Calorimetry                          0.000490606   0.00348926     0.0200954    0.00154115     0.0055964       10     
reco:pandorapid:Chi2ParticleID                        0.000246332   0.00105605    0.00748117    0.00034086     0.0021429       10     
reco:cvnmap:CVNMapper                                  0.0938343     0.184359      0.298073      0.187509      0.063556        10     
reco:cvneva:CVNEvaluator                                1.73647       3.08632       11.4997       2.33152       2.8162         10     
reco:energyrecnumu:EnergyReco                          0.124005      0.185738      0.378648       0.1548       0.0703234       10     
reco:energyrecnue:EnergyReco                          0.00298049    0.00556948     0.0139082    0.00462062    0.00307976       10     
reco:energyrecnc:EnergyReco                            0.0134843     0.0165409     0.019458      0.0171059    0.00202563       10     
[art]:TriggerResults:TriggerResultInserter            2.9241e-05    7.88506e-05   0.000282446   6.14465e-05   6.96814e-05      10     
end_path:out1:RootOutput                              1.0987e-05    2.37959e-05   7.4833e-05    1.8944e-05    1.77913e-05      10     
end_path:out1:RootOutput(write)                        0.499992      0.618007      0.933173      0.558443      0.136818        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 11919.5 MB
  Peak resident set size usage (VmHWM): 2813.44 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 115.209 | hit correction : 1.30046
hit initial : 45.8702 | hit correction : 1.30093
hit initial : 77.7414 | hit correction : 1.29741
hit initial : 118.92 | hit correction : 1.30098
hit initial : 143.451 | hit correction : 1.30346
hit initial : 586.038 | hit correction : 1.30346
hit initial : 295.85 | hit correction : 1.30356
hit initial : 194.494 | hit correction : 1.3063
hit initial : 167.665 | hit correction : 1.30579
hit initial : 75.5902 | hit correction : 1.30372
hit initial : 64.8225 | hit correction : 1.30743
hit initial : 172.173 | hit correction : 1.30739
hit initial : 142.652 | hit correction : 1.30544
hit initial : 140.032 | hit correction : 1.30715
hit initial : 132.5 | hit correction : 1.30526
hit initial : 130.529 | hit correction : 1.30517
hit initial : 130.427 | hit correction : 1.30519
hit initial : 135.523 | hit correction : 1.30534
hit initial : 130.674 | hit correction : 1.30555
hit initial : 147.095 | hit correction : 1.3058
hit initial : 158.291 | hit correction : 1.30604
hit initial : 153.402 | hit correction : 1.30635
hit initial : 139.401 | hit correction : 1.30666
hit initial : 64.3909 | hit correction : 1.30687
 *** Total Charge : 5050.33
 Looking at view: 2 | Number of hits in view: 29
Begin processing the 9th record. run: 4696 subRun: 0 event: 1549 at 13-Dec-2025 18:37:36 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
True Electron Energy: 50.842
Completed MCParticle Loop
True Neutrino Energy: 56.2201
 Size of all hits: 0
Begin processing the 10th record. run: 4696 subRun: 0 event: 1550 at 13-Dec-2025 18:37:41 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  5
This pfp has size:  28
Added hit list length: 28 | Max was set to: -3.40282e+38
This pfp has size:  14
This pfp has size:  6
This pfp has size:  5
True Neutron Energy: 939.571
True Neutron Energy: 942.255
True Neutron Energy: 939.998
True Electron Energy: 18.2921
Completed MCParticle Loop
True Neutrino Energy: 38.33
 Size of all hits: 28
hit initial : 29.6871 | hit correction : 1.07083
hit initial : 25.6107 | hit correction : 1.07084
hit initial : 37.0763 | hit correction : 1.07221
hit initial : 32.2314 | hit correction : 1.07223
hit initial : 207.54 | hit correction : 1.07719
hit initial : 500.186 | hit correction : 1.07745
hit initial : 278.217 | hit correction : 1.07775
hit initial : 419.646 | hit correction : 1.07834
hit initial : 254.301 | hit correction : 1.07858
 *** Total Charge : 1922.67
 Looking at view: 0 | Number of hits in view: 9
hit initial : 17.0091 | hit correction : 1.07082
hit initial : 32.9936 | hit correction : 1.07222
hit initial : 272.532 | hit correction : 1.07717
hit initial : 410.674 | hit correction : 1.07737
hit initial : 400.224 | hit correction : 1.07783
hit initial : 402.302 | hit correction : 1.07825
hit initial : 340.577 | hit correction : 1.07851
hit initial : 171.094 | hit correction : 1.07873
 *** Total Charge : 2206.64
 Looking at view: 1 | Number of hits in view: 8
hit initial : 48.2268 | hit correction : 1.07082
hit initial : 68.1408 | hit correction : 1.07222
hit initial : 317.24 | hit correction : 1.07712
hit initial : 561.689 | hit correction : 1.07745
hit initial : 215.818 | hit correction : 1.07788
hit initial : 190.113 | hit correction : 1.07809
hit initial : 160.094 | hit correction : 1.0783
hit initial : 216.025 | hit correction : 1.07843
hit initial : 150.598 | hit correction : 1.07856
hit initial : 170.539 | hit correction : 1.07875
hit initial : 84.4017 | hit correction : 1.07896
 *** Total Charge : 2352.24
 Looking at view: 2 | Number of hits in view: 11
13-Dec-2025 18:37:46 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                          0.13346      0.164982      0.211399      0.161888      0.0218361       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000632387   0.00204895     0.0130831    0.000844818   0.00367986       10     
end_path:ana:AnalyzeEvents         0.132384      0.162647      0.197439      0.160832      0.0192763       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 9814.99 MB
  Peak resident set size usage (VmHWM): 1542.29 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 12:06:18 UTC       justIN version: 01.06.00