justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 261350.1@dunegpschedd02.fnal.gov

Jobsub ID261350.1@dunegpschedd02.fnal.gov
Workflow ID11108
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-11 23:22:42
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-12-12 01:30:31
From worker nodeHostnamedunegli-7877514-0-fnpc23037.fnal.gov
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-11 23:22:58
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T232127Z_gen_001557_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time2h (7637s)
CPU time2h (7569s = 99%)
Max RSS bytes2919190528 (2783 MiB)
Outputting started2025-12-12 01:30:16
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11108/1/001/analysisLowE_20251211T232259Z.root
Finished2025-12-12 01:30:31
Saved logsjustin-logs:261350.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

, 0.165966, 0.639459, 0.0361423, 
Output 3: 0.983145, 0.00773321, 0.000800842, 0.00832048, 
Output 4: 0.541543, 0.440703, 0.0156325, 0.00212194, 
Output 5: 0.428903, 0.498437, 0.0551205, 0.0175398, 
Output 6: 0.0276042, 0.0679674, 0.073005, 0.831423, 

Begin processing the 10th record. run: 5027 subRun: 0 event: 15570 at 12-Dec-2025 01:17:16 UTC
Error: A CaloHitList is empty
Boundary wire vector sizes: 6725, 6642, 6628
minwire 0: 31
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 887, 1386
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.12093, 
Output 1: 0.0959781, 0.113118, 0.135865, 0.655039, 
Output 2: 0.0364992, 0.196828, 0.734432, 0.0322408, 
Output 3: 0.966536, 0.0174753, 0.00451838, 0.0114705, 
Output 4: 0.719482, 0.242048, 0.0332, 0.00527024, 
Output 5: 0.387868, 0.353433, 0.201597, 0.0571016, 
Output 6: 0.040515, 0.0756208, 0.0789757, 0.804888, 

12-Dec-2025 01:29:45 UTC  Closed output file "matt_reco2.root"
12-Dec-2025 01:29:45 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/1a/a6/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T232127Z_gen_001557_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              700.733       753.164       833.55        747.674       37.9491        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000690206   0.00558648     0.0205999    0.00404953     0.0055135       10     
reco:pandora:StandardPandora                            699.415       751.818       832.204       746.462       37.9236        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.00134488     0.0111654     0.0555578    0.00309554     0.0163497       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.00101857    0.00519111     0.0292728    0.00292538    0.00808196       10     
reco:pandoracalo:Calorimetry                          0.000162061   0.00347489     0.0145882    0.000680481    0.0050811       10     
reco:pandorapid:Chi2ParticleID                        3.4477e-05    0.000189081   0.000725984    6.77e-05     0.000221683      10     
reco:cvnmap:CVNMapper                                  0.0182197     0.0377632     0.0662833     0.0307563     0.0154636       10     
reco:cvneva:CVNEvaluator                               0.710353      0.931752       2.0419       0.765062      0.382824        10     
reco:energyrecnumu:EnergyReco                          0.0025891     0.0654678     0.122469      0.0757011     0.036944        10     
reco:energyrecnue:EnergyReco                          0.00082793    0.00364312    0.00588207    0.00502103    0.00203419       10     
reco:energyrecnc:EnergyReco                           0.00483042     0.0192302     0.0753931    0.00535406      0.02735        10     
[art]:TriggerResults:TriggerResultInserter             9.028e-06    1.69877e-05   4.6139e-05    1.32105e-05   1.08887e-05      10     
end_path:out1:RootOutput                               3.577e-06    1.29398e-05   6.0376e-05     5.896e-06    1.64348e-05      10     
end_path:out1:RootOutput(write)                        0.212114      0.262223      0.367637       0.2366       0.0575769       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21685.4 MB
  Peak resident set size usage (VmHWM): 2919.19 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 179.963 | hit correction : 1.37268
hit initial : 162.872 | hit correction : 1.37319
hit initial : 165.282 | hit correction : 1.3736
hit initial : 174.168 | hit correction : 1.374
hit initial : 325.259 | hit correction : 1.37425
hit initial : 291.4 | hit correction : 1.37466
hit initial : 192.423 | hit correction : 1.37511
hit initial : 217.921 | hit correction : 1.37543
hit initial : 142.445 | hit correction : 1.37619
hit initial : 149.723 | hit correction : 1.37651
hit initial : 131.314 | hit correction : 1.37673
hit initial : 123.886 | hit correction : 1.3769
hit initial : 119.768 | hit correction : 1.37707
hit initial : 114.068 | hit correction : 1.37724
 *** Total Charge : 4976.28
 Looking at view: 2 | Number of hits in view: 25
Begin processing the 7th record. run: 5027 subRun: 0 event: 15567 at 12-Dec-2025 01:30:08 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
This pfp has size:  29
Added hit list length: 29 | Max was set to: -3.40282e+38
True Electron Energy: 17.3682
Completed MCParticle Loop
True Neutrino Energy: 20.653
 Size of all hits: 29
hit initial : 104.458 | hit correction : 1.11509
hit initial : 247.845 | hit correction : 1.11501
hit initial : 64.104 | hit correction : 1.11392
hit initial : 157.569 | hit correction : 1.11484
hit initial : 62.1144 | hit correction : 1.11366
hit initial : 48.1945 | hit correction : 1.11365
hit initial : 88.9987 | hit correction : 1.11178
hit initial : 140.787 | hit correction : 1.11184
hit initial : 29.34 | hit correction : 1.11235
hit initial : 630.999 | hit correction : 1.11308
hit initial : 82.6318 | hit correction : 1.11308
hit initial : 180.355 | hit correction : 1.11392
 *** Total Charge : 2046.09
 Looking at view: 0 | Number of hits in view: 12
hit initial : 167.917 | hit correction : 1.11489
hit initial : 129.808 | hit correction : 1.11507
hit initial : 66.1416 | hit correction : 1.11509
hit initial : 144.206 | hit correction : 1.11185
hit initial : 362.526 | hit correction : 1.11224
hit initial : 338.597 | hit correction : 1.11308
hit initial : 155.15 | hit correction : 1.1137
hit initial : 157.224 | hit correction : 1.11392
 *** Total Charge : 1694.07
 Looking at view: 1 | Number of hits in view: 8
hit initial : 30.2026 | hit correction : 1.1151
hit initial : 471.82 | hit correction : 1.115
hit initial : 237.751 | hit correction : 1.11432
hit initial : 236.298 | hit correction : 1.11174
hit initial : 23.7812 | hit correction : 1.11177
hit initial : 400.677 | hit correction : 1.1125
hit initial : 242.634 | hit correction : 1.11319
hit initial : 251.85 | hit correction : 1.11362
hit initial : 172.362 | hit correction : 1.11388
 *** Total Charge : 2302.14
 Looking at view: 2 | Number of hits in view: 9
Begin processing the 8th record. run: 5027 subRun: 0 event: 15568 at 12-Dec-2025 01:30:10 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Electron Energy: 45.5854
Completed MCParticle Loop
True Neutrino Energy: 52.2617
 Size of all hits: 0
Begin processing the 9th record. run: 5027 subRun: 0 event: 15569 at 12-Dec-2025 01:30:12 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
True Electron Energy: 21.4786
Completed MCParticle Loop
True Neutrino Energy: 26.8606
 Size of all hits: 0
Begin processing the 10th record. run: 5027 subRun: 0 event: 15570 at 12-Dec-2025 01:30:14 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.574
True Electron Energy: 7.59014
Completed MCParticle Loop
True Neutrino Energy: 23.0601
 Size of all hits: 0
12-Dec-2025 01:30:16 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0493379     0.0577658     0.0702169     0.0532022    0.00775229       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000291249   0.000474838   0.00084916    0.000421454   0.000165983      10     
end_path:ana:AnalyzeEvents         0.0487846     0.0571517     0.069492      0.0526891    0.00759443       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19472.2 MB
  Peak resident set size usage (VmHWM): 1530.18 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 15:00:04 UTC       justIN version: 01.06.00