justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 249101.10@dunegpschedd02.fnal.gov

Jobsub ID249101.10@dunegpschedd02.fnal.gov
Workflow ID10433
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-11-21 08:20:11
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-11-21 10:07:32
From worker nodeHostnamedunegli-7677209-0-fnpc22020.fnal.gov
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-21 08:21:21
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T194331Z_gen_000990_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time1h (6358s)
CPU time1h (6170s = 97%)
Max RSS bytes2922266624 (2786 MiB)
Outputting started2025-11-21 10:07:19
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/10433/1/001/analysisLowE_20251121T082121Z.root
Finished2025-11-21 10:07:32
Saved logsjustin-logs:249101.10-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

utput 2: 0.0498449, 0.198931, 0.718088, 0.0331365, 
Output 3: 0.953281, 0.0191189, 0.00618052, 0.02142, 
Output 4: 0.778182, 0.195614, 0.0219194, 0.00428485, 
Output 5: 0.766735, 0.186757, 0.0354768, 0.0110312, 
Output 6: 0.0165452, 0.0414714, 0.0500936, 0.89189, 

Begin processing the 10th record. run: 3799 subRun: 0 event: 9900 at 21-Nov-2025 09:56:43 UTC
Error: A CaloHitList is empty
Boundary wire vector sizes: 6712, 6676, 6638
minwire 0: 10
minwire 1: 1
minwire 2: 3
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 1818, 2317
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.160374, 
Output 1: 0.0860275, 0.0484675, 0.111144, 0.754361, 
Output 2: 0.13221, 0.243946, 0.571912, 0.0519327, 
Output 3: 0.976871, 0.00737741, 0.00150745, 0.0142438, 
Output 4: 0.395591, 0.557903, 0.0418738, 0.00463254, 
Output 5: 0.584251, 0.361687, 0.0424992, 0.0115622, 
Output 6: 0.0172115, 0.0435197, 0.0621455, 0.877123, 

21-Nov-2025 10:06:54 UTC  Closed output file "matt_reco2.root"
21-Nov-2025 10:06:54 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/ea/93/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T194331Z_gen_000990_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              597.329       612.658       708.292       601.354       32.139         10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00137962    0.00593601     0.0209125    0.00350652     0.0060551       10     
reco:pandora:StandardPandora                            596.357       611.474       706.052       600.02        31.7925        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.00139846    0.00969586     0.0535629    0.00343994     0.0152897       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.00169592    0.00324818     0.0086203    0.00273276    0.00201695       10     
reco:pandoracalo:Calorimetry                          0.000182534   0.00510128     0.0160222    0.00146885    0.00644253       10     
reco:pandorapid:Chi2ParticleID                        7.3489e-05    0.000321194   0.00067825    0.000281401    0.000171        10     
reco:cvnmap:CVNMapper                                  0.0158752     0.0186592     0.0278386     0.0175767    0.00341173       10     
reco:cvneva:CVNEvaluator                               0.673447      0.788263       1.78516       0.6778       0.332309        10     
reco:energyrecnumu:EnergyReco                         0.00504864     0.135697      0.758337      0.0619942      0.20983        10     
reco:energyrecnue:EnergyReco                          0.000756558   0.00392032    0.00496438     0.0045063    0.00136781       10     
reco:energyrecnc:EnergyReco                           0.00441377    0.00977014     0.0571493    0.00452091     0.0157931       10     
[art]:TriggerResults:TriggerResultInserter             8.025e-06    1.16702e-05   3.3704e-05     9.248e-06    7.41263e-06      10     
end_path:out1:RootOutput                               4.248e-06    6.5534e-06    1.9387e-05     5.32e-06     4.2969e-06       10     
end_path:out1:RootOutput(write)                        0.201287      0.203361      0.212422      0.202286     0.00308885       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21681.5 MB
  Peak resident set size usage (VmHWM): 2922.27 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 270.98 | hit correction : 1.0695
 *** Total Charge : 2863.45
 Looking at view: 0 | Number of hits in view: 14
hit initial : 229.14 | hit correction : 1.06956
hit initial : 685.341 | hit correction : 1.06953
hit initial : 57.1307 | hit correction : 1.06933
hit initial : 56.2656 | hit correction : 1.06749
hit initial : 144.109 | hit correction : 1.06751
hit initial : 31.6981 | hit correction : 1.068
hit initial : 313.102 | hit correction : 1.06922
hit initial : 191.586 | hit correction : 1.06928
hit initial : 570.295 | hit correction : 1.06939
 *** Total Charge : 2436.42
 Looking at view: 1 | Number of hits in view: 9
hit initial : 140.5 | hit correction : 1.06965
hit initial : 211.43 | hit correction : 1.06957
hit initial : 303.348 | hit correction : 1.06951
hit initial : 194.922 | hit correction : 1.06949
hit initial : 152.928 | hit correction : 1.06955
hit initial : 132.024 | hit correction : 1.06954
hit initial : 136.815 | hit correction : 1.06913
hit initial : 165.682 | hit correction : 1.06914
hit initial : 198.646 | hit correction : 1.06748
hit initial : 48.4162 | hit correction : 1.06757
hit initial : 155.952 | hit correction : 1.06921
hit initial : 56.6085 | hit correction : 1.06936
hit initial : 153.673 | hit correction : 1.06931
hit initial : 154.256 | hit correction : 1.06925
hit initial : 140.697 | hit correction : 1.06941
 *** Total Charge : 2508.25
 Looking at view: 2 | Number of hits in view: 15
Begin processing the 4th record. run: 3799 subRun: 0 event: 9894 at 21-Nov-2025 10:07:08 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Electron Energy: 30.765
Completed MCParticle Loop
True Neutrino Energy: 36.1605
 Size of all hits: 0
Begin processing the 5th record. run: 3799 subRun: 0 event: 9895 at 21-Nov-2025 10:07:09 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Electron Energy: 5.97691
Completed MCParticle Loop
True Neutrino Energy: 9.26069
 Size of all hits: 0
Begin processing the 6th record. run: 3799 subRun: 0 event: 9896 at 21-Nov-2025 10:07:11 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.576
True Electron Energy: 17.7708
Completed MCParticle Loop
True Neutrino Energy: 21.0662
 Size of all hits: 0
Begin processing the 7th record. run: 3799 subRun: 0 event: 9897 at 21-Nov-2025 10:07:12 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Electron Energy: 44.8933
Completed MCParticle Loop
True Neutrino Energy: 48.2065
 Size of all hits: 0
Begin processing the 8th record. run: 3799 subRun: 0 event: 9898 at 21-Nov-2025 10:07:14 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Electron Energy: 1.94785
Completed MCParticle Loop
True Neutrino Energy: 5.2315
 Size of all hits: 0
Begin processing the 9th record. run: 3799 subRun: 0 event: 9899 at 21-Nov-2025 10:07:16 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Electron Energy: 30.7601
Completed MCParticle Loop
True Neutrino Energy: 53.7817
 Size of all hits: 0
Begin processing the 10th record. run: 3799 subRun: 0 event: 9900 at 21-Nov-2025 10:07:17 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.569
True Electron Energy: 7.03872
Completed MCParticle Loop
True Neutrino Energy: 10.3224
 Size of all hits: 0
21-Nov-2025 10:07:19 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0454009     0.0514192     0.0609633     0.0501527    0.00512981       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000294325   0.000361821   0.000461451   0.000361802   4.81649e-05      10     
end_path:ana:AnalyzeEvents         0.0449728     0.0509162     0.0604186     0.0497547    0.00509609       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19470.5 MB
  Peak resident set size usage (VmHWM): 1529.89 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-09 01:58:41 UTC       justIN version: 01.06.00