justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 261358.0@dunegpschedd02.fnal.gov

Jobsub ID261358.0@dunegpschedd02.fnal.gov
Workflow ID11115
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-11 23:42:43
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-12-12 01:31:37
From worker nodeHostnamedunegli-8033740-0-fnpc22020.fnal.gov
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-11 23:44:24
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T013152Z_gen_003112_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time1h (6418s)
CPU time1h (6365s = 99%)
Max RSS bytes2935525376 (2799 MiB)
Outputting started2025-12-12 01:31:23
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11115/1/001/analysisLowE_20251211T234424Z.root
Finished2025-12-12 01:31:37
Saved logsjustin-logs:261358.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

.697898, 0.0310143, 
Output 3: 0.96798, 0.0199209, 0.0034464, 0.00865282, 
Output 4: 0.545809, 0.403594, 0.0454635, 0.00513414, 
Output 5: 0.318681, 0.457971, 0.183424, 0.0399238, 
Output 6: 0.0327501, 0.0806946, 0.0916349, 0.79492, 

Begin processing the 10th record. run: 5027 subRun: 0 event: 31120 at 12-Dec-2025 01:20:05 UTC
Error: A CaloHitList is empty
Error: A CaloHitList is empty
Boundary wire vector sizes: 6868, 6753, 6659
minwire 0: 19
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.174002, 
Output 1: 0.0887261, 0.0624649, 0.151099, 0.69771, 
Output 2: 0.0345482, 0.153819, 0.792421, 0.0192117, 
Output 3: 0.979191, 0.0110019, 0.00177971, 0.0080273, 
Output 4: 0.236016, 0.676848, 0.0803348, 0.00680076, 
Output 5: 0.272615, 0.460756, 0.213704, 0.0529258, 
Output 6: 0.0258693, 0.061711, 0.0793911, 0.833029, 

12-Dec-2025 01:30:53 UTC  Closed output file "matt_reco2.root"
12-Dec-2025 01:30:53 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/d4/60/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T013152Z_gen_003112_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              620.942       632.884       654.133       630.082       10.6561        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00098585    0.00424349    0.00928932    0.00355054    0.00220854       10     
reco:pandora:StandardPandora                            619.949       631.787       653.162       628.545       10.6537        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.00128218    0.00847683     0.0479092    0.00336905     0.0137534       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.00110875    0.00457291     0.017719     0.00270571    0.00499399       10     
reco:pandoracalo:Calorimetry                          0.000169821   0.00497846     0.0184841    0.00248228    0.00679379       10     
reco:pandorapid:Chi2ParticleID                        3.9745e-05    0.000179171   0.000786334   7.2632e-05    0.000216612      10     
reco:cvnmap:CVNMapper                                  0.0146352     0.0198775     0.0291836     0.0192422    0.00391777       10     
reco:cvneva:CVNEvaluator                               0.669413      0.784129       1.70122      0.678297      0.305846        10     
reco:energyrecnumu:EnergyReco                         0.00182723     0.0476174     0.0690472     0.0541898     0.0197903       10     
reco:energyrecnue:EnergyReco                          0.000795591   0.00282047    0.00508312    0.00275299    0.00195472       10     
reco:energyrecnc:EnergyReco                           0.00448855     0.0164007     0.0705185    0.00471651     0.0236436       10     
[art]:TriggerResults:TriggerResultInserter             8.406e-06    1.48289e-05   5.3381e-05     9.853e-06    1.30004e-05      10     
end_path:out1:RootOutput                               4.469e-06    6.6818e-06    1.8395e-05    5.5255e-06    3.95627e-06      10     
end_path:out1:RootOutput(write)                        0.196827      0.202667      0.216788      0.202089     0.00519509       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21691.9 MB
  Peak resident set size usage (VmHWM): 2935.53 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 22.3716 | hit correction : 1.14718
hit initial : 83.4933 | hit correction : 1.14714
hit initial : 61.1927 | hit correction : 1.14712
hit initial : 48.7047 | hit correction : 1.14645
hit initial : 133.102 | hit correction : 1.14651
hit initial : 235.928 | hit correction : 1.14679
hit initial : 463.685 | hit correction : 1.14666
hit initial : 224.137 | hit correction : 1.14687
hit initial : 233.268 | hit correction : 1.1469
hit initial : 233.867 | hit correction : 1.1469
hit initial : 215.133 | hit correction : 1.14693
hit initial : 208.398 | hit correction : 1.14704
hit initial : 219.598 | hit correction : 1.14722
hit initial : 220.861 | hit correction : 1.14738
hit initial : 216.832 | hit correction : 1.1475
hit initial : 222.569 | hit correction : 1.14755
hit initial : 225.188 | hit correction : 1.14759
hit initial : 213.598 | hit correction : 1.14768
hit initial : 194.956 | hit correction : 1.14776
hit initial : 197.884 | hit correction : 1.14778
hit initial : 219.584 | hit correction : 1.14774
hit initial : 244.625 | hit correction : 1.14765
hit initial : 239.473 | hit correction : 1.14753
hit initial : 143.992 | hit correction : 1.14745
hit initial : 31.168 | hit correction : 1.14742
 *** Total Charge : 5453.5
 Looking at view: 1 | Number of hits in view: 25
hit initial : 36.797 | hit correction : 1.14718
hit initial : 254.514 | hit correction : 1.14718
hit initial : 165.135 | hit correction : 1.14709
hit initial : 138.755 | hit correction : 1.14694
hit initial : 144.059 | hit correction : 1.14675
hit initial : 203.909 | hit correction : 1.14648
hit initial : 204.507 | hit correction : 1.14648
hit initial : 135.881 | hit correction : 1.14644
hit initial : 154.578 | hit correction : 1.14669
hit initial : 164.934 | hit correction : 1.14653
hit initial : 148.84 | hit correction : 1.14684
hit initial : 155.818 | hit correction : 1.14688
hit initial : 186.758 | hit correction : 1.14688
hit initial : 167.837 | hit correction : 1.1469
hit initial : 129.912 | hit correction : 1.1469
hit initial : 141.657 | hit correction : 1.14689
hit initial : 139.818 | hit correction : 1.14692
hit initial : 139.426 | hit correction : 1.14699
hit initial : 141.721 | hit correction : 1.14712
hit initial : 169.013 | hit correction : 1.14725
hit initial : 144.454 | hit correction : 1.14738
hit initial : 126.576 | hit correction : 1.14748
hit initial : 132.218 | hit correction : 1.14753
hit initial : 123.833 | hit correction : 1.14755
hit initial : 128.642 | hit correction : 1.14755
hit initial : 147.078 | hit correction : 1.14757
hit initial : 152.572 | hit correction : 1.14765
hit initial : 145.046 | hit correction : 1.14772
hit initial : 137.146 | hit correction : 1.14778
hit initial : 136.029 | hit correction : 1.14781
hit initial : 125.858 | hit correction : 1.1478
 *** Total Charge : 5303.42
 Looking at view: 2 | Number of hits in view: 31
Begin processing the 8th record. run: 5027 subRun: 0 event: 31118 at 12-Dec-2025 01:31:17 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Electron Energy: 32.542
Completed MCParticle Loop
True Neutrino Energy: 37.9659
 Size of all hits: 0
Begin processing the 9th record. run: 5027 subRun: 0 event: 31119 at 12-Dec-2025 01:31:19 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
True Electron Energy: 59.278
Completed MCParticle Loop
True Neutrino Energy: 63.1172
 Size of all hits: 0
Begin processing the 10th record. run: 5027 subRun: 0 event: 31120 at 12-Dec-2025 01:31:21 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.577
True Electron Energy: 3.914
Completed MCParticle Loop
True Neutrino Energy: 7.63851
 Size of all hits: 0
12-Dec-2025 01:31:22 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0467997     0.0591147     0.0842684     0.0576373     0.0123735       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000324522   0.000480506   0.000730519   0.000413521   0.000140914      10     
end_path:ana:AnalyzeEvents          0.04636      0.0584963     0.0831188     0.0570283     0.0122231       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19477.5 MB
  Peak resident set size usage (VmHWM): 1538.28 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 14:59:41 UTC       justIN version: 01.06.00