justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 267036.0@dunegpschedd01.fnal.gov

Jobsub ID267036.0@dunegpschedd01.fnal.gov
Workflow ID11112
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-11 21:52:38
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-12-12 03:03:11
From worker nodeHostnamedunegli-7875980-0-fnpc19111.fnal.gov
cpuinfoAMD EPYC 7502 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-11 21:54:12
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T025648Z_gen_004428_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time5h (18522s)
CPU time5h (18380s = 99%)
Max RSS bytes2918834176 (2783 MiB)
Outputting started2025-12-12 03:02:54
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11112/1/001/analysisLowE_20251211T215413Z.root
Finished2025-12-12 03:03:11
Saved logsjustin-logs:267036.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

C run: 5027 subRun: 0 event: 44280
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  12-Dec-2025 03:01:53 UTC run: 5027 subRun: 0 event: 44280
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
Boundary wire vector sizes: 6905, 6826, 6646
minwire 0: 22
minwire 1: 1
minwire 2: 2
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 1260, 1759
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.108934, 
Output 1: 0.0579827, 0.0939891, 0.223053, 0.624975, 
Output 2: 0.0187356, 0.0630241, 0.91208, 0.00615979, 
Output 3: 0.97607, 0.0164328, 0.00185808, 0.00563885, 
Output 4: 0.240442, 0.629827, 0.111425, 0.0183069, 
Output 5: 0.306293, 0.446406, 0.189379, 0.0579216, 
Output 6: 0.0338517, 0.087399, 0.0853687, 0.793381, 

12-Dec-2025 03:01:56 UTC  Closed output file "matt_reco2.root"
12-Dec-2025 03:01:56 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/d0/3c/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T025648Z_gen_004428_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1734.26       1836.33       1952.99       1818.35       60.7904        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00175666    0.00663724     0.0202255    0.00375984     0.0063251       10     
reco:pandora:StandardPandora                            1730.92       1834.43       1951.46       1816.43       61.1433        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.00193033     0.0197725     0.0806365    0.00660072     0.0235906       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.00156874    0.00606688     0.0190077    0.00554074    0.00471464       10     
reco:pandoracalo:Calorimetry                          0.000201362   0.00712298     0.0280811    0.00161104     0.0103713       10     
reco:pandorapid:Chi2ParticleID                        0.000100701   0.000313345   0.000883149   0.000187218   0.000275796      10     
reco:cvnmap:CVNMapper                                  0.0304024     0.0419277     0.0517466     0.0415474    0.00641983       10     
reco:cvneva:CVNEvaluator                                0.89681       1.26381       2.57565       1.15713      0.468103        10     
reco:energyrecnumu:EnergyReco                         0.00931848     0.0761312     0.121626       0.10004      0.0434953       10     
reco:energyrecnue:EnergyReco                          0.00140218    0.00734989     0.0115163     0.010853     0.00464548       10     
reco:energyrecnc:EnergyReco                            0.0105146     0.0347688     0.0939831     0.0111141     0.0363067       10     
[art]:TriggerResults:TriggerResultInserter             1.477e-05    2.68571e-05   9.0931e-05     1.94e-05     2.17198e-05      10     
end_path:out1:RootOutput                               7.461e-06    1.21863e-05   2.9601e-05     8.755e-06    6.65182e-06      10     
end_path:out1:RootOutput(write)                        0.402626      0.430218      0.459316      0.427297      0.0167046       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21686.5 MB
  Peak resident set size usage (VmHWM): 2918.83 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 160.447 | hit correction : 1.37509
hit initial : 174.229 | hit correction : 1.37527
hit initial : 201.993 | hit correction : 1.37543
hit initial : 202.928 | hit correction : 1.37568
hit initial : 181.75 | hit correction : 1.3759
hit initial : 156.334 | hit correction : 1.37606
hit initial : 156.735 | hit correction : 1.37619
hit initial : 155.987 | hit correction : 1.37633
hit initial : 177.673 | hit correction : 1.37647
hit initial : 177.577 | hit correction : 1.37661
hit initial : 176.288 | hit correction : 1.37682
hit initial : 183.358 | hit correction : 1.37708
hit initial : 187.595 | hit correction : 1.37739
hit initial : 179.52 | hit correction : 1.37771
hit initial : 169.222 | hit correction : 1.37808
hit initial : 166.589 | hit correction : 1.37839
hit initial : 107.9 | hit correction : 1.37858
hit initial : 30.312 | hit correction : 1.37867
 *** Total Charge : 5914.18
 Looking at view: 1 | Number of hits in view: 27
hit initial : 43.5781 | hit correction : 1.37374
hit initial : 152.086 | hit correction : 1.37488
hit initial : 316.38 | hit correction : 1.37495
hit initial : 271.815 | hit correction : 1.37517
hit initial : 201.433 | hit correction : 1.37538
hit initial : 172.57 | hit correction : 1.37566
hit initial : 163.275 | hit correction : 1.37585
hit initial : 151.68 | hit correction : 1.376
hit initial : 162.153 | hit correction : 1.37612
hit initial : 158.144 | hit correction : 1.37625
hit initial : 153.298 | hit correction : 1.3764
hit initial : 179.458 | hit correction : 1.37652
hit initial : 202.035 | hit correction : 1.37671
hit initial : 184.371 | hit correction : 1.37696
hit initial : 176.213 | hit correction : 1.37728
hit initial : 160.621 | hit correction : 1.37756
hit initial : 143.397 | hit correction : 1.37791
hit initial : 155.23 | hit correction : 1.37822
hit initial : 161.395 | hit correction : 1.37849
hit initial : 81.6058 | hit correction : 1.37867
 *** Total Charge : 4666.87
 Looking at view: 2 | Number of hits in view: 20
Begin processing the 10th record. run: 5027 subRun: 0 event: 44280 at 12-Dec-2025 03:02:50 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  3
This pfp has size:  14
Added hit list length: 14 | Max was set to: -3.40282e+38
This pfp has size:  22
Added hit list length: 22 | Max was set to: 14
True Neutron Energy: 939.572
True Electron Energy: 28.8206
Completed MCParticle Loop
True Neutrino Energy: 34.2316
 Size of all hits: 22
hit initial : 119.105 | hit correction : 1.26256
hit initial : 145.248 | hit correction : 1.26258
hit initial : 179.728 | hit correction : 1.26171
hit initial : 208.482 | hit correction : 1.26189
hit initial : 53.5643 | hit correction : 1.26198
hit initial : 83.3402 | hit correction : 1.26241
hit initial : 43.2957 | hit correction : 1.26166
 *** Total Charge : 1051.04
 Looking at view: 0 | Number of hits in view: 7
hit initial : 81.2316 | hit correction : 1.26236
hit initial : 123.343 | hit correction : 1.26181
hit initial : 68.1891 | hit correction : 1.2625
hit initial : 88.2614 | hit correction : 1.26169
hit initial : 95.1551 | hit correction : 1.26254
hit initial : 54.4498 | hit correction : 1.26168
 *** Total Charge : 644.463
 Looking at view: 1 | Number of hits in view: 6
hit initial : 33.9679 | hit correction : 1.26249
hit initial : 160.186 | hit correction : 1.26254
hit initial : 319.424 | hit correction : 1.26249
hit initial : 99.4884 | hit correction : 1.26174
hit initial : 120.734 | hit correction : 1.26312
hit initial : 429.24 | hit correction : 1.26179
hit initial : 269.003 | hit correction : 1.26348
hit initial : 39.7806 | hit correction : 1.26174
hit initial : 353.385 | hit correction : 1.26361
 *** Total Charge : 2304.65
 Looking at view: 2 | Number of hits in view: 9
12-Dec-2025 03:02:54 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0998749     0.115936      0.131263      0.117516     0.00943203       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000430984   0.000671852    0.0015526    0.000520495   0.000325476      10     
end_path:ana:AnalyzeEvents         0.099281      0.115055      0.130647      0.116688     0.00950764       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19480.7 MB
  Peak resident set size usage (VmHWM): 1543.38 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 14:58:55 UTC       justIN version: 01.06.00