justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 262211.0@dunegpschedd02.fnal.gov

Jobsub ID262211.0@dunegpschedd02.fnal.gov
Workflow ID11174
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-13 13:34:40
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-12-13 18:25:06
From worker nodeHostnamedunegli-7890608-0-fnpc17112.fnal.gov
cpuinfoIntel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-13 13:36:31
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T234000Z_gen_004867_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time4h (17306s)
CPU time4h (17142s = 99%)
Max RSS bytes2791526400 (2662 MiB)
Outputting started2025-12-13 18:24:57
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11174/1/001/analysisLowE_20251213T133632Z.root
Finished2025-12-13 18:25:06
Saved logsjustin-logs:262211.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

biston/18_10_24_Official_LowE_Build/srcs/larpandoracontent/larpandoracontent/LArMonitoring/EventValidationBaseAlgorithm.cc line#: 64
iter->second->Run() throw STATUS_CODE_NOT_INITIALIZED
    in function: RunAlgorithm
    in file:     /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0015, LArNeutrinoEventValidation, STATUS_CODE_NOT_INITIALIZED
Boundary wire vector sizes: 6853, 6769, 6745
minwire 0: 51
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 137, 636
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0918468, 
Output 1: 0.0505931, 0.0582445, 0.13772, 0.753442, 
Output 2: 0.0459378, 0.203392, 0.727262, 0.0234087, 
Output 3: 0.92092, 0.0661597, 0.00743076, 0.00548948, 
Output 4: 0.251872, 0.635633, 0.104532, 0.00796264, 
Output 5: 0.538314, 0.345349, 0.0964134, 0.0199227, 
Output 6: 0.0361429, 0.116507, 0.111861, 0.735489, 

13-Dec-2025 18:23:59 UTC  Closed output file "matt_reco2.root"
13-Dec-2025 18:23:59 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/d4/d9/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T234000Z_gen_004867_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1368.17       1702.76       2944.93       1453.95       493.361        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000658153   0.00333469     0.0152705    0.000802809   0.00509695       10     
reco:pandora:StandardPandora                            1366.71       1700.49       2936.91       1452.48       491.646        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.00223747     0.0497766     0.125351      0.0561437     0.0358928       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.00209112     0.0133002     0.0469866     0.0057175     0.0155787       10     
reco:pandoracalo:Calorimetry                          0.000351763   0.00195978     0.0100373    0.000655511    0.0029558       10     
reco:pandorapid:Chi2ParticleID                        7.4007e-05    0.000531933   0.00417603    0.000112818   0.00121559       10     
reco:cvnmap:CVNMapper                                  0.0924134     0.114412       0.16372      0.105025      0.0232061       10     
reco:cvneva:CVNEvaluator                               0.848765       1.56603       6.6308       0.884276       1.70039        10     
reco:energyrecnumu:EnergyReco                          0.0681039     0.101394      0.235361      0.0852843     0.0477321       10     
reco:energyrecnue:EnergyReco                          0.00134123    0.00193326    0.00537991     0.0014506    0.00118038       10     
reco:energyrecnc:EnergyReco                           0.00801715     0.0100274     0.0157878    0.00884548    0.00268056       10     
[art]:TriggerResults:TriggerResultInserter            1.5318e-05    3.96661e-05   0.000223709   1.74475e-05   6.1561e-05       10     
end_path:out1:RootOutput                               6.038e-06    1.25748e-05   6.0971e-05    6.8625e-06    1.61634e-05      10     
end_path:out1:RootOutput(write)                        0.295556      0.414038       0.78852      0.329958      0.168312        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 14054.8 MB
  Peak resident set size usage (VmHWM): 2791.53 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 118.194 | hit correction : 1.00877
 *** Total Charge : 119.231
 Looking at view: 1 | Number of hits in view: 1
hit initial : 42.8515 | hit correction : 1.0273
hit initial : 197.43 | hit correction : 1.00878
 *** Total Charge : 243.184
 Looking at view: 2 | Number of hits in view: 2
Begin processing the 9th record. run: 5026 subRun: 0 event: 48669 at 13-Dec-2025 18:24:49 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  4
This pfp has size:  10
Added hit list length: 10 | Max was set to: -3.40282e+38
This pfp has size:  9
True Electron Energy: 6.64628
Completed MCParticle Loop
True Neutrino Energy: 12.8333
 Size of all hits: 10
hit initial : 29.9807 | hit correction : 1.01641
hit initial : 45.4878 | hit correction : 1.01255
hit initial : 471.092 | hit correction : 1.01349
 *** Total Charge : 553.98
 Looking at view: 0 | Number of hits in view: 3
hit initial : 33.3754 | hit correction : 1.01252
hit initial : 231.27 | hit correction : 1.0135
hit initial : 135.143 | hit correction : 1.01343
 *** Total Charge : 405.143
 Looking at view: 1 | Number of hits in view: 3
hit initial : 39.2983 | hit correction : 1.01644
hit initial : 57.8498 | hit correction : 1.01252
hit initial : 313.302 | hit correction : 1.01354
hit initial : 145.062 | hit correction : 1.01334
 *** Total Charge : 563.059
 Looking at view: 2 | Number of hits in view: 4
Begin processing the 10th record. run: 5026 subRun: 0 event: 48670 at 13-Dec-2025 18:24:52 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  4
This pfp has size:  31
Added hit list length: 31 | Max was set to: -3.40282e+38
This pfp has size:  18
True Neutron Energy: 939.578
True Neutron Energy: 940.127
True Electron Energy: 52.4198
Completed MCParticle Loop
True Neutrino Energy: 69.8094
 Size of all hits: 31
hit initial : 132.062 | hit correction : 1.01785
hit initial : 134.595 | hit correction : 1.01783
hit initial : 298.806 | hit correction : 1.0103
hit initial : 494.608 | hit correction : 1.0104
hit initial : 367.741 | hit correction : 1.01047
hit initial : 258.672 | hit correction : 1.01041
hit initial : 169.302 | hit correction : 1.01032
hit initial : 421.645 | hit correction : 1.00679
hit initial : 209.557 | hit correction : 1.00662
hit initial : 381.199 | hit correction : 1.00695
hit initial : 147.826 | hit correction : 1.00572
 *** Total Charge : 3045.03
 Looking at view: 0 | Number of hits in view: 11
hit initial : 20.9053 | hit correction : 1.03276
hit initial : 167.162 | hit correction : 1.01783
hit initial : 397.211 | hit correction : 1.01037
hit initial : 715.321 | hit correction : 1.01041
hit initial : 431.051 | hit correction : 1.01038
hit initial : 491.451 | hit correction : 1.0068
hit initial : 325.854 | hit correction : 1.00692
hit initial : 166.052 | hit correction : 1.00572
 *** Total Charge : 2741.26
 Looking at view: 1 | Number of hits in view: 8
hit initial : 34.5647 | hit correction : 1.03791
hit initial : 27.8617 | hit correction : 1.03277
hit initial : 306.498 | hit correction : 1.01783
hit initial : 708.888 | hit correction : 1.01029
hit initial : 401.361 | hit correction : 1.01037
hit initial : 451.811 | hit correction : 1.0106
hit initial : 99.293 | hit correction : 1.01025
hit initial : 418.366 | hit correction : 1.00683
hit initial : 480.002 | hit correction : 1.00656
hit initial : 476.046 | hit correction : 1.00697
hit initial : 403.311 | hit correction : 1.00598
hit initial : 335.522 | hit correction : 1.00565
 *** Total Charge : 4182.11
 Looking at view: 2 | Number of hits in view: 12
13-Dec-2025 18:24:56 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0894071     0.211953       0.59208      0.124218      0.171639        10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000325636    0.0494675     0.476742     0.000448649    0.142486        10     
end_path:ana:AnalyzeEvents         0.0888658     0.162342      0.494765      0.118755      0.116196        10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 11958.5 MB
  Peak resident set size usage (VmHWM): 1535.93 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 12:06:21 UTC       justIN version: 01.06.00