Jobsub ID 259884.1@dunegpschedd02.fnal.gov
| Jobsub ID | 259884.1@dunegpschedd02.fnal.gov |
| Workflow ID | 11085 |
| Stage ID | 1 |
| User name | osbiston@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 2097152000 (2000 MiB) |
| Wall seconds limit | 80000 (22 hours) |
| Submitted time | 2025-12-09 21:34:11 |
| Site | US_FNAL-FermiGrid |
| Entry | FNAL_GPGrid_ce03_mcore_op_duneonly |
| Last heartbeat | 2025-12-11 06:56:07 |
| From worker node | Hostname | dunegli-7867874-0-fnpc8002.fnal.gov |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2670 v3 @ 2.30GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 2097152000 (2000 MiB) |
| Wall seconds limit | 172800 (48 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2025-12-10 14:00:08 |
| Input files | fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T020055Z_gen_003495_supernova_g4stage1_g4stage2_detsim_reco.root
|
| Jobscript | Exit code | 0 |
| Real time | 16h (60901s) |
| CPU time | 16h (59586s = 97%) |
| Max RSS bytes | 3017318400 (2877 MiB) |
| Outputting started | 2025-12-11 06:55:09 |
| Output files | https://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11085/1/001/analysisLowE_20251210T140009Z.root
|
| Finished | 2025-12-11 06:56:07 |
| Saved logs | justin-logs:259884.1-dunegpschedd02.fnal.gov.logs.tgz |
| List job events (HTCondor job logs unavailable) |
Jobscript log (last 10,000 characters)
c line#: 181
iter->second->Run() throw STATUS_CODE_NOT_INITIALIZED
in function: RunAlgorithm
in file: /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0005, LArDLVertexing, STATUS_CODE_NOT_INITIALIZED
Boundary wire vector sizes: 6472, 6406, 6462
minwire 0: 2
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 960, 1459
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary:
Output 0: 0.10056,
Output 1: 0.0975777, 0.0969485, 0.0901284, 0.715345,
Output 2: 0.0496697, 0.237848, 0.676839, 0.035644,
Output 3: 0.960309, 0.0136198, 0.00398803, 0.0220832,
Output 4: 0.765993, 0.204194, 0.0250489, 0.00476387,
Output 5: 0.51597, 0.339776, 0.111946, 0.0323075,
Output 6: 0.0273497, 0.0507634, 0.0643308, 0.857556,
11-Dec-2025 06:52:17 UTC Closed output file "matt_reco2.root"
11-Dec-2025 06:52:17 UTC Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/dd/b4/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T020055Z_gen_003495_supernova_g4stage1_g4stage2_detsim_reco.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 4929.61 6034.77 7139.02 6008.15 823.24 10
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00261544 0.024776 0.0438073 0.0262294 0.0132296 10
reco:pandora:StandardPandora 4925.01 6029.08 7133.51 5999.12 823.272 10
reco:pandoraTrack:LArPandoraTrackCreation 0.00399125 0.124653 0.406217 0.0947679 0.124053 10
reco:pandoraShower:LArPandoraModularShowerCreation 0.00396331 0.0350945 0.160321 0.0183084 0.0449709 10
reco:pandoracalo:Calorimetry 0.00157464 0.0375293 0.134134 0.00261802 0.0506208 10
reco:pandorapid:Chi2ParticleID 0.000161485 0.00176392 0.0130909 0.000437208 0.00378456 10
reco:cvnmap:CVNMapper 0.135555 0.18699 0.221502 0.190362 0.0236683 10
reco:cvneva:CVNEvaluator 2.95988 4.25055 9.58198 3.69592 1.87522 10
reco:energyrecnumu:EnergyReco 0.0147517 0.112337 0.231183 0.118052 0.0742164 10
reco:energyrecnue:EnergyReco 0.00408752 0.0115289 0.0210156 0.00654548 0.00723733 10
reco:energyrecnc:EnergyReco 0.0155629 0.11216 0.266968 0.0967945 0.0960944 10
[art]:TriggerResults:TriggerResultInserter 3.964e-05 9.47978e-05 0.000283499 7.1099e-05 6.6277e-05 10
end_path:out1:RootOutput 1.5678e-05 2.75616e-05 8.0787e-05 2.0809e-05 1.85207e-05 10
end_path:out1:RootOutput(write) 0.531085 0.790776 1.0203 0.793663 0.115554 10
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 11041.9 MB
Peak resident set size usage (VmHWM): 3017.32 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 757.156 | hit correction : 1.32928
hit initial : 471.049 | hit correction : 1.33041
hit initial : 481.932 | hit correction : 1.33043
hit initial : 143.277 | hit correction : 1.33265
*** Total Charge : 3734.37
Looking at view: 0 | Number of hits in view: 9
hit initial : 26.8891 | hit correction : 1.32908
hit initial : 22.5483 | hit correction : 1.32902
hit initial : 238.472 | hit correction : 1.32843
hit initial : 29.5266 | hit correction : 1.32828
hit initial : 282.366 | hit correction : 1.32787
hit initial : 53.193 | hit correction : 1.32972
hit initial : 956.555 | hit correction : 1.33008
hit initial : 980.968 | hit correction : 1.33018
hit initial : 61.1292 | hit correction : 1.33098
hit initial : 123.807 | hit correction : 1.33256
*** Total Charge : 3690.89
Looking at view: 1 | Number of hits in view: 10
hit initial : 30.0601 | hit correction : 1.33361
hit initial : 29.6284 | hit correction : 1.32753
hit initial : 293.09 | hit correction : 1.32758
hit initial : 350.161 | hit correction : 1.32793
hit initial : 62.5573 | hit correction : 1.32786
hit initial : 40.36 | hit correction : 1.32774
hit initial : 390.838 | hit correction : 1.32824
hit initial : 49.8794 | hit correction : 1.32828
hit initial : 71.1709 | hit correction : 1.32653
hit initial : 367.873 | hit correction : 1.32872
hit initial : 674.304 | hit correction : 1.32949
hit initial : 311.534 | hit correction : 1.33088
hit initial : 220.64 | hit correction : 1.33174
hit initial : 204.493 | hit correction : 1.33242
hit initial : 135.696 | hit correction : 1.33286
hit initial : 19.6732 | hit correction : 1.33297
*** Total Charge : 4323.25
Looking at view: 2 | Number of hits in view: 16
Begin processing the 9th record. run: 5027 subRun: 0 event: 34949 at 11-Dec-2025 06:54:51 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles: 2
This pfp has size: 23
Added hit list length: 23 | Max was set to: -3.40282e+38
True Electron Energy: 9.00613
Completed MCParticle Loop
True Neutrino Energy: 14.3857
Size of all hits: 23
hit initial : 49.5562 | hit correction : 1.41905
hit initial : 94.0168 | hit correction : 1.42
hit initial : 128.402 | hit correction : 1.41917
hit initial : 226.305 | hit correction : 1.42017
hit initial : 213.463 | hit correction : 1.42031
hit initial : 36.796 | hit correction : 1.42155
hit initial : 87.0856 | hit correction : 1.42033
hit initial : 15.3377 | hit correction : 1.42154
*** Total Charge : 1208.43
Looking at view: 0 | Number of hits in view: 8
hit initial : 25.2354 | hit correction : 1.42153
hit initial : 28.792 | hit correction : 1.42153
hit initial : 149.69 | hit correction : 1.41929
hit initial : 60.0279 | hit correction : 1.41915
hit initial : 86.1964 | hit correction : 1.42024
hit initial : 231.28 | hit correction : 1.42026
hit initial : 225.325 | hit correction : 1.42023
*** Total Charge : 1145.36
Looking at view: 1 | Number of hits in view: 7
hit initial : 31.0417 | hit correction : 1.41889
hit initial : 197.287 | hit correction : 1.41934
hit initial : 195.319 | hit correction : 1.41996
hit initial : 167.362 | hit correction : 1.42031
hit initial : 47.3355 | hit correction : 1.42151
hit initial : 166.346 | hit correction : 1.42034
hit initial : 21.7491 | hit correction : 1.42155
hit initial : 90.5872 | hit correction : 1.42028
*** Total Charge : 1302.25
Looking at view: 2 | Number of hits in view: 8
Begin processing the 10th record. run: 5027 subRun: 0 event: 34950 at 11-Dec-2025 06:54:59 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles: 0
True Neutron Energy: 939.572
True Electron Energy: 2.18155
Completed MCParticle Loop
True Neutrino Energy: 5.46508
Size of all hits: 0
11-Dec-2025 06:55:07 UTC Closed input file "matt_reco2.root"
====================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
====================================================================================================================
Full event 0.177603 0.378298 0.939147 0.327523 0.220108 10
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00140668 0.0699937 0.631726 0.00166302 0.187444 10
end_path:ana:AnalyzeEvents 0.175746 0.307757 0.555175 0.302149 0.116518 10
====================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 8737.97 MB
Peak resident set size usage (VmHWM): 1535.86 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0 