justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 264312.2@dunegpschedd01.fnal.gov

Jobsub ID264312.2@dunegpschedd01.fnal.gov
Workflow ID11023
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-06 09:29:49
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-12-06 15:01:22
From worker nodeHostnamedunegli-7989158-0-fnpc19120.fnal.gov
cpuinfoAMD EPYC 7502 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-06 09:30:42
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T202635Z_gen_000636_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time5h (19829s)
CPU time5h (19634s = 99%)
Max RSS bytes3069059072 (2926 MiB)
Outputting started2025-12-06 15:01:12
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11023/1/001/analysisLowE_20251206T093043Z.root
Finished2025-12-06 15:01:22
Saved logsjustin-logs:264312.2-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

in file:     /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0005, LArDLVertexing, STATUS_CODE_NOT_INITIALIZED
Boundary wire vector sizes: 6754, 6721, 6640
minwire 0: 22
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 925, 1424
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.110054, 
Output 1: 0.0590243, 0.0518444, 0.0970622, 0.792069, 
Output 2: 0.0831668, 0.269819, 0.610432, 0.036582, 
Output 3: 0.97768, 0.0135172, 0.00173741, 0.0070655, 
Output 4: 0.319362, 0.616544, 0.0599197, 0.00417523, 
Output 5: 0.749022, 0.216034, 0.0277943, 0.00715009, 
Output 6: 0.0172833, 0.0409932, 0.0565389, 0.885185, 

06-Dec-2025 15:00:31 UTC  Closed output file "matt_reco2.root"
06-Dec-2025 15:00:31 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/12/a8/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T202635Z_gen_000636_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1684.3        1967.07       2178.97       1991.81       158.647        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00181097    0.00867279     0.0368853    0.00421555     0.0100716       10     
reco:pandora:StandardPandora                            1682.45       1965.09       2177.08       1990.06       158.385        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.00154068     0.0260715     0.106785      0.0161923     0.0298712       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.00123028    0.00590096     0.0172879    0.00548999    0.00427729       10     
reco:pandoracalo:Calorimetry                          0.000317401   0.00410761     0.0334374    0.000504977   0.00982542       10     
reco:pandorapid:Chi2ParticleID                         8.362e-05    0.000291122   0.00180479    0.000116536   0.000505309      10     
reco:cvnmap:CVNMapper                                  0.0277586     0.037142      0.0466218      0.03553     0.00555385       10     
reco:cvneva:CVNEvaluator                               0.974057       1.39192       3.34404       1.22485      0.663156        10     
reco:energyrecnumu:EnergyReco                         0.00746389     0.0486923     0.110585      0.0196677     0.0439997       10     
reco:energyrecnue:EnergyReco                          0.00110922    0.00266401     0.0117771    0.00178038     0.0030545       10     
reco:energyrecnc:EnergyReco                           0.00835014     0.0570762     0.107462      0.0747266     0.039721        10     
[art]:TriggerResults:TriggerResultInserter             1.514e-05    2.51541e-05    7.535e-05     1.888e-05    1.71601e-05      10     
end_path:out1:RootOutput                               7.16e-06     1.13841e-05   3.1091e-05     8.815e-06    6.92259e-06      10     
end_path:out1:RootOutput(write)                        0.328616      0.394746      0.467123      0.407285      0.0545362       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21778.6 MB
  Peak resident set size usage (VmHWM): 3069.06 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 41.7145 | hit correction : 1.22802
hit initial : 1082.54 | hit correction : 1.22856
hit initial : 60.6782 | hit correction : 1.22898
hit initial : 43.9635 | hit correction : 1.22901
hit initial : 67.3168 | hit correction : 1.22908
hit initial : 166.587 | hit correction : 1.22912
hit initial : 51.7928 | hit correction : 1.23038
hit initial : 111.422 | hit correction : 1.23123
hit initial : 327.47 | hit correction : 1.23125
hit initial : 168.184 | hit correction : 1.23137
 *** Total Charge : 3990.13
 Looking at view: 0 | Number of hits in view: 21
hit initial : 93.3909 | hit correction : 1.2255
hit initial : 23.4966 | hit correction : 1.22455
hit initial : 40.7997 | hit correction : 1.22455
hit initial : 29.1014 | hit correction : 1.22467
hit initial : 53.3321 | hit correction : 1.22901
hit initial : 40.2184 | hit correction : 1.22901
hit initial : 226.267 | hit correction : 1.22562
hit initial : 248.535 | hit correction : 1.22603
hit initial : 268.552 | hit correction : 1.22651
hit initial : 236.975 | hit correction : 1.22704
hit initial : 222.233 | hit correction : 1.22728
hit initial : 212.701 | hit correction : 1.22748
hit initial : 209.228 | hit correction : 1.22766
hit initial : 216.795 | hit correction : 1.22785
hit initial : 230.617 | hit correction : 1.22806
hit initial : 235.057 | hit correction : 1.22834
hit initial : 231.427 | hit correction : 1.22863
hit initial : 239.072 | hit correction : 1.22895
hit initial : 242.653 | hit correction : 1.22926
hit initial : 260.905 | hit correction : 1.22964
hit initial : 253.11 | hit correction : 1.23008
hit initial : 266.498 | hit correction : 1.23059
hit initial : 261.057 | hit correction : 1.23105
hit initial : 241.864 | hit correction : 1.23135
hit initial : 167.719 | hit correction : 1.23151
hit initial : 43.0231 | hit correction : 1.23157
 *** Total Charge : 5890.08
 Looking at view: 1 | Number of hits in view: 26
hit initial : 174.68 | hit correction : 1.22532
hit initial : 83.7838 | hit correction : 1.22456
hit initial : 47.2377 | hit correction : 1.22457
hit initial : 39.2918 | hit correction : 1.22902
hit initial : 224.266 | hit correction : 1.22569
hit initial : 198.322 | hit correction : 1.226
hit initial : 186.426 | hit correction : 1.22631
hit initial : 208.042 | hit correction : 1.22674
hit initial : 192.186 | hit correction : 1.2271
hit initial : 186.264 | hit correction : 1.22729
hit initial : 154.208 | hit correction : 1.22744
hit initial : 154.043 | hit correction : 1.22756
hit initial : 146.69 | hit correction : 1.22772
hit initial : 155.24 | hit correction : 1.22785
hit initial : 173.908 | hit correction : 1.22799
hit initial : 170.184 | hit correction : 1.22817
hit initial : 187.064 | hit correction : 1.22838
hit initial : 154.397 | hit correction : 1.2286
hit initial : 148.807 | hit correction : 1.22879
hit initial : 162.628 | hit correction : 1.22898
hit initial : 159.945 | hit correction : 1.22921
hit initial : 173.573 | hit correction : 1.22943
hit initial : 149.285 | hit correction : 1.22969
hit initial : 168.11 | hit correction : 1.22997
hit initial : 150.442 | hit correction : 1.23021
hit initial : 138.258 | hit correction : 1.23052
hit initial : 146.715 | hit correction : 1.2308
hit initial : 172.569 | hit correction : 1.23104
hit initial : 128.485 | hit correction : 1.23123
hit initial : 136.595 | hit correction : 1.2314
hit initial : 181.195 | hit correction : 1.23154
hit initial : 43.7318 | hit correction : 1.2316
 *** Total Charge : 6014.88
 Looking at view: 2 | Number of hits in view: 32
Begin processing the 10th record. run: 5026 subRun: 0 event: 6360 at 06-Dec-2025 15:01:09 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.578
True Electron Energy: 36.5897
Completed MCParticle Loop
True Neutrino Energy: 39.8761
 Size of all hits: 0
06-Dec-2025 15:01:12 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0722777     0.0909067      0.11389      0.0877673     0.0117652       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000402712   0.000520867   0.000910423   0.000477491   0.000134641      10     
end_path:ana:AnalyzeEvents         0.071677      0.0902068     0.112478      0.0871638     0.0116077       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19472.3 MB
  Peak resident set size usage (VmHWM): 1536.24 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 19:09:02 UTC       justIN version: 01.06.00