justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 257767.1@dunegpschedd02.fnal.gov

Jobsub ID257767.1@dunegpschedd02.fnal.gov
Workflow ID11006
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-05 05:24:41
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-12-05 09:45:04
From worker nodeHostnamedunegli-7828113-0-fnpc18007.fnal.gov
cpuinfoAMD EPYC 7551 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-05 05:26:02
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T183955Z_gen_001681_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time4h (15509s)
CPU time4h (15336s = 98%)
Max RSS bytes3074383872 (2931 MiB)
Outputting started2025-12-05 09:44:32
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11006/1/001/analysisLowE_20251205T052603Z.root
Finished2025-12-05 09:45:04
Saved logsjustin-logs:257767.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

t(*this, m_inputVertexListName, pVertexList) throw STATUS_CODE_NOT_INITIALIZED
    in function: GetHitRegion
    in file:     /exp/dune/app/users/osbiston/18_10_24_Official_LowE_Build/srcs/larpandoracontent/larpandoradlcontent/LArVertex/DlVertexingAlgorithm.cc line#: 624
Failure in algorithm Alg0005, LArDLVertexing, STATUS_CODE_NOT_INITIALIZED
Boundary wire vector sizes: 6874, 6867, 6865
minwire 0: 23
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 2342, 2841
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.103554, 
Output 1: 0.0478867, 0.0601076, 0.0982989, 0.793707, 
Output 2: 0.0762484, 0.265117, 0.632793, 0.0258416, 
Output 3: 0.969746, 0.0229956, 0.00230855, 0.00494958, 
Output 4: 0.325285, 0.597666, 0.0713486, 0.00569984, 
Output 5: 0.544527, 0.405874, 0.0432955, 0.00630381, 
Output 6: 0.0219234, 0.0702136, 0.0832203, 0.824643, 

05-Dec-2025 09:43:38 UTC  Closed output file "matt_reco2.root"
05-Dec-2025 09:43:38 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/b9/d7/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T183955Z_gen_001681_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1469.56       1535.25       1593.98       1531.19       35.4486        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00122137    0.00574059     0.0163608     0.0036413    0.00519823       10     
reco:pandora:StandardPandora                            1467.22       1532.76       1591.62       1528.89       35.3449        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.00152728    0.00751063     0.0365622    0.00213575     0.0101967       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.00203503    0.00478225     0.0169319    0.00239571    0.00435119       10     
reco:pandoracalo:Calorimetry                          0.000242287   0.00120843    0.00599352    0.000348248   0.00184839       10     
reco:pandorapid:Chi2ParticleID                        6.6124e-05    0.000280444   0.00141086    0.000109663   0.00038763       10     
reco:cvnmap:CVNMapper                                  0.029308      0.0348284     0.0428108     0.0340432    0.00435158       10     
reco:cvneva:CVNEvaluator                                1.72951       1.95258       3.30795       1.81411      0.453775        10     
reco:energyrecnumu:EnergyReco                          0.0144089     0.0816239     0.121707      0.0922336     0.0321789       10     
reco:energyrecnue:EnergyReco                          0.00125588    0.00675709     0.0119326    0.00930442    0.00433854       10     
reco:energyrecnc:EnergyReco                           0.00921892     0.0270201     0.109184     0.00941995     0.0346948       10     
[art]:TriggerResults:TriggerResultInserter             1.582e-05    2.53789e-05   8.3808e-05    1.9477e-05    1.9572e-05       10     
end_path:out1:RootOutput                               3.937e-06    8.9409e-06    2.6821e-05    7.4445e-06    6.06375e-06      10     
end_path:out1:RootOutput(write)                        0.357247       0.36836      0.382904      0.369395     0.00781756       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21783 MB
  Peak resident set size usage (VmHWM): 3074.38 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 322.266 | hit correction : 1.45973
 *** Total Charge : 5331.59
 Looking at view: 2 | Number of hits in view: 18
Begin processing the 8th record. run: 5026 subRun: 0 event: 16808 at 05-Dec-2025 09:44:21 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Electron Energy: 9.30546
Completed MCParticle Loop
True Neutrino Energy: 17.0928
 Size of all hits: 0
Begin processing the 9th record. run: 5026 subRun: 0 event: 16809 at 05-Dec-2025 09:44:24 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
This pfp has size:  49
Added hit list length: 49 | Max was set to: -3.40282e+38
True Electron Energy: 33.7777
Completed MCParticle Loop
True Neutrino Energy: 39.2944
 Size of all hits: 49
hit initial : 72.698 | hit correction : 1.38095
hit initial : 103.928 | hit correction : 1.38154
hit initial : 680.454 | hit correction : 1.38173
hit initial : 106.82 | hit correction : 1.38188
hit initial : 895.987 | hit correction : 1.38333
hit initial : 235.005 | hit correction : 1.38386
hit initial : 238.534 | hit correction : 1.38599
hit initial : 84.9588 | hit correction : 1.38727
hit initial : 307.104 | hit correction : 1.38745
hit initial : 425.153 | hit correction : 1.3871
hit initial : 128.595 | hit correction : 1.38787
 *** Total Charge : 4539.21
 Looking at view: 0 | Number of hits in view: 11
hit initial : 52.4204 | hit correction : 1.38091
hit initial : 239.985 | hit correction : 1.38096
hit initial : 276.08 | hit correction : 1.38103
hit initial : 48.1778 | hit correction : 1.38293
hit initial : 90.8654 | hit correction : 1.39015
hit initial : 341.667 | hit correction : 1.38169
hit initial : 139.691 | hit correction : 1.38171
hit initial : 236.616 | hit correction : 1.38192
hit initial : 171.47 | hit correction : 1.38222
hit initial : 158.885 | hit correction : 1.38246
hit initial : 93.2441 | hit correction : 1.38262
hit initial : 93.986 | hit correction : 1.38332
hit initial : 7.02964 | hit correction : 1.38352
hit initial : 100.964 | hit correction : 1.38413
hit initial : 96.5156 | hit correction : 1.38425
hit initial : 135.202 | hit correction : 1.38563
hit initial : 397.832 | hit correction : 1.38611
hit initial : 341.549 | hit correction : 1.38723
hit initial : 139.399 | hit correction : 1.38733
hit initial : 290.91 | hit correction : 1.3874
 *** Total Charge : 4778.35
 Looking at view: 1 | Number of hits in view: 20
hit initial : 153.993 | hit correction : 1.38519
hit initial : 133.491 | hit correction : 1.38087
hit initial : 360.591 | hit correction : 1.38104
hit initial : 373.781 | hit correction : 1.38149
hit initial : 231.849 | hit correction : 1.38175
hit initial : 115.291 | hit correction : 1.38189
hit initial : 113.277 | hit correction : 1.38206
hit initial : 147.288 | hit correction : 1.38223
hit initial : 156.217 | hit correction : 1.38242
hit initial : 211.393 | hit correction : 1.38288
hit initial : 233.411 | hit correction : 1.38376
hit initial : 109.5 | hit correction : 1.38432
hit initial : 367.286 | hit correction : 1.38598
hit initial : 119.296 | hit correction : 1.38725
hit initial : 185.733 | hit correction : 1.38732
hit initial : 23.1555 | hit correction : 1.38748
hit initial : 431.418 | hit correction : 1.38719
hit initial : 153.216 | hit correction : 1.38774
 *** Total Charge : 5010.11
 Looking at view: 2 | Number of hits in view: 18
Begin processing the 10th record. run: 5026 subRun: 0 event: 16810 at 05-Dec-2025 09:44:28 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.578
True Electron Energy: 30.0907
Completed MCParticle Loop
True Neutrino Energy: 35.5027
 Size of all hits: 0
05-Dec-2025 09:44:31 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0882981     0.103886       0.1629       0.0991995     0.0210481       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000415124   0.000575907   0.00122144    0.00051931    0.000225657      10     
end_path:ana:AnalyzeEvents         0.0876699     0.103119      0.161139      0.0985524     0.0207264       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19475.8 MB
  Peak resident set size usage (VmHWM): 1540.52 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 20:41:06 UTC       justIN version: 01.06.00