justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 267081.0@dunegpschedd01.fnal.gov

Jobsub ID267081.0@dunegpschedd01.fnal.gov
Workflow ID11108
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-11 22:40:40
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-12-12 02:53:07
From worker nodeHostnamedunegli-8035305-0-fnpc19114.fnal.gov
cpuinfoAMD EPYC 7502 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-11 22:42:36
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T192904Z_gen_002339_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time4h (15016s)
CPU time4h (14881s = 99%)
Max RSS bytes2729181184 (2602 MiB)
Outputting started2025-12-12 02:52:52
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11108/1/001/analysisLowE_20251211T224236Z.root
Finished2025-12-12 02:53:07
Saved logsjustin-logs:267081.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

404937, 
Output 4: 0.705672, 0.272901, 0.018586, 0.00284064, 
Output 5: 0.765052, 0.209631, 0.0197151, 0.00560245, 
Output 6: 0.00846645, 0.0231934, 0.0378137, 0.930526, 

Begin processing the 10th record. run: 5026 subRun: 0 event: 23390 at 12-Dec-2025 02:26:05 UTC
Error: A CaloHitList is empty
Error: A CaloHitList is empty
Boundary wire vector sizes: 6554, 6468, 6736
minwire 0: 11
minwire 1: 1
minwire 2: 1
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 1853, 2352
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.119697, 
Output 1: 0.205084, 0.0589301, 0.242712, 0.493273, 
Output 2: 0.0477457, 0.0891943, 0.843469, 0.0195911, 
Output 3: 0.975962, 0.0167918, 0.00221241, 0.00503365, 
Output 4: 0.588556, 0.37863, 0.0279209, 0.0048932, 
Output 5: 0.268521, 0.449675, 0.207179, 0.0746246, 
Output 6: 0.0992501, 0.187885, 0.112668, 0.600197, 

12-Dec-2025 02:52:09 UTC  Closed output file "matt_reco2.root"
12-Dec-2025 02:52:09 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/76/a8/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T192904Z_gen_002339_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1398.17       1488.54       1559.8        1501.85       46.1961        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00118096    0.00602466     0.0184808    0.00331792    0.00576968       10     
reco:pandora:StandardPandora                            1395.24       1487.08       1558.52       1500.57       46.5227        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000478474    0.0173767     0.0698389    0.00388258     0.0214978       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000261712   0.00375263     0.0105214    0.00253897    0.00321496       10     
reco:pandoracalo:Calorimetry                          0.000207042   0.00761309     0.0225574    0.00147911    0.00950975       10     
reco:pandorapid:Chi2ParticleID                         4.537e-05    0.000187867   0.000676685   8.9081e-05    0.000203367      10     
reco:cvnmap:CVNMapper                                  0.0250182     0.0300084     0.0420216     0.0283524    0.00545196       10     
reco:cvneva:CVNEvaluator                               0.837108       1.00125       2.24696      0.858283      0.415835        10     
reco:energyrecnumu:EnergyReco                         0.00797839     0.0687291     0.189145      0.0752258     0.0497674       10     
reco:energyrecnue:EnergyReco                          0.000896819   0.00478105    0.00757167    0.00705559    0.00295996       10     
reco:energyrecnc:EnergyReco                           0.00689945     0.0268622     0.0813296    0.00717492     0.0303486       10     
[art]:TriggerResults:TriggerResultInserter             1.14e-05     1.9168e-05     7.154e-05     1.315e-05    1.7506e-05       10     
end_path:out1:RootOutput                               6.17e-06     9.0741e-06     2.965e-05     6.705e-06    6.87577e-06      10     
end_path:out1:RootOutput(write)                         0.27432      0.290092      0.342602      0.283078      0.0186847       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21693 MB
  Peak resident set size usage (VmHWM): 2729.18 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
Begin processing the 9th record. run: 5026 subRun: 0 event: 23389 at 12-Dec-2025 02:52:45 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
This pfp has size:  59
Added hit list length: 59 | Max was set to: -3.40282e+38
True Electron Energy: 48.949
Completed MCParticle Loop
True Neutrino Energy: 52.3207
 Size of all hits: 59
hit initial : 35.5615 | hit correction : 1.3528
hit initial : 31.032 | hit correction : 1.35282
hit initial : 224.241 | hit correction : 1.3558
hit initial : 29.7846 | hit correction : 1.35589
hit initial : 107.583 | hit correction : 1.35371
hit initial : 501.522 | hit correction : 1.35378
hit initial : 915.729 | hit correction : 1.354
hit initial : 1141.42 | hit correction : 1.35449
hit initial : 1166.16 | hit correction : 1.35489
hit initial : 316.045 | hit correction : 1.35495
 *** Total Charge : 6053.27
 Looking at view: 0 | Number of hits in view: 10
hit initial : 71.6035 | hit correction : 1.35602
hit initial : 87.3868 | hit correction : 1.35357
hit initial : 277.116 | hit correction : 1.35373
hit initial : 220.645 | hit correction : 1.35382
hit initial : 72.281 | hit correction : 1.35391
hit initial : 326.971 | hit correction : 1.35376
hit initial : 488.898 | hit correction : 1.35406
hit initial : 588.956 | hit correction : 1.3543
hit initial : 378.939 | hit correction : 1.35444
hit initial : 246.004 | hit correction : 1.35457
hit initial : 228.689 | hit correction : 1.35471
hit initial : 223.139 | hit correction : 1.35481
hit initial : 235.196 | hit correction : 1.35493
hit initial : 267.125 | hit correction : 1.35505
hit initial : 237.74 | hit correction : 1.35524
hit initial : 206.062 | hit correction : 1.3555
hit initial : 194.125 | hit correction : 1.35571
hit initial : 157.602 | hit correction : 1.3559
 *** Total Charge : 6106.97
 Looking at view: 1 | Number of hits in view: 18
hit initial : 33.7606 | hit correction : 1.35346
hit initial : 212.702 | hit correction : 1.35356
hit initial : 64.1506 | hit correction : 1.35359
hit initial : 52.2307 | hit correction : 1.35309
hit initial : 38.3449 | hit correction : 1.35285
hit initial : 208.366 | hit correction : 1.35365
hit initial : 337.325 | hit correction : 1.35374
hit initial : 130.127 | hit correction : 1.35386
hit initial : 221.826 | hit correction : 1.35378
hit initial : 156.995 | hit correction : 1.3539
hit initial : 106.863 | hit correction : 1.35413
hit initial : 138.225 | hit correction : 1.35431
hit initial : 212.51 | hit correction : 1.35435
hit initial : 286.629 | hit correction : 1.35438
hit initial : 198.758 | hit correction : 1.35444
hit initial : 129.714 | hit correction : 1.35447
hit initial : 121.297 | hit correction : 1.3545
hit initial : 137.988 | hit correction : 1.35459
hit initial : 123.843 | hit correction : 1.35467
hit initial : 126.271 | hit correction : 1.35475
hit initial : 153.967 | hit correction : 1.3548
hit initial : 135.77 | hit correction : 1.35485
hit initial : 161.675 | hit correction : 1.35495
hit initial : 217.563 | hit correction : 1.35502
hit initial : 181.017 | hit correction : 1.35511
hit initial : 147.895 | hit correction : 1.35533
hit initial : 133.817 | hit correction : 1.35549
hit initial : 138.485 | hit correction : 1.35566
hit initial : 159.536 | hit correction : 1.35583
hit initial : 124.373 | hit correction : 1.356
hit initial : 32.8717 | hit correction : 1.35609
 *** Total Charge : 6264.39
 Looking at view: 2 | Number of hits in view: 31
Begin processing the 10th record. run: 5026 subRun: 0 event: 23390 at 12-Dec-2025 02:52:48 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.576
True Electron Energy: 5.29705
Completed MCParticle Loop
True Neutrino Energy: 8.58112
 Size of all hits: 0
12-Dec-2025 02:52:51 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0709945     0.0827132     0.100509      0.0808039     0.0105355       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000331453   0.000613168   0.00112543    0.000497839   0.000249708      10     
end_path:ana:AnalyzeEvents         0.0705167     0.0819276     0.0998506     0.0798234     0.0104576       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19477.1 MB
  Peak resident set size usage (VmHWM): 1534.97 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 15:01:01 UTC       justIN version: 01.06.00