justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 266897.1@dunegpschedd01.fnal.gov

Jobsub ID266897.1@dunegpschedd01.fnal.gov
Workflow ID11109
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-11 16:40:19
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-12-12 03:18:46
From worker nodeHostnamedunegli-8029141-0-fnpc19104.fnal.gov
cpuinfoAMD EPYC 7502 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-11 16:41:48
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T205830Z_gen_003558_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time10h (38204s)
CPU time10h (37691s = 98%)
Max RSS bytes2923483136 (2788 MiB)
Outputting started2025-12-12 03:18:33
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11109/1/001/analysisLowE_20251211T164150Z.root
Finished2025-12-12 03:18:46
Saved logsjustin-logs:266897.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

0.160435, 0.170419, 0.517183, 
Output 2: 0.0274203, 0.229482, 0.702327, 0.0407702, 
Output 3: 0.962235, 0.0161611, 0.00684332, 0.0147603, 
Output 4: 0.819648, 0.150927, 0.0250686, 0.00435649, 
Output 5: 0.402362, 0.298756, 0.23193, 0.0669522, 
Output 6: 0.0321043, 0.0574617, 0.0676101, 0.842824, 

Begin processing the 10th record. run: 5026 subRun: 0 event: 35580 at 12-Dec-2025 02:46:36 UTC
Error: A CaloHitList is empty
Error: A CaloHitList is empty
Boundary wire vector sizes: 6792, 6760, 6650
minwire 0: 38
minwire 1: 1
minwire 2: 2
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 1652, 2151
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0689478, 
Output 1: 0.051449, 0.0241516, 0.0457708, 0.878629, 
Output 2: 0.142321, 0.388855, 0.427996, 0.0408276, 
Output 3: 0.969297, 0.0227836, 0.00169285, 0.00622654, 
Output 4: 0.19204, 0.745524, 0.0597343, 0.00270134, 
Output 5: 0.920656, 0.0754494, 0.00308111, 0.000813924, 
Output 6: 0.0107053, 0.0479724, 0.071002, 0.87032, 

12-Dec-2025 03:17:54 UTC  Closed output file "matt_reco2.root"
12-Dec-2025 03:17:54 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/d9/85/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T205830Z_gen_003558_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1874.63       3788.64       5146.59       4000.92       931.964        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00565456     0.017307      0.0314437     0.0165141    0.00753258       10     
reco:pandora:StandardPandora                            1873.4        3785.57       5143.97       3994.24       931.041        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.00093872     0.0840377     0.467189      0.0286256     0.134865        10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000448025    0.0178674     0.0794839    0.00671721     0.0259195       10     
reco:pandoracalo:Calorimetry                          0.000224722   0.00695052     0.0279787    0.00286718    0.00862287       10     
reco:pandorapid:Chi2ParticleID                         6.381e-05    0.00116084    0.00956665    0.000236762   0.00280591       10     
reco:cvnmap:CVNMapper                                  0.0284139     0.0789977     0.242129      0.0608012     0.0576002       10     
reco:cvneva:CVNEvaluator                               0.771523       2.24151       9.55191       1.55503       2.46693        10     
reco:energyrecnumu:EnergyReco                          0.0096186     0.0714248     0.139226      0.0892007     0.0498434       10     
reco:energyrecnue:EnergyReco                          0.000947928   0.00484622     0.0127659    0.00416246    0.00349457       10     
reco:energyrecnc:EnergyReco                           0.00628716     0.0670428     0.167403      0.041676      0.0611938       10     
[art]:TriggerResults:TriggerResultInserter             1.49e-05     8.07177e-05   0.000372264   5.2565e-05    9.98745e-05      10     
end_path:out1:RootOutput                               6.49e-06     2.44363e-05   7.4031e-05    1.8705e-05    1.91403e-05      10     
end_path:out1:RootOutput(write)                        0.285403      0.478082      0.697275      0.477001      0.136183        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21689.7 MB
  Peak resident set size usage (VmHWM): 2923.48 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
Completed MCParticle Loop
True Neutrino Energy: 19.3197
 Size of all hits: 7
hit initial : 108.415 | hit correction : 1.44655
hit initial : 42.4518 | hit correction : 1.44655
hit initial : 48.1266 | hit correction : 1.44768
 *** Total Charge : 287.908
 Looking at view: 0 | Number of hits in view: 3
hit initial : 30.0243 | hit correction : 1.44624
hit initial : 69.0411 | hit correction : 1.44624
hit initial : 16.1644 | hit correction : 1.44478
hit initial : 44.0163 | hit correction : 1.4448
 *** Total Charge : 230.221
 Looking at view: 1 | Number of hits in view: 4
Begin processing the 9th record. run: 5026 subRun: 0 event: 35579 at 12-Dec-2025 03:18:27 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  3
This pfp has size:  38
Added hit list length: 38 | Max was set to: -3.40282e+38
This pfp has size:  43
Added hit list length: 43 | Max was set to: 38
True Electron Energy: 52.3866
Completed MCParticle Loop
True Neutrino Energy: 57.8031
 Size of all hits: 43
hit initial : 65.3973 | hit correction : 1.35709
hit initial : 148.32 | hit correction : 1.35709
hit initial : 54.7005 | hit correction : 1.35708
hit initial : 56.8713 | hit correction : 1.35572
hit initial : 32.475 | hit correction : 1.35347
hit initial : 79.7274 | hit correction : 1.35358
hit initial : 230.333 | hit correction : 1.35517
hit initial : 199.64 | hit correction : 1.35379
hit initial : 91.1635 | hit correction : 1.35489
hit initial : 33.2068 | hit correction : 1.3541
hit initial : 19.2941 | hit correction : 1.35369
hit initial : 36.3088 | hit correction : 1.35644
hit initial : 73.0217 | hit correction : 1.35646
hit initial : 102.164 | hit correction : 1.35644
hit initial : 37.0309 | hit correction : 1.35643
hit initial : 158.682 | hit correction : 1.3557
hit initial : 186.866 | hit correction : 1.35584
hit initial : 171.259 | hit correction : 1.35588
hit initial : 188.996 | hit correction : 1.35588
hit initial : 195.994 | hit correction : 1.35585
hit initial : 144.121 | hit correction : 1.35578
 *** Total Charge : 3125.45
 Looking at view: 0 | Number of hits in view: 21
hit initial : 99.2921 | hit correction : 1.35645
hit initial : 108.486 | hit correction : 1.35642
hit initial : 96.0189 | hit correction : 1.35494
hit initial : 113.363 | hit correction : 1.35362
hit initial : 266.55 | hit correction : 1.35393
hit initial : 70.3256 | hit correction : 1.35482
hit initial : 37.7685 | hit correction : 1.35576
hit initial : 205.404 | hit correction : 1.3555
hit initial : 168.262 | hit correction : 1.3558
hit initial : 274.837 | hit correction : 1.35585
hit initial : 236.16 | hit correction : 1.3558
hit initial : 254.144 | hit correction : 1.35587
 *** Total Charge : 2616.73
 Looking at view: 1 | Number of hits in view: 12
hit initial : 130.164 | hit correction : 1.35351
hit initial : 203.757 | hit correction : 1.35386
hit initial : 176.942 | hit correction : 1.35511
hit initial : 123.482 | hit correction : 1.35426
hit initial : 128.201 | hit correction : 1.35644
hit initial : 85.1311 | hit correction : 1.35644
hit initial : 278.294 | hit correction : 1.35569
hit initial : 322.412 | hit correction : 1.35587
hit initial : 194.603 | hit correction : 1.35578
hit initial : 354.477 | hit correction : 1.35586
 *** Total Charge : 2707.3
 Looking at view: 2 | Number of hits in view: 10
Begin processing the 10th record. run: 5026 subRun: 0 event: 35580 at 12-Dec-2025 03:18:30 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.567
True Electron Energy: 11.1766
Completed MCParticle Loop
True Neutrino Energy: 16.5596
 Size of all hits: 0
12-Dec-2025 03:18:32 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.064125      0.0795011     0.100609      0.0798554     0.0115519       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000349953   0.00044837    0.000825957   0.000411114   0.000133694      10     
end_path:ana:AnalyzeEvents         0.063628      0.0789124     0.0997275     0.0793578     0.0114577       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19479.5 MB
  Peak resident set size usage (VmHWM): 1541.59 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 14:58:10 UTC       justIN version: 01.06.00