justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 266865.1@dunegpschedd01.fnal.gov

Jobsub ID266865.1@dunegpschedd01.fnal.gov
Workflow ID11108
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-11 16:18:18
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-12-12 03:05:58
From worker nodeHostnamedunegli-8029141-0-fnpc19104.fnal.gov
cpuinfoAMD EPYC 7502 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-11 16:19:49
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T014422Z_gen_004267_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time10h (38757s)
CPU time10h (37852s = 97%)
Max RSS bytes2914377728 (2779 MiB)
Outputting started2025-12-12 03:05:47
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11108/1/001/analysisLowE_20251211T161951Z.root
Finished2025-12-12 03:05:58
Saved logsjustin-logs:266865.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

357, 0.79213, 
Output 2: 0.0744526, 0.271188, 0.608655, 0.0457044, 
Output 3: 0.963559, 0.0222154, 0.00355585, 0.0106699, 
Output 4: 0.844816, 0.135338, 0.0165309, 0.00331576, 
Output 5: 0.843971, 0.121652, 0.0246918, 0.00968486, 
Output 6: 0.00896737, 0.0259959, 0.0406713, 0.924365, 

Begin processing the 10th record. run: 5026 subRun: 0 event: 42670 at 12-Dec-2025 02:31:39 UTC
Error: A CaloHitList is empty
Error: A CaloHitList is empty
Boundary wire vector sizes: 6823, 6805, 6728
minwire 0: 46
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.118111, 
Output 1: 0.0927181, 0.0564298, 0.079866, 0.770986, 
Output 2: 0.111256, 0.113871, 0.759284, 0.0155884, 
Output 3: 0.991855, 0.00497226, 0.000173058, 0.00299929, 
Output 4: 0.363286, 0.604524, 0.0279947, 0.00419487, 
Output 5: 0.263366, 0.55145, 0.127098, 0.0580862, 
Output 6: 0.0552386, 0.121528, 0.113033, 0.710201, 

12-Dec-2025 03:05:06 UTC  Closed output file "matt_reco2.root"
12-Dec-2025 03:05:07 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/bb/52/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T014422Z_gen_004267_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2004.12       3847.55       5180.16       4047.54       1015.37        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00508116     0.0176465     0.0530015     0.0144809     0.0131079       10     
reco:pandora:StandardPandora                            2002.92       3804.5        5105.02       3971.12       985.672        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000897048    0.0494634     0.293012      0.0049078     0.0922931       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000456624   0.00843694     0.0362651    0.00168387     0.0116445       10     
reco:pandoracalo:Calorimetry                          0.000411674    0.0292418     0.110747      0.0191531     0.0338594       10     
reco:pandorapid:Chi2ParticleID                        7.1591e-05    0.000533756    0.003689     0.000180527   0.00105466       10     
reco:cvnmap:CVNMapper                                  0.0319916     0.0921865     0.172541      0.084629      0.0401339       10     
reco:cvneva:CVNEvaluator                               0.770374       1.86969       6.7598        1.32744       1.66421        10     
reco:energyrecnumu:EnergyReco                         0.00706554      14.9144       74.3102      0.0913231      29.6934        10     
reco:energyrecnue:EnergyReco                          0.00246202    0.00878037     0.0178753    0.00765269    0.00450082       10     
reco:energyrecnc:EnergyReco                           0.00693096      25.6331       72.3652      0.0133955      31.5305        10     
[art]:TriggerResults:TriggerResultInserter             1.379e-05    7.24556e-05   0.000314083   3.6985e-05    8.67687e-05      10     
end_path:out1:RootOutput                               6.63e-06     3.08213e-05   0.000123411    1.82e-05     3.33687e-05      10     
end_path:out1:RootOutput(write)                        0.275638      0.418058      0.546401      0.451164      0.0875499       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21680.5 MB
  Peak resident set size usage (VmHWM): 2914.38 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
 Looking at view: 0 | Number of hits in view: 19
hit initial : 34.719 | hit correction : 1.11449
hit initial : 100.589 | hit correction : 1.11637
hit initial : 47.1826 | hit correction : 1.11638
hit initial : 34.5134 | hit correction : 1.11171
hit initial : 96.5877 | hit correction : 1.11001
hit initial : 108.579 | hit correction : 1.11003
hit initial : 98.1817 | hit correction : 1.10967
hit initial : 121.364 | hit correction : 1.11015
hit initial : 42.6859 | hit correction : 1.10963
hit initial : 176.146 | hit correction : 1.11062
hit initial : 58.1278 | hit correction : 1.11125
hit initial : 39.0575 | hit correction : 1.11122
hit initial : 119.638 | hit correction : 1.11128
hit initial : 234.209 | hit correction : 1.11147
hit initial : 234.633 | hit correction : 1.11173
hit initial : 245.06 | hit correction : 1.11199
hit initial : 257.79 | hit correction : 1.11223
hit initial : 227.155 | hit correction : 1.11255
hit initial : 238.99 | hit correction : 1.1128
hit initial : 268.06 | hit correction : 1.11316
hit initial : 295.082 | hit correction : 1.11349
hit initial : 276.002 | hit correction : 1.11386
hit initial : 150.747 | hit correction : 1.11425
 *** Total Charge : 3898.82
 Looking at view: 1 | Number of hits in view: 23
hit initial : 253.197 | hit correction : 1.10971
hit initial : 309.758 | hit correction : 1.11068
hit initial : 172.626 | hit correction : 1.1101
hit initial : 59.6021 | hit correction : 1.11033
hit initial : 57.1205 | hit correction : 1.11636
hit initial : 126.219 | hit correction : 1.11124
hit initial : 544.695 | hit correction : 1.11161
hit initial : 771.932 | hit correction : 1.1122
hit initial : 951.525 | hit correction : 1.11329
hit initial : 286.801 | hit correction : 1.11419
 *** Total Charge : 3929.76
 Looking at view: 2 | Number of hits in view: 10
Begin processing the 5th record. run: 5026 subRun: 0 event: 42665 at 12-Dec-2025 03:05:30 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Electron Energy: 1.44548
Completed MCParticle Loop
True Neutrino Energy: 5.16946
 Size of all hits: 0
Begin processing the 6th record. run: 5026 subRun: 0 event: 42666 at 12-Dec-2025 03:05:33 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.619
True Electron Energy: 1.58986
Completed MCParticle Loop
True Neutrino Energy: 4.87343
 Size of all hits: 0
Begin processing the 7th record. run: 5026 subRun: 0 event: 42667 at 12-Dec-2025 03:05:35 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Electron Energy: 54.4564
Completed MCParticle Loop
True Neutrino Energy: 57.8571
 Size of all hits: 0
Begin processing the 8th record. run: 5026 subRun: 0 event: 42668 at 12-Dec-2025 03:05:38 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Electron Energy: 19.9545
Completed MCParticle Loop
True Neutrino Energy: 28.7654
 Size of all hits: 0
Begin processing the 9th record. run: 5026 subRun: 0 event: 42669 at 12-Dec-2025 03:05:41 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Electron Energy: 5.30325
Completed MCParticle Loop
True Neutrino Energy: 9.02748
 Size of all hits: 0
Begin processing the 10th record. run: 5026 subRun: 0 event: 42670 at 12-Dec-2025 03:05:44 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 942.106
True Electron Energy: 3.72242
Completed MCParticle Loop
True Neutrino Energy: 7.44711
 Size of all hits: 0
12-Dec-2025 03:05:46 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0643365     0.0739096     0.0951567     0.0668396     0.0108321       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000348133   0.000423827   0.000497935   0.000423763   4.89091e-05      10     
end_path:ana:AnalyzeEvents         0.0638033     0.0733414     0.0942865     0.0663164     0.0107754       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19472.4 MB
  Peak resident set size usage (VmHWM): 1532.06 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 14:58:10 UTC       justIN version: 01.06.00