justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 266849.0@dunegpschedd01.fnal.gov

Jobsub ID266849.0@dunegpschedd01.fnal.gov
Workflow ID11110
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-11 15:30:16
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-12-12 02:59:58
From worker nodeHostnamedunegli-8029141-0-fnpc19104.fnal.gov
cpuinfoAMD EPYC 7502 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-11 15:31:44
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T230828Z_gen_001213_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time11h (41280s)
CPU time11h (40714s = 98%)
Max RSS bytes2922442752 (2787 MiB)
Outputting started2025-12-12 02:59:44
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11110/1/001/analysisLowE_20251211T153146Z.root
Finished2025-12-12 02:59:58
Saved logsjustin-logs:266849.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

962974, 0.0241845, 0.0040361, 0.00880576, 
Output 4: 0.831312, 0.135624, 0.0264537, 0.00660976, 
Output 5: 0.750078, 0.198508, 0.0404324, 0.0109813, 
Output 6: 0.0104345, 0.0351075, 0.0520728, 0.902385, 

Begin processing the 10th record. run: 5027 subRun: 0 event: 12130 at 12-Dec-2025 02:08:09 UTC
Error: A CaloHitList is empty
Error: A CaloHitList is empty
Boundary wire vector sizes: 7062, 6988, 6870
minwire 0: 39
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 1532, 2031
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0916227, 
Output 1: 0.0575148, 0.0717697, 0.16121, 0.709505, 
Output 2: 0.0702637, 0.214674, 0.685733, 0.0293292, 
Output 3: 0.93525, 0.0556082, 0.00492815, 0.00421327, 
Output 4: 0.362058, 0.554733, 0.0762184, 0.00699073, 
Output 5: 0.724963, 0.225494, 0.0393473, 0.010196, 
Output 6: 0.0247361, 0.0803606, 0.08485, 0.810053, 

12-Dec-2025 02:58:59 UTC  Closed output file "matt_reco2.root"
12-Dec-2025 02:58:59 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/00/04/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T230828Z_gen_001213_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              3046.74       4099.98       4634.26       4214.42       506.69         10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00349519     0.0193229     0.0365037     0.0181332    0.00923943       10     
reco:pandora:StandardPandora                            3045.45       4096.95       4631.7        4211.89       505.671        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000692796    0.0847048     0.423279      0.0423284     0.122195        10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000367013    0.0104524      0.04793      0.0025378     0.0156495       10     
reco:pandoracalo:Calorimetry                          0.000324293   0.00370766     0.0194758    0.000836033   0.00625767       10     
reco:pandorapid:Chi2ParticleID                        9.6251e-05    0.000349113   0.00182713    0.000191536   0.000498112      10     
reco:cvnmap:CVNMapper                                  0.0377711     0.0829226     0.184449      0.0789131     0.0406475       10     
reco:cvneva:CVNEvaluator                               0.864169       2.27567       7.09269       1.86178       1.67224        10     
reco:energyrecnumu:EnergyReco                          0.0128022     0.0397832     0.120257      0.0193067     0.0372261       10     
reco:energyrecnue:EnergyReco                          0.000917618   0.00427978     0.0129723    0.00260847    0.00351643       10     
reco:energyrecnc:EnergyReco                           0.00702877      0.10216       0.17881      0.110923      0.0550848       10     
[art]:TriggerResults:TriggerResultInserter             1.71e-05     8.21659e-05   0.000294093   5.51955e-05   7.84686e-05      10     
end_path:out1:RootOutput                               8.01e-06     3.27762e-05   9.0421e-05     2.247e-05    2.44846e-05      10     
end_path:out1:RootOutput(write)                         0.27798      0.406058      0.632709       0.42522      0.0989365       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21686.7 MB
  Peak resident set size usage (VmHWM): 2922.44 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
 Looking at view: 0 | Number of hits in view: 17
hit initial : 166.556 | hit correction : 1.12266
hit initial : 150.891 | hit correction : 1.12248
hit initial : 46.4231 | hit correction : 1.12239
hit initial : 47.7508 | hit correction : 1.11854
hit initial : 775.416 | hit correction : 1.11857
hit initial : 876.586 | hit correction : 1.11887
hit initial : 582.335 | hit correction : 1.11913
hit initial : 177.964 | hit correction : 1.11919
 *** Total Charge : 3160.91
 Looking at view: 1 | Number of hits in view: 8
hit initial : 349.713 | hit correction : 1.11932
hit initial : 124.289 | hit correction : 1.11852
hit initial : 218.274 | hit correction : 1.11856
hit initial : 358.982 | hit correction : 1.11841
hit initial : 167.179 | hit correction : 1.11807
hit initial : 180.233 | hit correction : 1.1178
hit initial : 149.582 | hit correction : 1.11754
hit initial : 192.773 | hit correction : 1.11728
hit initial : 195.992 | hit correction : 1.11696
hit initial : 169.121 | hit correction : 1.11877
hit initial : 162.014 | hit correction : 1.11891
hit initial : 141.107 | hit correction : 1.11898
hit initial : 370.494 | hit correction : 1.11882
hit initial : 138.241 | hit correction : 1.11903
hit initial : 436.086 | hit correction : 1.11919
 *** Total Charge : 3751.6
 Looking at view: 2 | Number of hits in view: 15
Begin processing the 9th record. run: 5027 subRun: 0 event: 12129 at 12-Dec-2025 02:59:38 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
This pfp has size:  31
Added hit list length: 31 | Max was set to: -3.40282e+38
True Electron Energy: 22.8904
Completed MCParticle Loop
True Neutrino Energy: 28.4083
 Size of all hits: 31
hit initial : 95.4144 | hit correction : 1.1262
hit initial : 119.088 | hit correction : 1.12619
hit initial : 38.2798 | hit correction : 1.12618
hit initial : 63.3519 | hit correction : 1.12727
hit initial : 159.698 | hit correction : 1.12721
hit initial : 214.725 | hit correction : 1.12707
hit initial : 260.998 | hit correction : 1.12672
hit initial : 236.077 | hit correction : 1.12649
hit initial : 389.116 | hit correction : 1.12632
hit initial : 483.619 | hit correction : 1.12621
hit initial : 151.333 | hit correction : 1.12617
 *** Total Charge : 2491.48
 Looking at view: 0 | Number of hits in view: 11
hit initial : 26.085 | hit correction : 1.12577
hit initial : 68.9571 | hit correction : 1.12617
hit initial : 385.865 | hit correction : 1.12623
hit initial : 576.968 | hit correction : 1.12633
hit initial : 335.815 | hit correction : 1.12653
hit initial : 280.507 | hit correction : 1.12699
hit initial : 211.767 | hit correction : 1.1272
hit initial : 72.6252 | hit correction : 1.12727
hit initial : 123.077 | hit correction : 1.12618
 *** Total Charge : 2345.06
 Looking at view: 1 | Number of hits in view: 9
hit initial : 31.3885 | hit correction : 1.12735
hit initial : 444.561 | hit correction : 1.12317
hit initial : 161.501 | hit correction : 1.12309
hit initial : 15.1741 | hit correction : 1.12727
hit initial : 406.016 | hit correction : 1.12719
hit initial : 147.361 | hit correction : 1.12618
hit initial : 102.944 | hit correction : 1.12618
hit initial : 678.456 | hit correction : 1.12621
hit initial : 105.801 | hit correction : 1.1262
hit initial : 534.617 | hit correction : 1.1265
hit initial : 220.814 | hit correction : 1.12662
 *** Total Charge : 3207
 Looking at view: 2 | Number of hits in view: 11
Begin processing the 10th record. run: 5027 subRun: 0 event: 12130 at 12-Dec-2025 02:59:41 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 939.572
True Electron Energy: 44.6548
Completed MCParticle Loop
True Neutrino Energy: 60.2687
 Size of all hits: 0
12-Dec-2025 02:59:44 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0656991     0.0901639     0.145292      0.0856684     0.0214611       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000433984    0.0013173    0.00797775    0.000511294   0.00222609       10     
end_path:ana:AnalyzeEvents         0.0650783     0.0886443     0.136686      0.0850549     0.0194032       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19474.4 MB
  Peak resident set size usage (VmHWM): 1534.22 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 14:58:55 UTC       justIN version: 01.06.00