justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 257906.122@dunegpschedd02.fnal.gov

Jobsub ID257906.122@dunegpschedd02.fnal.gov
Workflow ID11024
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-05 11:19:42
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-12-06 08:37:05
From worker nodeHostnamedunegli-7832314-0-fnpc18010.fnal.gov
cpuinfoAMD EPYC 7551 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-05 22:33:05
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T010623Z_gen_004988_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time10h (36201s)
CPU time9h (35550s = 98%)
Max RSS bytes3095531520 (2952 MiB)
Outputting started2025-12-06 08:36:26
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11024/1/001/analysisLowE_20251205T223308Z.root
Finished2025-12-06 08:37:05
Saved logsjustin-logs:257906.122-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

0.00397818, 
Output 4: 0.381597, 0.54078, 0.0699103, 0.00771305, 
Output 5: 0.393226, 0.457125, 0.120252, 0.0293966, 
Output 6: 0.0830137, 0.169009, 0.131063, 0.616914, 

Begin processing the 10th record. run: 5026 subRun: 0 event: 49880 at 06-Dec-2025 07:52:00 UTC
Boundary wire vector sizes: 6734, 6627, 6586
minwire 0: 14
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 813, 1312
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.12936, 
Output 1: 0.0472437, 0.0334142, 0.095466, 0.823876, 
Output 2: 0.0700596, 0.181419, 0.723041, 0.0254798, 
Output 3: 0.977788, 0.0105351, 0.00162747, 0.0100493, 
Output 4: 0.274898, 0.655001, 0.0639599, 0.00614164, 
Output 5: 0.380235, 0.505927, 0.095116, 0.0187218, 
Output 6: 0.0178628, 0.0447022, 0.0622445, 0.87519, 

06-Dec-2025 08:35:15 UTC  Closed output file "matt_reco2.root"
06-Dec-2025 08:35:15 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/47/c1/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T010623Z_gen_004988_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2588.62       3583.42       4456.68       3768.9        566.168        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00692396     0.0146924     0.0295975     0.011186     0.00811261       10     
reco:pandora:StandardPandora                             2586         3578.86       4446.59       3764.29       564.775        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.00105908     0.0543213     0.227727      0.0492223     0.063249        10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000985047    0.0123649     0.0611607    0.00574992     0.0171609       10     
reco:pandoracalo:Calorimetry                          0.000457562   0.00232003    0.00893944    0.000860467   0.00272996       10     
reco:pandorapid:Chi2ParticleID                        0.000119175   0.00081848    0.00337258    0.000211679   0.00119038       10     
reco:cvnmap:CVNMapper                                  0.0455038     0.0806798     0.114507      0.0818201     0.0188499       10     
reco:cvneva:CVNEvaluator                                1.95672       3.56948       8.56166       3.33032       1.76665        10     
reco:energyrecnumu:EnergyReco                          0.010146      0.107224      0.188303      0.131052      0.0574086       10     
reco:energyrecnue:EnergyReco                          0.00181379    0.00642785     0.0184862    0.00380809    0.00582359       10     
reco:energyrecnc:EnergyReco                            0.0102593     0.0554054     0.193538      0.0180409     0.0626058       10     
[art]:TriggerResults:TriggerResultInserter            2.5758e-05    7.92064e-05   0.000337016   4.1914e-05    9.09619e-05      10     
end_path:out1:RootOutput                               1.107e-05    2.90117e-05   0.000102824   1.54245e-05   2.82288e-05      10     
end_path:out1:RootOutput(write)                        0.391208      0.653898      0.908562      0.692365      0.147262        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21759 MB
  Peak resident set size usage (VmHWM): 3095.53 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 34.1582 | hit correction : 1.02606
hit initial : 60.2992 | hit correction : 1.03783
hit initial : 234.455 | hit correction : 1.03777
hit initial : 66.9403 | hit correction : 1.03722
hit initial : 393.811 | hit correction : 1.03729
hit initial : 168.187 | hit correction : 1.03726
hit initial : 179.358 | hit correction : 1.029
hit initial : 116.954 | hit correction : 1.02905
hit initial : 56.1425 | hit correction : 1.02902
hit initial : 35.5419 | hit correction : 1.02836
hit initial : 174.789 | hit correction : 1.02831
hit initial : 73.0912 | hit correction : 1.02833
hit initial : 42.3585 | hit correction : 1.02778
hit initial : 185.278 | hit correction : 1.02747
hit initial : 51.1772 | hit correction : 1.02679
hit initial : 477.929 | hit correction : 1.02683
hit initial : 488.125 | hit correction : 1.02645
hit initial : 31.3623 | hit correction : 1.02603
hit initial : 446.968 | hit correction : 1.02599
hit initial : 459.19 | hit correction : 1.02555
hit initial : 364.835 | hit correction : 1.0251
hit initial : 299.134 | hit correction : 1.02458
hit initial : 303.662 | hit correction : 1.02426
hit initial : 233.045 | hit correction : 1.0241
hit initial : 638.06 | hit correction : 1.02392
hit initial : 435.567 | hit correction : 1.02391
hit initial : 314.038 | hit correction : 1.02372
hit initial : 133.943 | hit correction : 1.02357
 *** Total Charge : 6885.6
 Looking at view: 0 | Number of hits in view: 32
hit initial : 122.588 | hit correction : 1.03784
hit initial : 322.167 | hit correction : 1.03753
hit initial : 310.373 | hit correction : 1.03726
hit initial : 146.61 | hit correction : 1.03722
hit initial : 218.583 | hit correction : 1.02897
hit initial : 31.6468 | hit correction : 1.02904
hit initial : 722.38 | hit correction : 1.02879
hit initial : 202.404 | hit correction : 1.02831
hit initial : 70.5553 | hit correction : 1.02831
hit initial : 13.2805 | hit correction : 1.02756
hit initial : 42.3905 | hit correction : 1.02755
hit initial : 217.793 | hit correction : 1.02744
hit initial : 100.021 | hit correction : 1.02673
hit initial : 602.808 | hit correction : 1.02662
hit initial : 462.909 | hit correction : 1.02627
hit initial : 45.8494 | hit correction : 1.02587
hit initial : 114.667 | hit correction : 1.02564
hit initial : 59.202 | hit correction : 1.02564
hit initial : 270.791 | hit correction : 1.0253
hit initial : 66.688 | hit correction : 1.02518
hit initial : 141.726 | hit correction : 1.02507
hit initial : 134.406 | hit correction : 1.02413
hit initial : 942.628 | hit correction : 1.02404
hit initial : 976.27 | hit correction : 1.02384
hit initial : 373.245 | hit correction : 1.02364
 *** Total Charge : 6895.63
 Looking at view: 1 | Number of hits in view: 25
hit initial : 128.049 | hit correction : 1.02177
hit initial : 30.6492 | hit correction : 1.03827
hit initial : 481.566 | hit correction : 1.03764
hit initial : 52.3172 | hit correction : 1.03722
hit initial : 411.034 | hit correction : 1.03723
hit initial : 156.173 | hit correction : 1.02907
hit initial : 223.648 | hit correction : 1.02897
hit initial : 328.507 | hit correction : 1.02893
hit initial : 205.866 | hit correction : 1.0285
hit initial : 417.567 | hit correction : 1.02834
hit initial : 226.542 | hit correction : 1.02824
hit initial : 41.6214 | hit correction : 1.02751
hit initial : 724.479 | hit correction : 1.0275
hit initial : 797.552 | hit correction : 1.02663
hit initial : 801.267 | hit correction : 1.02589
hit initial : 453.231 | hit correction : 1.0252
hit initial : 226.993 | hit correction : 1.02473
hit initial : 298.453 | hit correction : 1.02438
hit initial : 345.55 | hit correction : 1.02419
hit initial : 536.169 | hit correction : 1.02399
hit initial : 504.95 | hit correction : 1.02395
hit initial : 542.92 | hit correction : 1.02367
hit initial : 431.698 | hit correction : 1.02348
 *** Total Charge : 8594.43
 Looking at view: 2 | Number of hits in view: 23
06-Dec-2025 08:36:25 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.100987      0.124081      0.141524      0.124697      0.0132239       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000684431   0.00128125    0.00367439    0.00081167    0.000997068      10     
end_path:ana:AnalyzeEvents         0.0999391     0.122476      0.140559      0.123588      0.0132401       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19471.7 MB
  Peak resident set size usage (VmHWM): 1535.54 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 19:09:45 UTC       justIN version: 01.06.00