justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 268019.0@dunegpschedd01.fnal.gov

Jobsub ID268019.0@dunegpschedd01.fnal.gov
Workflow ID11175
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-13 12:10:36
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-12-13 21:41:00
From worker nodeHostnamedunegli-8047088-0-fnpc17005.fnal.gov
cpuinfoIntel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-13 12:11:52
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T191625Z_gen_002060_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time9h (34135s)
CPU time9h (33787s = 98%)
Max RSS bytes2800246784 (2670 MiB)
Outputting started2025-12-13 21:40:48
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11175/1/001/analysisLowE_20251213T121154Z.root
Finished2025-12-13 21:41:00
Saved logsjustin-logs:268019.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

er
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  13-Dec-2025 21:39:33 UTC run: 5026 subRun: 0 event: 20600
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
Boundary wire vector sizes: 6673, 6644, 6671
minwire 0: 58
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 1233, 1732
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0289068, 
Output 1: 0.00662562, 0.0701277, 0.0392949, 0.883952, 
Output 2: 0.149046, 0.249531, 0.493414, 0.108009, 
Output 3: 0.890599, 0.022378, 0.0163401, 0.0706829, 
Output 4: 0.989336, 0.00756634, 0.00142783, 0.00167015, 
Output 5: 0.958796, 0.0363928, 0.00326112, 0.00154998, 
Output 6: 0.00181055, 0.00648883, 0.0135311, 0.97817, 

13-Dec-2025 21:39:37 UTC  Closed output file "matt_reco2.root"
13-Dec-2025 21:39:37 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/35/96/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T191625Z_gen_002060_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1966.32       3372.67       4697.44       3610.26       900.79         10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000889916   0.00670807     0.0348473    0.00146076     0.0109742       10     
reco:pandora:StandardPandora                            1963.82       3369.41       4694.46       3607.25       900.319        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.00235347     0.0877891     0.179869      0.0686119     0.0630222       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.00223457    0.00904912     0.0378756    0.00612988     0.0099869       10     
reco:pandoracalo:Calorimetry                          0.000380742   0.00239541     0.0102124    0.000705408    0.0034282       10     
reco:pandorapid:Chi2ParticleID                        0.000115343   0.000516347   0.00310861    0.000250579   0.000868252      10     
reco:cvnmap:CVNMapper                                  0.0789934     0.122409      0.174245      0.131952      0.0326524       10     
reco:cvneva:CVNEvaluator                                1.66679       2.32284       6.91298       1.77456       1.53564        10     
reco:energyrecnumu:EnergyReco                           0.1032       0.140946      0.222206      0.128196      0.0329955       10     
reco:energyrecnue:EnergyReco                          0.00167472    0.00516463     0.0140817    0.00296437    0.00457104       10     
reco:energyrecnc:EnergyReco                           0.00999799     0.0129056     0.0149023     0.0137348    0.00167143       10     
[art]:TriggerResults:TriggerResultInserter            2.3084e-05    4.75051e-05   0.000174103   3.3652e-05    4.26188e-05      10     
end_path:out1:RootOutput                               7.152e-06    1.48146e-05   5.2219e-05    1.09995e-05   1.26596e-05      10     
end_path:out1:RootOutput(write)                        0.478759       0.54434      0.716962      0.532697      0.0667508       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 11911 MB
  Peak resident set size usage (VmHWM): 2800.25 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 148.576 | hit correction : 1.33775
hit initial : 35.694 | hit correction : 1.33764
hit initial : 33.1161 | hit correction : 1.37118
hit initial : 157.294 | hit correction : 1.37119
hit initial : 248.552 | hit correction : 1.37127
hit initial : 261.025 | hit correction : 1.37162
hit initial : 242.896 | hit correction : 1.37188
hit initial : 45.8171 | hit correction : 1.37184
hit initial : 204.377 | hit correction : 1.3727
hit initial : 105.332 | hit correction : 1.37416
hit initial : 243.826 | hit correction : 1.37403
hit initial : 404.079 | hit correction : 1.37372
hit initial : 437.529 | hit correction : 1.3739
hit initial : 433.03 | hit correction : 1.37438
hit initial : 391.912 | hit correction : 1.37512
hit initial : 289.367 | hit correction : 1.37579
hit initial : 245.127 | hit correction : 1.37658
hit initial : 241.38 | hit correction : 1.3773
hit initial : 127.44 | hit correction : 1.37767
hit initial : 22.5089 | hit correction : 1.37782
 *** Total Charge : 6546.72
 Looking at view: 0 | Number of hits in view: 28
hit initial : 23.0561 | hit correction : 1.30429
hit initial : 56.3285 | hit correction : 1.30431
hit initial : 25.2509 | hit correction : 1.30434
hit initial : 18.4104 | hit correction : 1.32155
hit initial : 52.3664 | hit correction : 1.32154
hit initial : 38.8436 | hit correction : 1.32154
hit initial : 69.5207 | hit correction : 1.33781
hit initial : 203.914 | hit correction : 1.33781
hit initial : 138.147 | hit correction : 1.33783
hit initial : 54.4851 | hit correction : 1.37118
hit initial : 16.0384 | hit correction : 1.37121
hit initial : 94.7189 | hit correction : 1.37142
hit initial : 314.016 | hit correction : 1.37144
hit initial : 311.033 | hit correction : 1.37167
hit initial : 123.076 | hit correction : 1.37199
hit initial : 572.346 | hit correction : 1.37407
hit initial : 82.9602 | hit correction : 1.37416
hit initial : 104.659 | hit correction : 1.37379
hit initial : 21.494 | hit correction : 1.37502
hit initial : 959.833 | hit correction : 1.37567
hit initial : 39.1299 | hit correction : 1.37561
hit initial : 253.707 | hit correction : 1.37697
hit initial : 192.378 | hit correction : 1.37741
hit initial : 58.6236 | hit correction : 1.378
hit initial : 26.6664 | hit correction : 1.37757
hit initial : 25.7016 | hit correction : 1.37759
 *** Total Charge : 5300
 Looking at view: 1 | Number of hits in view: 26
hit initial : 23.4732 | hit correction : 1.30431
hit initial : 77.9579 | hit correction : 1.30432
hit initial : 80.4981 | hit correction : 1.32152
hit initial : 206.318 | hit correction : 1.33795
hit initial : 20.6874 | hit correction : 1.33803
hit initial : 155.452 | hit correction : 1.3377
hit initial : 49.3678 | hit correction : 1.33762
hit initial : 33.1848 | hit correction : 1.37121
hit initial : 160.483 | hit correction : 1.37119
hit initial : 174.734 | hit correction : 1.37116
hit initial : 143.592 | hit correction : 1.37142
hit initial : 151.793 | hit correction : 1.37168
hit initial : 184.139 | hit correction : 1.3718
hit initial : 142.587 | hit correction : 1.37207
hit initial : 224.071 | hit correction : 1.37267
hit initial : 24.3015 | hit correction : 1.37405
hit initial : 265.98 | hit correction : 1.37399
hit initial : 359.117 | hit correction : 1.37357
hit initial : 116.09 | hit correction : 1.37375
hit initial : 279.301 | hit correction : 1.37394
hit initial : 359.655 | hit correction : 1.37408
hit initial : 20.226 | hit correction : 1.37757
hit initial : 45.688 | hit correction : 1.37759
hit initial : 470.591 | hit correction : 1.37467
hit initial : 407.842 | hit correction : 1.37547
hit initial : 296.311 | hit correction : 1.37628
hit initial : 239.732 | hit correction : 1.3771
hit initial : 153.893 | hit correction : 1.37771
hit initial : 20.4172 | hit correction : 1.37798
hit initial : 113.177 | hit correction : 1.37827
 *** Total Charge : 6844.79
 Looking at view: 2 | Number of hits in view: 30
13-Dec-2025 21:40:45 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.122143      0.145125      0.173211      0.143441      0.0132797       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000480761   0.000644388   0.00121406    0.000584185   0.000196963      10     
end_path:ana:AnalyzeEvents         0.121362      0.144287       0.1714       0.142775      0.0130523       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 9811.62 MB
  Peak resident set size usage (VmHWM): 1538.92 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 12:22:22 UTC       justIN version: 01.06.00