justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 262292.0@dunegpschedd02.fnal.gov

Jobsub ID262292.0@dunegpschedd02.fnal.gov
Workflow ID11168
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-13 19:26:59
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-12-14 02:59:39
From worker nodeHostnamedunegli-8047129-0-fnpc19101.fnal.gov
cpuinfoAMD EPYC 7502 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-13 19:27:53
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T005555Z_gen_002851_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time7h (27096s)
CPU time7h (26761s = 98%)
Max RSS bytes2793398272 (2663 MiB)
Outputting started2025-12-14 02:59:30
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11168/1/001/analysisLowE_20251213T192754Z.root
Finished2025-12-14 02:59:39
Saved logsjustin-logs:262292.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ement holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  14-Dec-2025 02:58:48 UTC run: 5027 subRun: 0 event: 28510
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  14-Dec-2025 02:58:48 UTC run: 5027 subRun: 0 event: 28510
Trying to add data product: ShowerPCA. This element does not exist in the element holder
%MSG
Boundary wire vector sizes: 6996, 7004, 6782
minwire 0: 23
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 2100, 2599
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.106942, 
Output 1: 0.052307, 0.051152, 0.131348, 0.765193, 
Output 2: 0.0914332, 0.250586, 0.617, 0.0409807, 
Output 3: 0.949844, 0.035207, 0.00555047, 0.00939842, 
Output 4: 0.390347, 0.544112, 0.0601732, 0.00536858, 
Output 5: 0.602575, 0.341527, 0.0472389, 0.0086596, 
Output 6: 0.0225921, 0.056823, 0.0684045, 0.85218, 

14-Dec-2025 02:58:50 UTC  Closed output file "matt_reco2.root"
14-Dec-2025 02:58:50 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/86/9c/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T005555Z_gen_002851_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1715.79       2688.65       3365.31       2853.78       520.347        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000863578   0.00929855     0.0475649    0.00142181     0.0161154       10     
reco:pandora:StandardPandora                            1714.06       2686.34       3363.18       2851.93       520.688        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000823798    0.0537343     0.116897      0.0577675     0.038487        10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.00190005    0.00945375     0.0385676    0.00628103    0.00987725       10     
reco:pandoracalo:Calorimetry                          0.000382583   0.00204802     0.0117508    0.000595041   0.00340424       10     
reco:pandorapid:Chi2ParticleID                        0.000107081   0.000438872    0.0030035    0.000131226   0.000857168      10     
reco:cvnmap:CVNMapper                                  0.0456511     0.101597      0.260923      0.085719      0.0595972       10     
reco:cvneva:CVNEvaluator                                1.18124       1.66817       4.92542       1.3459        1.08788        10     
reco:energyrecnumu:EnergyReco                          0.0942157     0.118714      0.174945      0.115527      0.0205547       10     
reco:energyrecnue:EnergyReco                          0.00124115    0.00198797    0.00344903    0.00175047    0.000739217      10     
reco:energyrecnc:EnergyReco                           0.00715078    0.00866976     0.0128375    0.00806865    0.00169709       10     
[art]:TriggerResults:TriggerResultInserter             1.773e-05    3.24012e-05   0.000106991    2.237e-05    2.56012e-05      10     
end_path:out1:RootOutput                               8.78e-06     1.47761e-05    3.735e-05    1.22105e-05   7.98561e-06      10     
end_path:out1:RootOutput(write)                        0.281677       0.33377      0.555351       0.31231      0.0749598       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21571.4 MB
  Peak resident set size usage (VmHWM): 2793.4 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 137.225 | hit correction : 1.46725
hit initial : 134.441 | hit correction : 1.46694
hit initial : 144.263 | hit correction : 1.46662
hit initial : 150.499 | hit correction : 1.46615
hit initial : 220.413 | hit correction : 1.46565
hit initial : 296.198 | hit correction : 1.4653
 *** Total Charge : 4022.96
 Looking at view: 2 | Number of hits in view: 19
Begin processing the 9th record. run: 5027 subRun: 0 event: 28509 at 14-Dec-2025 02:59:24 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  5
This pfp has size:  18
Added hit list length: 18 | Max was set to: -3.40282e+38
This pfp has size:  11
This pfp has size:  6
True Electron Energy: 7.27616
Completed MCParticle Loop
True Neutrino Energy: 12.6548
 Size of all hits: 18
hit initial : 33.2478 | hit correction : 1.42001
hit initial : 78.2709 | hit correction : 1.42003
hit initial : 143.666 | hit correction : 1.41919
hit initial : 204.439 | hit correction : 1.41914
hit initial : 71.2494 | hit correction : 1.42006
hit initial : 50.9815 | hit correction : 1.41912
hit initial : 26.2326 | hit correction : 1.41998
 *** Total Charge : 863.153
 Looking at view: 0 | Number of hits in view: 7
hit initial : 34.0488 | hit correction : 1.42
hit initial : 36.6715 | hit correction : 1.41931
hit initial : 21.069 | hit correction : 1.41872
hit initial : 97.2891 | hit correction : 1.42001
hit initial : 174.565 | hit correction : 1.41916
hit initial : 121.811 | hit correction : 1.41997
hit initial : 80.3644 | hit correction : 1.41895
hit initial : 30.1306 | hit correction : 1.41991
 *** Total Charge : 845.959
 Looking at view: 1 | Number of hits in view: 8
hit initial : 42.0485 | hit correction : 1.42009
hit initial : 341.708 | hit correction : 1.41899
hit initial : 199.08 | hit correction : 1.41996
 *** Total Charge : 827.277
 Looking at view: 2 | Number of hits in view: 3
Begin processing the 10th record. run: 5027 subRun: 0 event: 28510 at 14-Dec-2025 02:59:27 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  6
This pfp has size:  22
Added hit list length: 22 | Max was set to: -3.40282e+38
This pfp has size:  20
True Neutron Energy: 939.573
True Electron Energy: 19.4045
Completed MCParticle Loop
True Neutrino Energy: 26.54
 Size of all hits: 22
hit initial : 33.3153 | hit correction : 1.33863
hit initial : 65.7465 | hit correction : 1.3387
hit initial : 26.4319 | hit correction : 1.3387
hit initial : 32.4953 | hit correction : 1.347
hit initial : 38.4282 | hit correction : 1.34699
hit initial : 62.0207 | hit correction : 1.36463
hit initial : 107.179 | hit correction : 1.3646
hit initial : 40.6941 | hit correction : 1.36455
 *** Total Charge : 549.95
 Looking at view: 0 | Number of hits in view: 8
hit initial : 16.9118 | hit correction : 1.33869
hit initial : 55.8648 | hit correction : 1.33868
hit initial : 38.0658 | hit correction : 1.33867
hit initial : 24.9742 | hit correction : 1.347
hit initial : 38.1442 | hit correction : 1.34701
hit initial : 87.2097 | hit correction : 1.36459
hit initial : 107.171 | hit correction : 1.36459
hit initial : 31.1882 | hit correction : 1.36461
 *** Total Charge : 541.213
 Looking at view: 1 | Number of hits in view: 8
hit initial : 71.4223 | hit correction : 1.33866
hit initial : 52.3256 | hit correction : 1.33869
hit initial : 51.4416 | hit correction : 1.34701
hit initial : 67.3997 | hit correction : 1.36466
hit initial : 29.0935 | hit correction : 1.36455
hit initial : 137.176 | hit correction : 1.36456
 *** Total Charge : 553.813
 Looking at view: 2 | Number of hits in view: 6
14-Dec-2025 02:59:29 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0800852     0.0856481     0.0966425     0.0838251    0.00500323       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000347054   0.000448399   0.000673467   0.000440044   9.0976e-05       10     
end_path:ana:AnalyzeEvents         0.0795655     0.0849871     0.0954373     0.0831449    0.00487696       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19472.9 MB
  Peak resident set size usage (VmHWM): 1531.12 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 12:05:50 UTC       justIN version: 01.06.00