Про швидкiсть збiжностi субградiєнтих методiв з кроком Поляка

Abstract

Subgradient methods (method A and method B) are considered for finding the minimum point of a convex function for the known optimal value of the function. Both methods use known Polyak’s step, which depends on scalar parameter m ≥ 1 and guarantees monotonous decrease of the distance to the minimum point. If m = 1 the methods A and B are applicable to arbitrary convex function. Parameter m > 1 permits to take into account special classes of convex functions – convex quadratic functions (m = 2), differentiable homogeneous of degree σ (m = σ), etc. For both methods theorems are proven about the convergence rate O 1/ √ k for arbitrary convex function and convergence with geometric progression rate for a convex function with acute minimum. Method A is a subgradient method with Polyak’s step in original space of variables. The parameter m = 1 corresponds to classical Polyak’s step for arbitrary convex function. Parameter m = 2 can be used for quadratic functions minimization, for which the value of Polyak’s step is doubled comparing to the classical Polyak’s step with m = 1. For onedimentional functions method A finds the minimum point in one step from the arbitrary start point, which corresponds to one iteration of the Newton’s method. Proof of theorems about convergence rate of the method A is based on convex function characteristics in original space of variables, among which a constant c1 plays a crucial role – it bounds a subgradient norm. Method B is a subgradient method with space transformation, where the Polyak’s step is calculated in space transformed by linear operator. The method is defined by nonsingular matrix B. The parameter m = 1 corresponds to classical Polyak’s step for arbitrary convex function in transformed space of variables. If m = 2, for one-dimentional quadratic function, the method B finds the minimum point in one step with both arbitrary matrix B and start point. Proof of theorems about convergence rate of the method B is based on function characteristics in transformed space of variables, which are analogous to the method A characteristics in original space.