A general minimization method for unsmooth extremal problems

Abstract

IN many constrained extremal problems the functional to be minimized or one of the functionals specifying the permissible set, is not even Gateaux-differentiable, and has only directional derivatives. We shall describe such problems as unsmooth. Various aspects of the theory of these problems have been dealt with for example, the necessary conditions for a minimum [1–4], existence theorems and strong convergence of the minimizing sequences [5, 6], and correct stipulation of the constraints and stability [7]. Extremely general minimization methods have been developed for the case when the functional to be minimized is differentiable, and the permissible set has a reasonably simple structure (e.g. linear functionals can easily be projected or minimized on it), i.e. for the smooth case ([8], and [9], Sections 5–7). As regards unsmooth problems, all investigations (with the exception of [10]) of minimization methods for then have been under much less general assumptions than those under which the necessary conditions for a minimum have been obtained (mainly for the finite-dimensional case [11–15]or for unsmooth functionals of a special type [16–19]). In [10], an extremely general and computationally simple minimization method in Hilbert space was described. In this method, however, the values of the functional do not necessarily decrease from iteration to iteration, nor do all the points belong to the permissible set, i.e. it is not a monotonic method of permissible directions. This factor complicates its numerical realization, since it is difficult to check the computations. Other general methods applicable to unsmooth problems include the cutting-plane methods devised by Kelley [20] for the finite-dimensional convex case, and extended to infinite-dimensional problems in [9], Section 10. In these methods, however, the successive approximations again do not belong to the permissible set, and in addition, the initial problem is reduced to a sequence of linear programming problems, the number of constraints in which increases with the number of iterations. The present paper describes a monotonic method of permissible directions for a wide class of unsmooth problems in arbitrary Banach space. In its fundamental idea, the algorithm is similar to those in [11–16, 18,19] and represents an extension of them to the case of arbitrary Banach spaces, unsmooth functions of a more general type, and the presence of a number of constraints. The convergence of the method is proved, and its applications to various unsmooth extremal problems (of mathematical programming, theory of games, approximation theory, and optimal control) considered.