Inexact Gradient Research Articles

A wavelet-based nested iteration–inexact conjugate gradient algorithm for adaptively solving elliptic PDEs

In Cohen et al. (Math Comput 70:27–75, 2001), a new paradigm for the adaptive solution of linear elliptic partial differential equations (PDEs) was proposed, based on wavelet discretizations. Starting from a well-conditioned representation of the linear operator equation in infinite wavelet coordinates, one performs perturbed gradient iterations involving approximate matrix–vector multiplications of finite portions of the operator. In a bootstrap-type fashion, increasingly smaller tolerances guarantee convergence of the adaptive method. In addition, coarsening performed on the iterates allow one to prove asymptotically optimal complexity results when compared to the wavelet best N-term approximation. In the present paper, we study adaptive wavelet schemes for symmetric operators employing inexact conjugate gradient routines. Inspired by fast schemes on uniform grids, we incorporate coarsening and the adaptive application of the elliptic operator into a nested iteration algorithm. Our numerical results demonstrate that the runtime of the algorithm is linear in the number of unknowns and substantial savings in memory can be achieved in two and three space dimensions.

Adaptive wavelet schemes for an elliptic control problem with Dirichlet boundary control

An adaptive algorithm based on wavelets is proposed for the fast numerical solution of control problems governed by elliptic boundary value problems with Dirichlet boundary control. A quadratic cost functional representing Sobolev norms of the state and a regularization in terms of the control is to be minimized subject to linear constraints in weak form. In particular, the constraints are formulated as a saddle point problem that allows to handle the varying boundary conditions explicitly. In the framework of (biorthogonal) wavelets, a representer for the functional is derived in terms of l2-norms of wavelet expansion coefficients and the constraints are written in form of an l2 automorphism. Standard techniques from optimization are then used to deduce the resulting first order necessary conditions as a (still infinite) system in l2. Applying the machinery developed in [8,9] which has been extended to control problems in [14], an adaptive method is proposed which can be interpreted as an inexact gradient method for the control. In each iteration step, in turn the primal and the adjoint saddle point system are solved up to a prescribed accuracy by an adaptive iterative Uzawa algorithm for saddle point problems which has been proposed in [10]. Under these premises, it can be shown that the adaptive algorithm containing now three layers of iterations is asymptotically optimal. This means that the convergence rate achieved for computing the solution up to a desired target tolerance is asymptotically the same as the wavelet-best N-term approximation of the solution, and the total computational work is proportional to the number of computational unknowns.

Adaptive Wavelet Methods for Linear-Quadratic Elliptic Control Problems: Convergence Rates

We propose an adaptive algorithm based on wavelets for the fast numerical solution of control problems governed by elliptic boundary value problems with distributed or Neumann boundary control. A quadratic cost functional that may involve fractional Sobolev norms of the state and the control is to be minimized subject to linear constraints in weak form. Placing the problem into the framework of (biorthogonal) wavelets allows us to formulate the functional and the constraints equivalently in terms of $\ell_2$-norms of wavelet expansion coefficients and constraints in the form of an $\ell_2$ automorphism. The resulting first order necessary conditions are then derived as a (still infinite) system in $\ell_2$. Applying the machinery developed in [A. Cohen, W. Dahmen, and R. DeVore, Math. Comp., 70 (2001), pp. 27--75; A. Cohen, W. Dahmen, and R. DeVore, Found. Comput. Math., 2 (2002), pp. 203--245], we propose an adaptive method which can be interpreted as an inexact gradient descent method, where in each iteration step the primal and the adjoint system need to be solved up to a prescribed accuracy. Convergence of the adaptive algorithm is proved. In addition, we show that the adaptive algorithm is asymptotically optimal, that is, the convergence rate achieved for computing the solution up to a desired target tolerance is asymptotically the same as the wavelet-best N-term approximation of the solution, and the total computational work is proportional to the number of computational unknowns.

Analysis of Inexact Trust-Region SQP Algorithms

In this paper we extend the design of a class of composite-step trust-region SQP methods and their global convergence analysis to allow inexact problem information. The inexact problem information can result from iterative linear system solves within the trust-region SQP method or from approximations of first-order derivatives. Accuracy requirements in our trust-region SQP methods are adjusted based on feasibility and optimality of the iterates. Our accuracy requirements are stated in general terms, but we show how they can be enforced using information that is already available in matrix-free implementations of SQP methods. In the absence of inexactness our global convergence theory is equal to that of Dennis, El-Alem, and Maciel [SIAM J. Optim., 7 (1997), pp. 177--207]. If all iterates are feasible, i.e., if all iterates satisfy the equality constraints, then our results are related to the known convergence analyses for trust-region methods with inexact gradient information for unconstrained optimization.

Fast Iterative Solution of Saddle Point Problems in Optimal Control Based on Wavelets

In this paper, wavelet techniques are employed for the fast numerical solution of a control problem governed by an elliptic boundary value problem with boundary control. A quadratic cost functional involving natural norms of the state and the control is to be minimized. Firstly the constraint, the elliptic boundary value problem, is formulated in an appropriate weak form that allows to handle varying boundary conditions explicitly: the boundary conditions are treated by Lagrange multipliers, leading to a saddle point problem. This is combined with a fictitious domain approach in order to cover also more complicated boundaries. Deviating from standard approaches, we then use (biorthogonal) wavelets to derive an equivalent infinite discretized control problem which involves only e2-norms and -operators. Classical methods from optimization yield the corresponding optimality conditions in terms of two weakly coupled (still infinite) saddle point problems for which a unique solution exists. For deriving finite-dimensional systems which are uniformly invertible, stability of the discretizations has to be ensured. This together with the e2-setting circumvents the problem of preconditioning: all operators have uniformly bounded condition numbers independent of the discretization. In order to numerically solve the resulting (finite-dimensional) linear system of the weakly coupled saddle point problems, a fully iterative method is proposed which can be viewed as an inexact gradient scheme. It consists of a gradient algorithm as an outer iteration which alternatingly picks the two saddle point problems, and an inner iteration to solve each of the saddle point problems, exemplified in terms of the Uzawa algorithm. It is proved here that this strategy converges, provided that the inner systems are solved sufficiently well. Moreover, since the system matrix is well-conditioned, it is shown that in combination with a nested iteration strategy this iteration is asymptotically optimal in the sense that it provides the solution on discretization level J with an overall amount of arithmetic operations that is proportional to the number of unknows NJ on that level. Finally, numerical results are provided.

GLOBAL CONVERGENCE OF A TRUST REGION ALGORITHM USING INEXACT GRADIENT FOR EQUALITY-CONSTRAINED OPTIMIZATION

A trust-region algorithm is presented for a nonlinear optimization problem of equality-constraints. The characterization of the algorithm is using inexact gradient information. Global convergence results are demonstrated where the gradient values are obeyed a simple relative error condition.

A Trust Region Method for Norm Constrained Problems

A trust region method for the solution of nonlinear optimization problems with norm constraints is analyzed. Such optimization problems arise in parameter identification and nonlinear eigenvalue problems. The characteristics of these applications motivate the formulation and analysis of the trust region method. The algorithms studied here allow for inexact gradient information and the use of subspace methods for the approximate solution of subproblems. Characterizations and the descent properties of trust region steps are given, criteria for the existence of successful iterations under inexact gradient information and under the use of subspace methods are established, and global convergence of the method is proven.