Standard Steepest Descent Research Articles

Fast Multiobjective Gradient Methods with Nesterov Acceleration via Inertial Gradient-Like Systems

We derive efficient algorithms to compute weakly Pareto optimal solutions for smooth, convex and unconstrained multiobjective optimization problems in general Hilbert spaces. To this end, we define a novel inertial gradient-like dynamical system in the multiobjective setting, which trajectories converge weakly to Pareto optimal solutions. Discretization of this system yields an inertial multiobjective algorithm which generates sequences that converge weakly to Pareto optimal solutions. We employ Nesterov acceleration to define an algorithm with an improved convergence rate compared to the plain multiobjective steepest descent method (Algorithm 1). A further improvement in terms of efficiency is achieved by avoiding the solution of a quadratic subproblem to compute a common step direction for all objective functions, which is usually required in first-order methods. Using a different discretization of our inertial gradient-like dynamical system, we obtain an accelerated multiobjective gradient method that does not require the solution of a subproblem in each step (Algorithm 2). While this algorithm does not converge in general, it yields good results on test problems while being faster than standard steepest descent.

Strong substitutes: structural properties, and a new algorithm for competitive equilibrium prices

We show the strong substitutes product-mix auction bidding language provides an intuitive and geometric interpretation of strong substitutes as Minkowski differences between sets that are easy to identify. We prove that competitive equilibrium prices for agents with strong substitutes preferences can be computed by minimizing the difference between two linear programs for the positive and the negative bids with suitably relaxed resource constraints. This also leads to a new algorithm for computing competitive equilibrium prices which is competitive with standard steepest descent algorithms in extensive experiments.

Riemannian methods for optimization in a shape space of triangular meshes

In this paper, the applicability of two optimization algorithms in the context of Riemannian manifolds is investigated. More precisely, the steepest descent method and the nonlinear conjugate gradient method are used to solve the Shape-From-Shading problem in a shape space of triangular meshes. For this reason, appropriate Riemannian metrics are introduced in this shape space. Instead of taking steps along straight lines, propagation along geodesics with respect to the Riemannian metric is employed. This requires to formulate the geodesic equation and the equation of parallel translation in the shape space of triangular meshes. In general, the proposed geodesic optimization algorithms outperform the standard steepest descent method concerning various quality measures and the visual impression of the reconstructed surface. In addition, it is shown that an appropriately chosen Riemannian metric can guide the optimization process towards a desired class of surfaces.

New Scaled Three-Term Memoryless vm-Methods for Solving Nonlinear Unconstraint Problems

Four new scaled three-term memoryless VM methods for solving nonlinear unconstrained problems are presented. The basic idea is to deal with Al-Bayati's (1991) and Biggs's (1983) self-scaling VMupdates in the frame of new scaled CG-methods. Birgin-Martinez (2001) and Abbo (2007) positive parameters are used to scale these spectral CG-methods. The new search directions are reset to the standard Steepest Descent (SD) direction when Powell's (1977) restarting criterion holds. Andrei's (2010) acceleration scheme of the step-size parameter has been employed in the new proposed methods to improve the efficiency of such methods. Under common assumptions; the new methods are proved to be globally convergent. Computational results for a set consisting of 100 unconstrained optimization test problems show that the new methods substantially outperforms the scaled memoryless BFGS method.

Convergence Estimates for the Generalized Davidson Method for Symmetric Eigenvalue Problems II: The Subspace Acceleration

The generalized Davidson (GD) method can be viewed as a generalization of the preconditioned steepest descent (PSD) method for solving symmetric eigenvalue problems. In the GD method, the new approximation is sought in the subspace that spans all the previous approximate eigenvectors, in addition to the current one and the preconditioned residual thereof used in PSD. In this respect, the relation between the GD and PSD methods is similar to that between the standard steepest descent method for linear systems and methods in Krylov subspaces. This paper presents convergence estimates for the (restarted) GD method that demonstrate convergence acceleration compared to the PSD method, similar to that achieved by methods in Krylov subspaces compared to the standard steepest descent.

Steepest descent using smoothed gradients

This paper reviews solving differential equations using the least-squares steepest descent method with Sobolev gradients. The method's superiority over standard steepest descent with a Euclidean gradient is explained in terms of stability and the classical Courant–Freiderichs–Lewy condition for the path of steepest descent. The spectra for several of the operators arising from a cononical example are also computed. Kantorovich's inequality then gives explicit estimates on the rate of convergence for the two processes. In this way, use of the Sobolev gradient is viewed as a very effective preconditioning strategy for the linear part of the differential equation.