Armijo Type Research Articles

An explicit three-term Polak–Ribière–Polyak conjugate gradient method for bicriteria optimization

A three-term Polak–Ribière–Polyak conjugate gradient-like method for bicriteria optimization without scalarization is proposed in this paper. Three advantages are to be noted. First, the descent directions are given explicitly and can then be directly computed. Second, the descent property turns out to be sufficient and independent of the line search. Third, without Lipschitzian hypotheses, global convergence towards Pareto stationary points is proved under an Armijo type condition. Numerical experiments including comparisons with other methods are also reported.

Double inertial steps extragadient-type methods for solving optimal control and image restoration problems

<abstract><p>In order to approximate the common solution of quasi-nonexpansive fixed point and pseudo-monotone variational inequality problems in real Hilbert spaces, this paper presented three new modified sub-gradient extragradient-type methods. Our algorithms incorporated viscosity terms and double inertial extrapolations to ensure strong convergence and to speed up convergence. No line search methods of the Armijo type were required by our algorithms. Instead, they employed a novel self-adaptive step size technique that produced a non-monotonic sequence of step sizes while also correctly incorporating a number of well-known step sizes. The step size was designed to lessen the algorithms' reliance on the initial step size. Numerical tests were performed, and the results showed that our step size is more effective and that it guarantees that our methods require less execution time. We stated and proved the strong convergence of our algorithms under mild conditions imposed on the control parameters. To show the computational advantage of the suggested methods over some well-known methods in the literature, several numerical experiments were provided. To test the applicability and efficiencies of our methods in solving real-world problems, we utilized the proposed methods to solve optimal control and image restoration problems.</p></abstract>

A coupled FETI-BDNM for solving 3D elastic frictional contact problem

Based on the B-differentiable Newton method (BDNM) and finite element tearing and interconnecting (FETI) algorithm, a coupled approach named FETI-BDNM is developed to solve the 3D elastic contact problem with friction. In this method, the contact bodies are decomposed into a number of subdomains. The interfaces between the two adjacent substructures will be skillfully considered as the special contact surfaces that are always in the stick state. By resorting to the generalized inverse of stiffness matrices of the subdomains, the relationships between the relative displacements and forces on contact/interface, i.e., contact flexibility matrices will be computed efficiently. The contact equations, displacement continuity equations and equilibrium equations for each subdomain are expressed in the form of B-differentiable equations (BDEs) uniformly, which can be solved by the damped Newton-Raphson method with 1D Armijo type linear search. The proposed method makes it possible to solve large-scale contact problems without compromising the solution accuracy compared to the original BDNM, and the good convergence of BDEs and parallel scalability of the FETI algorithm will be preserved. Numerical examples validate the accuracy, efficiency, robustness and effects of the number of subdomains on the computational time of the proposed method.

A Newton-type proximal gradient method for nonlinear multi-objective optimization problems

In this paper, a globally convergent Newton-type proximal gradient method is developed for composite multi-objective optimization problems where each objective function can be represented as the sum of a smooth function and a nonsmooth function. The proposed method deals with unconstrained convex multi-objective optimization problems. This method is free from any kind of priori chosen parameters or ordering information of objective functions. At every iteration of the proposed method, a subproblem is solved to find a suitable descent direction. The subproblem uses a quadratic approximation of each smooth function. An Armijo type line search is conducted to find a suitable step length. A sequence is generated using the descent direction and the step length. The global convergence of this method is justified under some mild assumptions. The proposed method is verified and compared with some existing methods using a set of test problems.

An inertial subgradient extragradient method with Armijo type step size for pseudomonotone variational inequalities with non-Lipschitz operators in Banach spaces

In this paper, we study the pseudomonotone variational inequality problems with non-Lipschitz operators. We propose an inertial subgradient extragradient method with Halpern technique and Armijo type step size for approximating the solution of the problem in the framework of 2-uniformly convex real Banach spaces. We prove that the sequence generated by our proposed method converges strongly to the solution of the problem under some mild conditions and without the weak sequential continuity condition often assumed by authors in solving pseudomonotone variational inequality problems. Finally, we provide some numerical experiments for the proposed method in comparison with other existing methods in the literature. Our result extends and improves several of the existing results in the current literature in this direction.

An extragradient method for non-monotone equilibrium problems on Hadamard manifolds with applications

In the present paper, we consider a non-monotone equilibrium problem on Hadamard manifolds and define Armijo's type extragradient algorithm for the proposed equilibrium problem. The introduced algorithm does not require objective bifunction's monotonicity and the solution set's nonemptiness. A convergence result of our algorithm under very mild assumptions is presented. Moreover, we investigate the applications of the established results to the non-monotone set-valued variational inequalities and the generalized Nash equilibrium problems from the considered method.

A Riemannian under-determined BFGS method for least squares inverse eigenvalue problems

This paper is concerned with the parameterized least squares inverse eigenvalue problems for the case that the number of parameters to be constructed is greater than the number of prescribed realizable eigenvalues. Intrinsically, this is a specific problem of finding a zero of an under-determined nonlinear map defined between a Riemannian product manifold and a matrix space. To solve this problem, we propose a Riemannian under-determined BFGS algorithm with a specialized update formula for iterative linear operators, and an Armijo type line search is used. Global convergence properties of this algorithm are established under some mild assumptions. In addition, we also generalize a Riemannian inexact Newton method for solving this problem. Specially, the explicit form of the inverse of the linear operator corresponding to the perturbed normal Riemannian Newton equation is obtained, which improves the efficiency of Riemannian inexact Newton method. Numerical experiments are provided to illustrate the efficiency of the proposed method.

Shamanskii-Like Levenberg-Marquardt Method with a New Line Search for Systems of Nonlinear Equations

To save the calculations of Jacobian, a multi-step Levenberg-Marquardt method named Shamanskii-like LM method for systems of nonlinear equations was proposed by Fan (2013). Its convergence properties have been proved by using a trust region technique under the local error bound condition. However, the authors wonder whether the similar convergence properties are still true with standard line searches since the direction may not be a descent direction. For this purpose, the authors present a new nonmonotone m-th order Armijo type line search to guarantee the global convergence. Under the same condition as trust region case, the convergence rate also has been shown to be m + 1 by using this line search technique. Numerical experiments show the new algorithm can save much running time for the large scale problems, so it is efficient and promising.

A globally convergent QP-free algorithm for nonlinear semidefinite programming

In this paper, we present a QP-free algorithm for nonlinear semidefinite programming. At each iteration, the search direction is yielded by solving two systems of linear equations with the same coefficient matrix; l_{1} penalty function is used as merit function for line search, the step size is determined by Armijo type inexact line search. The global convergence of the proposed algorithm is shown under suitable conditions. Preliminary numerical results are reported.

Reliability-based design optimization using SORM and SQP

In this work a second order approach for reliability-based design optimization (RBDO) with mixtures of uncorrelated non-Gaussian variables is derived by applying second order reliability methods (SORM) and sequential quadratic programming (SQP). The derivation is performed by introducing intermediate variables defined by the incremental iso-probabilistic transformation at the most probable point (MPP). By using these variables in the Taylor expansions of the constraints, a corresponding general first order reliability method (FORM) based quadratic programming (QP) problem is formulated and solved in the standard normal space. The MPP is found in the physical space in the metric of Hasofer-Lind by using a Newton algorithm, where the efficiency of the Newton method is obtained by introducing an inexact Jacobian and a line-search of Armijo type. The FORM-based SQP approach is then corrected by applying four SORM approaches: Breitung, Hohenbichler, Tvedt and a recent suggested formula. The proposed SORM-based SQP approach for RBDO is accurate, efficient and robust. This is demonstrated by solving several established benchmarks, with values on the target of reliability that are considerable higher than what is commonly used, for mixtures of five different distributions (normal, lognormal, Gumbel, gamma and Weibull). Established benchmarks are also generalized in order to study problems with large number of variables and several constraints. For instance, it is shown that the proposed approach efficiently solves a problem with 300 variables and 240 constraints within less than 20 CPU minutes on a laptop. Finally, a most well-know deterministic benchmark of a welded beam is treated as a RBDO problem using the proposed SORM-based SQP approach.

The higher-order Levenberg–Marquardt method with Armijo type line search for nonlinear equations

To save more Jacobian calculations and achieve a faster convergence rate, Yang [A higher-order Levenberg-Marquardt method for nonlinear equations, Appl. Math. Comput. 219(22)(2013), pp. 10682–10694, doi:10.1016/j.amc.2013.04.033, 65H10] proposed a higher-order Levenberg–Marquardt (LM) method by computing the LM step and another two approximate LM steps for nonlinear equations. Under the local error bound condition, global and local convergence of this method is proved by using trust region technique. However, it is clear that the last two approximate LM steps may be not necessarily a descent direction, and standard line search technique cannot be used directly to obtain the convergence properties of this higher-order LM method. Hence, in this paper, we employ the nonmonotone second-order Armijo line search proposed by Zhou [On the convergence of the modified Levenberg-Marquardt method with a nonmonotone second order Armijo type line search, J. Comput. Appl. Math. 239 (2013), pp. 152–161] to guarantee the global convergence of this higher-order LM method. Moreover, the local convergence is also preserved under the local error bound condition. Numerical results show that the new method is efficient.

Three-steps modified Levenberg–Marquardt method with a new line search for systems of nonlinear equations

Three steps modified Levenberg–Marquardt method for nonlinear equations was introduced by Yang (2013). This method uses the addition of the Levenberg–Marquardt (LM) step and two approximate LM steps as the trial step at every iteration. Using trust region technique, the global and biquadratic convergence of the method is proved by Yang. The main aim of this paper is to introduce a new line search strategy while investigating the convergence properties of the method with this line search technique. Since the search direction of Yang method may be not a descent direction, standard line searches cannot be used directly. In this paper we propose a new nonmonotone third order Armijo type line search technique which guarantees the global convergence of this method while we use an adaptive LM parameter. It is proved that the convergence order of the new method is biquadratic. Numerical results show the new algorithm is efficient and promising.

Numerical Behaviors of Conjugate Gradient Methods with an Armijo Type Line Search

In this paper, we consider several modified nonlinear conjugate gradient methods with an Armijo type line search proposed in [Y. D. Dong, A practical PR+ conjugate gradient method only using gradient, Appl. Math. Comput., 219(2012)2041-2052.]. Their numerical behaviors are investigated by utilizing a class of unconstrained nonlinear problems from the CUTEr test library and a class of boundary value problems. Numerical experiments for boundary value problems illustrate that the nonlinear conjugate gradient methods with the new line search can be directly applied to some problems whose original functions may be implicit while gradient information is available. And the comparisons of numerical performances among these methods illustrate that the hybrid versions are more efficient.

Further studies on the Wei–Yao–Liu nonlinear conjugate gradient method

The Wei–Yao–Liu (WYL) method proposed by Wei et al. [Z. Wei, S. Yao, L. Liu, The convergence properties of some new conjugate gradient methods, Appl. Math. Comput. 183 (2006) 1341–1350] has been proved to converge globally for uniformly convex functions with exact line search and nonconvex functions with the strong Wolfe line search if the parameter σ∈(0,1/4). In this paper, we further study the global convergence properties of the WYL method. We show that this method still converges globally for nonconvex optimization when the strong Wolfe line search with the parameter σ=1/4 or some Armijo type line searches are used.

On the convergence of the modified Levenberg–Marquardt method with a nonmonotone second order Armijo type line search

Recently, Fan [J. Fan, The modified Levenberg–Marquardt method for nonlinear equations with cubic convergence, Math. Comput. 81 (2012) 447–466] proposed a modified Levenberg–Marquardt (MLM) method for nonlinear equations. Using a trust region technique, global and cubic convergence of the MLM method is proved by Fan (2012) [12] under the local error bound condition, which is weaker than nonsingularity. The purpose of the paper is to investigate the convergence properties of the MLM method with a line search technique. Since the search direction of the MLM method may be not a descent direction, standard line searches can not be used directly. In this paper, we propose a nonmonotone second order Armijo line search which guarantees the global convergence of the MLM method. Moreover, we prove that the unit step will be always accepted finally. Then cubic convergence of the MLM method is preserved under the local error bound condition. Some preliminary numerical results are also reported.

A short note on the global convergence of the unmodified PRP method

To guarantee global convergence of the standard (unmodified) PRP nonlinear conjugate gradient method for unconstrained optimization, the exact line search or some Armijo type line searches which force the PRP method to generate descent directions have been adopted. In this short note, we propose a non-descent PRP method in another way. We prove that the unmodified PRP method converges globally even for nonconvex minimization by the use of an approximate descent inexact line search.

Some Goldstein's type methods for co-coercive variant variational inequalities

The classical Goldstein's method has been well studied in the context of variational inequalities (VIs). In particular, it has been shown in the literature that the Goldstein's method works well for co-coercive VIs where the underlying mapping is co-coercive. In this paper, we show that the Goldstein's method can be extended to solve co-coercive variant variational inequalities (VVIs). We first show that when the Goldstein's method is applied to solve VVIs, the iterative scheme can be improved by identifying a refined step-size if the involved co-coercive modulus is known. By doing so, the allowable range of the involved scaling parameter ensuring convergence is enlarged compared to that in the context of VVIs with Lipschitz and strongly monotone operators. Then, we show that for such a VVI whose co-coercive modulus is unknown, the Goldstein's method is still convergent provided that an easily-implementable Armijo's type strategy of adjusting the scaling parameter self-adaptively is employed. Some numerical results are reported to verify that the proposed Goldstein's type methods are efficient for solving VVIs.

A nonmonotone truncated Newton–Krylov method exploiting negative curvature directions, for large scale unconstrained optimization

We propose a new truncated Newton method for large scale unconstrained optimization, where a Conjugate Gradient (CG)-based technique is adopted to solve Newton’s equation. In the current iteration, the Krylov method computes a pair of search directions: the first approximates the Newton step of the quadratic convex model, while the second is a suitable negative curvature direction. A test based on the quadratic model of the objective function is used to select the most promising between the two search directions. Both the latter selection rule and the CG stopping criterion for approximately solving Newton’s equation, strongly rely on conjugacy conditions. An appropriate linesearch technique is adopted for each search direction: a nonmonotone stabilization is used with the approximate Newton step, while an Armijo type linesearch is used for the negative curvature direction. The proposed algorithm is both globally and superlinearly convergent to stationary points satisfying second order necessary conditions. We carry out a significant numerical experience in order to test our proposal.

A descent method with inexact linear search for nonsmooth equilibrium problems

A descent method with respect to the gap function is formulated and justified for the nonsmooth equilibrium problem. It uses the procedure of inexact linear search of the Armijo type. The proposed method converges under the same assumptions as the methods with exact linear search.

A New Superlinearly Convergent Strongly Subfeasible Sequential Quadratic Programming Algorithm for Inequality-Constrained Optimization

Combining the ideas of generalized projection and the strongly subfeasible sequential quadratic programming (SQP) method, we present a new strongly subfeasible SQP algorithm for nonlinearly inequality-constrained optimization problems. The algorithm, in which a new unified step-length search of Armijo type is introduced, starting from an arbitrary initial point, produces a feasible point after a finite number of iterations and from then on becomes a feasible descent SQP algorithm. At each iteration, only one quadratic program needs to be solved, and two correctional directions are obtained simply by explicit formulas that contain the same inverse matrix. Furthermore, the global and superlinear convergence results are proved under mild assumptions without strict complementarity conditions. Finally, some preliminary numerical results show that the proposed algorithm is stable and promising.