Backtracking Line Search Technique Research Articles

A Five-Parameter Class of Derivative-Free Spectral Conjugate Gradient Methods for Systems of Large-Scale Nonlinear Monotone Equations

In this paper, we consider a system of large-scale nonlinear monotone equations and propose a class of derivative-free spectral conjugate gradient methods to solve it efficiently. We demonstrate the appropriate analytical features of this class and prove the global convergence theorems under a backtracking line search technique. In order to illustrate the numerical effectiveness of our class, we organize a competition in which 405 test problems will be solved by some members of the new class and four other similar derivative-free conjugate gradient methods. All the analytical and numerical results indicate that the presented class is promising.

Spectral Gradient Method with Log-determinant Norm for Solving Non-Linear System of Equations

Solving a system of non-linear equations has always been a complex issue whereby various methods were carried out. However, most of the methods used are optimization-based methods. This paper has modified the spectral gradient method with the backtracking line search technique to solve the non-linear systems. The efficiency of the modified spectral gradient method is tested by comparing the number of iterations, the number of function calls, and computational time with some existing methods. As a result, the proposed method shows better performance and gives more stable results than some existing methods. Moreover, it can be useful in solving some non-linear application problems. Therefore, the proposed method can be considered an alternative for solving non-linear systems.

A Derivative-Free Affine Scaling LP Algorithm Based on Probabilistic Models for Linear Inequality Constrained Optimization

In this paper, a derivative-free affine scaling linear programming algorithm based on probabilistic models is considered for solving linear inequality constrainted optimization problems. The proposed algorithm is designed to build probabilistic linear polynomial interpolation models using only n + 1 interpolation points for the objective function and reduce the computation cost for building interpolation function. We build the affine scaling linear programming methods which use probabilistic or random models and affine matrix within a classical linear programming framework. The backtracking line search technique can guarantee monotone descent of the objective function, and by using this technique, the new iterative points are located within the feasible region. Under some reasonable conditions, the global and local fast convergence of the algorithm is shown, and the results of numerical experiments are reported to show the effectiveness of the proposed algorithm.

The Riemannian two-step perturbed Gauss–Newton method for least squares inverse eigenvalue problems

In this paper, we are concerned with the parameterized least squares inverse eigenvalue problems for the case that the number of parameters to be constructed is less than the number of prescribed realizable eigenvalues. Through equivalent transformation, the original problem becomes a nonlinear least squares problem associated with a specific over-determined mapping defined between a Riemannian manifold and a Euclidean space. We propose the Riemannian two-step perturbed Gauss–Newton method combined with a specific second-order nonmonotone backtracking line search technique for solving general nonlinear least squares problem on Riemannian manifold. Global convergence of this algorithm is discussed under some mild assumptions. Meanwhile, a cubical convergence rate is obtained under injectivity of the differential of the underlying map and zero residue of this map at an accumulation point. To apply the proposed method to solving the parameterized least squares inverse eigenvalue problems, exact solution of the perturbed Riemannian Gauss–Newton equation is constructed. Finally, numerical experiments show the efficiency of the proposed method.

A Modified Spectral Gradient Projection Method for Solving Non-linear Monotone Equations with Convex Constraints and Its Application

In this paper, we propose a derivative free algorithm for solving non-linear monotone equations with convex constraints. The proposed algorithm combines the method of spectral gradient and the projection method. We also modify the backtracking line search technique. The global convergence of the proposed method is guaranteed, under the mild conditions. Further, the numerical experiments show that the large-scale non-linear equations with convex constraints can be effectively solved with our method. The $L_{1}$ -norm regularized problems in signal reconstruction are studied by using our method.

A note on hybridization process applied on transformed double step size model

We introduce a hybrid gradient model for solving unconstrained optimization problems based on one specific accelerated gradient iteration. Having applied a three term hybridization relation on transformed accelerated double step size model, we develop an efficient hybrid accelerated scheme. We determine an iterative step size variable using Backtracking line search technique in which we take an optimally calculated starting value for the posed method. In convergence analysis, we show that the proposed method is at least linearly convergent on the sets of uniformly convex functions and strictly convex quadratic functions. Numerical computations confirm a significant improvement compared with some relevant hybrid and accelerated gradient processes. More precisely, subject to the number of iterations, the CPU time metric and the number of evaluations of the objective function, defined process outperforms comparative schemes multiple times.

A modified conjugate gradient algorithm with backtracking line search technique for large-scale nonlinear equations

ABSTRACTConjugate gradient methods are widely used for solving unconstrained optimization and nonlinear equations, specially in large-scale cases. Since they own the attractive practical factors of simple computation and low memory requirement, interesting theoretical features of curvature information and strong global convergence. In this paper, we present a modified conjugate gradient algorithm by line search method with acceleration scheme for nonlinear symmetric equations. Furthermore, the proposed method not only possess descent property but also owns global convergence in mild conditions. Numerical results also indicate that the presented method is much more effective than the other methods for the test problems.

An inexact derivative-free Levenberg–Marquardt method for linear inequality constrained nonlinear systems under local error bound conditions

In this paper, a derivative-free affine scaling inexact Levenberg–Marquardt method with interior backtracking line search technique is considered for solving linear inequality constrained nonlinear systems. The proposed algorithm is designed to take advantage of the problem structure by building polynomial interpolation models for each function of nonlinear systems subject to the linear inequality constraints on variables. Each iterate switches to backtracking step generated by affine scaling inexact Levenberg–Marquardt method and satisfies strict interior point feasibility by line search backtracking technique. Under local error bounded assumption, the method is superlinear and quadratic convergent on F(x). The results of numerical experiments are reported to show the effectiveness of the proposed algorithm.

A three-terms Polak–Ribière–Polyak conjugate gradient algorithm for large-scale nonlinear equations

In this paper, a conjugate gradient algorithm for systems of large-scale nonlinear equations is designed by the following steps: (i) A three-terms conjugate gradient direction dk is presented where the direction possesses the sufficient descent property and the trust region property independent of line search technique; (ii) A backtracking line search technique along the direction is proposed to get the step length αk and construct a point; (iii) If the point satisfies the given condition then it is the next point, otherwise the projection-proximal technique is used and get the next point. Both the direction and the line search technique are the derivative-free approaches, then the large-scale nonlinear equations are successfully solved (100,000 variables). The global convergence of the given algorithm is established under suitable conditions. Numerical results show that the proposed method is efficient for large-scale problems.

Computing Weighted Analytic Center for Linear Matrix Inequalities Using Infeasible Newton’s Method

We study the problem of computing weighted analytic center for system of linear matrix inequality constraints. The problem can be solved using Standard Newton’s method. However, this approach requires that a starting point in the interior point of the feasible region be given or a Phase I problem be solved. We address the problem by using Infeasible Newton’s method applied to the KKT system of equations which can be started from any point. We implement the method using backtracking line search technique and also study the effect of large weights on the method. We use numerical experiments to compare Infeasible Newton’s method with Standard Newton’s method. The results show that Infeasible Newton’s method moves in the interior of the feasible regions often very quickly, starting from any point. We recommend it as a method for finding an interior point by setting each weight to be 1. It appears to work better than Standard Newton’s method in finding the weighted analytic center when none of weights is very large relative to the other weights. However, we find that Infeasible Newton’s method is more sensitive than Standard Newton’s method to large variation in the weights.

An affine-scaling derivative-free trust-region method for solving nonlinear systems subject to linear inequality constraints

In this paper, an affine-scaling derivative-free trust-region method with interior backtracking line search technique is considered for solving nonlinear systems subject to linear inequality constraints. The proposed algorithm is designed to take advantage of the problem structured by building polynomial interpolation models for each function in the nonlinear system function F. The proposed approach is developed by forming a quadratic model with an appropriate quadratic function and scaling matrix: there is no need to handle the constraints explicitly. By using both trust-region strategy and interior backing line search technique, each iteration switches to backtracking step generated by the trust-region subproblem and satisfies strict interior point feasibility by line search backtracking technique. Under reasonable conditions, the global convergence and fast local convergence rate of the proposed algorithm are established. The results of numerical experiments are reported to show the effectiveness of the proposed algorithms.

Non Monotone Backtracking Inexact BFGS Method for Regression Analysis

In this article, we propose a non monotone backtracking line search technique combining with BFGS (Broyden et al., 1970) method for solving regression problems. The convergent results of the given method are established under mild conditions. Numerical results show that the proposed algorithm is interesting.

A modified Fletcher-Reeves-Type derivative-free method for symmetric nonlinear equations

In this paper, we propose a descent derivative-free method for solving symmetric nonlinear equations. The method is an extension of the modified Fletcher-Reeves (MFR) method proposed by Zhang, Zhou and Li [25] to symmetric nonlinear equations. It can be applied to solve large-scale symmetric nonlinear equations due to lower storage requirement. An attractive property of the method is that the directions generated by the method are descent for the residual function. By the use of some backtracking line search technique, the generated sequence of function values is decreasing. Under appropriate conditions, we show that the proposed method is globally convergent. The preliminary numerical results show that the method is practically effective.

Inexact Newton method via Lanczos decomposed technique for solving box-constrained nonlinear systems

This paper proposes an inexact Newton method via the Lanczos decomposed technique for solving the box-constrained nonlinear systems. An iterative direction is obtained by solving an affine scaling quadratic model with the Lanczos decomposed technique. By using the interior backtracking line search technique, an acceptable trial step length is found along this direction. The global convergence and the fast local convergence rate of the proposed algorithm are established under some reasonable conditions. Furthermore, the results of the numerical experiments show the effectiveness of the proposed algorithm.

Superlinearly convergent affine scaling interior trust-region method for linear constrained LC1 minimization

We extend the classical affine scaling interior trust region algorithm for the linear constrained smooth minimization problem to the nonsmooth case where the gradient of objective function is only locally Lipschitzian. We propose and analyze a new affine scaling trust-region method in association with nonmonotonic interior backtracking line search technique for solving the linear constrained LC1 optimization where the second-order derivative of the objective function is explicitly required to be locally Lipschitzian. The general trust region subproblem in the proposed algorithm is defined by minimizing an augmented affine scaling quadratic model which requires both first and second order information of the objective function subject only to an affine scaling ellipsoidal constraint in a null subspace of the augmented equality constraints. The global convergence and fast local convergence rate of the proposed algorithm are established under some reasonable conditions where twice smoothness of the objective function is not required. Applications of the algorithm to some nonsmooth optimization problems are discussed.

An affine scaling optimal path method with interior backtracking curvilinear technique for linear constrained optimization

This paper presents an affine scaling optimal path approach in association with nonmonotonic interior backtracking line search technique for nonlinear optimization subject to linear constraints. We shall employ the eigensystem decomposition and affine scaling mapping to form affine scaling optimal curvilinear path very easily. By using interior backtracking line search technique, each iterate switches to trial step of strict interior feasibility. The nonmonotone criterion is used to speed up the convergence progress in the contours of objective function with large curvature. Theoretical analysis is given which prove that the proposed algorithm is globally convergent and has a local superlinear convergence rate under some reasonable conditions. The results of numerical experiments are reported to show the effectiveness of the proposed algorithm.

Nonmonotonic Reduced Projected Hessian Method Via an Affine Scaling Interior Modified Gradient Path for Bounded-Constrained Optimization

The authors propose an affine scaling modified gradient path method in association with reduced projective Hessian and nonmonotonic interior backtracking line search techniques for solving the linear equality constrained optimization subject to bounds on variables. By employing the QR decomposition of the constraint matrix and the eigensystem decomposition of reduced projective Hessian matrix in the subproblem, the authors form affine scaling modified gradient curvilinear path very easily. By using interior backtracking line search technique, each iterate switches to trial step of strict interior feasibility. The global convergence and fast local superlinear/quadratical convergence rates of the proposed algorithm are established under some reasonable conditions. A nonmonotonic criterion should bring about speeding up the convergence progress in some ill-conditioned cases. The results of numerical experiments are reported to show the effectiveness of the proposed algorithm.

Affine scaling interior Levenberg–Marquardt method for bound-constrained semismooth equations under local error bound conditions

We develop and analyze a new affine scaling Levenberg–Marquardt method with nonmonotonic interior backtracking line search technique for solving bound-constrained semismooth equations under local error bound conditions. The affine scaling Levenberg–Marquardt equation is based on a minimization of the squared Euclidean norm of linear model adding a quadratic affine scaling matrix to find a solution that belongs to the bounded constraints on variable. The global convergence results are developed in a very general setting of computing trial directions by a semismooth Levenberg–Marquardt method where a backtracking line search technique projects trial steps onto the feasible interior set. We establish that close to the solution set the affine scaling interior Levenberg–Marquardt algorithm is shown to converge locally Q-superlinearly depending on the quality of the semismooth and Levenberg–Marquardt parameter under an error bound assumption that is much weaker than the standard nonsingularity condition, that is, BD-regular condition under nonsmooth case. A nonmonotonic criterion should bring about speed up the convergence progress in the contours of objective function with large curvature.

A new backtracking inexact BFGS method for symmetric nonlinear equations

A BFGS method, in association with a new backtracking line search technique, is presented for solving symmetric nonlinear equations. The global and superlinear convergences of the given method are established under mild conditions. Preliminary numerical results show that the proposed method is better than the normal technique for the given problems.

An affine scaling reduced preconditional conjugate gradient path method for linear constrained optimization

This paper presents an affine scaling reduced preconditional conjugate gradient path approach in association with nonmonotonic interior backtracking line search technique for the linear constrained optimization. Employing the affine scaling preconditional conjugate gradient to form the curvilinear path and using interior backtracking line search technique, each iterate switches to trial step of strict interior feasibility. The nonmonotone criterion is used to speed up the convergence progress in the contours of objective function with large curvature. Theoretical analysis are given which prove that the proposed algorithm is globally convergent and has a local superlinear convergence rate under some reasonable conditions.