Nonmonotone Line Search Technique Research Articles

Accelerated nonmonotone line search technique for multiobjective optimization

In order to increase the probability of applying more recent information, a forgetting factor is embedded in the nonmonotone line search technique for minimization of the multiobjective problem concerning the partial order induced by a closed, convex, and pointed cone. The method is shown to be globally convergent without convexity assumption on the objective function. Moreover, to improve behavior of the classical steepest descent method, an accelerated scheme is presented. Ultimately, computational advantages of the algorithms are depicted on a class of standard test problems.

A scalar Hessian estimation with a sparse nonmonotone line search technique for the sparse recovery problem

A scalar Hessian estimation with a sparse nonmonotone line search technique for the sparse recovery problem

A Barzilai-Borwein descent method for multiobjective optimization problems

The steepest descent method proposed by Fliege and Svaiter has motivated the research on descent methods for multiobjective optimization, which has received increasing attention in recent years. However, empirical results show that the Armijo line search often results in a very small stepsize along the steepest descent direction, which decelerates the convergence seriously. This paper points out the issue is mainly due to imbalances among objective functions. To address this issue, we propose a Barzilai-Borwein descent method for multiobjective optimization (BBDMO), which dynamically tunes gradient magnitudes using Barzilai-Borwein’s rule in direction-finding subproblem. We emphasize that the BBDMO produces a sequence of new descent directions compared to Barzilai-Borwein’s method proposed by Morovati et al. With monotone and nonmonotone line search techniques, we prove that accumulation points generated by BBDMO are Pareto critical points, respectively. Furthermore, theoretical results indicate that the Armijo line search can achieve a better stepsize in BBDMO. Finally, comparative results of numerical experiments are reported to illustrate the efficiency of BBDMO and verify the theoretical results.

An accelerated alternating directional method with non-monotone technique for matrix recovery

<abstract><p>Low-rank and sparse structures have been frequently exploited in matrix recovery and robust PCA problems. In this paper, we develop an alternating directional method and its variant equipped with the non-monotone search procedure for solving a non-convex optimization model of low-rank and sparse matrix recovery problems, where the concerned matrix with incomplete data is separable into a low-rank part and a sparse part. The main idea is to use the alternating minimization method for the low-rank matrix part, and to use the non-monotone line search technique for the sparse matrix part to iteratively update, respectively. To some extent, the non-monotone strategy relaxes the single-step descent into a multi-step descent and then greatly improves the performance of the alternating directional method. Theoretically, we prove the global convergence of the two proposed algorithms under some mild conditions. Finally, the comparison of numerical experiments shows that the alternate directional method with non-monotone technique is more effective than the original monotone method and the previous method. The efficiency and effectiveness of the proposed algorithms are demonstrated by solving some instances of random incomplete matrix recovery problems and some problems of the background modeling in video processing.</p></abstract>

Variable Metric Method for Unconstrained Multiobjective Optimization Problems

In this paper, we propose a variable metric method for unconstrained multiobjective optimization problems (MOPs). First, a sequence of points is generated using different positive definite matrices in the generic framework. It is proved that accumulation points of the sequence are Pareto critical points. Then, without convexity assumption, strong convergence is established for the proposed method. Moreover, we use a common matrix to approximate the Hessian matrices of all objective functions, along which a new nonmonotone line search technique is proposed to achieve a local superlinear convergence rate. Finally, several numerical results demonstrate the effectiveness of the proposed method.

An Adaptive Nonmonotone Line Search Technique for Solving Systems of Nonlinear Equations

In this article, a new nonmonotone line search technique is proposed for solving a system of nonlinear equations. We attempt to answer this question how to control the degree of the nonmonotonicity of line search rules in order to reach a more efficient algorithm? Therefore, we present a novel algorithm that can avoid the increase of unsuccessful iterations. For this purpose, we show the robust behavior of the proposed algorithm by solving a few numerical examples. Under some suitable assumptions, the global convergence of our strategy is proved.

An inexact projected LM type algorithm for solving convex constrained nonlinear equations

In this paper, we propose two complementary variants of the projected Levenberg–Marquardt (LM) algorithm for solving convex constrained nonlinear equations. Since the orthogonal projection onto the feasible set may be computationally expensive, we first propose a local LM algorithm in which inexact projections are allowed. The feasible inexact projections used in our algorithm can be easily obtained by means of iterative methods, such as conditional gradient. Local convergence of the proposed algorithm is established by using an error bound condition which is weaker than the standard full-rank assumption. We further present and analyze a global version of this algorithm by means of a nonmonotone line search technique. Numerical experiments are reported to showcase the effectiveness of the proposed algorithms, especially when the projection onto the feasible set is difficult to compute.

Two adaptive scaled gradient projection methods for Stiefel manifold constrained optimization

This article is concerned with the problem of minimizing a smooth function over the Stiefel manifold. In order to address this problem, we introduce two adaptive scaled gradient projection methods that incorporate scaling matrices that depend on the step-size and a parameter that controls the search direction. These iterative algorithms use a projection operator based on the QR factorization to preserve the feasibility in each iteration. However, for some particular cases, the proposals do not require the use of any projection operator. In addition, we consider a Barzilai and Borwein-like step-size combined with the Zhang–Hager nonmonotone line-search technique in order to accelerate the convergence of the proposed procedures. We proved the global convergence for these schemes, and we evaluate their effectiveness and efficiency through an extensive computational study, comparing our approaches with other state-of-the-art gradient-type algorithms.

An improvement of adaptive cubic regularization method for unconstrained optimization problems

In this paper, we present two nonmonotone versions of adaptive cubic regularized (ARC) method for unconstrained optimization problems. The proposed methods are a combination of the ARC algorithm with the nonmonotone line search methods introduced by Zhang and Hager [A nonmonotone line search technique and its application to unconstrained optimization, SIAM J. Optim. 14 (2004), pp. 1043–1056] and Ahookhosh et al. [A nonmonotone trust-region line search method for large-scale unconstrained optimization, Appl. Math. Model. 36 (2012), pp. 478–487]. The global convergence analysis for these iterative algorithms is established under suitable conditions. Several numerical examples are given to illustrate the efficiency and robustness of the newly suggested methods. The obtained results show the satisfactory performance of the proposed algorithms when compared to the basic ARC algorithm.

A Riemannian Optimization Approach for Solving the Generalized Eigenvalue Problem for Nonsquare Matrix Pencils

In this paper, based on the Riemannian optimization approach we propose a Riemannian nonlinear conjugate gradient method with nonmonotone line search technique for solving the l parameterized original problem on generalized eigenvalue problems for nonsquare matrix pencils, which was first proposed by Chu and Golub (SIAM J Matrix Anal Appl 28:770–787, 2006). The new innovative approach is to reformulate the original optimization problem as a feasible optimization problem over a certain real product manifold. The global convergence of the proposed method is then established. Some numerical tests are given to demonstrate the efficiency of the proposed method. Comparisons with some latest methods are also given.

An alternating iterative algorithm for sound source identification based on equivalent source method

Near-field acoustical holography (NAH) based on equivalent source method (ESM) is an efficient technique for sound source identification. Conventional ESM with Tikhonov regularization (TRESM), ESM based on CVX MATLAB toolbox (CVX) and wideband acoustic holography (WBH) are commonly used methods for calculating equivalent source strengths. However, all of them have their respective limitations. To address some of these, an alternating iterative algorithm for sound source identification based on equivalent source method (AIESM) is proposed in this article, which is a combination of alternating direction method and a non-monotone line search technique. The method makes use of sparse regularization under the principle of compressive sensing (CS) to calculate equivalent source strengths. Moreover, inspired by the idea of functional beamforming (FB), AIESM with order n can yield an improved dynamic range when detecting the source location. Numerical simulations are carried out at different frequencies, and the results suggest that the computational efficiency of the proposed method is close to that of TRESM. In addition, AIESM has a better reconstruction accuracy than TRESM and WBH in a relatively wide frequency range. Compared with ESM based on CVX, AIESM is slightly better in reconstruction accuracy and has a higher computational efficiency. Meanwhile, AIESM with order n can provide more accurate source position and better resolution. The validity and practicality of the proposed method are further supported by experimental results.

A Scaled Conjugate Gradient Method Based on New BFGS Secant Equation with Modified Nonmonotone Line Search

In this paper, we provide and analyze a new scaled conjugate gradient method and its performance, based on the modified secant equation of the Broyden-Fletcher-Goldfarb-Shanno (BFGS) method and on a new modified nonmonotone line search technique. The method incorporates the modified BFGS secant equation in an effort to include the second order information of the objective function. The new secant equation has both gradient and function value information, and its update formula inherits the positive definiteness of Hessian approximation for general convex function. In order to improve the likelihood of finding a global optimal solution, we introduce a new modified nonmonotone line search technique. It is shown that, for nonsmooth convex problems, the proposed algorithm is globally convergent. Numerical results show that this new scaled conjugate gradient algorithm is promising and efficient for solving not only convex but also some large scale nonsmooth nonconvex problems in the sense of the Dolan-Moré performance profiles.

A Simulated Annealing-Based Barzilai–Borwein Gradient Method for Unconstrained Optimization Problems

In this paper, we propose a simulated annealing-based Barzilai–Borwein (SABB) gradient method for unconstrained optimization problems. The SABB method accepts the Barzilai–Borwein (BB) step by a simulated annealing rule. If the BB step cannot be accepted, the Armijo line search is used. The global convergence of the SABB method is established under some mild conditions. Numerical experiments indicate that, compared to some existing BB methods using nonmonotone line search technique, the SABB method performs well with high efficiency.

An amplitude-preserving deblending approach for simultaneous sources

A goal of simultaneous shooting is to acquire high-quality seismic data more efficiently, while reducing operational costs and improving acquisition efficiency. Effective sampling and deblending techniques are essential to achieve this goal. Inspired by compressive sensing (CS), we have formulated deblending as an analysis-based sparse inversion problem. We solve the inversion problem with an algorithm derived from the classic alternating direction method (ADM), associated with variable splitting and nonmonotone line-search techniques. In our testing, the analysis-based formulation together with nonmonotone ADM algorithm provides improved performance compared with synthesis-based approaches. A major issue for all deblending approaches is how to deal with real-world variations in seismic data caused by static shifts and amplitude imbalances. We evaluate the concept of including static and amplitude corrections obtained from surface-consistent solutions into the deblending formulation. We implement solutions that use a multistage inversion scheme to overcome the practical issues embedded in the field-blended data, such as strong coherent noise, statics, and shot-amplitude variations. The combination of these techniques gives high-fidelity deblending results for marine and land data. We use two field-data examples acquired with simultaneous sources to demonstrate the effectiveness of the proposed approach. Imaging and amplitude variation with offset quantitative analysis are carried out to indicate the amplitude-preserving character of deblended data with this methodology.

A class of accelerated conjugate-gradient-like methods based on a modified secant equation

This paper proposes a new class of accelerated conjugate-gradient-like algorithms for solving large scale unconstrained optimization problems, which combine the idea of accelerated adaptive Perry conjugate gradient algorithms proposed by Andrei (2017) with the modified secant condition and the nonmonotone line search technique. An attractive property of the proposed methods is that the search direction always provides sufficient descent step which is independent of the line search used and the convexity of objective function. Under common assumptions, it is proven that the proposed methods possess global convergence for nonconvex smooth functions, and R-linear convergence for uniformly convex functions. The numerical experiments show the efficiency of the proposed method in practical computations.

Hypergraph Matching With an Entropy Barrier Function

Graph matching problem has been widely used in many fields. Due to the discrete property of graph matching problem, the continuous relaxation will result in a bad influence on the solution. Thus, we propose a hypergraph matching model with an entropy barrier function considering the discrete property of the assignment matrix. The model is solved by a nonmonotone active set projected Newton method whose nonmonotone line search technique has a good ability to accept the projected Newton stepsize. We evaluate our method through several experiments, which indicates that our method has better matching results than others.

A Levenberg–Marquardt Method for Nonlinear Complementarity Problems Based on Nonmonotone Trust Region and Line Search Techniques

Using the FB function, we propose a new Levenberg–Marquardt algorithm for nonlinear complementarity problem. To obtain the global convergence, the algorithm uses the nonmonotone trust region and line search techniques under a convenient boundedness assumption. Furthermore, we get local superlinear/quadratic convergence of the algorithm under a nonsingularity condition. Some numerical examples are given to illustrate the performance and efficiency of the presented algorithm.

A variant nonmonotone smoothing algorithm with improved numerical results for large-scale LWCPs

This paper considers the linear weighted complementarity problem (denoted by LWCP) introduced by Potra (SIAM J Optim 22:1634–1654, 2012). Based on two weighted smoothing functions, we propose a new nonmonotone smoothing algorithm for solving the LWCP and establish its global and local quadratic convergence without the strict complementarity assumption. Compared to existing nonmonotone smoothing algorithms, the proposed algorithm solves the linear system only approximately which can save the computation work when one solves large-scale LWCPs. Moreover, the nonmonotone line search technique adopted in this paper includes the usual monotone line search and some existing nonmonotone line searches as special cases. Numerical results show that our algorithm is considerably efficient for solving large-scale LWCPs.

An extended nonmonotone line search technique for large-scale unconstrained optimization

In this paper, an extended nonmonotone line search is proposed to improve the efficiency of the existing line searches. This line search is first proved to be an extension of the classical line search rules. On the one hand, under mild assumptions, global convergence and R-linear convergence are established for the new line search rule. On the other hand, by numerical experiments, it is shown that the line search can integrate the advantages of the existing methods in searching for a suitable step-size. Combined with the spectral step-size, a class of spectral gradient algorithms are developed and employed to solve a large number of benchmark test problems from CUTEst. Numerical results show that the new line search is promising in solving large-scale optimization problems, and outperforms the other similar ones as it is combined with a spectral gradient method.

A modified nonmonotone BFGS algorithm for unconstrained optimization

In this paper, a modified BFGS algorithm is proposed for unconstrained optimization. The proposed algorithm has the following properties: (i) a nonmonotone line search technique is used to obtain the step size alpha_{k} to improve the effectiveness of the algorithm; (ii) the algorithm possesses not only global convergence but also superlinear convergence for generally convex functions; (iii) the algorithm produces better numerical results than those of the normal BFGS method.