Line Search Technique Research Articles

A novel double inertial viscosity algorithm for convex bilevel optimization problems applied to image restoration problems

Convex bilevel optimization holds a crucial role in the decision-making process in a hierarchical setting, while image restoration plays a critical role in signal processing and computer vision. In this paper, a new double inertial viscosity approximation method with a linesearch technique is introduced for solving convex bilevel optimization problems and then its strong convergence is analyzed under moderate conditions. As an application, our proposed algorithm is employed for solving image restoration problems. The experiment results highlight the specific advantages and potential infect of the algorithm over the existing algorithms in the literature, particularly in the domain of image restoration.

An improved descent Perry‐type algorithm for large‐scale unconstrained nonconvex problems and applications to image restoration problems

AbstractConjugate gradient methods are much effective for large‐scale unconstrained optimization problems by their simple computations and low memory requirements. The Perry conjugate gradient method has been considered to be one of the most efficient methods in the context of unconstrained minimization. However, a globally convergent result for general functions has not been established yet. In this paper, an improved three‐term Perry‐type algorithm is proposed which automatically satisfies the sufficient descent property independent of the accuracy of line search strategy. Under the standard Wolfe line search technique and a modified secant condition, the proposed algorithm is globally convergent for general nonlinear functions without convexity assumption. Numerical results compared with the Perry method for stability, two modified Perry‐type conjugate gradient methods and two effective three‐term conjugate gradient methods for large‐scale problems up to 300,000 dimensions indicate that the proposed algorithm is more efficient and reliable than the other methods for the testing problems. Additionally, we also apply it to some image restoration problems.

A modified quasi-Newton method for uncertain multiobjective optimization problems under a finite uncertainty set

ABSTRACT In this article, a modified quasi-Newton method is developed for the robust counterpart (RC) of an uncertain multiobjective optimization problem (UMOP). Specifically, the RC of a UMOP is the minimum of an objective-wise worst-case (OWWC) type RC. To determine the descent direction for the RC, a subproblem using a common Hessian approximation is constructed. An inexact line search technique based on the Armijo technique is employed to find an appropriate step length. Furthermore, a modified version of the Davidon–Fletcher–Powell (DFP) update formula is developed for the RC. A modified quasi-Newton algorithm is built using descent direction step length, and DFP Hessian update formula. Furthermore, the convergence of the modified quasi-Newton algorithm is also discussed. Finally, the modified quasi-Newton algorithm is illustrated with a set of test problems and compared with the weighted sum method. Moreover, it is demonstrated that the algorithm is effective for both convex and non-convex problems.

Accelerated nonmonotone line search technique for multiobjective optimization

In order to increase the probability of applying more recent information, a forgetting factor is embedded in the nonmonotone line search technique for minimization of the multiobjective problem concerning the partial order induced by a closed, convex, and pointed cone. The method is shown to be globally convergent without convexity assumption on the objective function. Moreover, to improve behavior of the classical steepest descent method, an accelerated scheme is presented. Ultimately, computational advantages of the algorithms are depicted on a class of standard test problems.

Solving nonlinear structural problems by associating line-search and path-following techniques

The demand for computational tools to simulate the nonlinear behavior of structures has intensified. Regarding nonlinear static or dynamic analysis, it is fundamental to use numerical strategies to trace the structure equilibrium paths, overcoming critical points (limit and bifurcations points). The nonlinear solvers must have a high level of efficiency in the two phases of the solution process (predictor and corrector), for each load step. In solving the nonlinear equations, it is quite common that Newton-Raphson's iterations do not converge or require an excessive number of iterations near equilibrium path critical points. Therefore, the line-search optimization technique appears as an additional sophistication. Basically, this technique aims to stagger the corrective displacements vector in the iterative phase, seeking to guarantee and accelerate the convergence of the process. This paper aims to verify the efficiency of the line-search technique coupled with Newton-Raphson iterations and different path-following methods and to verify their influence on the efficiency of the nonlinear solver. The effectiveness of the implemented line-search algorithm is verified by solving two slender structures with accentuated geometric nonlinearity. Such a resource is perceived to be triggered near load limit points (with more success when applied to structures with these critical points), accelerate the iterative process and increase the chances of convergence.

Pre-conditioning CQ algorithm for solving the split feasibility problem and its application to image restoration problem

In this study, we define two pre-conditioning CQ algorithms for the split feasibility problem by using self-adaptive and line-search techniques in the real Hilbert space. One of the self-adaptive techniques is used to obtain optimal step size. These techniques do not require prior knowledge of the operator norm or estimation of the matrix norm. We also establish weak convergence theorems with respect to particular norms. As an application, numerical experiments show that the proposed algorithm is effective in image restoration problem.

A line search technique for a class of multi-objective optimization problems using subgradient

A line search technique for a class of multi-objective optimization problems using subgradient

A globally convergent SQP-type method with least constraint violation for nonlinear semidefinite programming

A globally convergent SQP-type method with least constraint violation for nonlinear semidefinite programming

An inertial‐like Tseng's extragradient method for solving pseudomonotone variational inequalities in reflexive Banach spaces

A survey of the existing results in the literature shows that several of the results on variational inequality problem were established under some stringent conditions and employed some form of linesearch technique even in the framework of Hilbert spaces. However, due to the loop nature of the linesearch technique, the implementation of such algorithms might be economically nonviable. In this article, we study the pseudomonotone variational inequality problem. We propose a new inertial‐like Tseng's extragradient method with non‐monotonic self‐adaptive step size that does not involve any linesearch technique for finding the solution of the problem in reflexive Banach spaces. Under some mild conditions, we prove a strong convergence result for the proposed method without the sequentially weakly continuity condition often assumed by authors to guarantee convergence when studying the pseudomonotone variational inequality problems. Moreover, we apply our result to study a constrained convex minimization problem. Finally, we present several theoretical and real applications to optimal control numerical experiments to illustrate and show the performance of our method in comparison with related methods in the literature.

A modified two-parameter scaled Broyden-type algorithm for unconstrained optimization problems

The Broyden-type algorithm is one of the most effective quasi-Newton methods for solving unconstrained optimization problems. However, the study of the global convergence of the Broyden-type algorithm is not enough. Motivated by Andrei (J. Comput. Appl. Math. 332: 26–44, 2018), we present a modified two-parameter scaled Broyden-type algorithm. Some approaches are used in the designed method: (1) we introduce a modified Broyden-type formula with two parameters; (2) we use approximated parameter settings to reduce the effect of eigenvalues of sequence {Bk}; (3) we propose a modified weak Wolfe–Powell line search technique with projection technique to avoid using failed search direction. The parameters of our study have the same functionality, that of correcting the structure of the eigenvalues of the approximate Hessian matrix. The difference is in the approach of 4-term Broyden updates instead of BFGS. Numerical performance shows that the modified Broyden-type algorithm is competitive with the classical Broyden approach.

Strong Convergent Inertial Two-subgradient Extragradient Method for Finding Minimum-norm Solutions of Variational Inequality Problems

AbstractIn 2012, Censor et al. (Extensions of Korpelevich’s extragradient method for the variational inequality problem in Euclidean space. Optimization 61(9):1119–1132, 2012b) proposed the two-subgradient extragradient method (TSEGM). This method does not require computing projection onto the feasible (closed and convex) set, but rather the two projections are made onto some half-space. However, the convergence of the TSEGM was puzzling and hence posted as open question. Very recently, some authors were able to provide a partial answer to the open question by establishing weak convergence result for the TSEGM though under some stringent conditions. In this paper, we propose and study an inertial two-subgradient extragradient method (ITSEGM) for solving monotone variational inequality problems (VIPs). Under more relaxed conditions than the existing results in the literature, we prove that proposed method converges strongly to a minimum-norm solution of monotone VIPs in Hilbert spaces. Unlike several of the existing methods in the literature for solving VIPs, our method does not require any linesearch technique, which could be time-consuming to implement. Rather, we employ a simple but very efficient self-adaptive step size method that generates a non-monotonic sequence of step sizes. Moreover, we present several numerical experiments to demonstrate the efficiency of our proposed method in comparison with related results in the literature. Finally, we apply our result to image restoration problem. Our result in this paper improves and generalizes several of the existing results in the literature in this direction.

Solving Large-Scale Unconstrained Optimization Problems with an Efficient Conjugate Gradient Class

The main goal of this paper is to introduce an appropriate conjugate gradient class to solve unconstrained optimization problems. The presented class enjoys the benefits of having three free parameters, its directions are descent, and it can fulfill the Dai–Liao conjugacy condition. Global convergence property of the new class is proved under the weak-Wolfe–Powell line search technique. Numerical efficiency of the proposed class is confirmed in three sets of experiments including 210 test problems and 11 disparate conjugate gradient methods.

A class of derivative free three-term descent Hestenes-Stiefel conjugate gradient algorithms for constrained nonlinear problems

This paper presents a novel method for solving large-scale nonlinear monotone equations with convex constraints. The method builds upon the conjugate gradient approach and incorporates a search direction of three terms, complemented by a modified line search. The proposed method generates a search direction that guarantees sufficient descent property, irrespective of the line search technique used. We establish the global convergence of the method under mild conditions and conduct numerical experiments to compare it with existing algorithms from the literature. The results demonstrate the method’s effectiveness and robustness, highlighting its potential for practical applications.

A scalar Hessian estimation with a sparse nonmonotone line search technique for the sparse recovery problem

A scalar Hessian estimation with a sparse nonmonotone line search technique for the sparse recovery problem

A novel nonmonotone trust region method based on the Metropolis criterion for solving unconstrained optimization

<p>In this paper, we propose a novel nonmonotone trust region method that incorporates the Metropolis criterion to construct a new function sequence. This sequence is used to update both the trust region ratio and the iteration criterion, increasing the likelihood of accepting the current trial step and introducing randomness into the iteration process. When the current trial step is not accepted, we introduce an improved nonmonotone line search technique to continue the iteration. This approach significantly reduces the number of subproblems that need to be solved, thereby saving computational resources. The stochastic nonmonotone technique helps the algorithm avoid being trapped in the local optima, and a global convergence is guaranteed under certain conditions. Numerical experiments demonstrate that the algorithm can be more effectively applied to a broader range of problems.</p>

A new accelerated algorithm with a linesearch technique for convex bilevel optimization problems with applications

A new accelerated algorithm with a linesearch technique for convex bilevel optimization problems with applications

A recent proximal gradient algorithm for convex minimization problem using double inertial extrapolations

<abstract><p>In this study, we suggest a new class of forward-backward (FB) algorithms designed to solve convex minimization problems. Our method incorporates a linesearch technique, eliminating the need to choose Lipschitz assumptions explicitly. Additionally, we apply double inertial extrapolations to enhance the algorithm's convergence rate. We establish a weak convergence theorem under some mild conditions. Furthermore, we perform numerical tests, and apply the algorithm to image restoration and data classification as a practical application. The experimental results show our approach's superior performance and effectiveness, surpassing some existing methods in the literature.</p></abstract>

A class of CG algorithms overcoming jamming of the iterative solving process and its application in image restoration

Intending to improve the numerical performance of the classic Fletcher–Reeves conjugate gradient method, Jiang and Jian embedded a shrinkage multiplier on the Fletcher–Reeves parameter which enhances its computational merits. As a matter of fact, we show that when the iterations jam, the line search turns to be approximately exact and so, search direction of the Jiang–Jian method automatically approaches the steepest descent direction which may lead to overcoming undesirable jamming effect of the classic Fletcher–Reeves method. A more detailed analysis of the numerical behavior of the Fletcher–Reeves method under the jamming phenomenon is presented as well. Then, on account of the advantages of the Jiang–Jian method, a conjugate gradient parameter is devised. Additionally, a descent spectral version of the proposed method is developed which fulfills the sufficient descent property regardless of the line search technique. Convergence of the proposed methods is addressed under classical suppositions. To gain support for our theoretical arguments, the computational merits of the given algorithms on a set of CUTEr test functions and the image restoration problem are depicted.

A Stochastic Algorithm Based on Cable-stayed Bridges Model for Stochastic Optimization in Machine Learning

This paper takes the economic cost of road design as the objective function to study the optimization of road construction cost. It is well-know that stochastic optimization is widely regarded as one of the most important and difficult problems in optimization and machine learning. It is noted that various methods for stochastic optimization problems are sensitive to the learning rate (step-size), which is usually underestimated. Therefore, two new stochastic conjugate gradient algorithms are proposed in this paper. We adapt two modified line search techniques to set the stochastic optimization. One uses the modified Armijo line search technology, the other uses the modified weak Wolfe-Powell line search technology. The new search direction has the properties of sufficient descent and trust region, which makes the establishment of global convergence of the two algorithms very understandable and clear. Finally, numerical experiments show that the two new stochastic conjugate gradient methods are competitive compared with other classical methods.

A return mapping algorithm based on the hyper dual step derivative approximation for elastoplastic models

Accurately evaluating derivatives poses a key challenge when numerically implementing complex constitutive models. This work presents an implicit stress update algorithm that utilizes the hyper dual step derivative approximation to address derivative evaluations in elastoplastic problems. Initially, the performance of various numerical differentiation methods is discussed and compared by examining their numerical errors in the representative example. Subsequently, the hyper dual step derivative approximation, without truncation and subtractive cancellation errors, is employed to compute the Jacobian matrix and consistent tangent operator, ensuring quadratic convergence in both local and global computations. The size of the Newton search step is optimized by the line search technique, thereby enhancing the convergence in solving nonlinear stress integral equations. Finally, the proposed stress update algorithm is used to implement the non-associated Mohr–Coulomb plastic model in the ABAQUS software using the UMAT subroutine. The stress update algorithm’s performance and its practical application in geotechnical engineering problems are demonstrated using five boundary value problems.