Wolfe Line Search Research Articles

Global Convergence Analysis of a new Hybrid Conjugate Gradient Method for Unconstraint Optimization Problems

In this study, we introduce a novel hybrid conjugate gradient [CG] to solve an efficient and effective unconstrained optimization problem. The parameter θk is a convex combination of the conjugate gradient method and . Under strong Wolfe line search conditions (SWC), we have shown that this method is globally convergent, and the proposed hybrid CG method is able to generate a descending search direction at each iteration. The numerical results presented, in this paper demonstrate that the proposed strategy is both effective and promising.

A New Hybrid Conjugates Gradient Algorithm for Unconstraint Optimization Problems

In this paper, we present a new hybrid conjugate gradient strategy that is both efficient and effective for solving unconstrained optimization problems. The parameter is derived from a convex combination of the and the conjugate gradient methods. We demonstrated that this strategy is globally convergent under strong Wolfe line search conditions, and that the recommended hybrid CG method is capable of creating a descending search direction at each iteration. Numerical results are presented in this study, demonstrating that the proposed technique is both efficient and promising.

A Quasi-Newton Method with Wolfe Line Searches for Multiobjective Optimization

A Quasi-Newton Method with Wolfe Line Searches for Multiobjective Optimization

New iterative conjugate gradient method for nonlinear unconstrained optimization

Conjugate gradient methods (CG) are an important class of methods for solving unconstrained optimization problems, especially for large-scale problems. Recently, they have been much studied. In this paper, we propose a new conjugate gradient method for unconstrained optimization. This method is a convex combination of Fletcher and Reeves (abbreviated FR), Polak–Ribiere–Polyak (abbreviated PRP) and Dai and Yuan (abbreviated DY) methods. The new conjugate gradient methods with the Wolfe line search is shown to ensure the descent property of each search direction. Some general convergence results are also established for this method. The numerical experiments are done to test the efficiency of the proposed method, which confirms its promising potentials.

Accurate and fast reconstruction for bioluminescence tomography based on adaptive Newton hard thresholding pursuit algorithm.

As a promising noninvasive medical imaging technique, bioluminescence tomography (BLT) dynamically offers three-dimensional visualization of tumor distribution in living animals. However, due to the high ill-posedness caused by the strong scattering property of biological tissues and the limited boundary measurements with noise, BLT reconstruction still cannot meet actual preliminary clinical application requirements. In our research, to recover 3D tumor distribution quickly and precisely, an adaptive Newton hard thresholding pursuit (ANHTP) algorithm is proposed to improve the performance of BLT. The ANHTP algorithm fully combines the advantages of sparsity constrained optimization and convex optimization to guarantee global convergence. More precisely, an adaptive sparsity adjustment strategy was developed to obtain the support set of the inverse system matrix. Based on the strong Wolfe line search criterion, a modified damped Newton algorithm was constructed to obtain optimal source distribution information. A series of numerical simulations and phantom and in vivo experiments show that ANHTP has high reconstruction accuracy, fast reconstruction speed, and good robustness. Our proposed algorithm can further increase the practicality of BLT in biomedical applications.

Performance of New Line Search Methods with Three-Term Hybrid Descent Approach for Unconstrained Optimization Problems

In this paper, l demonstrates the performance and efficiency of new line search methods with three-term hybrid descent method for the solution of unconstrained optimization problems (UOPs). The techniques advanced the sustainable range of step-length to a broader level than the previous studies and gave a suitable initial step-length at each step of the iterations. The global convergence rate of the new line with three-term hybrid descent method search is carried studied. Some numerical results through performance profile shows that among the new search method modified Wolfe line search method in CPU time and iterations is best in practical computation.

A regularized limited memory BFGS method for large-scale unconstrained optimization and its efficient implementations

The limited memory BFGS (L-BFGS) method is one of the popular methods for solving large-scale unconstrained optimization. Since the standard L-BFGS method uses a line search to guarantee its global convergence, it sometimes requires a large number of function evaluations. To overcome the difficulty, we propose a new L-BFGS with a certain regularization technique. We show its global convergence under the usual assumptions. In order to make the method more robust and efficient, we also extend it with several techniques such as the nonmonotone technique and simultaneous use of the Wolfe line search. Finally, we present some numerical results for test problems in CUTEst, which show that the proposed method is robust in terms of solving more problems.

An efficient hybrid conjugate gradient method for unconstrained optimization

In this paper, we propose a hybrid conjugate gradient method for unconstrained optimization, obtained by a convex combination of the LS and KMD conjugate gradient parameters. A favourite property of the proposed method is that the search direction satisfies the Dai–Liao conjugacy condition and the quasi-Newton direction. In addition, this property does not depend on the line search. Under a modified strong Wolfe line search, we establish the global convergence of the method. Numerical comparison using a set of 109 unconstrained optimization test problems from the CUTEst library show that the proposed method outperforms the Liu–Storey and Hager–Zhang conjugate gradient methods.

Development and Evaluation of Geometry Optimization Algorithms in Conjunction with ANI Potentials.

An efficient yet accurate method for producing a large amount of energy data for molecular mechanical force field (MMFF) parameterization is on demand, especially for torsional angle parameters which are typically derived to reproduce ab initio rotational profiles or torsional potential energy surfaces (PESs). Recently, an active learning potential (ANI-1x) for organic molecules which can produce smooth and physically meaningful PESs has been developed. The high efficiency and accuracy make ANI-1x especially attractive for geometry optimization at low cost. To apply the ANI-1x potential in MMFF parameterization, one needs to perform constrained geometry optimization. In this work, we first developed a computational protocol to constrain rotatable torsional angles and other geometric parameters for a molecule whose geometry is described by Cartesian coordinates. The constraint is successfully achieved by force projection for the two conjugated gradient (CG) algorithms. We then conducted large-scale assessments on ANI-1x along with four different optimization algorithms in reproducing DFT energies and geometries for two CG algorithms, CG backtracking line search (CG-BS) and CG Wolfe line search (CG-WS), and two quasi-Newton algorithms, Broyden-Fletcher-Goldfarb-Shanno (BFGS) and low-memory BFGS (L-BFGS). Note that CG-BS is a new algorithm we developed in this work. All four algorithms take the ANI energies and forces to optimize a molecule geometry. Last, we conducted a large-scale assessment of applying ANI-1x in MMFF development in three aspects. First, we performed full optimizations for 100 drug molecules, each consisting of five distinct conformations. The average root-mean-square error (RMSE) between ANI-1x and DFT is about 1.3 kcal/mol, and the root-mean-square displacement (RMSD) of heavy atoms is about 0.35 Å. Second, we generated torsional PESs for 160 organic molecules, and constrained optimizations were performed for up to 18 conformations for each PES. We found that the RMSE of all the conformers is 1.23 kcal/mol. Last, we carried out constrained optimizations for alanine dipeptide with both ϕ and φ angles being frozen. The Ramachandran plots indicate that the two CG algorithms in conjunction with the ANI-1x potential could well reproduce the DFT-optimized geometries and torsional PESs. We concluded that CG-BS and CG-WS are good choices for generating PESs, while CG-WS or BFGS is ideal for performing full geometry optimization. With the continuously increased quality of ANI, it is expected that the computational algorithms and protocols presented in this work will have great applications in improving the quality of an existing small-molecule MMFF.

A Globally Convergent Hybrid FR-PRP Conjugate Gradient Method for Unconstrained Optimization Problems

In this paper, a new conjugate gradient (CG) parameter is proposed through the convex combination of the Fletcher-Reeves (FR) and Polak-Ribiére-Polyak (PRP) CG update parameters such that the conjugacy condition of Dai-Liao is satisfied. The computational efficiency of the PRP method and the convergence profile of the FR method motivated the choice of these two CG methods. The corresponding CG algorithm satisfies the sufficient descent property and was shown to be globally convergent under the strong Wolfe line search procedure. Numerical tests on selected benchmark test functions show that the algorithm is efficient and very competitive in comparison with some existing classical methods.

Improvement of conjugate gradient methods for removing impulse noise images

Optimization problems occur in most disciplines like engineering, physics, mathematics, economics, administration, commerce, social sciences, and even politics. The conjugate coefficient is the cornerstone of conjugate gradient algorithms with the desired conjugate property. In this study, we discovered fresh second order information for the Hessian from the target function, which might lead to a new search direction. Based on a unique search direction, we proposed the update formula and nonlinear conjugate gradient technique. Under Wolfe line search and moderate objective function assumptions, the strategy has acceptable descent property and is always globally convergent. According to numerical results, the technique is successful and competitive in recovering the original picture from an image corrupted by impulsive noise.

Nonlinear Conjugate Gradient Coefficients with Exact and Strong Wolfe Line Searches Techniques

Nonlinear conjugate gradient (CG) methods are very important for solving unconstrained optimization problems. These methods have been subjected to extensive researches in terms of enhancing them. Exact and strong Wolfe line search techniques are usually used in practice for the analysis and implementation of conjugate gradient methods. For better results, several studies have been carried out to modify classical CG methods. The method of Fletcher and Reeves (FR) is one of the most well‐known CG methods. It has strong convergence properties, but it gives poor numerical results in practice. The main goal of this paper is to enhance this method in terms of numerical performance via a convexity type of modification on its coefficient βk. We ensure that with this modification, the method is still achieving the sufficient descent condition and global convergence via both exact and strong Wolfe line searches. The numerical results show that this modified FR is more robust and effective.

CONJUGATE GRADIENT WITH SUBSPACE MINIMIZATION BASED ON CUBIC REGULARIZATION MODEL OF THE MINIMIZING FUNCTION

A new algorithm for unconstrained optimization based on the cubic regularization in two dimensional subspace is developed. Different strategies for search direction are also discussed. The stepsize is computed by means of the weak Wolfe line search. Under classical assumptions it is proved that the algorithm is convergent. Intensive numerical experiments with 800 unconstrained optimization test functions with the number of variables in the range [1000 - 10,000] show that the suggested algorithm is more efficient and more robust than the well established conjugate gradient algorithms CG-DESCENT, CONMIN and L-BFGS (m=5). Comparisons of the suggested algorithm versus CG-DESCENT for solving five applications from MINPACK-2 collection, each of them with 40,000 variables, show that CUBIC is 3.35 times faster than CG-DESCENT.

A convergent hybrid three-term conjugate gradient method with sufficient descent property for unconstrained optimization

Abstract Conjugate gradient methods are very popular for solving large scale unconstrained optimization problems because of their simplicity to implement and low memory requirements. In this paper, we present a hybrid three-term conjugate gradient method with a direction that always satisfies the sufficient descent condition. We establish global convergence of the new method under the weak Wolfe line search conditions. We also report some numerical results of the proposed method compared to relevant methods in the literature.

A new modified conjugate gradient method under the strong Wolfe line search for solving unconstrained optimization problems

Conjugate gradient (CG) method is well-known due to efficiency to solve the problems of unconstrained optimization because of its convergence properties and low computation cost. Nowadays, the method is widely developed to compete with existing methods in term of their efficiency. In this paper, a modification of CG method will be proposed under strong Wolfe line search. A new CG coefficient is presented based on the idea of make use some parts of the previous existing CG methods to retain the advantages. The proposed method guarantees that the sufficient descent condition holds and globally convergent under inexact line search. Numerical testing provides strong indication that the proposed method has better capability when solving unconstrained optimization compared to the other methods under inexact line search specifically strong Wolfe–Powell line search.

Global convergence of a descent PRP type conjugate gradient method for nonconvex optimization

Nonlinear conjugate gradient methods are often used to solve large-scale unconstrained optimization problems owing to their lower memory requirement and less computation cost. In this paper, we propose a new Polak-Ribiére-Polyak type conjugate gradient method, which satisfies the sufficient descent condition independent of any line search. A remarkable property about this method is its strong global convergence with the standard Wolfe line search conditions as well as the standard Armijo line search strategy without convexity assumption of the objective function. The numerical results demonstrating the efficiency of the proposed method are reported.

A new family of Dai-Liao conjugate gradient methods with modified secant equation for unconstrained optimization

In this paper, a new family of Dai-Liao–type conjugate gradient methods are proposed for unconstrained optimization problem. In the new methods, the modified secant equation used in [H. Yabe and M. Takano, Comput. Optim. Appl. 28 (2004) 203–225] is considered in Dai and Liao’s conjugacy condition. Under some certain assumptions, we show that our methods are globally convergent for general functions with strong Wolfe line search. Numerical results illustrate that our proposed methods can outperform some existing ones.

Two Improved Nonlinear Conjugate Gradient Methods with the Strong Wolfe Line Search

Two Improved Nonlinear Conjugate Gradient Methods with the Strong Wolfe Line Search

A new class of nonlinear conjugate gradient coefficients for unconstrained optimization

The nonlinear Conjugate gradient method (CGM) is a very effective way in solving large-scale optimization problems. Zhang et al. proposed a new CG coefficient which is defined by [Formula: see text]. They proved the sufficient descent condition and the global convergence for nonconvex minimization in strong Wolfe line search. In this paper, we prove that this CG coefficient possesses sufficient descent conditions and global convergence properties under the exact line search.

A dai-liao hybrid conjugate gradient method for unconstrained optimization

One of todays’ best-performing CG methods is Dai-Liao (DL) method which depends on non-negative parameter and conjugacy conditions for its computation. Although numerous optimal selections for the parameter were suggested, the best choice of remains a subject of consideration. The pure conjugacy condition adopts an exact line search for numerical experiments and convergence analysis. Though, a practical mathematical experiment implies using an inexact line search to find the step size. To avoid such drawbacks, Dai and Liao substituted the earlier conjugacy condition with an extended conjugacy condition. Therefore, this paper suggests a new hybrid CG that combines the strength of Liu and Storey and Conjugate Descent CG methods by retaining a choice of Dai-Liao parameterthat is optimal. The theoretical analysis indicated that the search direction of the new CG scheme is descent and satisfies sufficient descent condition when the iterates jam under strong Wolfe line search. The algorithm is shown to converge globally using standard assumptions. The numerical experimentation of the scheme demonstrated that the proposed method is robust and promising than some known methods applying the performance profile Dolan and Mor´e on 250 unrestricted problems. Numerical assessment of the tested CG algorithms with sparse signal reconstruction and image restoration in compressive sensing problems, file restoration, image video coding and other applications. The result shows that these CG schemes are comparable and can be applied in different fields such as temperature, fire, seismic sensors, and humidity detectors in forests, using wireless sensor network techniques.