Spectral Conjugate Gradient Research Articles

A New Spectral Conjugate Gradient Method for Nonlinear Unconstrained Optimization

Theconjugate gradient method is widely used to solve large scale unconstrainedoptimization problems. However, the rate of convergence conjugate gradient method is linearunless it restarted. In this study, we present a new spectral conjugategradient modification formula with restart property obtains the globalconvergence and descent properties.In addition, we proposed a new restart condition for Fletcher-Reeves conjugate gradient formula. The numerical resultsdemonstrated that the modified Fletcher-Reeves parameter and the new CG formulawith their restart conditions are more efficient and robustness than otherconventional methods.

A new three-term spectral conjugate gradient algorithm with higher numerical performance for solving large scale optimization problems based on Quasi-Newton equation

A new spectral three-term conjugate gradient algorithm in virtue of the Quasi-Newton equation is developed for solving large-scale unconstrained optimization problems. It is proved that the search directions in this algorithm always satisfy a sufficiently descent condition independent of any line search. Global convergence is established for general objective functions if the strong Wolfe line search is used. Numerical experiments are employed to show its high numerical performance in solving large-scale optimization problems. Particularly, the developed algorithm is implemented to solve the 100 benchmark test problems from CUTE with different sizes from 1000 to 10,000, in comparison with some similar ones in the literature. The numerical results demonstrate that our algorithm outperforms the state-of-the-art ones in terms of less CPU time, less number of iteration or less number of function evaluation.

Global Convergence Condition for a New Spectral Conjugate Gradient Method for Large-Scale Optimization

The spectral conjugate gradient (SCG) method is an effective method to solve large-scale nonlinear unconstrained optimization problems. In this work, a new spectral conjugate gradient method is proposed with a strong Wolfe-Powell line search (SWP). The idea of the new one is using the βBZA formula which is proposed by Baluch and et al., with suitable parameter φ denoted by (SCGBZA). Under the usual assumptions, the descent properties and overall global convergence of the proposed method (SCGBZA) are proved. The proposed method is numerically proven to be effective.

A fast spectral conjugate gradient method for solving nonlinear optimization problems

<span>This paper proposes a new spectral conjugate gradient (SCG) approach for solving unregulated nonlinear optimization problems. Our approach proposes Using Wolfe's rapid line scan to adjust the standard conjugate descent (CD) algorithm. A new spectral parameter is a mixture of new gradient and old search path. The path provided by the modified method provides a path of descent for the solution of objective functions. The updated method fits the traditional CD method if the line check is correct. The stability and global convergence properties of the current new SCG are technically obtained from applying certain well-known and recent mild assumptions. We test our approach with eight recently published CD and SCG methods on 55 optimization research issues from the CUTE library. The suggested and all other algorithms included in our experimental research were implemented in FORTRAN language with double precision arithmetic and all experiments were conducted on a PC with 8 GB ram Processor Intel Core i7. The results indicate that our proposed solution outperforms recently reported algorithms by processing and performing fewer iterations in a shorter time.</span>

An efficient DY-type spectral conjugate gradient method for system of nonlinear monotone equations with application in signal recovery

<abstract> Many problems in engineering and social sciences can be transformed into system of nonlinear equations. As a result, a lot of methods have been proposed for solving the system. Some of the classical methods include Newton and Quasi Newton methods which have rapid convergence from good initial points but unable to deal with large scale problems due to the computation of Jacobian matrix or its approximation. Spectral and conjugate gradient methods proposed for unconstrained optimization, and later on extended to solve nonlinear equations do not require any computation of Jacobian matrix or its approximation, thus, are suitable to handle large scale problems. In this paper, we proposed a spectral conjugate gradient algorithm for solving system of nonlinear equations where the operator under consideration is monotone. The search direction of the proposed algorithm is constructed by taking the convex combination of the Dai-Yuan (DY) parameter and a modified conjugate descent (CD) parameter. The proposed search direction is sufficiently descent and under some suitable assumptions, the global convergence of the proposed algorithm is proved. Numerical experiments on some test problems are presented to show the efficiency of the proposed algorithm in comparison with an existing one. Finally, the algorithm is successfully applied in signal recovery problem arising from compressive sensing. </abstract>

A Spectral RMIL+ Conjugate Gradient Method for Unconstrained Optimization With Applications in Portfolio Selection and Motion Control

The Spectral conjugate gradient (SCG) methods are among the efficient variants of CG algorithms which are obtained by combining the spectral gradient parameter and CG parameter. The success of SCG methods relies on effective choices of the step-size $\alpha _{k}$ and the search direction $d_{k}$ . This paper presents an SCG method for unconstrained optimization models. The search directions generated by the new method possess sufficient descent property without the restart condition and independent of the line search procedure used. The global convergence of the new method is proved under the weak Wolfe line search. Preliminary numerical results are presented which show that the method is efficient and promising, particularly for large-scale problems. Also, the method was applied to solve the robotic motion control problem and portfolio selection problem.

A spectral conjugate gradient method for convex constrained monotone equations

A spectral conjugate gradient method for convex constrained monotone equations

Taking the 4D Nature of fMRI Data Into Account Promises Significant Gains in Data Completion.

Functional magnetic resonance imaging (fMRI) is a powerful, noninvasive tool that has significantly contributed to the understanding of the human brain. FMRI data provide a sequence of whole-brain volumes over time and hence are inherently four dimensional (4D). Missing data in fMRI experiments arise from image acquisition limits, susceptibility and motion artifacts or during confounding noise removal. Hence, significant brain regions may be excluded from the data, which can seriously undermine the quality of subsequent analyses due to the significant number of missing voxels. We take advantage of the four dimensional (4D) nature of fMRI data through a tensor representation and introduce an effective algorithm to estimate missing samples in fMRI data. The proposed Riemannian nonlinear spectral conjugate gradient (RSCG) optimization method uses tensor train (TT) decomposition, which enables compact representations and provides efficient linear algebra operations. Exploiting the Riemannian structure boosts algorithm performance significantly, as evidenced by the comparison of RSCG-TT with state-of-the-art stochastic gradient methods, which are developed in the Euclidean space. We thus provide an effective method for estimating missing brain voxels and, more importantly, clearly show that taking the full 4D structure of fMRI data into account provides important gains when compared with three-dimensional (3D) and the most commonly used two-dimensional (2D) representations of fMRI data.

The Performance Analysis of a New Modification of Conjugate Gradient Parameter for Unconstrained Optimization Models

Conjugate Gradient (CG) method is the most prominent iterative mathematical technique that can be useful for the optimization of both linear and non-linear systems due to its simplicity, low memory requirement, computational cost, and global convergence properties. However, some of the classical CG methods have some drawbacks which include weak global convergence, poor numerical performance both in terms of number of iterations and the CPU time. To overcome these drawbacks, researchers proposed new variants of the CG parameters with efficient numerical results and nice convergence properties. Some of the variants of the CG method include the scale CG method, hybrid CG method, spectral CG method, three-term CG method, and many more. The hybrid conjugate gradient (CG) algorithm is among the efficient variant in the class of the conjugate gradient methods mentioned above. Some interesting features of the hybrid modifications include inherenting the nice convergence properties and efficient numerical performance of the existing CG methods. In this paper, we proposed a new hybrid CG algorithm that inherits the features of the Rivaie et al. (RMIL*) and Dai (RMIL+) conjugate gradient methods. The proposed algorithm generates a descent direction under the strong Wolfe line search conditions. Preliminary results on some benchmark problems show that the proposed method efficient and promising.

A New Spectral on the Gradient Methods for Unconstrained Optimization Minimization Problem

One simple and well-known method for minimizing the functions is a spectral conjugate gradient method. In this paper, we derive a new spectral on the method of gradient, which can give a new path of search. The new spectral method holds the property of descent and we have shown that the spectral method is convergent globally. The empirical results show that for the test problems, the given approach is competitive with the other conjugate gradient methods.

Application of Spectral PRP Conjugate Gradient Parameter for Unconstrained Optimization Problems

Conjugate Gradient (CG) method have been utilised to solve nonlinear unconstrained optimization problems because of less storage locations and fewer computational cost in dealing with large-scale problems. In this paper, we present a real life application of spectral PRP Conjugate Gradient method in regression analysis, the proposed method is suitably deriving from the CG search direction without secant condition. Some benchmark functions with several variables have been use to prove the global convergence properties and satisfies sufficient descent condition. The numerical results are certifying by exact line search techniques; the method outperform the prominent least square method

A new spectral conjugate gradient method with descent condition and global convergence property for unconstrained optimization

The Spectral conjugate gradient method is an efficient method for solving large-scale unconstrained optimization problems. In this paper, we propose a new spectral conjugate gradient method in which performance is analyzed numerically. We establish the descent condition and global convergence property under some assumptions and the strong Wolfe line search. Numerical experiments to evaluate the method’s efficiency are conducted using 98 problems with various dimensions and initial points. The numerical results based on the number of iterations and central processing unit time show that the new method has a high performance computational.

An integrated stochastic model and algorithm for multi-product newsvendor problems

In this paper, a multi-product newsvendor problem is formulated as a random nonlinear integrated optimization model by taking into consideration the selling price, the producing and outsourcing quantities, and the nonlinear budget constraint. Different from the existing models, the demands of products depend on the prices, as well as being time-varying due to random market fluctuation. In addition, outsourcing strategy is adopted to deal with possible shortage caused by the limited capacity. Consequently, the constructed model is involved with joint optimization of the producing and outsourcing quantities, and the selling prices of all the products. For this model with continuous random demands, we first transform it into a nonlinear programming problem by expectation method. Then, an efficient algorithm, called the feasible-direction-based spectral conjugate gradient algorithm, is developed to find a robust solution of the model. By case study and sensitivity analysis, some interesting conclusions are drawn as follows: (a) Budget is a critical constraint for optimizing the decision-making of the retailer, and there exist different threshold values of the budget for the substitute and complementarity scenarios. (b) The price sensitivity matrix seriously affects the maximal expected profit mainly through affecting the optimal outsourcing quantity.

A Modified Bat Algorithm with Conjugate Gradient Method for Global Optimization

Metaheuristic algorithms are used to solve many optimization problems. Firefly algorithm, particle swarm improvement, harmonic search, and bat algorithm are used as search algorithms to find the optimal solution to the problem field. In this paper, we have investigated and analyzed a new scaled conjugate gradient algorithm and its implementation, based on the exact Wolfe line search conditions and the restart Powell criterion. The new spectral conjugate gradient algorithm is a modification of the Birgin and Martínez method, a manner to overcome the lack of positive definiteness of the matrix defining the search direction. The preliminary computational results for a set of 30 unconstrained optimization test problems show that this new spectral conjugate gradient outperforms a standard conjugate gradient in this field and we have applied the newly proposed spectral conjugate gradient algorithm in bat algorithm to reach the lowest possible goal of bat algorithm. The newly proposed approach, namely, the directional bat algorithm (CG-BAT), has been then tested using several standard and nonstandard benchmarks from the CEC’2005 benchmark suite with five other algorithms and has been then tested using nonparametric statistical tests and the statistical test results show the superiority of the directional bat algorithm, and also we have adopted the performance profiles given by Dolan and More which show the superiority of the new algorithm (CG-BAT).

Application of spectral conjugate gradient methods for solving unconstrained optimization problems

Conjugate gradient (CG) methods are among the most efficient numerical methods for solving unconstrained optimization problems. This is due to their simplicty and less computational cost in solving large-scale nonlinear problems. In this paper, we proposed some spectral CG methods using the classical CG search direction. The proposed methods are applied to real-life problems in regression analysis. Their convergence proof was establised under exact line search. Numerical results has shown that the proposed methods are efficient and promising.

The new spectral conjugate gradient method for large-scale unconstrained optimisation

The spectral conjugate gradient methods are very interesting and have been proved to be effective for strictly convex quadratic minimisation. In this paper, a new spectral conjugate gradient method is proposed to solve large-scale unconstrained optimisation problems. Motivated by the advantages of approximate optimal stepsize strategy used in the gradient method, we design a new scheme for the choices of the spectral and conjugate parameters. Furthermore, the new search direction satisfies the spectral property and sufficient descent condition. Under some suitable assumptions, the global convergence of the developed method is established. Numerical comparisons show better behaviour of the proposed method with respect to some existing methods for a set of 130 test problems.

The spectral conjugate gradient method in variational adjoint assimilation for model terrain correction I: Theoretical frame

A spectral conjugate gradient (SCG) method is proposed within the mathematical framework of a variational adjoint assimilation system to correct the bottom terrain of a shallow-water equations model. The formulation of this method is described from the mathematical point of view with determination of the descent direction by using Andrei’s limited-memory form and the step length by solving the tangent linear model. It is proved to benefit from (i) the iterative regularization strategy, (ii) the inverse Hessian approximation involving the second-order information from Broyden-Fletcher-Goldfarb-Shanno class (BFGS), and (iii) the optimal step length. The regularization term introduced to the cost function will allow the incorporation of the known information about the desired bottom terrain and guarantee the uniqueness of the optimal solution.

The spectral conjugate gradient method in variational adjoint assimilation for model terrain correction II: Numerical test

The performance of the spectral conjugate gradient (SCG) method proposed in part I of this paper is evaluated by comparison with that of the limited-memory Broyden-Fletcher-Goldfarb-Shanno (LBFGS) method and Hager and Zhang’s CG_DESCENT method. The step length is determined by solving the tangent linear model and a line search algorithm with cubic interpolation is introduced for comparison. Numerical tests indicate that the SCG method is effective at correcting the bottom terrain. It is robust, providing small root mean square (RMS) errors and smooth profiles for the corrected bottom terrain for all tests conducted. It has much higher optimizing efficiency than Hager and Zhang’s CG_DESCENT and Liu’s LBFGS methods. The SCG method reduces the norm of the gradient sufficiently and the value of the cost function very quickly. It always provides linear convergence rates, especially when including a regularization term. This study suggests the combined utilization of the SCG method and line search strategy to ensure high efficiency when solving the large-scale unconstrained problems.

A Spectral Conjugate Gradient Method with Descent Property

Spectral conjugate gradient method (SCGM) is an important generalization of the conjugate gradient method (CGM), and it is also one of the effective numerical methods for large-scale unconstrained optimization. The designing for the spectral parameter and the conjugate parameter in SCGM is a core work. And the aim of this paper is to propose a new and effective alternative method for these two parameters. First, motivated by the strong Wolfe line search requirement, we design a new spectral parameter. Second, we propose a hybrid conjugate parameter. Such a way for yielding the two parameters can ensure that the search directions always possess descent property without depending on any line search rule. As a result, a new SCGM with the standard Wolfe line search is proposed. Under usual assumptions, the global convergence of the proposed SCGM is proved. Finally, by testing 108 test instances from 2 to 1,000,000 dimensions in the CUTE library and other classic test collections, a large number of numerical experiments, comparing with both SCGMs and CGMs, for the presented SCGM are executed. The detail results and their corresponding performance profiles are reported, which show that the proposed SCGM is effective and promising.

Self adaptive spectral conjugate gradient method for solving nonlinear monotone equations

In this paper, we propose a self adaptive spectral conjugate gradient-based projection method for systems of nonlinear monotone equations. Based on its modest memory requirement and its efficiency, the method is suitable for solving large-scale equations. We show that the method satisfies the descent condition F_{k}^{T}d_{k}leq -c|F_{k}|^{2}, c>0, and also prove its global convergence. The method is compared to other existing methods on a set of benchmark test problems and results show that the method is very efficient and therefore promising.