Global Convergence Properties Research Articles

A new class of nonlinear conjugate gradient coefficients with exact and inexact line searches

Conjugate gradient (CG) methods have played an important role in solving large-scale unconstrained optimization. In this paper, we propose a new family of CG coefficients (βk) that possess sufficient descent conditions and global convergence properties. This new βk is an extension of the already proven βkRMIL from Rivaie et al. [19] (A new class of nonlinear conjugate gradient coefficient with global convergence properties, Appl. Math. Comp. 218(2012) 11323-11332). Global convergence result is established using both exact and inexact line searches. Numerical results show that the performance of the new proposed formula is quite similar to βkRMIL and suited to both line searches. Importantly, the performance of this βk is more efficient and superior than the other well-known βk.

CMOS analogue amplifier circuits optimisation using hybrid backtracking search algorithm with differential evolution

This paper proposes a novel hybrid optimisation algorithm which combines the recently proposed evolutionary algorithm Backtracking Search Algorithm (BSA) with another widely accepted evolutionary algorithm, namely, Differential Evolution (DE). The proposed algorithm called BSA-DE is employed for the optimal designs of two commonly used analogue circuits, namely Complementary Metal Oxide Semiconductor (CMOS) differential amplifier circuit with current mirror load and CMOS two-stage operational amplifier (op-amp) circuit. BSA has a simple structure that is effective, fast and capable of solving multimodal problems. DE is a stochastic, population-based heuristic approach, having the capability to solve global optimisation problems. In this paper, the transistors’ sizes are optimised using the proposed BSA-DE to minimise the areas occupied by the circuits and to improve the performances of the circuits. The simulation results justify the superiority of BSA-DE in global convergence properties and fine tuning ability, and prove it to be a promising candidate for the optimal design of the analogue CMOS amplifier circuits. The simulation results obtained for both the amplifier circuits prove the effectiveness of the proposed BSA-DE-based approach over DE, harmony search (HS), artificial bee colony (ABC) and PSO in terms of convergence speed, design specifications and design parameters of the optimal design of the analogue CMOS amplifier circuits. It is shown that BSA-DE-based design technique for each amplifier circuit yields the least MOS transistor area, and each designed circuit is shown to have the best performance parameters such as gain, power dissipation, etc., as compared with those of other recently reported literature.

A new regularized limited memory BFGS-type method based on modified secant conditions for unconstrained optimization problems

In the framework of large-scale optimization problems, the standard BFGS method is not affordable due to memory constraints. The so-called limited-memory BFGS (L-BFGS) method is an adaption of the BFGS method for large-scale settings. However, the standard BFGS method and therefore the standard L-BFGS method only use the gradient information of the objective function and neglect function values. In this paper, we propose a new regularized L-BFGS method for solving large scale unconstrained optimization problems in which more available information from the function and gradient values are employed to approximate the curvature of the objective function. The proposed method utilizes a class of modified quasi-Newton equations in order to achieve higher order accuracy in approximating the second order curvature of the objective function. Under some standard assumptions, we provide the global convergence property of the new method. In order to provide an efficient method for finding global minima of a continuously differentiable function, a hybrid algorithm that combines a genetic algorithm (GA) with the new proposed regularized L-BFGS method has been proposed. This combination leads the iterates to a stationary point of the objective function with higher chance of being global minima. Numerical results show the efficiency and robustness of the new proposed regularized L-BFGS and its hybridized version with GA in practice.

An Affine Scaling Interior Trust-Region Algorithm Combining Backtracking Line Search with Filter Technique for Nonlinear Constrained Optimization

In this article, an affine scaling interior trust-region algorithm which employs backtracking line search with filter technique is presented for solving nonlinear equality constrained programming with nonnegative constraints on variables. At current iteration, the general full affine scaling trust-region subproblem is decomposed into a pair of trust-region subproblems in vertical and horizontal subspaces, respectively. The trial step is given by the solutions of the pair of trust-region subproblems. Then, the step size is decided by backtracking line search together with filter technique. This is different from traditional trust-region methods and has the advantage of decreasing the number of times that a trust-region subproblem must be resolved in order to determine a new iteration point. Meanwhile, using filter technique instead of merit function to determine a new iteration point can avoid the difficult decisions regarding the choice of penalty parameters. Under some reasonable assumptions, the new method possesses the property of global convergence to the first-order critical point. Preliminary numerical results show the effectiveness of the proposed algorithm.

Primal–dual algorithm based on Gauss–Seidel scheme with application to multiplicative noise removal

Due to the strong edge preserving ability and low computational cost, the total variation (TV) regularization has been developed as one promising approach to solve the multiplicative denoising problem. In recent years, many efficient algorithms have been proposed for computing the numerical solution of TV-based convex variational models. Among these methods, the (linearized) augmented Lagrangian algorithm (ALM) and the primal–dual hybrid gradient (PDHG) algorithm are two of the most effective and most widely used techniques. In this paper, inspired by the connection of the ALM and PDHG algorithms, we develop an improved primal–dual algorithm for multiplicative noise removal. In the proposed algorithm, an auxiliary variable, which is updated by the Gauss–Seidel scheme, is introduced to accelerate the original primal–dual framework. The global convergence property of the proposed algorithm is also investigated. Numerical experiments on the multiplicative denoising show that the proposed algorithm outperforms the current state-of-the-art methods.

Trajectory planning and tracking control for autonomous lane change maneuver based on the cooperative vehicle infrastructure system

Lane change maneuver is one of the most conventional behaviors in driving. Unsafe lane change maneuvers are key factor for traffic accidents and traffic congestion. For drivers’ safety, comfort and convenience, advanced driver assistance systems (ADAS) are presented. The main problem discussed in this paper is the development of an autonomous lane change system. The system can be extended applied in intelligent vehicles in future. It solves two crucial issues – trajectory planning and trajectory tracking. Polynomial method was adopted for describing the trajectory planning issue. Movement of a host vehicle was abstracted into time functions. Moreover, collision detection was mapped into a parameter space by adopting infinite dynamic circles. The second issue was described by backstepping principle. According to the Lyapunov function, a tracking controller with global convergence property was verified. Both the simulations and the experimental results demonstrate the feasibility and effectiveness of the designed method for autonomous lane change.

Globally convergent evolution strategies for constrained optimization

In this paper we propose, analyze, and test algorithms for constrained optimization when no use of derivatives of the objective function is made. The proposed methodology is built upon the globally convergent evolution strategies previously introduced by the authors for unconstrained optimization. Two approaches are encompassed to handle the constraints. In a first approach, feasibility is first enforced by a barrier function and the objective function is then evaluated directly at the feasible generated points. A second approach projects first all the generated points onto the feasible domain before evaluating the objective function. The resulting algorithms enjoy favorable global convergence properties (convergence to stationarity from arbitrary starting points), regardless of the linearity of the constraints. The algorithmic implementation (i) includes a step where previously evaluated points are used to accelerate the search (by minimizing quadratic models) and (ii) addresses the particular cases of bounds on the variables and linear constraints. Our solver is compared to others, and the numerical results confirm its competitiveness in terms of efficiency and robustness.

Global behavior of solutions of the generalized Lyness difference equations under quadratic perturbations

We study the global asymptotic stability of solutions of the following two difference equationsxn+2xn=a+bxn+1+(1-c)xn+12+cxn2,n=0,1,2,…andxn+2xn=a+bxn+1+d(1-c)xn+12d+xn+1+cxn2,n=0,1,2,…,where a∈(0,+∞),d∈[0,+∞),c∈(0,1] and the initial values x0,x1∈(0,+∞). Bastien and Rogalski (2004) showed if c=0 then there exist all the possible periods for the solutions of the above equations. Using an extension of the quasi-Lyapunov method, we prove that the sequences generated by the first difference equation are globally asymptotically stable where 0<c<1,0>b>-2a(1-c) and the initial values x0,x1∈(0,+∞). The global convergence property of the second difference equation has also been obtained for b>0 and 0<c<1. Conversely, the sequences generated by the above equations are increasing to +∞ for c=1 and b>0.

A new class of efficient and globally convergent conjugate gradient methods in the Dai–Liao family

In this paper, we propose a new conjugate gradient (CG) method which belongs to the CG methods of Dai–Liao family [New conjugacy conditions and related nonlinear conjugate gradient methods, Appl. Math. Optim. 43 (2001), pp. 87–101]. Babaie-Kafaki et al. [Two new conjugate gradient methods based on modified secant equations, J. Comput. Appl. Math. 234 (2010), pp. 1374–1386] made some modifications on the Yabe and Takano's CG approach [Global convergence properties of nonlinear conjugate gradient methods with modified secant condition, Comput. Optim. Appl. 28 (2004), pp. 203–225] and received some appealing results in theory and practice. Here, we introduce an efficient updating rule for the parameters of the Yabe and Takano's CG algorithm. Under some standard assumptions, we establish the global convergence property of the new suggested algorithm on uniformly convex and general functions. Numerical results on some testing problems from CUTEr collection show the priority of the proposed method to some existing CG methods in practice.

A Regularized Newton Method with Correction for Unconstrained Nonconvex Optimization

In this paper, we present a modified regularized Newton method for minimizing a nonconvex function whose Hessian matrix may be singular. We show that if the gradient and Hessian of the objective function are Lipschitz continuous, then the method has a global convergence property. Under the local error bound condition which is weaker than nonsingularity, the method has cubic convergence.

Augmented Lagrangian methods for nonlinear programming with possible infeasibility

In this paper, we consider a nonlinear programming problem for which the constraint set may be infeasible. We propose an algorithm based on a large family of augmented Lagrangian functions and analyze its global convergence properties taking into account the possible infeasibility of the problem. We show that, in a finite number of iterations, the algorithm stops detecting the infeasibility of the problem or finds an approximate feasible/optimal solution with any required precision. We illustrate, by means of numerical experiments, that our algorithm is reliable for different Lagrangian/penalty functions proposed in the literature.

A new nonmonotone filter Barzilai–Borwein method for solving unconstrained optimization problems

In this paper, a finite filter is used in the structure of the Barzilai–Browein (BB) gradient method in order to propose a new modified BB algorithm for solving large-scale unconstrained optimization problems. Our algorithm is equipped with a relaxed nonmonotone line search technique which allows the algorithm to enjoy the nonmonotonicity properties from scratch. Under some suitable conditions, the global convergence property of the new proposed algorithm is established. Numerical results on some test problems in CUTEr library show the efficiency and effectiveness of the new algorithm in practice too.

A globally convergent numerical method for a coefficient inverse problem for a parabolic equation

In this work a Multidimensional Coefficient Inverse Problem (MCIP) for a parabolic PDE with the data resulting from a single measurement event is considered. This measured data is obtained using a single position of the point source. The most important property of the method presented here is that even though we do not need any advanced knowledge of a small neighborhood of the solution, we still obtain points in this neighborhood. This is the reason why the numerical algorithm for this method is called globally convergent. In the literature a globally convergent numerical method for the MCIP with single measurement data has inclusively been considered in the book (Beilina and Klibanov, 2012) and some other publications of Beilina and Klibanov. In those publications a globally convergent algorithm for MCIP for a hyperbolic PDE was developed. Here we modify their technique to prove the global convergence property of their method for the parabolic case.

A relaxed nonmonotone adaptive trust region method for solving unconstrained optimization problems

In this paper, we present a new relaxed nonmonotone trust region method with adaptive radius for solving unconstrained optimization problems. The proposed method combines a relaxed nonmonotone technique with a modified version of the adaptive trust region strategy proposed by Shi and Guo (J Comput Appl Math 213:509---520, 2008). Under some suitable and standard assumptions, we establish the global convergence property as well as the superlinear convergence rate for the new method. Numerical results on some test problems show the efficiency and effectiveness of the new proposed method in practice.

Another modified conjugate gradient coefficient with global convergence properties

Conjugate gradient [CG] methods are considered in solving nonlinear unconstrained optimization problem, because of their simplicity, low memory requirement and global convergence properties. Different reviews and modification have been carried out in order to upgrade the method. In this paper, a new type of CG parameter, which satisfies the sufficient descent condition and global convergences property under exact line search, is proposed. The numerical outcomes indicate that our new modified parameter perform well when compare with other CG parameters for a given standards test function.

The convergence properties of a new kind of conjugate gradient method for unconstrained optimization

Conjugate gradient (CG) methods are the most prominent technique for solving large-scale unconstrained optimization problems, due to its robustness, low memory requirement, and global convergence properties. Numerous studies and modifications have been carried out recently to improve these methods. In this paper, a new modification of a CG coefficient that possesses the global convergence properties is presented. The global convergence result is validated using exact line search. Several numerical experiments showed that, the proposed formula is found to be robust and efficient when compared to other CG coefficients.

A new simple conjugate gradient coefficient for unconstrained optimization

Abstract Conjugate gradient method holds an important role in solving unconstrained optimizations, especially for large scale problems. Numerous studies and modifications have been done to improve this method. In this paper, we propose a fundamentally different conjugate gradient method in which the well known coefficient β k is computed using the eigenvalues generated by using exact Hessian matrix of f ( x ) . This relatively easy formula for β k has made conjugate gradient method very simple, but still possesses global convergence properties. Numerical results have shown that this formula performs better than the original conjugate gradient methods. Mathematics Subject Classification : 65K10 Keywords : conjugate gradient methods, conjugate gradient coefficient, eigenvalues, Hessian matrix, global convergence properties 1. Introduction The conjugate gradient method (CG) plays an important role in solving the unconstrained optimization problem. In general, the method has the following form min f (

The global convergence properties of an improved conjugate gradient method

Conjugate gradient (CG) methods have played a significant role in solving large scale unconstrained optimization. This is due to its simplicity, low memory requirement, and global convergence properties. Various studies and modifications have been done recently to improve this method. In this paper, we proposed a new conjugate gradient parameter ) ( k  which possesses global convergence properties under the exact line search. Numerical result shows that our new formula performs better when compared to other classical conjugate gradient methods.

A SURVEY OF NUMERICAL METHODS FOR NONLINEAR SEMIDEFINITE PROGRAMMING

Nonlinear semidefinite programming (SDP) problems have received a lot of attentions because of large variety of applications. In this paper, we survey numerical methods for solving nonlinear SDP problems. Three kinds of typical numerical methods are described; augmented Lagrangian methods, sequential SDP methods and primal-dual interior point methods. We describe their typical algorithmic forms and discuss their global and local convergence properties which include rate of convergence.

A Nonmonotone Filter SQP Method: Local Convergence and Numerical Results

The work by Gould, Loh, and Robinson [SIAM J. Optim., 24 (2014), pp. 175--209] established global convergence of a new filter line search method for finding local first-order solutions to nonlinear and nonconvex constrained optimization problems. A key contribution of that work was that the search direction was computed using the same procedure during every iteration from subproblems that were always feasible and computationally tractable. This contrasts previous filter methods that require a separate restoration phase based on subproblems solely designed to reduce infeasibility. In this paper, we present a nonmonotone variant of our previous algorithm that inherits the previously established global convergence property. In addition, we establish local superlinear convergence of the iterates and provide the results of numerical experiments. The numerical tests validate our method and highlight an interesting numerical trade-off between accepting more (on average lower quality) steps versus fewer (on avera...