Global Convergence Results Research Articles

A practical relative error criterion for augmented Lagrangians

This paper develops a new error criterion for the approximate minimization of augmented Lagrangian subproblems. This criterion is practical since it is readily testable given only a gradient (or subgradient) of the augmented Lagrangian. It is also “relative” in the sense of relative error criteria for proximal point algorithms: in particular, it uses a single relative tolerance parameter, rather than a summable parameter sequence. Our analysis first describes an abstract version of the criterion within Rockafellar’s general parametric convex duality framework, and proves a global convergence result for the resulting algorithm. Specializing this algorithm to a standard formulation of convex programming produces a version of the classical augmented Lagrangian method with a novel inexact solution condition for the subproblems. Finally, we present computational results drawn from the CUTE test set—including many nonconvex problems—indicating that the approach works well in practice.

A smoothing Newton algorithm for solving the monotone second-order cone complementarity problems

In this paper, a new smoothing function is given by smoothing the symmetric perturbed Fischer-Burmeister function. Based on this function, a smoothing Newton algorithm is proposed for solving the monotone second-order cone complementarity problems. The global and local quadratic convergence results of the algorithm are established under suitable assumptions. The theory of Euclidean Jordan algebras is a basic tool which is extensively used in our analysis. Numerical results indicate that the proposed algorithm is effective.

An improved trust region method for unconstrained optimization

In this paper, a new trust region method for unconstrained optimization is proposed. In the new method, the trust radius adjusts itself adaptively. In our algorithm, we use the convex combination of the Hessian matrix at a previous iteration and current iteration to define a suitable trust region radius at each iteration. The global, superlinear and quadratic convergence results of the algorithm are established under reasonable assumptions. Finally, some numerical results are given.

A new modified nonmonotone adaptive trust region method for unconstrained optimization

In this paper, we present an adaptive trust region method for solving unconstrained optimization problems which combines nonmonotone technique with a new update rule for the trust region radius. At each iteration, our method can adjust the trust region radius of related subproblem. We construct a new ratio to adjust the next trust region radius which is different from the ratio in the traditional trust region methods. The global and superlinear convergence results of the method are established under reasonable assumptions. Numerical results show that the new method is efficient for unconstrained optimization problems.

An Augmented Lagrangian Algorithm for Solving SemiinfiniteProgramming

We present a smooth augmented Lagrangian algorithm for semiinfinite programming (SIP). For this algorithm, we establish a perturbation theorem under mild conditions. As a corollary of the perturbation theorem, we obtain the global convergence result, that is, any accumulation point of the sequence generated by the algorithm is the solution of SIP. We get this global convergence result without any boundedness condition or coercive condition. Another corollary of the perturbation theorem shows that the perturbation function at zero point is lower semi‐continuous if and only if the algorithm forces the sequence of objective function convergence to the optimal value of SIP. Finally, numerical results are given.

Global Attractivity and Periodic Character of Difference Equation of Order Four

We investigate the global convergence result, boundedness, and periodicity of solutions of the recursive sequence xn+1 = axn + ((bxn−1 + cxn−2 + dxn−3)/(αxn−1 + βxn−2 + γxn−3)), n = 0,1,…, where the parameters a, b, c, d, α, β, and γ are positive real numbers and the initial conditions x−3, x−2, x−1, and x0 are positive real numbers.

A balanced truncation-based strategy for optimal control of evolution problems

In this paper, we present a balanced truncation based strategy for the numerical solution of optimal control problems governed by nonlinear evolution partial differential equations. The idea consists in utilizing a balanced truncation model reduction method for the efficient solution of the semi-discretized adjoint system, while the nonlinear state equations are fully solved. Our strategy is analysed as a descent method in function spaces and global convergence results are presented. In combination with a Broyden–Fletcher–Goldfarb–Shanno update also superlinear convergence is verified. Numerical examples are given to illustrate the behaviour of the proposed method for different problems.

Convergence of the descent Dai–Yuan conjugate gradient method for unconstrained optimization

The conjugate gradient method is one of the most useful and the earliest-discovered techniques for solving large-scale nonlinear optimization problems. Many variants of this method have been proposed, and some are widely used in practice. In this article, we study the descent Dai–Yuan conjugate gradient method which guarantees the sufficient descent condition for any line search. With exact line search, the introduced conjugate gradient method reduces to the Dai–Yuan conjugate gradient method. Finally, a global convergence result is established when the line search fulfils the Goldstein conditions.

Modified nonlinear conjugate gradient method with sufficient descent condition for unconstrained optimization

In this paper, an efficient modified nonlinear conjugate gradient method for solving unconstrained optimization problems is proposed. An attractive property of the modified method is that the generated direction in each step is always descending without any line search. The global convergence result of the modified method is established under the general Wolfe line search condition. Numerical results show that the modified method is efficient and stationary by comparing with the wellknown Polak-Ribiere-Polyak method, CG-DESCENT method and DSP-CG method using the unconstrained optimization problems from More and Garbow (ACM Trans Math Softw 7, 17-41, 1981), so it can be widely used in scientific computation. Mathematics Subject Classification (2010) 90C26 · 65H10

Numerical Implementation of Nonlinear Implicit Iterative Method for Solving Ill-posed Problems

Many nonlinear regularization methods may converge to local minima in numerical implementation for the complexity of nonlinear operator. Under some not very strict assumptions, we implement our proposed nonlinear implicit iterative method and have a global convergence results. Using the convexity property of the modified Tikhonov functional, it combines nonlinear implicit iterative method with a gradient method for solving ill-posed problems. Finally we present two numerical results for integral equation and parameter identification.

Projected gradient trust-region method for solving nonlinear systems with convex constraints

In this paper, a projected gradient trust region algorithm for solving nonlinear equality systems with convex constraints is considered. The global convergence results are developed in a very general setting of computing trial directions by this method combining with the line search technique. Close to the solution set this method is locally Q-superlinearly convergent under an error bound assumption which is much weaker than the standard nonsingularity condition.

Aggregate Constraint Homotopy Method for Nonlinear Programming on Unbounded Set

In [1], an aggregate constraint aggregate (ACH) method for nonconvex nonlinear programming problems was presented and global convergence result was obtained when the feasible set is bounded and satisfies a weak normal cone condition with some standard constraint qualifications. In this paper, without assuming the boundedness of feasible set, the global convergence of ACH method is proven under a suitable additional assumption.

A least-square semismooth Newton method for the second-order cone complementarity problem

We present a nonlinear least-square formulation for the second-order cone complementarity problem based on the Fischer–Burmeister (FB) function and the plus function. This formulation has two-fold advantages. First, the operator involved in the over-determined system of equations inherits the favourable properties of the FB function for local convergence, for example, the (strong) semi-smoothness; second, the natural merit function of the over-determined system of equations share all the nice features of the class of merit functions f YF studied in [J.-S. Chen and P. Tseng, An unconstrained smooth minimization reformulation of the second-order cone complementarity problem, Math. Program. 104 (2005), pp. 293–327] for global convergence. We propose a semi-smooth Levenberg–Marquardt method to solve the arising over-determined system of equations, and establish the global and local convergence results. Among others, the superlinear (quadratic) rate of convergence is obtained under strict complementarity of the solution and a local error bound assumption, respectively. Numerical results verify the advantages of the least-square reformulation for difficult problems.

The Levenberg‐Marquardt‐Type Methods for a Kind of Vertical Complementarity Problem

Two kinds of the Levenberg‐Marquardt‐type methods for the solution of vertical complementarity problem are introduced. The methods are based on a nonsmooth equation reformulation of the vertical complementarity problem for its solution. Local and global convergence results and some remarks about the two kinds of the Levenberg‐Marquardt‐type methods are also given. Finally, numerical experiments are reported.

A linearly convergent derivative-free descent method for the second-order cone complementarity problem

We consider a class of derivative-free descent methods for solving the second-order cone complementarity problem (SOCCP). The algorithm is based on the Fischer–Burmeister (FB) unconstrained minimization reformulation of the SOCCP, and utilizes a convex combination of the negative partial gradients of the FB merit function ψFB as the search direction. We establish the global convergence results of the algorithm under monotonicity and the uniform Jordan P-property, and show that under strong monotonicity the merit function value sequence generated converges at a linear rate to zero. Particularly, the rate of convergence is dependent on the structure of second-order cones. Numerical comparisons are also made with the limited BFGS method used by Chen and Tseng (An unconstrained smooth minimization reformulation of the second-order cone complementarity problem, Math. Program. 104(2005), pp. 293–327), which confirm the theoretical results and the effectiveness of the algorithm.

Nonmonotone trust region algorithm for unconstrained optimization problems

In this paper, a nonmonotone trust region algorithm for unconstrained optimization problems is presented. In the algorithm, a kind of nonmonotone technique, which is evidently different from Grippo, Lampariello and Lucidi’s approach, is used. Under mild conditions, global and local convergence results of the algorithm are established. Preliminary numerical results show that the new algorithm is efficient.

A filter algorithm for nonlinear semidefinite programming

This paper proposes a filter method for solving nonlinear semidefinite programming problems. Our method extends to this setting the filter SQP (sequential quadratic programming) algorithm, recently introduced for solving nonlinear programming problems, obtaining the respective global convergence results. Mathematical subject classification: 90C30, 90C55.

Generalized gradient projection neural networks for nonsmooth optimization problems

Generalized gradient projection neural network models are proposed to solve nonsmooth convex and nonconvex nonlinear programming problems over a closed convex subset of ℝ n . By using Clarke’s generalized gradient, the neural network modeled by a differential inclusion is developed, and its dynamical behavior and optimization capabilities both for convex and nonconvex problems are rigorously analyzed in the framework of nonsmooth analysis and the differential inclusion theory. First for nonconvex optimization problems, the quasiconvergence results similar to those achieved by previous work are proved. For convex optimization problems, the global convergence results are proved, i.e. any trajectory of the neural network converges to an asymptotically stable equilibrium point, which is an optimal solution of the primal problem whenever it exists. This result remains valid even if the initial condition is chosen outside the feasible set. In addition, an asymptotic control result involving a Tykhonov-like regularization shows that any trajectory of the revised neural network can be forced to converge toward a particular optimal solution of the primal problem. Finally, two simulation optimization algorithms for solving the optimization and control problems are designed respectively, and three typical simulation experiments are illustrated to present the accuracy and efficiency of theoretical convergence results of this paper.

New Scaled Sufficient Descent Conjugate Gradient Algorithm for Solving Unconstraint Optimization Problems

Problem statement: The scaled hybrid Conjugate Gradient (CG) algorithm which usually used for solving non-linear functions was presented and was compared with two standard well-Known NAG routines, yielding a new fast comparable algorithm. Approach: We proposed, a new hybrid technique based on the combination of two well-known scaled (CG) formulas for the quadratic model in unconstrained optimization using exact line searches. A global convergence result for the new technique was proved, when the Wolfe line search conditions were used. Results: Computational results, for a set consisting of 1915 combinations of (unconstrained optimization test problems/dimensions) were implemented in this research making a comparison between the new proposed algorithm and the other two similar algorithms in this field. Conclusion: Our numerical results showed that this new scaled hybrid CG-algorithm substantially outperforms Andrei-sufficient descent condition (CGSD) algorithm and the well-known Andrei standard sufficient descent condition from (ACGA) algorithm.

A descent spectral conjugate gradient method for impulse noise removal

In most applications, denoising image is fundamental to subsequent image processing operations. This paper proposes a spectral conjugate gradient (CG) method for impulse noise removal, which is based on a two-phase scheme. The noise candidates are first identified by the adaptive (center-weighted) median filter; then these noise candidates are restored by minimizing an edge-preserving regularization functional, which is accomplished by the proposed spectral CG method. A favorite property of the proposed method is that the search direction generated at each iteration is descent. Under strong Wolfe line search conditions, its global convergence result could be established. Numerical experiments are given to illustrate the efficiency of the spectral conjugate gradient method for impulse noise removal.