Global Convergence Properties Research Articles

A hybrid assembly sequence planning approach based on discrete particle swarm optimization and evolutionary direction operation

Assembly sequence planning (ASP) has always been an important part of the product development process, and ASP problem can usually be understood as to determine the sequence of assembly. A good assembly sequence can reduce the time and cost of the manufacturing process. In view of the local convergence problem with basic discrete particle swarm optimization (DPSO) in ASP, this paper presents a hybrid algorithm to solve ASP problem. First, a chosen strategy of global optimal particle in DPSO is introduced, and then an improved discrete particle swarm optimization (IDPSO) is proposed for solving ASP problems. Through an example study, the results show that the IDPSO algorithm can obtain the global optimum efficiently, but it converges slowly compared with the basic DPSO. Subsequently, a modified evolutionary direction operator (MEDO) is used to accelerate the convergence rate of IDPSO. The results of the case study show that the new hybrid algorithm MEDO-IDPSO is more efficient for solving ASP problems, with excellent global convergence properties and fast convergence rate.

Global convergence and finite termination of a class of smooth penalty function algorithms

In this paper, based on a new class of asymptotic l 1 exact penalty functions, we propose a smooth penalty function for solving nonlinear programming problems. One of the main features of our algorithm is that at each iteration, we do not need to solve the global minimum of penalty functions. Furthermore, global convergence property is established without requiring any constraint qualifications. By addressing perturbation functions, we obtain that the lower semi-continuity of the perturbation function at zero is a necessary and sufficient condition to ensure the convergence of the objective function values generated by the algorithm to the optimal value of the primal problem. Since the perturbation function is only dependent on the data of the primal problem, these results enable us to check the convergence property of the algorithm in advance. Finally, we discuss the finite termination of the algorithm when the solution set is non-degenerate or weakly sharp.

An affine scaling method for optimization problems with polyhedral constraints

Recently an affine scaling, interior point algorithm ASL was developed for box constrained optimization problems with a single linear constraint (Gonzalez-Lima et al., SIAM J. Optim. 21:361---390, 2011). This note extends the algorithm to handle more general polyhedral constraints. With a line search, the resulting algorithm ASP maintains the global and R-linear convergence properties of ASL. In addition, it is shown that the unit step version of the algorithm (without line search) is locally R-linearly convergent at a nondegenerate local minimizer where the second-order sufficient optimality conditions hold. For a quadratic objective function, a sublinear convergence property is obtained without assuming either nondegeneracy or the second-order sufficient optimality conditions.

Fewer Bernoulli Measurements Satisfying the Constraint of Reconstruction Probability

In the compressed sensing (CS) framework, the previous methods statistically provide the probability of signal reconstruction when the number of measurements is given. However, when the reconstruction probability of the signal is constrained, these existing methods do not provide an effective mechanism to find the measurements satisfying the constraint. To handle the problem, we focus on Bernoulli measurements and utilize the reconstruction feature of Bernoulli measurements and sequential CS to determine the measurements such that they can satisfy the constraint of reconstruction probability. Moreover, the measurements obtained by the proposed method are fewer than those by the classical methods because some redundant measurements are removed from the obtained measurement sequence. In summary, the proposed method has three unique features: satisfying the constraint of reconstruction probability, containing fewer measurements, and having the property of global convergence. Both theoretical analysis and experimental results suggest that the proposed method can obtain fewer measurements which satisfy the required reconstruction probability for different sparse signals and different initial measurements.

Nonlinear conjugate gradient methods with sufficient descent properties for unconstrained optimization

It is very important to generate a descent search direction independent of line searches in showing the global convergence of conjugate gradient methods. The method of Hager and Zhang (2005) satisfies the sufficient descent condition. In this paper, we treat two subjects. We first consider a unified formula of parameters which establishes the sufficient descent condition and follows the modification technique of Hager and Zhang. In order to show the global convergence of the conjugate gradient method with the unified formula of parameters, we define some property (say Property A). We prove the global convergence of the method with Property A. Next, we apply the unified formula to a scaled conjugate gradient method and show its global convergence property. Finally numerical results are given.

Digital FIR Filter Design Using Hybrid Random Particle Swarm Optimization with Differential Evolution

Abstract This paper presents a novel approach of designing linear phase FIR low pass and high pass filter using Random PSO in hybrid with DE known as Random PSODE (RPSODE). In this paper, the Random PSO is used which utilises the weighted particle to guide the search direction for both explorative and exploitative searches. Differential evolution (DE) is one of the very fast and robust evolutionary algorithms which has shown superior performance for continuous global optimization; uses differential information to guide its search direction but sometime causes instability problem; whereas, PSO is a robust, population based stochastic search technique but has the problem of sub-optimality. This paper efficiently combines the Random PSO and DE so as to overcome the disadvantages faced by both the algorithms individually and is used for the design of linear phase low pass and high pass FIR filters. The simulation results show the superiority of RPSODE in global convergence properties and local search ability,...

A Globally Convergent Line Search Filter SQP Method for Inequality Constrained Optimization

A line search filter SQP method for inequality constrained optimization is presented. This method makes use of a backtracking line search procedure to generate step size and the efficiency of the filter technique to determine step acceptance. At each iteration, the subproblem is always consistent, and it only needs to solve one QP subproblem. Under some mild conditions, the global convergence property can be guaranteed. In the end, numerical experiments show that the method in this paper is effective.

Proximal point algorithm for nonlinear complementarity problem based on the generalized Fischer-Burmeister merit function

This paper is devoted to the study of the proximal point algorithm for solving monotone and nonmonotone nonlinear complementarity problems. The proximal point algorithm is to generate a sequence by solving subproblems that are regularizations of the original problem. After given an appropriate criterion for approximate solutions of subproblems by adopting a merit function, the proximal point algorithm is verified to have global and superlinear convergence properties. For the purpose of solving the subproblems efficiently, we introduce a generalized Newton method and show that only one Newton step is eventually needed to obtain a desired approximate solution that approximately satisfies the appropriate criterion under mild conditions. The motivations of this paper are twofold. One is analyzing the proximal point algorithm based on the generalized Fischer-Burmeister function which includes the Fischer-Burmeister function as special case, another one is trying to see if there are relativistic change on numerical performance when we adjust the parameter in the generalized Fischer-Burmeister.

A modified scaling BFGS method for nonconvex minimization

In this paper, we propose a modified scaling BFGS method for unconstrained minimization. A remarkable feature of the proposed method is that it can improve the performance of the BFGS method and possesses a global convergence property without convexity assumption on the objective function. Under certain assumptions, we also establish superlinear convergence of the method. Finally we show numerical results.

Double Flight-Modes Particle Swarm Optimization

Getting inspiration from the real birds in flight, we propose a new particle swarm optimization algorithm that we call the double flight modes particle swarm optimization (DMPSO) in this paper. In the DMPSO, each bird (particle) can use both rotational flight mode and nonrotational flight mode to fly, while it is searching for food in its search space. There is a King in the swarm of birds, and the King controls each bird’s flight behavior in accordance with certain rules all the time. Experiments were conducted on benchmark functions such as Schwefel, Rastrigin, Ackley, Step, Griewank, and Sphere. The experimental results show that the DMPSO not only has marked advantage of global convergence property but also can effectively avoid the premature convergence problem and has good performance in solving the complex and high-dimensional optimization problems.

On the strong convergence of a modified Hestenes-Stiefel method for nonconvex optimization

In [8], Zhang et al. proposed a modified three-term HS(MTTHS) conjugate gradient method and proved that this method converges globally for nonconvex minimizationin the sense that $\liminf_{k\to\infty}\|\nabla f(x_k)\|=0$ when the Armijo or Wolfe line search is used. In this paper, we further study the convergence property of the MTTHS method. We show that the MTTHS method has strongly global convergence property (i.e., $\lim_{k\to\infty}\|\nabla f(x_k)\|=0$) for nonconvex optimization by the use of the backtracking type line search in [7]. Some preliminary numerical results are reported.

A Globally Convergent Stabilized SQP Method

Sequential quadratic programming (SQP) methods are a popular class of methods for nonlinearly constrained optimization. They are particularly effective for solving a sequence of related problems, such as those arising in mixed-integer nonlinear programming and the optimization of functions subject to differential equation constraints. Recently, there has been considerable interest in the formulation of stabilized SQP methods, which are specifically designed to handle degenerate optimization problems. Existing stabilized SQP methods are essentially local in the sense that both the formulation and analysis focus on the properties of the methods in a neighborhood of a solution. A new SQP method is proposed that has favorable global convergence properties yet, under suitable assumptions, is equivalent to a variant of the conventional stabilized SQP method in the neighborhood of a solution. The method combines a primal-dual generalized augmented Lagrangian function with a flexible line search to obtain a seque...

The Limited Memory Conjugate Gradient Method

In theory, the successive gradients generated by the conjugate gradient method applied to a quadratic should be orthogonal. However, for some ill-conditioned problems, orthogonality is quickly lost due to rounding errors, and convergence is much slower than expected. A limited memory version of the nonlinear conjugate gradient method is developed. The memory is used to both detect the loss of orthogonality and to restore orthogonality. An implementation of the algorithm is presented based on the CG_DESCENT nonlinear conjugate gradient method. Limited memory CG_DESCENT (L-CG_DESCENT) possesses a global convergence property similar to that of the memoryless algorithm but has much better practical performance. Numerical comparisons to the limited memory BFGS method (L-BFGS) are given using the CUTEr test problems.

A Novel Quantum-behaved Particle Swarm Optimization Algorithm and Its Application to Parameter Optimization of Fuzzy Neural Networks

A novel Quantum-behaved Particle Swarm Optimization Algorithm with comprehensive learning and cooperative learning approach (CCQPSO) was introduced to improve the global convergence property of QPSO. In the proposed algorithm, the updating method of local attractor, particle’s previous best position and swarm’s global best position were performed in each dimension of the solution vector to avoid loss some components that had moved closer to the global optimal solution in the vector. Then the novel algorithm was applied to parameter optimization of fuzzy neural networks. The introduction of cooperative learning strategy is the originality in the proposed method. The results of prediction of chaotic time series experiment show that the proposed technique can converge more rapidly than other evolutionary computation methods, and the novel method is effective and efficient.

Hybrid Optimized Back propagation Learning Algorithm for Multi-layer Perceptron

Standard neural network based on general back propagation learning using delta method or gradient descent method has some great faults like poor optimization of error-weight objective function, low learning rate, instability .This paper introduces a hybrid supervised back propagation learning algorithm which uses trust-region method of unconstrained optimization of the error objective function by using quasi-newton method .This optimization leads to more accurate weight update system for minimizing the learning error during learning phase of multi-layer perceptron.[13][14][15] In this paper augmented line search is used for finding points which satisfies Wolfe condition. In this paper, This hybrid back propagation algorithm has strong global convergence properties & is robust & efficient in practice.

Globally convergent algorithms for solving unconstrained optimization problems

New algorithms for solving unconstrained optimization problems are presented based on the idea of combining two types of descent directions: the direction of anti-gradient and either the Newton or quasi-Newton directions. The use of latter directions allows one to improve the convergence rate. Global and superlinear convergence properties of these algorithms are established. Numerical experiments using some unconstrained test problems are reported. Also, the proposed algorithms are compared with some existing similar methods using results of experiments. This comparison demonstrates the efficiency of the proposed combined methods.

A New Preconditioned Inexact Line-Search Technique for Unconstrained Optimization

In this paper, we study the global convergence properties of the new class of preconditioned conjugate gradient descent algorithm, when applied to convex objective non-linear unconstrained optimization functions. We assume that a new inexact line search rule which is similar to the Armijo line-search rule is used. It's an estimation formula to choose a large step-size at each iteration and use the same formula to find the direction search. A new preconditioned conjugate gradient direction search is used to replace the conjugate gradient descent direction of ZIR-algorithm. Numerical results on twenty five well-know test functions with various dimensions show that the new inexact line-search and the new preconditioned conjugate gradient search directions are efficient for solving unconstrained nonlinear optimization problem in many situations.

Global Convergence of a Closed-Loop Regularized Newton Method for Solving Monotone Inclusions in Hilbert Spaces

We analyze the global convergence properties of some variants of regularized continuous Newton methods for convex optimization and monotone inclusions in Hilbert spaces. The regularization term is of Levenberg–Marquardt type and acts in an open-loop or closed-loop form. In the open-loop case the regularization term may be of bounded variation.

Global Exponential Convergence Properties for the Open-Loop Induction Motor

It has been recently shown by Oteafy and Chiasson that global asymptotic convergence properties can be guaranteed under certain conditions for the open-loop induction motor, that is for the induction motor which is fed by balanced sinusoidal voltages of constant electrical frequency. The slightly different analysis of this note not only confirms those results from a different point of view (involving the natural controlled variables rotor speed and flux modulus) but also clarifies that, under similar conditions, global exponential convergence properties can be actually established.

Smoothing and worst-case complexity for direct-search methods in nonsmooth optimization

For smooth objective functions it has been shown that the worst case cost of direct-search methods is of the same order as the one of steepest descent, when measured in number of iterations to achieve a certain threshold of stationarity. Motivated by the lack of such a result in the non-smooth case, we propose, analyze, and test a class of smoothing direct-search methods for the optimization of non-smooth functions. Given a parameterized family of smoothing functions for the non-smooth objective function, this class of methods consists of applying a direct-search algorithm for a fixed value of the smoothing parameter until the step size is relatively small, after which the smoothing parameter is reduced and the process is repeated. One can show that the worst case complexity (or cost) of this procedure is roughly one order of magnitude worse than the one for direct search or steepest descent on smooth functions. The class of smoothing direct-search methods is also showed to enjoy asymptotic global convergence properties. Some preliminary numerical experience indicates that this approach leads to better values of the objective function, pushing in some cases the optimization further, apparently without an additional cost in the number of function evaluations.