Global Convergence Theorem Research Articles

A Double Exponential Particle Swarm Optimization with non-uniform variates as stochastic tuning and guaranteed convergence to a global optimum with sample applications to finding optimal exact designs in biostatistics

Nature-inspired metaheuristic algorithms, like Particle Swarm Optimization (PSO), are powerful general-purpose optimization tools but they invariably do not come with rigorous theoretical justifications and can fail to find a global optimum. By treating PSO as a random search optimization process and repairing the famous Global Search Convergence Theorem by imposing an additional condition in the proof, we create a novel theory-based algorithm called the Double Exponential Particle Swarm optimization algorithm (DExPSO) that converges to a global optimum. In particular, we show the common practice of using uniform variates as stochastic components in PSO and related algorithms does not satisfy the sufficient conditions in DExPSO and hence may provide a reason why PSO and other nature-inspired algorithms, like QPSO, LcRiPSO, and CSO can fail to converge. Additionally, in more complicated design problems, we show that DExpSO tends to converge to the support points of the optimal design more frequently and faster than PSO and its variants do. Moreover, there is a possibility to modify other PSO variants to DExPSO variants, and such hybridization offers promising improvement in the quality of the global search. Our applications include finding designs that minimize the integrated mean squared prediction error and locally D-optimal exact designs for a 68-compartmental model to assess radioactive particles retained in the human lung after exposure. Because PSO, and more generally, metaheuristics are used across disciplines, including ecology, pharmacokinetics and pharmacodynamics studies, agriculture, engineering, and computer science, there are potentially broad and deep implications of our results.

New Order 2.0 Simplified Weak Itô–Taylor Symmetrical Scheme for Stochastic Delay Differential Equations

In this article, we construct a new order 2.0 simplified weak Itô–Taylor symmetrical scheme for stochastic delay differential equations. By the new local weak convergence lemma and the connection inequality, we theoretically prove the global weak convergence theorem in two parts on the basis of Malliavin stochastic analysis. Meanwhile, numerical examples are presented to illustrate the error and convergence results. Furthermore, the obtained results display the influence of the delay coefficient on global errors.

A Jump-Gain Integral Recurrent Neural Network for Solving Noise-Disturbed Time-Variant Nonlinear Inequality Problems.

Nonlinear inequalities are widely used in science and engineering areas, attracting the attention of many researchers. In this article, a novel jump-gain integral recurrent (JGIR) neural network is proposed to solve noise-disturbed time-variant nonlinear inequality problems. To do so, an integral error function is first designed. Then, a neural dynamic method is adopted and the corresponding dynamic differential equation is obtained. Third, a jump gain is exploited and applied to the dynamic differential equation. Fourth, the derivatives of errors are substituted into the jump-gain dynamic differential equation, and the corresponding JGIR neural network is set up. Global convergence and robustness theorems are proposed and proved theoretically. Computer simulations verify that the proposed JGIR neural network can solve noise-disturbed time-variant nonlinear inequality problems effectively. Compared with some advanced methods, such as modified zeroing neural network (ZNN), noise-tolerant ZNN, and varying-parameter convergent-differential neural network, the proposed JGIR method has smaller computational errors, faster convergence speed, and no overshoot when disturbance exists. In addition, physical experiments on manipulator control have verified the effectiveness and superiority of the proposed JGIR neural network.

New spectral LS conjugate gradient method for nonlinear unconstrained optimization

ABSTRACT In this work, we propose a novel algorithm to perform spectral conjugate gradient descent for an unconstrained, nonlinear optimization problem. First, we theoretically prove that the proposed method satisfies the sufficient descent condition, the conjugacy condition, and the global convergence theorem. The experimental setup uses Powell’s conjugacy condition coupled with a cubic polynomial line search using strong Wolfe conditions to ensure quick convergence. The experimental results demonstrate that the proposed method shows superior performance in terms of the number of iterations to convergence and the number of function evaluations when compared to traditional methods such as Liu and Storey (LS) and Conjugate Descent (CD).

A Five-Parameter Class of Derivative-Free Spectral Conjugate Gradient Methods for Systems of Large-Scale Nonlinear Monotone Equations

In this paper, we consider a system of large-scale nonlinear monotone equations and propose a class of derivative-free spectral conjugate gradient methods to solve it efficiently. We demonstrate the appropriate analytical features of this class and prove the global convergence theorems under a backtracking line search technique. In order to illustrate the numerical effectiveness of our class, we organize a competition in which 405 test problems will be solved by some members of the new class and four other similar derivative-free conjugate gradient methods. All the analytical and numerical results indicate that the presented class is promising.

New Spectral Idea for Conjugate Gradient Methods and its Global Convergence Theorems

Recently, the unconstrained optimization conjugate gradient methods have been widely utilized, especially for problems that are known as large-scale problems. This work proposes a new spectral gradient coefficient obtained from a convex linear combination of two different gradient coefficients to solve unconstrained optimization problems. One of the most essential features ofour suggested strategy is to guarantee the suitable subsidence direction of the line search precision. Furthermore, the proposed strategy is more effective than previous conjugate gradient approaches and stationery, which have been observed in the test problem. However, when it is compared to other conjugate gradient methods, such as FR methods, the proposed method confirmed the globally convergent, indicating that it can be used in scientific data computation.

Stochastic quasi-subgradient method for stochastic quasi-convex feasibility problems

<p style='text-indent:20px;'>The feasibility problem is at the core of the modeling of many problems in various disciplines of mathematics and physical sciences, and the quasi-convex function is widely applied in many fields such as economics, finance, and management science. In this paper, we consider the stochastic quasi-convex feasibility problem (SQFP), which is to find a common point of infinitely many sublevel sets of quasi-convex functions. Inspired by the idea of a stochastic index scheme, we propose a stochastic quasi-subgradient method to solve the SQFP, in which the quasi-subgradients of a random (and finite) index set of component quasi-convex functions at the current iterate are used to construct the descent direction at each iteration. Moreover, we introduce a notion of Hölder-type error bound property relative to the random control sequence for the SQFP, and use it to establish the global convergence theorem and convergence rate theory of the stochastic quasi-subgradient method. It is revealed in this paper that the stochastic quasi-subgradient method enjoys both advantages of low computational cost requirement and fast convergence feature.</p>

A new conjugate gradient algorithms using conjugacy condition for solving unconstrained optimization

The primarily objective of this paper which is indicated in the field of conjugate gradient algorithms for unconstrained optimization problems and algorithms is to show the advantage of the new proposed algorithm in comparison with the standard method which is denoted as. Hestenes Stiefel method, as we know the coefficient conjugate parameter is very crucial for this reason, we proposed a simple modification of the coefficient conjugate gradient which is used to derived the new formula for the conjugate gradient update parameter described in this paper. Our new modification is based on the conjugacy situation for nonlinear conjugate gradient methods which is given by the conjugacy condition for nonlinear conjugate gradient methods and added a nonnegative parameter to suggest the new extension of the method. Under mild Wolfe conditions, the global convergence theorem and lemmas are also defined and proved. The proposed method's efficiency is programming and demonstrated by the numerical instances, which were very encouraging.

Uniqueness and stability of bistable waves for monotone semiflows

This paper is devoted to the study of the uniqueness and stability of bistable traveling waves for monotone semiflows in an abstract setting. Under appropriate assumptions, we establish the uniqueness and stability of bistable waves for discrete and continuous-time semiflows in a continuous habitat by appealing to a global convergence theorem for monotone semiflows. We also extend such a result to time-periodic semiflows, and apply the general theory to a class of reaction-diffusion-advection systems in a cylinder.

The Global Convergence of the Nonlinear Power Method for Mixed-Subordinate Matrix Norms

We analyze the global convergence of the power iterates for the computation of a general mixed-subordinate matrix norm. We prove a new global convergence theorem for a class of entrywise nonnegative matrices that generalizes and improves a well-known results for mixed-subordinate ell ^p matrix norms. In particular, exploiting the Birkoff–Hopf contraction ratio of nonnegative matrices, we obtain novel and explicit global convergence guarantees for a range of matrix norms whose computation has been recently proven to be NP-hard in the general case, including the case of mixed-subordinate norms induced by the vector norms made by the sum of different ell ^p-norms of subsets of entries.

A gain-adjustment neural network based time-varying underdetermined linear equation solving method

To solve the time-varying underdetermined linear equation (TVULE), a gain-adjustment neural network (GANN) is proposed, designed and analyzed. First, based on the exploited error monitoring function and neural dynamic method, a GANN based linear equation solving method with gain-adjustment parameter is obtained. Second, the global convergence theorem is proved and shows that the state output of the GANN will converge to the solution of TVULE super-exponentially. Furthermore, the strong robustness of the GANN can be ensured theoretically. Third, compared with some traditional neural networks like the gradient neural network and the zeroing neural network, the computer simulations illustrate that the GANN has the fastest speed, the best accuracy and the best anti-noise capability when considering external disturbances. Meanwhile, a practical application to solve a motion planning scheme of redundant robot manipulators further verifies the flexibility and superiority of the GANN.

A Penalty Strategy Combined Varying-Parameter Recurrent Neural Network for Solving Time-Varying Multi-Type Constrained Quadratic Programming Problems.

To obtain the optimal solution to the time-varying quadratic programming (TVQP) problem with equality and multitype inequality constraints, a penalty strategy combined varying-parameter recurrent neural network (PS-VP-RNN) for solving TVQP problems is proposed and analyzed. By using a novel penalty function designed in this article, the inequality constraint of the TVQP can be transformed into a penalty term that is added into the objective function of TVQP problems. Then, based on the design method of VP-RNN, a PS-VP-RNN is designed and analyzed for solving the TVQP with penalty term. One of the greatest advantages of PS-VP-RNN is that it cannot only solve the TVQP with equality constraints but can also solve the TVQP with inequality and bounded constraints. The global convergence theorem of PS-VP-RNN is presented and proved. Finally, three numerical simulation experiments with different forms of inequality and bounded constraints verify the effectiveness and accuracy of PS-VP-RNN in solving the TVQP problems.

Convergence rates of subgradient methods for quasi-convex optimization problems

Quasi-convex optimization acts a pivotal part in many fields including economics and finance; the subgradient method is an effective iterative algorithm for solving large-scale quasi-convex optimization problems. In this paper, we investigate the quantitative convergence theory, including the iteration complexity and convergence rates, of various subgradient methods for solving quasi-convex optimization problems in a unified framework. In particular, we consider a sequence satisfying a general (inexact) basic inequality, and investigate the global convergence theorem and the iteration complexity when using the constant, diminishing or dynamic stepsize rules. More importantly, we establish the linear (or sublinear) convergence rates of the sequence under an additional assumption of weak sharp minima of Holderian order and upper bounded noise. These convergence theorems are applied to establish the iteration complexity and convergence rates of several subgradient methods, including the standard/inexact/conditional subgradient methods, for solving quasi-convex optimization problems under the assumptions of the Holder condition and/or the weak sharp minima of Holderian order.

On SOR-like iteration methods for solving weakly nonlinear systems

ABSTRACT In this paper, we introduce a class of SOR-like iteration methods for solving the systems of the weakly nonlinear equation, which is by reformulating equivalently the weakly nonlinear equation as a two-by-two block nonlinear equation. Two types of the global convergence theorems are given under suitable choices of the involved splitting matrix and parameter. Numerical results for the three-dimensional nonlinear convection-diffusion equation and the linear complementarity problem show that the proposed iteration methods are feasible and efficient for solving the weakly nonlinear equations.

A New Modified Conjugate Gradient Method and Its Global Convergence Theorem

In this article, we try to proposed a new conjugate gradient method for solving unconstrained optimization problems, we focus on conjugate gradient methods applied to the non-linear unconstrained optimization problems, the positive step size is obtained by a line search and the new scalar to the new direction for the conjugate gradient method is derived from the quadratic function and Taylor series and by using quasi newton condition and Newton direction while deriving the new formulae. We also prove that the search direction of the new conjugate gradient method satisfies the sufficient descent and all assumptions of the global convergence property are considered and proved .in order to complete the benefit of our research we should take into account studied the numerical results which are written in FORTRAN language when the objective function is compared our new algorithm with HS and PRP methods on the similar set of unconstrained optimization test problems which is very efficient and encouragement numerical results.

The Matrix Splitting Iteration Method for Nonlinear Complementarity Problems Associated with Second-Order Cone

For a class of second-order cone nonlinear complementarity problems, abbreviated as SOCNCPs, we establish the modulus-based matrix splitting relaxation iteration methods, which are obtained by reformulating equivalently SOCNCP as an implicit fixed-point equation based on Jordan algebra associated with the second-order cone. The global convergence theorems are given under suitable choices of the involved splitting matrix and parameter matrices. When the splitting matrix is symmetric positive definite, the strategy choice of the parameters is discussed. Numerical experiments have shown that the modulus-based iteration methods are effective for solving SOCNCPs.

Research on pose estimation for stereo vision measurement system by an improved method: uncertainty weighted stereopsis pose solution method based on projection vector.

We present UWSPSM, an algorithm of uncertainty weighted stereopsis pose solution method based on the projection vector which to solve the problem of pose estimation for stereo vision measurement system based on feature points. Firstly, we use a covariance matrix to represent the direction uncertainty of feature points, and utilize projection matrix to integrate the direction uncertainty of feature points into stereo-vision pose estimation. Then, the optimal translation vector is solved based on the projection vector of feature points, as well the depth is updated by the projection vector of feature points. In the absolute azimuth solution stage, the singular value decomposition algorithm is used to calculate the relative attitude matrix, and the above two stages are iteratively performed until the result converges. Finally, the convergence of the proposed algorithm is proved, from the theoretical point of view, by the global convergence theorem. Expanded into stereo-vision, the fixed relationship constraint between cameras is introduced into the stereoscopic pose estimation, so that only one pose parameter of the two images captured is optimized in the iterative process, and the two cameras are better bound as a camera, it can improve accuracy and efficiency while enhancing measurement reliability. The experimental results show that the proposed pose estimation algorithm can converge quickly, has high-precision and good robustness, and can tolerate different degrees of error uncertainty. So, it has useful practical application prospects.

A Family of Projection Gradient Methods for Solving the Multiple-Sets Split Feasibility Problem

In the present paper, we explore a family of projection gradient methods for solving the multiple-sets split feasibility problem, which include the cyclic/simultaneous iteration methods introduced in Wen et al. (J Optim Theory Appl 166:844–860, 2015) as special cases. For the general case, where the involved sets are given by level sets of convex functions, the calculation of the projection onto the level sets is complicated in general, and thus, the resulting projection gradient method cannot be implemented easily. To avoid this difficulty, we introduce a family of relaxed projection gradient methods, in which the projections onto the approximated halfspaces are adopted in place of the ones onto the level sets. They cover the relaxed cyclic/simultaneous iteration methods introduced in Wen et al. (J Optim Theory Appl 166:844–860, 2015) as special cases. Global weak convergence theorems are established for these methods. In particular, as direct applications of the established theorems, our results fill some gaps and deal with the imperfections that appeared in Wen et al. (J Optim Theory Appl 166:844–860, 2015) and hence improve and extend the corresponding results therein.

Two families of scaled three-term conjugate gradient methods with sufficient descent property for nonconvex optimization

In this paper, we present two families of modified three-term conjugate gradient methods for solving unconstrained large-scale smooth optimization problems. We show that our new families satisfy the Dai-Liao conjugacy condition and the sufficient descent condition under any line search technique which guarantees the positiveness of ${y_{k}^{T}} s_{k}$. For uniformly convex functions, we indicate that our families are globally convergent under weak-Wolfe-Powell line search technique and standard conditions on the objective function. We also establish a weaker global convergence theorem for general smooth functions under similar assumptions. Our numerical experiments for 260 standard problems and seven other recently developed conjugate gradient methods illustrate that the members of our families are numerically efficient and effective.

An Improved Pose Estimation Method Based on Projection Vector With Noise Error Uncertainty

Aiming at the problem of anomalous and non-independent distribution of the image errors in the feature-based visual pose estimation, a method of monocular visual pose estimation based on the uncertainty of noise error established by projection vector is proposed. First, by using the covariance matrix to describe the uncertainty of the feature point direction and integrating the uncertainty of the feature point direction into the pose estimation, characteristic point measurement error with different degrees of directional uncertainty can be adapted that can makes the algorithm robust. Then, by introducing the projection vector and combining the depth information of each feature point to represent the collinearity error, the model nonlinear problem caused by the camera perspective projection can be eliminated that can make the algorithm have global convergence. Finally, we use global convergence theorem to prove the global convergence of the proposed algorithm. The results show that the proposed method has good robustness and convergence while adapting to different degrees of error uncertainty, which can meet practical engineering applications.