Global Convergence Behavior Research Articles

Swarm-based gradient descent method for non-convex optimization

We introduce a new swarm-based gradient descent (SBGD) method for non-convex optimization. The swarm consists of agents, each is identified with a position, x \mathbf {x} , and mass, m m . The key to their dynamics is communication: masses are being transferred from agents at high ground to low(-est) ground. At the same time, agents change positions with step size adjusted to their relative mass. Accordingly, the crowd of agents is dynamically divided between heavier ‘leaders’, which proceed with small time steps and are expected to approach local minima, and lighter ‘explorers’ which proceed with larger-step protocol and are expected to encounter improved position for the swarm. If they do, then they assume the role of heavy swarm leaders and so on. We prove local convergence and we present convincing numerical simulations which demonstrate that the added layer of SBGD explorers performs well in improving its global convergence behavior in one-, two-, and 20-dimensional benchmarks.

A High-Order Numerical Scheme for Efficiently Solving Nonlinear Vectorial Problems in Engineering Applications

In scientific and engineering disciplines, vectorial problems involving systems of equations or functions with multiple variables frequently arise, often defying analytical solutions and necessitating numerical techniques. This research introduces an efficient numerical scheme capable of simultaneously approximating all roots of nonlinear equations with a convergence order of ten, specifically designed for vectorial problems. Random initial vectors are employed to assess the global convergence behavior of the proposed scheme. The newly developed method surpasses methods in the existing literature in terms of accuracy, consistency, computational CPU time, residual error, and stability. This superiority is demonstrated through numerical experiments tackling engineering problems and solving heat equations under various diffusibility parameters and boundary conditions. The findings underscore the efficacy of the proposed approach in addressing complex nonlinear systems encountered in diverse applied scenarios.

A New Approach to Multiroot Vectorial Problems: Highly Efficient Parallel Computing Schemes

In this article, we construct an efficient family of simultaneous methods for finding all the distinct as well as multiple roots of polynomial equations. Convergence analysis proves that the order of convergence of newly constructed family of simultaneous methods is seventeen. Fractal-based simultaneous iterative algorithms are thoroughly examined. Using self-similar features, fractal-based simultaneous schemes can converge to solutions faster, saving computational time and resources necessary for solving nonlinear equations. Fractals analysis illustrates the newly developed method’s global convergence behavior when compared to single root-finding procedures for solving fractional order polynomials that arise in complex engineering applications. Some real problems from various branches of engineering along with some higher degree polynomials are considered as test examples to show the global convergence property of simultaneous methods, performance and efficiency of the proposed family of methods. Further computational efficiencies, CPU time and residual graphs are also drawn to validate the efficiency, robustness of the newly introduced family of methods as compared to the existing methods in the literature.

On Highly Efficient Fractional Numerical Method for Solving Nonlinear Engineering Models

We proposed and analyzed the fractional simultaneous technique for approximating all the roots of nonlinear equations in this research study. The newly developed fractional Caputo-type simultaneous scheme’s order of convergence is 3ς+5, according to convergence analysis. Engineering-related numerical test problems are taken into consideration to demonstrate the efficiency and stability of fractional numerical schemes when compared to previously published numerical iterative methods. The newly developed fractional simultaneous approach converges on random starting guess values at random times, demonstrating its global convergence behavior. Although the newly developed method shows global convergent behavior when all starting guess values are distinct, the method diverges otherwise. The total computational time, number of iterations, error graphs and maximum residual error all clearly illustrate the stability and consistency of the developed scheme. The rate of convergence increases as the fractional parameter’s value rises from 0.1 to 1.0.

Chaotic Sand Cat Swarm Optimization

In this study, a new hybrid metaheuristic algorithm named Chaotic Sand Cat Swarm Optimization (CSCSO) is proposed for constrained and complex optimization problems. This algorithm combines the features of the recently introduced SCSO with the concept of chaos. The basic aim of the proposed algorithm is to integrate the chaos feature of non-recurring locations into SCSO’s core search process to improve global search performance and convergence behavior. Thus, randomness in SCSO can be replaced by a chaotic map due to similar randomness features with better statistical and dynamic properties. In addition to these advantages, low search consistency, local optimum trap, inefficiency search, and low population diversity issues are also provided. In the proposed CSCSO, several chaotic maps are implemented for more efficient behavior in the exploration and exploitation phases. Experiments are conducted on a wide variety of well-known test functions to increase the reliability of the results, as well as real-world problems. In this study, the proposed algorithm was applied to a total of 39 functions and multidisciplinary problems. It found 76.3% better responses compared to a best-developed SCSO variant and other chaotic-based metaheuristics tested. This extensive experiment indicates that the CSCSO algorithm excels in providing acceptable results.

Globally convergent iterative scheme for computing matrix sign function with numerical stability

In this article, we have proposed a novel matrix iteration for computing the matrix sign function numerically. An extensive convergence analysis is carried out in matrix case to achieve fourth‐order convergence. The basins of attraction are drawn to illustrate the global convergence behavior of the proposed method. Analytically, it is shown that the presented scheme is asymptotically stable. Further, the theoretical developments are verified numerically by discussing eigenvalues clustering around . Moreover, a class of numerical examples for matrices of various dimensions is assessed to exhibit the efficacy of proposed method.

Space-time geometric multigrid method for nonlinear advection–diffusion problems

Multigrid methods, algebraic or geometric, commonly suffer from high frequency residuals after prolongation. This paper develops a stable approach to remove high frequency residuals for geometric multigrid methods for solving nonlinear advection–diffusion problems with degenerate coefficients. Here, a local problem is treated by optimization on subdomains with mesh refinements. Newton's method is utilized in the procedure and the iteration is completed when the residual in the subdomain is reduced to the given magnitude, usually set to be the average of residuals in the non-high-frequency domains. An oversampling technique is employed to further improve the stability by providing a definite flow path in regions where coefficients have high contrast and complex structures. Removing high frequency residuals before continuing the global Newton iteration improves global convergence behavior.

Nonlinear feedback controller for the synchronization of (chaotic) systems with known parameters

This paper proposes, designs, and analyses a novel nonlinear feedback controller that realizes fast, and oscillation free convergence of the synchronization error to the equilibrium point. Oscillation free convergence lowers the failure chances of a closed-loop system due to the reduced chattering phenomenon in the actuator motion, which is a consequence of low energy sm ooth control signal. The proposed controller has a novel structure. This controller does not cancel nonlinear terms of the plant in the closed-loop; this attribute improves the robustness of the loop. The controller consists of linear and nonlinear parts; each part executes a specific task. The linear term in the controller keeps the closed-loop stable, while the nonlinear part of the controller facilitates the fast convergence of the error signal to the vicinity of the origin. Then the linear controller synthesizes a smooth control signal that moves the error signals to zero without oscillations. The nonlinear term of the controller does not contribute to this synthesis. The collaborative combination of linear and nonlinear controllers that drive the synchronization errors to zero is innovative. The paper establishes proof of global stability and convergence behavior by describing a detailed analysis based on the Lyapunov stability theory. Computer simulation results of two numerical examples verify the performance of the proposed controller approach. The paper also provides a comparative study with state-of-the-art controllers.

On Data Increasing in Phase Retrieval via Quadratic Inversion: Flattening Manifold and Local Minima

In this article, we address the question of traps in phase retrieval via quadratic inversion. First, we clarify the genesis of local minima from a geometrical perspective by showing how the shape of the manifold of data and the position of the data point are related to the presence of local minima in the objective functional. Later, we recall a mathematical condition for the lack of traps. Afterward, we illustrate how the ratio between the dimension of data space $M$ and the number of real unknowns $N_{r}$ impacts on the manifold of data and, consequently, on the presence of local minima. In particular, we show that if the dimension of data space is high enough, the manifold of data is sufficiently large and no traps appear in the functional. In the last part of this article, with reference to a particular unknown sequence, we establish the minimum value of the ratio ${M}/{N_{r}}$ such that no traps should appear in the functional. In this condition, phase retrieval via quadratic inversion exhibits a global convergence behavior, consequently, the actual solution of the problem can be reached regardless of the initial guess which can be chosen completely at random.

Computer Methodologies for the Comparison of Some Efficient Derivative Free Simultaneous Iterative Methods for Finding Roots of Non-Linear Equations

In this article, we construct the most powerful family of simultaneous iterative method with global convergence behavior among all the existing methods in literature for finding all roots of non-linear equations. Convergence analysis proved that the order of convergence of the family of derivative free simultaneous iterative method is nine. Our main aim is to check out the most regularly used simultaneous iterative methods for finding all roots of non-linear equations by studying their dynamical planes, numerical experiments and CPU time-methodology. Dynamical planes of iterative methods are drawn by using MATLAB for the comparison of global convergence properties of simultaneous iterative methods. Convergence behavior of the higher order simultaneous iterative methods are also illustrated by residual graph obtained from some numerical test examples. Numerical test examples, dynamical behavior and computational efficiency are provided to present the performance and dominant efficiency of the newly constructed derivative free family of simultaneous iterative method over existing higher order simultaneous methods in literature.

A fast iterative method to find the matrix geometric mean of two HPD matrices

The purpose of this research is to present a novel scheme based on a quick iterative scheme for calculating the matrix geometric mean of two Hermitian positive definite (HPD) matrices. To do this, an iterative scheme with global convergence is constructed for the sign function using a novel three‐step root‐solver. It is proved that the new scheme is convergent and shown to have global convergence behavior for this target, when square matrices having no pure imaginary eigenvalues. Next, the constructed scheme is used and extended through a well‐known identity for the calculation of the matrix geometric mean of two HPD matrices. Ultimately, several experiments are collected to show its usefulness.

Numerically stable improved Chebyshev–Halley type schemes for matrix sign function

A general family of iterative methods including a free parameter is derived and proved to be convergent for computing matrix sign function under some restrictions on the parameter. Several special cases including global convergence behavior are dealt with. It is analytically shown that they are asymptotically stable. A variety of numerical experiments for matrices with different sizes is considered to show the effectiveness of the proposed members of the family.

Dual and approximate dual basis functions for B-splines and NURBS – Comparison and application for an efficient coupling of patches with the isogeometric mortar method

This contribution defines and compares different methods for the computation of dual basis functions for B-splines and Non-Uniform Rational B-splines (NURBS). They are intended to be used as test functions for the isogeometric mortar method, but other fields of application are possible, too. Three different concepts are presented and compared. The first concept is the explicit formula for the computation of dual basis functions for NURBS proposed in the work of Carl de Boor. These dual basis functions entail minimal support, i.e., the support of the dual basis functions is equal to the support of the corresponding B-spline basis functions. In the second concept dual basis functions are derived from the inversion of the Gram matrix. These dual basis functions have global support along the interface. The third concept is the use of approximate dual basis functions, which were initially proposed for the use in harmonic analysis. The support of these functions is local but larger than the support of the associated B-spline basis functions. We propose an extension of the approximate dual basis functions for NURBS basis functions. After providing the general formulas, we elaborate explicit expressions for several degrees of spline basis functions. All three approaches are applied in the frame of the mortar method for the coupling of non-conforming NURBS patches. A method which allows complex discretizations with multiple intersecting interfaces is presented. Numerical examples show that the explicitly defined dual basis functions with minimal support severely deteriorate the global stress convergence behavior of the mechanical analysis. This fact is in accordance with mathematical findings in literature, which state that the optimal reproduction degree of arbitrary functions is not possible without extending the support of the dual basis functions. The dual basis functions computed from the inverse of the Gram matrix yield accurate numerical results but the global support yields significantly higher computational costs in comparison to computations of conforming meshes. Only the approximate dual basis functions yield accurate and efficient computations, where neither accuracy nor efficiency is significantly deteriorated in comparison to computations of conforming meshes. All basic cases of T-intersections and star-intersections are studied. Furthermore, an example which combines all basic cases in a complex discretization is given. The applicability of the presented method for the nonlinear case and for shell formulations is shown with the help of one numerical example.

Noncooperative registration for multistatic passive radars

Multistatic passive radars localize targets by using multiple bistatic range measurements. Multistatic localization requires all range measurements to be unbiased. It is proved by real data processing that registration is an important topic to be addressed. Traditionally, atomic clocks or GPS are used to achieve unbiased measurements. However, these approaches are unattainable under the case of multistatic passive radars as the transmitters of opportunity are beyond control. This paper proposes a novel registration method using noncooperative targets. The global convergence and accuracy of the proposed algorithm are obtained analytically. Computer simulations and experimental data are used to verify that the proposed algorithm possesses a good global convergence behavior and the performance can approach the Cram´er-Rao bound (CRB).

On a self-consistent-field-like iteration for maximizing the sum of the Rayleigh quotients

In this paper, we consider efficient methods for maximizing x⊤Bxx⊤Wx+x⊤Dx over the unit sphere, where B,D are symmetric matrices, and W is symmetric and positive definite. This problem can arise in the downlink of a multi-user MIMO system and in the sparse Fisher discriminant analysis in pattern recognition. It is already known that the problem of finding a global maximizer is closely associated with a nonlinear extreme eigenvalue problem. Rather than resorting to some general optimization methods, we introduce a self-consistent-field-like (SCF-like) iteration for directly solving the resulting nonlinear eigenvalue problem. The SCF iteration is widely used for solving the nonlinear eigenvalue problems arising in electronic structure calculations. One attractive feature of the SCF for our problem is that once it converges, the limit point not only satisfies the necessary local optimality conditions automatically, but also, and most importantly, satisfies a global optimality condition, which generally is not achievable in some optimization-based methods. The global convergence and local quadratic convergence rate are proved for certain situations. For the general case, we then discuss a trust-region SCF iteration for stabilizing the SCF iteration, which is of good global convergence behavior. Our preliminary numerical experiments show that these algorithms are more efficient than some optimization-based methods.

A majorization-minimization approach to the sparse generalized eigenvalue problem

Generalized eigenvalue (GEV) problems have applications in many areas of science and engineering. For example, principal component analysis (PCA), canonical correlation analysis (CCA) and Fisher discriminant analysis (FDA) are specific instances of GEV problems, that are widely used in statistical data analysis. The main contribution of this work is to formulate a general, efficient algorithm to obtain sparse solutions to a GEV problem. Specific instances of sparse GEV problems can then be solved by specific instances of this algorithm. We achieve this by solving the GEV problem while constraining the cardinality of the solution. Instead of relaxing the cardinality constraint using a ℓ 1-norm approximation, we consider a tighter approximation that is related to the negative log-likelihood of a Student’s t-distribution. The problem is then framed as a d.c. (difference of convex functions) program and is solved as a sequence of convex programs by invoking the majorization-minimization method. The resulting algorithm is proved to exhibit global convergence behavior, i.e., for any random initialization, the sequence (subsequence) of iterates generated by the algorithm converges to a stationary point of the d.c. program. Finally, we illustrate the merits of this general sparse GEV algorithm with three specific examples of sparse GEV problems: sparse PCA, sparse CCA and sparse FDA. Empirical evidence for these examples suggests that the proposed sparse GEV algorithm, which offers a general framework to solve any sparse GEV problem, will give rise to competitive algorithms for a variety of applications where specific instances of GEV problems arise.

Non-equidistant sampling for bounded bandlimited signals

In this paper non-equidistant sampling series are studied for bounded bandlimited signals. We consider sampling patterns that are made of the zeros of sine-type functions and analyze the local and global convergence behavior of the sampling series. It is shown that the series converge locally uniformly for bounded bandlimited signals that vanish at infinity. Moreover, we discuss the influence of oversampling on the global approximation behavior and the convergence speed of the sampling series.

Dynamic diversity control in genetic algorithm for mining unsearched solution space in TSP problems

The applications of genetic algorithms (GAs) in solving combinatorial problems are frequently faced with a problem of early convergence and the evolutionary processes are often trapped in a local optimum. This premature convergence occurs when the population of a genetic algorithm reaches a suboptimal state that the genetic operators can no longer produce offspring with a better performance than their parents. In the literature, plenty of work has been investigated to introduce new methods and operators in order to overcome this essential problem of genetic algorithms. As these methods and the belonging operators are rather problem specific in general. In this research, we take a different approach by constantly observing the progress of the evolutionary process and when the diversity of the population dropping below a threshold level then artificial chromosomes with high diversity will be introduced to increase the average diversity level thus to ensure the process can jump out the local optimum and to revolve again. A dynamic threshold control mechanism is built up during the evolutionary process to further improve the system performance. The proposed method is implemented independently of the problem characteristics and can be applied to improve the global convergence behavior of genetic algorithms. The experimental results using TSP instances show that the proposed approach is very effective in preventing the premature convergence when compared with other approaches.

Convergence behavior of non-equidistant sampling series

The convergence of sampling series with non-equidistant sampling points cannot be guaranteed for the Paley–Wiener space PW π 1 if the class of sampling patterns is not restricted. In this paper we consider sampling patterns that are made of the zeros of sine-type functions and analyze the local and global convergence behavior of the sampling series. It is shown that oversampling is necessary for global uniform convergence. If no oversampling is used there exists for every sampling pattern a signal such that the peak value of the approximation error grows arbitrarily large. Furthermore, we use the findings to derive results about the mean-square convergence behavior of the sampling series for bandlimited wide-sense stationary stochastic processes. Finally, a procedure is given to construct functions of sine type and possible sampling patterns.

Global Convergence and Limit Cycle Behavior of Weights of Perceptron

In this paper, it is found that the weights of a perceptron are bounded for all initial weights if there exists a nonempty set of initial weights that the weights of the perceptron are bounded. Hence, the boundedness condition of the weights of the perceptron is independent of the initial weights. Also, a necessary and sufficient condition for the weights of the perceptron exhibiting a limit cycle behavior is derived. The range of the number of updates for the weights of the perceptron required to reach the limit cycle is estimated. Finally, it is suggested that the perceptron exhibiting the limit cycle behavior can be employed for solving a recognition problem when downsampled sets of bounded training feature vectors are linearly separable. Numerical computer simulation results show that the perceptron exhibiting the limit cycle behavior can achieve a better recognition performance compared to a multilayer perceptron.