Maximum Rate Of Convergence Research Articles

Inexact accelerated high-order proximal-point methods

In this paper, we present a new framework of bi-level unconstrained minimization for development of accelerated methods in Convex Programming. These methods use approximations of the high-order proximal points, which are solutions of some auxiliary parametric optimization problems. For computing these points, we can use different methods, and, in particular, the lower-order schemes. This opens a possibility for the latter methods to overpass traditional limits of the Complexity Theory. As an example, we obtain a new second-order method with the convergence rate Oleft( k^{-4}right) , where k is the iteration counter. This rate is better than the maximal possible rate of convergence for this type of methods, as applied to functions with Lipschitz continuous Hessian. We also present new methods with the exact auxiliary search procedure, which have the rate of convergence Oleft( k^{-(3p+1)/ 2}right) , where p ge 1 is the order of the proximal operator. The auxiliary problem at each iteration of these schemes is convex.

Exponentially fitted finite difference method for singularly perturbed delay differential equations with integral boundary condition

In this paper, exponentially fitted finite difference method for solving singularly perturbed delay differential equation with integral boundary condition is considered. To treat the integral boundary condition, Simpson’s rule is applied. The stability and parameter uniform convergence of the proposed method are proved. To validate the applicability of the scheme, two model problems are considered for numerical experimentation and solved for different values of the perturbation parameter, and mesh size, The numerical results are tabulated in terms of maximum absolute errors and rate of convergence and it is observed that the present method is more accurate and -uniformly convergent for where the classical numerical methods fails to give good result and it also improves the results of the methods existing in the literature.

Analysis of convergence of adaptive single­step algorithms for the identification of non­stationary objects

The study deals with the problem of identification of non-stationary parameters of a linear object which can be described by first-order Markovian model, with the help of the simplest in computational terms single-step adaptive identification algorithms – modified algorithms by Kaczmarz and Nagumo-Noda. These algorithms do not require knowledge of information on the degree of non-stationarity of the studied object. When building the model, they use the information only about one step of measurements. Modification involves the use of the regularizing addition in the algorithms to improve their computing properties and avoid division by zero. Using a Markovian model is quite effective because it makes it possible to obtain analytic estimates of the properties of algorithms. It was shown that the use of regularizing additions in identification algorithms, while improving stability of algorithms, leads to some slowdown of the process of model construction. The conditions for convergence of regularizing algorithms by Kaczmarz and Nagumo-Noda at the evaluation of stationary parameters in mean and root-mean-square and existing measurement interference were determined. The obtained estimates differ from the existing ones by higher accuracy. Despite this, they are quite general and depend both on the degree of non-stationarity of an object, and on statistical characteristics of interference. In addition, the expressions for the optimal values of the parameters of algorithms, ensuring their maximum rate of convergence under conditions of non-stationarity and the presence of Gaussian interferences, were determined. The obtained analytical expressions contain a series of unknown parameters (estimation error, degree of non-stationarity of an object, statistical characteristics of interferences). For their practical application, it is necessary to use any recurrent procedure for estimation of these unknown parameters and apply the obtained estimates to refine the parameters that are included in the algorithms

The relaxed gradient based iterative algorithm for solving matrix equations [formula omitted

In this paper, we first present a relaxed gradient based iterative (RGI) algorithm for solving matrix equations A1XB1=F1 and A2XB2=F2. The idea is from (Niu et al., 2011; Xie and Ma, 2016) in which efficient algorithms were developed for solving the Sylvester matrix equations. Then the RGI algorithm is extended to solve the generalized linear matrix equations of the form AiXBi=Fi,(i=1,2,…,N). For any initial value, it is proved that the iterative solution converges to their true solution under some appropriate assumptions. Finally, a numerical example is given to illustrate that the introduced iterative algorithm is more efficient than gradient based maximal convergence rate iterative (GI) algorithm of Ding et al. (2010) in speed, elapsed time and iterative steps.

Nonlinear smoothing of N-dimensional data using successive over-relaxation method

Local smoothing of N-dimensional data can be performed in many ways. This paper is oriented to local penalization and its minimization which generates a system of nonlinear equations. This approach enables to realize trade-off between denoising, edge, and structure preserving. This is mainly useful in the case of discontinuous signals and images. Various penalization strategies can be used for this task, but only constrained penalizations (Tukey, Welsch, Andrews) are successful. Novel nonlinear method is inspired by successive over-relaxation scheme for linear systems of equations, but it is applied to nonlinear root-finding problem. The method is designed to be stable for several smoother types. Root bracketing inside inner loop is included in the procedure and extends the stability range in many applications. Numerical experiments are performed on 1D signal and 2D image. Optimum relaxation factors are found experimentally for maximum rate of convergence. The main results of experimental part are: preference of Tukey method in the case of discontinuous signal, similarity of proposed methods in the case of continuous signal, and efficiency of Tukey method followed by watershed transform in the case of image segmentation. Selected smoothers are recommended mainly for signals and images with discontinuities and can be useful in signal and image enhancement, analysis, segmentation, and classification.

Complete algebraic reconstruction of piecewise-smooth functions from Fourier data

In this paper we provide a reconstruction algorithm for piecewise-smooth functions with a-priori known smoothness and number of discontinuities, from their Fourier coefficients, posessing the maximal possible asymptotic rate of convergence -- including the positions of the discontinuities and the pointwise values of the function. This algorithm is a modification of our earlier method, which is in turn based on the algebraic method of K.Eckhoff proposed in the 1990s. The key ingredient of the new algorithm is to use a different set of Eckhoff's equations for reconstructing the location of each discontinuity. Instead of consecutive Fourier samples, we propose to use a decimated set which is evenly spread throughout the spectrum.

Finite Elements with mesh refinement for wave equations in polygons

Error estimates for the space semi-discrete approximation of solutions of the Wave equation in polygons G⊂R2 are presented. Based on corner asymptotics of the solution, it is shown that for continuous, simplicial Lagrangian Finite Elements of polynomial degree p≥1 with either suitably graded mesh refinement or with bisection tree mesh refinement towards the corners of G, the maximal rate of convergence O(N−p/2) which is afforded by the Lagrangian Finite Element approximations on quasiuniform meshes for smooth solutions is restored. Dirichlet, Neumann and mixed boundary conditions are considered. Numerical experiments which confirm the theoretical results are presented. Generalizations to nonhomogeneous coefficients and elasticity and electromagnetics are indicated.

Growth rates for persistently excited linear systems

We consider a family of linear control systems $$\dot{x}=Ax+\alpha Bu$$ on $$\mathbb {R}^d$$ , where $$\alpha $$ belongs to a given class of persistently exciting signals. We seek maximal $$\alpha $$ -uniform stabilization and destabilization by means of linear feedbacks $$u=Kx$$ . We extend previous results obtained for bidimensional single-input linear control systems to the general case as follows: if there exists at least one $$K$$ such that the Lie algebra generated by $$A$$ and $$BK$$ is equal to the set of all $$d\times d$$ matrices, then the maximal rate of convergence of $$(A,B)$$ is equal to the maximal rate of divergence of $$(-A,-B)$$ . We also provide more precise results in the general single-input case, where the above result is obtained under the simpler assumption of controllability of the pair $$(A,B)$$ .

Finding the pseudogradient of the objective function in procedures for estimation of interframe deformations of images

In the recursive estimate of parameters of interframe geometric deformations of digital images, calculating the pseudogradient from a local sample and current estimates of deformation parameters to be measured involves finding derivatives in terms of finite differences. The maximal convergence rate of estimates of the deformation parameters can be achieved when using the most informative readings of images in a local sample. Under this approach, there exist optimal values of increments in the parameters and basic axes, depending on the mismatch between estimates and correlation properties of the image. The performance of the procedures may be increased if the increments are optimized at each iteration. The potential accuracy of estimation of the parameters, as given by the Cramer-Rao inequality, serves as an optimality criterion when calculating the pseudogradient of the objective function. It is shown that the condition of maximum of information can be reduced to the condition of maximization of the ratio of the expectation of the pseudogradient to its mean square deviation. Application of optimized values of increments enables one to reduce considerably (up to several times) the computational cost with the same accuracy of estimation.

Optimization of convergence rate and stability margin of information flow in cooperative systems

The interplay between the convergence rate and stability margin (e.g. ability to reject disturbances) for a discrete-time information flow filter in cooperative systems is analyzed. For a given communication graph, the convergence rate is defined as the absolute value of the largest nonunit characteristic root of a matrix associated with the filter. The maximal convergence rate, obtained by “tuning” the control gains, is highly correlated to the number of distinct eigenvalues of the graph Laplacian (it is 1 for the complete graph). A stability margin is introduced for multiple-input–multiple-output (MIMO) systems and is then maximized with respect to the control gains subject to a constraint on the convergence rate. The optimal stability margin as a function of the convergence rate is bounded above for any order of the filter, and the bound is attained for the complete graph. For the zero-order filter and all strongly connected communication graphs, the optimal stability margin is found analytically, whereas for the first-order filter and undirected communication graphs, it is evaluated numerically. The results demonstrate the ability to distinguish graph topologies that dominate others in their ability to reject disturbances and converge rapidly to a consensus.

Finite elements with mesh refinement for wave equations in polygons

Error estimates for the space semi-discrete approximation of solutions of the Wave equation in polygons G ? R 2 are presented. Based on corner asymptotics of the solution, it is shown that for continuous, simplicial Lagrangian Finite Elements of polynomial degree p ? 1 with either suitably graded mesh refinement or with bisection tree mesh refinement towards the corners of G , the maximal rate of convergence O ( N - p / 2 ) which is afforded by the Lagrangian Finite Element approximations on quasiuniform meshes for smooth solutions is restored. Dirichlet, Neumann and mixed boundary conditions are considered. Numerical experiments which confirm the theoretical results are presented. Generalizations to nonhomogeneous coefficients and elasticity and electromagnetics are indicated.

An iterative method to compute Moore-Penrose inverse based on gradient maximal convergence rate

In this paper, we present an iterative method based on gradient maximal convergence rate to compute Moore-Penrose inverse A+ of a given matrix A. By this iterative method, when taken the initial matrix X0 = A*, the M-P inverse A+ can be obtained with maximal convergence rate in absence of round off errors. In the end, a numerical example is given to illustrate the effectiveness, accuracy and its computation time, which are all superior than the other methods for the large singular matrix.

Optimal exponential feedback stabilization of planar systems

This paper solves the problem of finding an optimal feedback control ensuring the maximal rate of convergence of system solutions to the origin for a general class of planar control systems including switched, bilinear systems and ones described by differential inclusions, etc. The prescribed control set is assumed to be compact but not necessarily convex. The developed approach is based on finding the minimal Lyapunov exponent of the system with an open loop control which provides an upper bound for the optimal convergence rate of the closed loop system. Then an optimal feedback controller is constructed for which the obtained bound is attained.

Gradient-based maximal convergence rate iterative method for solving linear matrix equations

This paper is concerned with numerical solutions to general linear matrix equations including the well-known Lyapunov matrix equation and Sylvester matrix equation as special cases. Gradient based iterative algorithm is proposed to approximate the exact solution. A necessary and sufficient condition guaranteeing the convergence of the algorithm is presented. A sufficient condition that is easy to compute is also given. The optimal convergence factor such that the convergence rate of the algorithm is maximized is established. The proposed approach not only gives a complete understanding on gradient based iterative algorithm for solving linear matrix equations, but can also be served as a bridge between linear system theory and numerical computing. Numerical example shows the effectiveness of the proposed approach.

Estimation of non-stationary delay for a linear discrete dynamic plant

A nonlinear recurrent gradient-type algorithm is proposed for estimation of nonstationary delay arising in finite impulse response of a linear discrete dynamic plant. Convergence of the algorithm is investigated. Using linear matrix inequalities, its optimal functioning is investigated in order to provide for the maximal rate of convergence of the estimation error to zero.

Efficient Strong Integrators for Linear Stochastic Systems

We present numerical schemes for the strong solution of linear stochastic differential equations driven by an arbitrary number of Wiener processes. These schemes are based on the Neumann (stochastic Taylor) and Magnus expansions. First, we consider the case when the governing linear diffusion vector fields commute with each other, but not with the linear drift vector field. We prove that numerical methods based on the Magnus expansion are more accurate in the mean-square sense than corresponding stochastic Taylor integration schemes. Second, we derive the maximal rate of convergence for arbitrary multidimensional stochastic integrals approximated by their conditional expectations. Consequently, for general nonlinear stochastic differential equations with noncommuting vector fields, we deduce explicit formulae for the relation between error and computational costs for methods of arbitrary order. Third, we consider the consequences in two numerical studies, one of which is an application arising in stochastic linear-quadratic optimal control.

A new algorithm with maximal rate convergence to obtain inverse matrix

This paper introduces a new efficient algorithm to obtain the accurate inverse of a given nonsingular matrix with maximal rate of convergence. The method is performed efficiently based on accurate inversion of the diagonally dominant matrix which is made from the given matrix, where it should be inverted. The presented algorithm can be employed in any numerical method.

Nonlinear control design for linear differential inclusions via convex hull of quadratics

This paper presents a nonlinear control design method for robust stabilization and robust performance of linear differential inclusions (LDIs). A recently introduced non-quadratic Lyapunov function, the convex hull of quadratics, will be used for the construction of nonlinear state feedback laws. Design objectives include stabilization with maximal convergence rate, disturbance rejection with minimal reachable set and least L 2 gain. Conditions for stabilization and performances are derived in terms of bilinear matrix inequalities (BMIs), which cover the existing linear matrix inequality (LMI) conditions as special cases. Numerical examples demonstrate the advantages of using nonlinear feedback control over linear feedback control for LDIs. It is also observed through numerical computation that nonlinear control strategies help to reduce control effort substantially.

On the optimal convergence rate of universal and nonuniversal algorithms for multivariate integration and approximation

We study the maximal rate of convergence (mrc) of algorithms for (multivariate) integration and approximation of d-variate functions from reproducing kernel Hilbert spaces H(K d ). Here K d is an arbitrary kernel all of whose partial derivatives up to order r satisfy a Holder-type condition with exponent 2β. Algorithms use n function values and we analyze their rate of convergence as n tends to infinity. We focus on universal algorithms which depend on d, r, and β but not on the specific kernel K d , and nonuniversal algorithms which may depend additionally on K d . For universal algorithms the mrc is (r + β)/d for both integration and approximation, and for nonuniversal algorithms it is 1/2+(r+β)/d for integration and a + (r + β)/d with a ∈ [1/(4 + 4(r + β)/d), 1/2] for approximation. Hence, the mrc for universal algorithms suffers from the curse of dimensionality if d is large relative to r + β, whereas the mrc for nonuniversal algorithms does not since it is always at least 1/2 for integration, and 1/4 for approximation. This is the price we have to pay for using universal algorithms. On the other hand, ifr r+β is large relative to d, then the mrc for universal and nonuniversal algorithms is approximately the same. We also consider the case when we have the additional knowledge that the kernel K d has product structure, K d,r,β = ⊗ d j=1 K rj,βj . Here K rj,βj are some univariate kernels whose all derivatives up to order rj satisfy a Holder-type condition with exponent 2 βj . Then the mrc for universal algorithms is q:= min j=1,2...,d (r j +β j ) for both integration and approximation, and for nonuniversal algorithms it is 1/2 + q for integration and a + q with a ∈ [1/(4 + 4q), 1/2] for approximation. If r j ≥ 1 or β j ≥ β > 0 for all j, then the mrc is at least min(1, β), and the curse of dimensionality is not present. Hence, the product form of reproducing kernels breaks the curse of dimensionality even for universal algorithms.

Variational Problems in Domains with Cusp-Points and the Finite Element Method

ABSTRACT A variational problem in a two-dimensional domain with cusp-points corresponding to a linear elliptic boundary value problem is formulated and the unique existence of its solution is proved. The corresponding finite element method using triangular finite C 0-elements with polynomials of the first degree is analyzed and both the convergence (under the assumptions sufficient for the existence of the exact solution) and the maximal rate of convergence 𝒪(h) are proved.