Strong Convergence Properties Research Articles

A null-space-based weightedl1minimization approach to compressed sensing

It has become an established fact that the constrained $\ell_1$ minimization is capable of recovering the sparse solution from a small number of linear observations and the reweighted version can significantly improve its numerical performance. The recoverability is closely related to the Restricted Isometry Constant (RIC) {of order $s$ ($s$ is an integer), often denoted as $\delta_{s}$. A class of sufficient conditions for successful $k$-sparse signal recovery often take the form $\delta_{tk} 1$, such a bound is often called RIC bound of high order.} There exist a number of such bounds of RICs, high order or not. For example, a high order bound {is recently given by Cai and Zhang \cite{CZ14}:} $\delta_{tk} < \sqrt{(t-1)/t}$, and this bound is known sharp for $t \ge 4/3$. In this paper, we propose a new weighted $\ell_1$ minimization which only requires the following RIC bound that is more relaxed (i.e., bigger) than the above mentioned bound: \[\delta_{tk} 1$ and $0< \omega \le 1$ is determined {by two optimizations of a similar type}over the null space of the linear observation operator. {In tackling the combinatorial nature of the two optimization problems,we develop a reweighted $\ell_1$ minimization that yields a sequence of approximate solutions,which enjoy strong convergence properties. Moreover, the numerical performance of the proposed method}is very satisfactory when compared to some of the state of-the-art methods incompressed sensing.

A Globally Convergent LP-Newton Method

We develop a globally convergent algorithm based on the LP-Newton method, which has been recently proposed for solving constrained equations, possibly nonsmooth and possibly with nonisolated solutions. The new algorithm makes use of linesearch for the natural merit function and preserves the strong local convergence properties of the original LP-Newton scheme. We also present computational experiments on a set of generalized Nash equilibrium problems, and a comparison of our approach with the previous hybrid globalization employing the potential reduction method.

Shape Optimization for a Class of Semilinear Variational Inequalities with Applications to Damage Models

The present contribution investigates shape optimization problems for a class of semilinear elliptic variational inequalities with Neumann boundary conditions. Sensitivity estimates and material derivatives are first derived in an abstract operator setting where the operators are defined on polyhedral subsets of reflexive Banach spaces. The results are then refined for variational inequalities arising from minimization problems for certain convex energy functionals considered over upper obstacle sets in $H^1$. One particularity is that we allow for dynamic obstacle functions which may arise from other optimization problems. We prove a strong convergence property for the material derivative and establish state-shape derivatives under regularity assumptions. Finally, as a concrete application from continuum mechanics, we show how the dynamic obstacle case can be used to treat shape optimization problems for time-discretised brittle damage models for elastic solids. We derive a necessary optimality system fo...

Kaplan–Meier estimator and hazard estimator for censored negatively superadditive dependent data

In this paper, we investigate the strong convergence properties for the Kaplan–Meier estimator and hazard estimator based on censored negatively superadditive dependent data. Under some mild conditions, the strong convergence rate of the Kaplan–Meier estimator and hazard estimator is established. In addition, the strong representation of the Kaplan–Meier estimator and hazard estimator is also obtained with the remainder of order Our results established in the paper generalize the corresponding ones for independent random variables and negatively associated random variables.

An efficient framework for the reliability-based design optimization of large-scale uncertain and stochastic linear systems

This paper is focused on the development of an efficient reliability-based design optimization algorithm for solving problems posed on uncertain linear dynamic systems characterized by large design variable vectors and driven by non-stationary stochastic excitation. The interest in such problems lies in the desire to define a new generation of tools that can efficiently solve practical problems, such as the design of high-rise buildings in seismic zones, characterized by numerous free parameters in a rigorously probabilistic setting. To this end a novel decoupling approach is developed based on defining and solving a limited sequence of deterministic optimization sub-problems. In particular, each sub-problem is formulated from information pertaining to a single simulation carried out exclusively in the current design point. This characteristic drastically limits the number of simulations necessary to find a solution to the original problem while making the proposed approach practically insensitive to the size of the design variable vector. To demonstrate the efficiency and strong convergence properties of the proposed approach, the structural system of a high-rise building defined by over three hundred free parameters is optimized under non-stationary stochastic earthquake excitation.

Speed adaptation for self-improvement of skills learned from user demonstrations

SUMMARYThe paper addresses the problem of speed adaptation of movements subject to environmental constraints. Our approach relies on a novel formulation of velocity profiles as an extension of dynamic movement primitives (DMP). The framework allows for compact representation of non-uniformly accelerated motion as well as simple modulation of the movement parameters. In the paper, we evaluate two model free methods by which optimal parameters can be obtained: iterative learning control (ILC) and policy search based reinforcement learning (RL). The applicability of each method is discussed and evaluated on two distinct cases, which are hard to model using standard techniques. The first deals with hard contacts with the environment while the second process involves liquid dynamics. We find ILC to be very efficient in cases where task parameters can be easily described with an error function. On the other hand, RL has stronger convergence properties and can therefore provide a solution in the general case.

Representation Discovery for MDPs Using Bisimulation Metrics

We provide a novel, flexible, iterative refinement algorithm to automatically construct an approximate statespace representation for Markov Decision Processes (MDPs). Our approach leverages bisimulation metrics, which have been used in prior work to generate features to represent the state space of MDPs.We address a drawback of this approach, which is the expensive computation of the bisimulation metrics. We propose an algorithm to generate an iteratively improving sequence of state space partitions. Partial metric computations guide the representation search and provide much lower space and computational complexity, while maintaining strong convergence properties. We provide theoretical results guaranteeing convergence as well as experimental illustrations of the accuracy and savings (in time and memory usage) of the new algorithm, compared to traditional bisimulation metric computation.

Representation Discovery for MDPs Using Bisimulation Metrics

We provide a novel, flexible, iterative refinement algorithm to automatically construct an approximate statespace representation for Markov Decision Processes (MDPs). Our approach leverages bisimulation metrics, which have been used in prior work to generate features to represent the state space of MDPs. We address a drawback of this approach, which is the expensive computation of the bisimulation metrics. We propose an algorithm to generate an iteratively improving sequence of state space partitions. Partial metric computations guide the representation search and provide much lower space and computational complexity, while maintaining strong convergence properties. We provide theoretical results guaranteeing convergence as well as experimental illustrations of the accuracy and savings (in time and memory usage) of the new algorithm, compared to traditional bisimulation metric computation.

An adaptive subspace trust-region method for frequency-domain seismic full waveform inversion

Full waveform inversion is currently considered as a promising seismic imaging method to obtain high-resolution and quantitative images of the subsurface. It is a nonlinear ill-posed inverse problem, the main difficulty of which that prevents the full waveform inversion from widespread applying to real data is the sensitivity to incorrect initial models and noisy data. Local optimization theories including Newton's method and gradient method always lead the convergence to local minima, while global optimization algorithms such as simulated annealing are computationally costly. To confront this issue, in this paper we investigate the possibility of applying the trust-region method to the full waveform inversion problem. Different from line search methods, trust-region methods force the new trial step within a certain neighborhood of the current iterate point. Theoretically, the trust-region methods are reliable and robust, and they have very strong convergence properties. The capability of this inversion technique is tested with the synthetic Marmousi velocity model and the SEG/EAGE Salt model. Numerical examples demonstrate that the adaptive subspace trust-region method can provide solutions closer to the global minima compared to the conventional Approximate Hessian approach and the L-BFGS method with a higher convergence rate. In addition, the match between the inverted model and the true model is still excellent even when the initial model deviates far from the true model. Inversion results with noisy data also exhibit the remarkable capability of the adaptive subspace trust-region method for low signal-to-noise data inversions. Promising numerical results suggest this adaptive subspace trust-region method is suitable for full waveform inversion, as it has stronger convergence and higher convergence rate.

Convergence and asymptotic stability of the explicit Steklov method for stochastic differential equations

In this paper, we develop a new numerical method with asymptotic stability properties for solving stochastic differential equations (SDEs). The foundations for the new solver are the Steklov mean and an exact discretization for the deterministic version of the SDEs. Strong consistency and convergence properties are demonstrated for the proposed method. Moreover, a rigorous linear and nonlinear asymptotic stability analysis is carried out for the multiplicative case in a mean-square sense and for the additive case in a path-wise sense using the pullback limit. In order to emphasize the characteristics of the Steklov discretization we use as benchmarks the stochastic logistic equation and the Langevin equation with a nonlinear potential of the Brownian dynamics. We show that the Steklov method has mild stability requirements and allows long-time simulations in several applications.

Uniform Semiglobal Exponential Stability of Integral Line-of-Sight Guidance Laws

This paper proves that an integral line-of-sight guidance law for path following control of underactuated marine vessels provides uniform semiglobal exponential stability. The stability result is stronger than what has been proved in previous literature, with stronger convergence properties and more robustness. The analysis is based on the 3-dimensional maneuvering control model of marine vessels, which describes both surface vessels and underwater vehicles moving in a horizontal plane. Both the kinematics and dynamics of the system is taken into account, as well as disturbances from constant and irrotational ocean currents. Simulation results are presented to validate the theoretical analysis.

Strong Convergence for Split-Step Methods in Stochastic Jump Kinetics

Mesoscopic models in the reaction-diffusion framework have gained recognition as a viable approach to describing chemical processes in cell biology. The resulting computational problem is a continuous-time Markov chain on a discrete and typically very large state space. Due to the many temporal and spatial scales involved many different types of computationally more effective multiscale models have been proposed, typically coupling different types of descriptions within the Markov chain framework. In this work we look at the strong convergence properties of the basic first order Strang, or Lie--Trotter, split-step method, which is formed by decoupling the dynamics in finite time steps. Thanks to its simplicity and flexibility, this approach has been tried in many different combinations. We develop explicit sufficient conditions for pathwise well-posedness and convergence of the method, including error estimates, and we illustrate our findings with numerical examples. In doing so, we also suggest a certain partition of unity representation for the split-step method, which in turn implies a concrete simulation algorithm under which trajectories may be compared in a pathwise sense.

On strong limit theorems for negatively superadditive dependent random variables

In this paper, we obtain strong convergence property for Jamison weighted sums of negatively superadditive dependent (NSD, in short) random variables, which extends the famous Jamison theorem. In addition, some sufficient conditions for complete convergence for weighed sums of NSD random variables are presented. These results generalize the corresponding results for independent identically distributed random variables to the case of NSD random variables without assumption of identical distribution.

Active constrained fuzzy clustering: A multiple kernels learning approach

In this paper, we address the problem of constrained clustering along with active selection of clustering constraints in a unified framework. To this aim, we extend the improved possibilistic c-Means algorithm (IPCM) with multiple kernels learning setting under supervision of side information. By incorporating multiple kernels, the limitation of improved possibilistic c-means to spherical clusters is addressed by mapping non-linear separable data to appropriate feature space. The proposed method is immune to inefficient kernels or irrelevant features by automatically adjusting the weight of kernels. Moreover, extending IPCM to incorporate constraints, its strong robustness and fast convergence properties are inherited by the proposed method. In order to avoid querying inefficient or redundant clustering constraints, an active query selection heuristic is embedded into the proposed method to query the most informative constraints. Experiments conducted on synthetic and real-world datasets demonstrate the effectiveness of the proposed method.

Improved error bound and a hybrid method for generalized Nash equilibrium problems

We exploit a recently proposed local error bound condition for a nonsmooth reformulation of the Karush---Kuhn---Tucker conditions of generalized Nash equilibrium problems (GNEPs) to weaken the theoretical convergence assumptions of a hybrid method for GNEPs that uses a smooth reformulation. Under the presented assumptions the hybrid method, which combines a potential reduction algorithm and an LP-Newton method, has global and fast local convergence properties. Furthermore we adapt the algorithm to a nonsmooth reformulation, prove under some additional strong assumptions similar convergence properties as for the smooth reformulation, and compare the two approaches.

On integrability and strong summability of Walsh-Kaczmarz series

In this paper we are concerned with integrability conditions and strong convergence properties of Walsh-Kaczmarz series that are derived from an inequality with respect to Walsh-Kaczmarz-Dirichlet kernels. Such type of inequalities, called Sidon type inequlities, have already been known for several other systems. The trigonometric and the Walsh-Paley-systems are among them. They turned to be useful in several areas including integrability conditions, strong convergence, strong approximation, and multipliers.

Study of a primal-dual algorithm for equality constrained minimization

The paper proposes a primal-dual algorithm for solving an equality constrained minimization problem. The algorithm is a Newton-like method applied to a sequence of perturbed optimality systems that follow naturally from the quadratic penalty approach. This work is first motivated by the fact that a primal-dual formulation of the quadratic penalty provides a better framework than the standard primal form. This is highlighted by strong convergence properties proved under standard assumptions. In particular, it is shown that the usual requirement of solving the penalty problem with a precision of the same size as the perturbation parameter, can be replaced by a much less stringent criterion, while guaranteeing the superlinear convergence property. A second motivation is that the method provides an appropriate regularization for degenerate problems with a rank deficient Jacobian of constraints. The numerical experiments clearly bear this out. Another important feature of our algorithm is that the penalty parameter is allowed to vary during the inner iterations, while it is usually kept constant. This alleviates the numerical problem due to ill-conditioning of the quadratic penalty, leading to an improvement of the numerical performances.

A Descent Algorithm-Based Parallel Variable Dual Matrix Model and Application to Blind Source Separation

In this paper, we focus on the problem of blind source separation (BSS). To solve the problem efficiently, a new algorithm is proposed. First, a parallel variable dual-matrix model (PVDMM) that considers all the numerical relations between a mixing matrix and a separating matrix is proposed. Different constrained terms are used to construct the cost function for every sub-algorithm. These constrained terms reflect a numerical relation. Therefore, a number of undesired solutions are excluded, the search region is reduced, and the convergence efficiency of the algorithm is ultimately improved. Parallel sub-algorithms are proven to converge to a separable matrix only if the cost function approaches zero. Second, a descent algorithm (DA) as a PVDMM optimizing approach is proposed in this paper as well. Apparently, the efficiency of the algorithm depends completely on the DA. Unlike the traditional descent method, the DA defines step length by solving inequality instead of merely utilizing the Wolfe- or Armijo-type search rule. Stimulation results indicate that the DA can improve computational efficiency. Under mild conditions, the DA has been proven to have strong convergence properties. Numerical results also show that the method is very efficient and robust. Finally, we applied the combinative algorithm to the BSS problem. Computer simulations illustrate its good performance.

Regularized Penalty Method for General Equilibrium Problems in Banach Spaces

We consider the regularized version of the penalty method for a general equilibrium problem in a Banach space setting. We suggest weak coercivity conditions instead of (generalized) monotonicity and show that they also provide weak and strong convergence properties of the method.

Strong local convergence properties of adaptive regularized methods for nonlinear least squares

Strong local convergence properties of adaptive regularized methods for nonlinear least squares