Stochastic Approximation Procedure Research Articles

Exact bounds for the rate of convergence in general stochastic approximation procedures

In this paper exact constants are given for the rate of convergence of the solution x n of the nonlinear difference equation towards the root θ of f under very general conditions on the steplengths an , on the weighting factors bn and on the random noise vn . Moreover, an almost sure representation is developed for the deviation xn – θ by weighted sums of (independent) random variables

A note on the empirical distribution of dependent random variables

Let {Y n, F n} n ⩾ 0 be an adapted stochastic sequence. Let D n(t) = (1/n)Σ i=1 n (1 {Y i ⩽ t} − P{Y i ⩽ t | F i − 1}) . In stochastic approximation and various other sequential procedures, the question of whether ▪ arises. For each t, D n ( t) converges to 0 a.s., and by the Glivenko-Cantelli theorem (1) holds if { Y n } is i.i.d. We give an example showing that without the assumption of independence (1) might fail to hold.

An almost sure invariance principle for stochastic approximation procedures in linear filtering theory

In this paper we consider a class of stochastic approximation procedures that arises in linear filtering and regression theory. Our main result asserts that the stochastic approximation process satisfies an almost sure invariance principle (with a certain rate of convergence) if the partial sums of the errors do.

An Empirical Study of Weak Convergence for Automatic Learning

The algorithms described in this paper use estimates of the sensitivities in a gradient-based stochastic approximation procedure for adaptive control. Learning rates are kept constant for adaptability reasons. For these procedures, the control processes do not converge almost surely, but in distribution. For a flexible machine where the control parameter is a probability vector, a new sensitivity estimator is described, generalizing the Phantom RPA to multi-valued decisions. Several updating schemes are built based on the weak convergence theory to ensure asymptotic optimality. Results show that weak convergence yields a dramatic improvement in the rate of convergence, in addition to the capability of tracking.

On a stochastic approximation procedure based on averaging

Based on the idea of averaging a new stochastic approximation algorithm has been proposed by Bather (1989), which shows a preferable performance for small to moderate sample sizes. In the present paper an almost sure representation is established for this procedure, which gives the optimal rate of convergence with minimal asymptotic variance.

On the convergence of linear stochastic approximation procedures

Many stochastic approximation procedures result in a stochastic algorithm of the form h/sub k+1/=h/sub k/+1/k(b/sub k/-A/sub k/h/sub k/), for all k=1,2,3,... . Here, {b/sub k/,k=1,2,3...} is a R/sup d/-valued process, {A/sub k/,k=1,2,3,...} is a symmetric, positive semidefinite Re/sup d/spl times/d/-valued process, and {h/sub k/,k=1,2,3,...} is a sequence of stochastic estimates which hopefully converges to h/sup /spl Delta//=[lim/sub N/spl rarr//spl infin//1/N/spl Sigma//sub k=1//sup N/EA/sub k/]/sup -1/ {lim/sub N/spl rarr//spl infin//1/N/spl Sigma//sub k=1//sup N/Eb/sub k/} (assuming everything here is well defined). We give an elementary proof which relates the almost sure convergence of {h/sub k/,k=1,2,3,...} to strong laws of large numbers for {b/sub k/,k=1,2,3,...} and {A/sub k/,k=1,2,3,...}.

The optimal number of kanbans in a manufacturing system with general machine breakdowns and stochastic demands

Considers manufacturing systems with a failure‐prone machine and a stochastic demand. The production is controlled by a kanban scheme. Although the kanban‐control enjoys many applications, in a stochastic setting, the problem of defining the optimal number of circulating kanbans remains unsolved except for some special cases. Seeks an optimal kanban‐controlled policy which minimizes long run average inventory and backlog costs. Uses perturbation analysis to estimate the gradients of the cost functional with respect to the number of circulating kanbans, and then employs an iterative algorithm, which is a constant step‐size stochastic approximation procedure, to find the optimal number of circulating kanbans. Proves that the perturbed path is a shifted version of the nominal path, and the gradient estimate is consistent. Also conducts numerical experiments to investigate the performance of the proposed algorithms.

Stability and convergence of large-scale stochastic approximation procedures

In practice, knowledge about input-output systems from various scientific fields such as engineering, biology and the social sciences is limited to output data. In this work we consider the continuous version of the discrete approximation procedure relative to such a system whose outcome is known at a time and a point. Convergence and stability results of the solution of continuous time approximation procedures are developed utilizing comparison principles in the context of vector Lyapunov-like functions. In particular, by considering continuous time approximation procedures as large-scale procedures, similar results are obtained using the decomposition-aggregation principle.

Non-standard limit theorems for urn models and stochastic approximation procedures

The adaptive processes of growth modeled by a generalized urn scheme have proved to be an efficient tool for the analysis of complex phenomena in economics, biology, and physical chemistry. They demonstrate non-ergodic limit behavior with multiple limit states. There are two major sources of complex feedbacks governing these processes: non-linearity (even local which is caused by non-differentiability of the functions driving them) and multiplicity of limit states stipulated by the non-linearity. The authors suggest an analytical approach for studying some of the patterns of complex limit behavior. The approach is based on conditional limit theorems. The corresponding limits are, in general, not infinitely divisible. They show that convergence rates could vary for different limit states. The rates depend upon the smoothness (in neighborhoods of the limit states) of the functions governing the processes. Since the mathematical machinery allows us to treat a quite general class of recursive stochastic discrete-time processes, we also derive corresponding limit theorems for stochastic approximation procedures. The theorems yield new insight into the limit behavior of stochastic approximation procedures in the case of non-differentiable regression functions with multiple roots.

Establishing connections between evolutionary algorithms and stochastic approximation

This work is our first attempt in establishing the connections between evolutionary computation algorithms and stochastic approximation procedures. By treating evolutionary algorithms as recursive stochastic procedures, we study both constant gain and decreasing step size algorithms. We formulate the problem in a rather general form, and supply the sufficient conditions for convergence (both with probability one, and in the weak sense). Among other things, our approach reveals the natural connection of the discrete iterations and the continuous dynamics (ordinary differential equations, and/or stochastic differential equations). We hope that this attempt will open up a new horizon for further research and lead to in depth understanding of the underlying algorithms.

Continuous action set learning automata for stochastic optimization

The problem of optimization with noisy measurements is common in many areas of engineering. The only available information is the noise corrupted value of the objective function at any chosen point in the parameter space. One well known method for solving this problem is the stochastic approximation procedure. In this paper we consider an adaptive random search procedure, based on the reinforcement learning paradigm. The learning model presented here generalizes the traditional model of a learning automaton [Narendra and Thathachar, Learning Automata : An Introduction, Prentice Hall, Englewood Cliffs, 1989]. This procedure requires a lesser number of function evaluations at each step compared to the stochastic approximation. The convergence properties of the algorithm are theoretically investigated. Simulation results are presented to show the efficacy of the learning method.

On the rate of convergence of stochastic approximation procedures

The rate of convergence of a linear stochastic approximation procedure inRd is studied under fairly general assumptions on the coefficients of the equation.

A class of robust image processors

In this paper robust recursive estimators for image restoration are developed. Image restoration for images corrupted by noise is carried out in two steps. To preserve true edges while restoring, edge detection using a 5 × 5 × 5 × 5 Graeco-Latin square is carried out as a first step. An edge is localized using an F-test on contrasts. The center pixel is then estimated as a second step. The method of estimation of a center pixel uses a multiple linear regression model fitted to the noisy image part on the same side of the edge. Parameters of a multiple linear regression model are estimated recursively using the Robbins-Monro Stochastic Approximation procedure applied to the least-squares estimator. When noise departs from a Gaussian assumption, robust techniques for restoration are sought. The recursive least-squares estimator is robustized using Huber's maximum likelihood estimator of location parameter of the - f′/ f type, where - f′/ f is approximated by an M-interval polynomial approximation algorithm, and f is the p.d.f. of noise. A minimax estimator based on a soft limiter is used to robustize the recursive least-squares estimator as a computationally simpler but slightly less efficient alternative. The theory developed in this paper was tested using computer simulations which verified the theory and evaluated the computational complexity/simplicity of the methods.

About Gaussian schemes in stochastic approximation

A family of one-dimensional linear stochastic approximation procedures in continuous time where processes of errors are Gaussian martingales is considered. Under some general assumptions the asymptotic behaviour of these procedures is studied concerning strong consistency, rate of convergence and limiting law of involved estimates and costs. At first some asymptotic results for Gaussian martingales, associated quadratic functionals and functions with finite variation are discussed.

UNSUPERVISED LEARNING OF PRIOR WEIGHTS FOR BAYES RULES IN PATTERN RECOGNITION

In the pattern classification problem, it is known that the Bayes decision rule, which separates k classes, gives a minimum probability of misclassification. In this study, the prior probability of each class is unknown and the conditional density functions are known. A set of unidentified input patterns is used to establish an empirical Bayes rule, which separates k classes and which leads to a stochastic approximation procedure for estimation of the priors. This classifier can adapt itself to a better decision rule by making use of unidentified input patterns while the system is in use. The resulting probability of misclassification can be made arbitrarily close to that of the Bayes rule. The results of a Monte Carlo simulation study with normal, uniform and exponential distributions are presented to demonstrate the favorable prior estimation for the empirical Bayes rule.

On a class of stochastic differential equations arising from the stochastic approximation theory

The paper dealt with generalized stochastic approximation procedures of Robbins-Monro type. We consider these procedures as strong solutions of some stochastic differential equations with respect to semimartingales and investigate their almost sure convergence and mean square convergence

Martingale transforms with non-atomic limits and stochastic approximation

A sufficient condition for the limit of a martingale transform to posses a continuous distribution is given. The result is used to show that for a stochastic approximation procedure, if the adjustment rate is too small then it would not converge to the target value a.s. Furthermore, if the adjustment rate is taken to be 1/n as usual but the derivative of the regression function at the target value is 0, then the convergence rate is shown to be logn instead of\(\sqrt n\), the rate obtained when the derivative is non-zero.

Exponential inequalities in stochastic approximation procedures

Exponential inequalities in stochastic approximation procedures

Semi-srochastic approximation by the response surface methodology (RMS)

Stochastic approximation procedures, e.g. stochastic gradient methods. for the minimization of a mean value unction. on a cover feasible domain can be accelerated considerably by using deterministic descent directions or more exact gradient estimates at certain iteration points. The gradient estimator estimator considered here is defined by the gradient of a so-called first or second order polynomial reponse surface model f of the unknow objective function f,where the estimate f of F is constructed by regression analysis.The accuracy of this gradient estimator is examined and the convergence behavior of the resulting hybride procedure—in comparison with standard stochastic approximation—is considered

On sampling controlled stochastic approximation

The authors examine a novel class of stochastic approximation procedures which are based on carefully controlling the number of observations or measurements taken before each computational iteration. This method, referred to as sampling controlled stochastic approximation, has advantages over standard stochastic approximation such as requiring less computation and the ability to handle bias in estimation. The authors address the growth rate required of the number of samples and prove a general convergence theorem for the proposed stochastic approximation method. In addition, they present applications to optimize and also derive a sampling controlled version of the classic Robbins-Munro algorithm.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>