Stochastic Approximation Algorithm Research Articles

Stochastic approximation algorithms for rumor source inference on graphs

We revisit the classical problem of identifying the source of rumor in a network, assumed unique, treating as given the network topology and the set of rumor infected nodes. In addition, it is assumed that some partial information about the order in which the nodes were infected, is also available. Such form of information is commonly available to network monitoring agencies. Under this premise, we propose a class of estimators that is agnostic to the underlying stochastic model of rumor spread and further, generalizes other estimators popular in literature, e.g., rumor center, distance center and Jordan center. We also develop an MCMC-based stochastic approximation framework to implement this class of estimators. Results from extensive simulations on the Erdős–Rényi and Barabási–Albert random graph models indicate promising empirical performance and robustness of the proposed estimators to different graph topologies. Our stochastic approximation framework also extends easily to general statistical inference problems on graphs that are of combinatorial nature. In particular, we demonstrate a suitable modification of our framework that makes it amenable to be used for the popular problem of rank aggregation from noisy pairwise comparisons.

Large and moderate deviation principles for recursive kernel estimators of a regression function for spatial data defined by stochastic approximation method

In the present paper, we are mainly concerned with a family of kernel type estimators based upon spatial data. More precisely, we establish large and moderate deviations principles for the recursive kernel estimators of a regression function for spatial data defined by the stochastic approximation algorithm.

Bandwidth selector for nonparametric recursive density estimation for spatial data defined by stochastic approximation method

In this article we propose an automatic selection of the bandwidth of the recursive kernel density estimators for spatial data defined by the stochastic approximation algorithm. We showed that, using the selected bandwidth and the stepsize which minimize the MWISE (Mean Weighted Integrated Squared Error), the recursive estimator will be quite similar to the nonrecursive one in terms of estimation error and much better in terms of computational costs. In addition, we obtain the central limit theorem for the nonparametric recursive density estimator under some mild conditions.

Block Trading: Building up a Stock Position Under a Regime Switching Model

This work is intended to systematically characterize the problem of block trading. The author characterizes it as a constrained optimal purchasing (or control) problem, on behalf of the vendee. The purpose of this work is to investigate the optimal purchasing strategies and give quantitative reference information on the pricing of block trade. Noting that large block of stock trading may result in the price fluctuation and different market environment may lead to distinct investing strategies, the controlled diffusion model with regime switching is adopted, which can be used to represent distinct environment of financial market, and the model is modified to allow the price impact, by allowing the drift depend on the purchase rate. Where the switching regimes can be completely observable or unobservable (hidden). Which is a significant difference in contrast to most market models in the existing literature. The objective is to maximize the total discounted shares of the underlying stock, subject to the expected fund for building up corresponding position with an upper bound. This work mainly focuses on the completely observable case, as to the case with hidden regimes, this work just briefly talk about how to reduce the unobservable model to a completely observable one. In the completely observable case, to solve this constrained control problem, the Lagrange multiplier methods and the dynamic programming approaches are used. Though optimality is obtained, what is still lacking is an explicit solution to the optimality equation. To overcome this difficulty, the two loop approximation scheme is given. For fixed Lagrange multiplier, the inner loop is to obtain optimal strategies, which is based mainly on the finite difference approximation. While, the outer loop is a stochastic recursive approximation algorithm to get the optimal Lagrange multiplier. Convergence results are obtained for both the inner and the outer approximations. The quantitative pricing information is related to the threshold levels of the optimal purchasing strategy. Numerical examples are also provided to illustrate our results. Finally, as to the unobservable case, by using the well known Wonham filter approach, one can reduce the unobservable case to a completely observable one.

Stochastic Successive Convex Approximation for Non-Convex Constrained Stochastic Optimization

This paper proposes a constrained stochastic successive convex approximation (CSSCA) algorithm to find a stationary point for a general non-convex stochastic optimization problem, whose objective and constraint functions are non-convex and involve expectations over random states. Most existing methods for non-convex stochastic optimization, such as the stochastic (average) gradient and stochastic majorization-minimization, only consider minimizing a stochastic non-convex objective over a deterministic convex set. The proposed CSSCA algorithm can also handle stochastic non-convex constraints in optimization problems, and it opens the way to solving more challenging optimization problems that occur in many applications. The algorithm is based on solving a sequence of convex objective/feasibility optimization problems obtained by replacing the objective/constraint functions in the original problems with some convex surrogate functions. The CSSCA algorithm allows a wide class of surrogate functions and thus provides many freedoms to design good surrogate functions for specific applications. Moreover, it also facilitates parallel implementation for solving large scale stochastic optimization problems, which arise naturally in today's signal processing such as machine learning and big data analysis. We establish the convergence of CSSCA algorithm with a feasible initial point, and customize the algorithmic framework to solve several important application problems. Simulations show that the CSSCA algorithm can achieve superior performance over existing solutions.

General multilevel adaptations for stochastic approximation algorithms of Robbins–Monro and Polyak–Ruppert type

In this article we present and analyse new multilevel adaptations of classical stochastic approximation algorithms for the computation of a zero of a function $$f:D \rightarrow {{\mathbb {R}}}^d$$ defined on a convex domain $$D\subset {{\mathbb {R}}}^d$$ , which is given as a parameterised family of expectations. The analysis of the error and the computational cost of our method is based on similar assumptions as used in Giles (Oper Res 56(3):607–617, 2008) for the computation of a single expectation. Additionally, we essentially only require that f satisfies a classical contraction property from stochastic approximation theory. Under these assumptions we establish error bounds in pth mean for our multilevel Robbins–Monro and Polyak–Ruppert schemes that decay in the computational time as fast as the classical error bounds for multilevel Monte Carlo approximations of single expectations known from Giles (Oper Res 56(3):607–617, 2008). Our approach is universal in the sense that having multilevel implementations for a particular application at hand it is straightforward to implement the corresponding stochastic approximation algorithm.

Unbalanced multi-drawing urn with random addition matrix

In this paper, we consider a two color multi-drawing urn model. At each discrete time step, we draw uniformly at random a sample of m balls (m≥1) and note their color, they will be returned to the urn together with a random number of balls depending on the sample’s composition. The replacement rule is a 2 × 2 matrix depending on bounded discrete positive random variables. Using a stochastic approximation algorithm and martingales methods, we investigate the asymptotic behavior of the urn after many draws.

Efficient Adaptive Speech Reception Threshold Measurements Using Stochastic Approximation Algorithms.

This study examines whether speech-in-noise tests that use adaptive procedures to assess a speech reception threshold in noise (SRT50n) can be optimized using stochastic approximation (SA) methods, especially in cochlear-implant (CI) users. A simulation model was developed that simulates intelligibility scores for words from sentences in noise for both CI users and normal-hearing (NH) listeners. The model was used in Monte Carlo simulations. Four different SA algorithms were optimized for use in both groups and compared with clinically used adaptive procedures. The simulation model proved to be valid, as its results agreed very well with existing experimental data. The four optimized SA algorithms all provided an efficient estimation of the SRT50n. They were equally accurate and produced smaller standard deviations (SDs) than the clinical procedures. In CI users, SRT50n estimates had a small bias and larger SDs than in NH listeners. At least 20 sentences per condition and an initial signal-to-noise ratio below the real SRT50n were required to ensure sufficient reliability. In CI users, bias and SD became unacceptably large for a maximum speech intelligibility score in quiet below 70%. In conclusion, SA algorithms with word scoring in adaptive speech-in-noise tests are applicable to various listeners, from CI users to NH listeners. In CI users, they lead to efficient estimation of the SRT50n as long as speech intelligibility in quiet is greater than 70%. SA procedures can be considered as a valid, more efficient, and alternative to clinical adaptive procedures currently used in CI users.

Optimal bandwidth selection for recursive Gumbel kernel density estimators

Abstract In this paper, we propose a data driven bandwidth selection of the recursive Gumbel kernel estimators of a probability density function based on a stochastic approximation algorithm. The choice of the bandwidth selection approaches is investigated by a second generation plug-in method. Convergence properties of the proposed recursive Gumbel kernel estimators are established. The uniform strong consistency of the proposed recursive Gumbel kernel estimators is derived. The new recursive Gumbel kernel estimators are compared to the non-recursive Gumbel kernel estimator and the performance of the two estimators are illustrated via simulations as well as a real application.

Inverse Material Characterization through Finite Element Simulation of Material Tests and Numerical Optimization

Abstract This paper presents a new method to characterize the material flow stress and failure models based on scratch test data. A group of parametric studies is first conducted to reveal the sensitivity of typical material responses in micro-scratch tests with respect to various materials constitutive and damage model parameters. The results were then used to guide the development of an inverse material characterization method by matching the finite element predictions with the corresponding test data. The simultaneous perturbation stochastic approximation (SPSA) algorithm was used for an efficient parameter search. The method was implemented to the identification of the Johnson-Cook (J-C) constitutive and damage model parameters for an aluminum alloy based on its micro-scratch tests conducted at room and elevated temperatures. The identification was validated with the morphological measurement of the scratches.

Stochastic approximation on noncompact measure spaces and application to measure-valued Pólya processes

Our main result is to prove almost-sure convergence of a stochastic-approximation algorithm defined on the space of measures on a non-compact space. Our motivation is to apply this result to measure-valued P\'olya processes (MVPPs, also known as infinitely-many P\'olya urns). Our main idea is to use Foster-Lyapunov type criteria in a novel way to generalize stochastic-approximation methods to measure-valued Markov processes with a non-compact underlying space, overcoming in a fairly general context one of the major difficulties of existing studies on this subject. From the MVPPs point of view, our result implies almost-sure convergence of a large class of MVPPs, this convergence was only obtained until now for specific examples, with only convergence in probability established for general classes. Furthermore, our approach allows us to extend the definition of MVPPs by adding weights to the different colors of the infinitely-many-color urn. We also exhibit a link between MVPPs and quasi-stationary distributions of Markovian processes, which allows us to treat, for the first time in the literature, the non-balanced case. Finally, we show how our result can be applied to designing stochastic-approximation algorithms for the approximation of quasi-stationary distributions of discrete- and continuous-time Markov processes on non-compact spaces.

On Stochastic Extremum Seeking via Adaptive Perturbation–Demodulation Loop

In this paper, we propose a stochastic approximation algorithm for optimization of functions based on an adaptive extremum seeking method. The essence of this method is to approximate the gradient direction by introduction of a probing sequence, that is added to approximations and subsequently demodulated using an adaptive gain. Assuming that the probing and the demodulation signals are martingale difference sequences with adaptive diminishing gains, it is proved that the approximations converge almost surely to the optimizing value, under mild constraints on the measurement disturbance, and without assuming a priori boundedness of the approximation sequence. The measurement disturbance can contain a stochastic component, as well as a mean-square bounded deterministic component. The stochastic component can be nonstationary colored noise or a state-dependent random sequence.

String-averaging incremental stochastic subgradient algorithms

ABSTRACTWe present a method to solve constrained convex stochastic optimization problems when the objective is a finite sum of convex functions . Our method is based on Incremental Stochastic Subgradient Algorithms and String-Averaging techniques, with an assumption that the subgradient directions are affected by random errors in each iteration. Our analysis allows the method to perform approximate projections onto the feasible set in each iteration. We provide convergence results for the case where a diminishing step-size rule is used. We test our method in a large set of random instances of a stochastic convex programming problem and we compare its performance with the robust mirror descent stochastic approximation algorithm proposed in Nemirovski et al. (Robust stochastic approximation approach to stochastic programming, SIAM J Optim 19 (2009), pp. 15741609).

A self-equilibrium Friedman-like urn via stochastic approximation

In this study, we propose a class of generalized Friedman urns with random entries. The class has the property of self-equilibrium. We prove the almost-sure convergence to the equilibrium point of this entire class by the method of stochastic approximation. We develop a central limit theorem for the proportion of white balls through the convergence theorem of stochastic approximation algorithms. We then apply this class of schemes to model the effect of trading on the price in the stock market under the efficient market hypothesis.

The Holling Type II Population Model Subjected to Rapid Random Attacks of Predator

We present the analysis of a mathematical model of the dynamics of interacting predator and prey populations with the Holling type random trophic function under the assumption of random time interval passage between predator attacks on prey. We propose a stochastic approximation algorithm for quantitative analysis of the above model based on the probabilistic limit theorem. If the predators’ gains and the time intervals between predator attacks are sufficiently small, our proposed method allows us to derive an approximative average dynamical system for mathematical expectations of population dynamics and the stochastic Ito differential equation for the random deviations from the average motion. Assuming that the averaged dynamical system is the classic Holling type II population model with asymptotically stable limit cycle, we prove that the dynamics of stochastic model may be approximated with a two-dimensional Gaussian Markov process with unboundedly increasing variances.

Production Optimization of Polymer Flooding Using Improved Monte Carlo Gradient Approximation Algorithm with Constraints

Aiming at optimizing polymer flooding, we establish an optimal control model of polymer flooding, which has an objective function of the net present value (NPV) involving the effect of polymer injection. An improved Monte Carlo gradient approximation (MCGA) algorithm, based on the idea of the ensemble-based optimization (EnOpt) scheme to solve the problem of strongly fluctuating perturbation gradients, is proposed by introducing the covariance matrix of the control vectors to filter and smooth the searching direction. A synthetic heterogeneous reservoir model is built to test the performance of the algorithms including the improved MCGA, standard MCGA and finite difference stochastic approximation (FDSA) algorithm. For the results, the improved MCGA gets closer to the optimal NPV of FDSA than the standard algorithm, and shows the high efficiency of saving calculation time compared with the FDSA. The value of NPV increases more than 20% for the improved algorithm, and the optimal production rates, injection rates, polymer concentrations, polymer slug sizes are obtained simultaneously. This paper subsequently discusses the influence of different time step sizes and oil prices. It can be concluded that moderate step size and relatively low oil price are applicable. Finally, an actual block application of the improved MCGA shows a 11.3% increase of NPV and a 6.5% increase of field oil production total (FOPT), showing the feasibility in optimizing real reservoirs.

Nonlinear randomized urn models: a stochastic approximation viewpoint

This paper extends the link between stochastic approximation (SA) theory and randomized urn models developed in Laruelle, Pages (2013), and their applications to clinical trials introduced in Bai, HU (1999,2005) and Bai, Hu, Shen (2002). We no longer assume that the drawing rule is uniform among the balls of the urn (which contains d colors), but can be reinforced by a function f. This is a way to model risk aversion. Firstly, by considering that f is concave or convex and by reformulating the dynamics of the urn composition as an SA algorithm with remainder, we derive the a.s. convergence and the asymptotic normality (Central Limit Theorem, CLT) of the normalized procedure by calling upon the so-called ODE and SDE methods. An in-depth analysis of the case d=2 exhibits two different behaviors: A single equilibrium point when f is concave, and when f is convex, a transition phase from a single attracting equilibrium to a system with two attracting and one repulsive equilibrium points. The last setting is solved using results on non-convergence toward noisy and noiseless ``traps in order to deduce the a.s. convergence toward one of the attracting points. Secondly, the special case of a Polya urn (when the addition rule is the identity matrix) is analyzed, still using result from SA theory about ``traps''. Finally, these results are applied to a function with regular variation and to an optimal asset allocation in Finance.

Decision rule-based method for flexible multi-facility capacity expansion problem

ABSTRACTStrategic capacity planning for multiple-facility systems with flexible designs is an important topic in the area of capacity expansion problems with random demands. The difficulties of this problem lie in the multidimensional nature of its random variables and action space. For a single-facility problem, the decision rule method has been shown to be efficient in deriving desirable solutions, but for a Multiple-facility Capacity Expansion Problem (MCEP), it has not been well studied. This article designs a novel decision rule–based method for the solution of an MCEP with multiple options, discrete capacity, and a concave capacity expansion cost. An if–then decision rule is designed and the original multi-stage problem is thus transformed into a master problem and a multi-period sub-problem. As the sub-problem contains non-binding constraints, we combine a stochastic approximation algorithm with a branch-and-cut technique so that the sub-problem can be further decomposed across scenarios and be solved efficiently. The proposed decision rule–based method is also extended to solving the MCEP with fixed costs. Numerical studies in this article illustrate that the proposed method affords not only improved performance relative to an inflexible design taken as benchmark but also time savings relative to approximate dynamic programming analysis.

Nonparametric recursive method for kernel-type function estimators for spatial data

In the present paper we propose recursive general kernel-type estimators for spatial data defined by the stochastic approximation algorithm. We obtain the central limit theorem and strong pointwise convergence rate for the nonparametric recursive general kernel-type estimators under some mild conditions. Finally, we investigate the MISE of the proposed estimators and provide the optimal bandwidth.

Recursive identification of systems with binary-valued outputs and with ARMA noises

This paper considers the identification problem of the ARMA system followed by a binary sensor, in which the internal variables are corrupted by additive ARMA noises. Recursive estimates for the parameters of the linear system and for the threshold of the binary sensor are given by the stochastic approximation algorithms with expanding truncations (SAAWET). Under reasonable conditions, all the estimates are proved to be convergent to the true values with probability one. The almost surely convergence rates are also investigated. A simulation example is included to justify the theoretical results.