Stochastic Recursions Research Articles

Neural based RSPN multi-agent strategy for biped motion control

In this paper fhe problem of motion control of a biped is considered. We develop a new method based on multi-agent associated Neural AIGLS (On-line Augmented Integration of Gradient and Last Sguare method) – RSPN (Recursive Stochastic Petri Nets) strategy. This method deals with organization and coordination aspects in an intelligent modeling of human motion. We propose a cooperative multi-agent model. Based on this model, we develop a control kernel named IMCOK (Intelligent Motion COntrol Kernel) which consists of a controller, a coordinator and an executor of different cycles of the motion of the biped. When walking, IMCOK receives messages and sends offers. A Decision Making of Actions (DMA) is developed at the supervisor level. The articulator agents partially planify the motion of the associated non-articulator agents. The system is hybrid and distributed functionally. The learning of the biped is performed using an On-line Augmented Integration of Gradient and Last Sguare Neural Networks based algorithm. In the conflictual situations of sending or receiving messages by the managers of MABS we apply a new strategy: Recursive Stochastic Petri Nets (RSPN). This module is fundamental in the On-line information processing between agents. It allows particularly the Recursive strategy concept. Cognitive agents communicate with reactive (non-articulator) agents in order to generate the motion.

Continuous-time monotone stochastic recursions and duality

A duality is presented for continuous-time, real-valued, monotone, stochastic recursions driven by processes with stationary increments. A given recursion defines the time evolution of a content process (such as a dam or queue), and it is shown that the existence of the content process implies the existence of a corresponding dual risk process that satisfies a dual recursion. The one-point probabilities for the content process are then shown to be related to the one-point probabilities of the risk process. In particular, it is shown that the steady-state probabilities for the content process are equivalent to the first passage time probabilities for the risk process. A number of applications are presented that flesh out the general theory. Examples include regulated processes with one or two barriers, storage models with general release rate, and jump and diffusion processes.

Continuous-time monotone stochastic recursions and duality

A duality is presented for continuous-time, real-valued, monotone, stochastic recursions driven by processes with stationary increments. A given recursion defines the time evolution of a content process (such as a dam or queue), and it is shown that the existence of the content process implies the existence of a corresponding dual risk process that satisfies a dual recursion. The one-point probabilities for the content process are then shown to be related to the one-point probabilities of the risk process. In particular, it is shown that the steady-state probabilities for the content process are equivalent to the first passage time probabilities for the risk process. A number of applications are presented that flesh out the general theory. Examples include regulated processes with one or two barriers, storage models with general release rate, and jump and diffusion processes.

Adaptive importance sampling for performance evaluation and parameter optimization of communication systems

We present new adaptive importance sampling techniques based on stochastic Newton recursions. Their applicability to the performance evaluation of communication systems is studied. Besides bit-error rate (BER) estimation, the techniques are used for system parameter optimization. Two system models that are analytically tractable are employed to demonstrate the validity of the techniques. As an application to situations that are analytically intractable and numerically intensive, the influence of crosstalk in a wavelength-division multiplexing (WDM) crossconnect is assessed. In order to consider a realistic system model, optimal setting of thresholds in the detector is carried out while estimating error rate performances. Resulting BER estimates indicate that the tolerable crosstalk levels are significantly higher than predicted in the literature. This finding has a strong impact on the design of WDM networks. Power penalties induced by the addition of channels can also be accurately predicted in short run-times.

Efficient identification of non-Gaussian mixtures

Non-Gaussian mixtures are commonly used as parametric models for impulsive interference superimposed on a Gaussian background. In this study, the problem of efficient batch and recursive estimation of the parameters of such mixtures via minimization of the Kullback-Leibler distance is considered. The maximum-likelihood estimator (MLE) provides the starting point for the development of these estimators. First, it is shown that the MLE yields consistent estimates of the parameters, despite the existence of multiple roots in the Kullback-Leibler distance function. Since direct implementation of the MLE is difficult, an alternative estimator, designed with the objective of maximizing the likelihood function, is proposed. A simulation study of the proposed estimator reveals that it performs very well in terms of attaining the Cramer-Rao lower bound. Stochastic approximation is also considered in this study. A modification to the standard recursion is presented, which greatly facilitates its implementation. Three initiating estimators are developed for these recursions: an iterative likelihood-based scheme, a batch estimator that uses a histogram of the data, and an iterative scheme that integrates the concept of complete data into the method-of-moments estimator. Upon initiating the modified stochastic approximation recursion with these estimators, it is seen that excellent estimates of the parameters can be obtained.

Stochastic approximation algorithms: Overview and recent trends

Stochastic approximation is a common paradigm for many stochastic recursions arising both as algorithms and as models of some stochastic dynamic phenomena. This article gives an overview of the known results about their asymptotic behaviour, highlights recent developments such as distributed and multiscale algorithms, and describes existing and potential applications, and other related issues.

Auxiliary-vector filters and adaptive steering for DS/CDMA single-user detection

Single-user detectors for direct-sequence code-division multiple-access (DS/CDMA) systems are characterized by their architecture, the design optimality criterion, the optimization computational complexity, and ultimately by their average bit error rate (BER) performance. With a fixed tap-weight architecture that implies zero optimization cost, the conventional signature-matched correlator positions itself as the low-cost low-performance solution. This paper submits a proposal that maintains the same tap-weight architecture and introduces the concept of maximum cross-correlation auxiliary-vector filtering associated with minimum BER adaptive steering. Calculation of the auxiliary vector taps involves standard sample average estimation of the data autocorrelation matrix, and no matrix inversion operation is required. The resulting scalar parametrized filter is optimized adaptively in the minimum BER sense by a fast stochastic approximation recursion on /spl Rscr/. Simplicity and BER comparisons place the proposed receiver competitively on the cost versus performance curve. Numerical results under realistic small sample support scenarios support the theoretical developments and indicate that performance-wise the receiver compares favorably not only to the conventional matched-filter correlator, but also to the minimum-output-energy (MOE) detector the LMS-implemented minimum-mean-square-error (MMSE) filter, and the multiuser decorrelating receiver that assumes perfectly known interfering user population.

Why do clocks move clockwise?

A mathematical model for collective learning by several autonomous agents is described. This is a stochastic recursion that arises in several disparate fields like statistics, engineering and economics

Stationary solutions of stochastic recursions describing discrete event systems

We consider recursions of the form x n + 1 = ϕ n [ x n ], where { ϕ n , n ≥ 0} is a stationary ergodic sequence of maps from a Polish space ( E, E ) into itself, and { x n , n ≥ 0} are random variables taking values in ( E, E ). Questions of existence and uniqueness of stationary solutions are of considerable interest in discrete event system applications. Currently available techniques use simplifying assumptions on the statistics of { ϕ n } n (such as Markov assumptions), or on the nature of these maps (such as monotonicity). We introduce a new technique, without such simplifying assumptions, by weakening the solution concept: instead of a pathwise solution, we construct a probability measure on another sample space and families of random variables on this space whose law gives a stationary solution. The existence of a stationary solution is then translated into tightness of a sequence of probability distributions. Uniqueness questions can be addressed using techniques familiar from the ergodic theory of positive Markov operators

Applications of Borovkov's Renovation Theory to Non-Stationary Stochastic Recursive Sequences and Their Control

We investigate in this paper the stability of non-stationary stochastic processes, arising typically in applications of control. The setting is known as stochastic recursive sequences, which allows us to construct on one probability space stochastic processes that correspond to different initial states and even different control policies. It does not require any Markovian assumptions. A natural criterion for stability for such processes is that the influence of the initial state disappears after some finite time; in other words, starting from different initial states, the process will couple after some finite time to the same limiting (not necessarily stationary nor ergodic) stochastic process. We investigate this as well as other types of coupling, and present conditions for them to occur uniformly in some class of control policies. We then use the coupling results to establish new theoretical aspects in the theory of non-Markovian control.

Applications of Borovkov's Renovation Theory to Non-Stationary Stochastic Recursive Sequences and Their Control

We investigate in this paper the stability of non-stationary stochastic processes, arising typically in applications of control. The setting is known as stochastic recursive sequences, which allows us to construct on one probability space stochastic processes that correspond to different initial states and even different control policies. It does not require any Markovian assumptions. A natural criterion for stability for such processes is that the influence of the initial state disappears after some finite time; in other words, starting from different initial states, the process will couple after some finite time to the same limiting (not necessarily stationary nor ergodic) stochastic process. We investigate this as well as other types of coupling, and present conditions for them to occur uniformly in some class of control policies. We then use the coupling results to establish new theoretical aspects in the theory of non-Markovian control.

A Dynamical System Approach to Stochastic Approximations

It is known that some problems of almost sure convergence for stochastic approximation processes can be analyzed via an ordinary differential equation (ODE) obtained by suitable averaging. The goal of this paper is to show that the asymptotic behavior of such a process can be related to the asymptotic behavior of the ODE without any particular assumption concerning the dynamics of this ODE. The main results are as follows: a) The limit sets of trajectory solutions to the stochastic approximation recursion are, under classical assumptions, almost surely nonempty compact connected sets invariant under the flow of the ODE and contained in its set of chain-recurrence. b) If the gain parameter goes to zero at a suitable rate depending on the expansion rate of the ODE, any trajectory solution to the recursion is almost surely asymptotic to a forward trajectory solution to the ODE.

Monotone Stochastic Recursions and their Duals

A duality is presented for real-valued stochastic sequences [Vn] defined by a general recursion of the form Vn+1 = f(Vn, Un), with [Un] a stationary driving sequence and f nonnegative, continuous, and monotone in its first variable. The duality is obtained by defining a dual function g of f, which if used recursively on the time reversal of [Un] defines a dual risk process. As a consequence, we prove that steady-state probabilities for Vn can always be expressed as transient probabilities of the dual risk process. The construction is related to duality of stochastically monotone Markov processes as studied by Siegmund (1976, The equivalence of absorbing and reflecting barrier problems for stochastically monotone Markov processes, Annals of Probability 4: 914–924). Our method of proof involves an elementary sample-path analysis. A variety of examples are given, including random walks with stationary increments and two reflecting barriers, reservoir models, autoregressive processes, and branching processes. Finally, general stability issues of the content process are dealt with by expressing them in terms of the dual risk process.

Limit laws for a stochastic process and random recursion arising in probabilistic modelling

We study certain stochastic processes arising in probabilistic modelling. We discuss the limit behavior of these processes and estimate the rate of convergence to the limit.

Limit laws for a stochastic process and random recursion arising in probabilistic modelling

We study certain stochastic processes arising in probabilistic modelling. We discuss the limit behavior of these processes and estimate the rate of convergence to the limit.

Stability of Recursive Stochastic Tracking Algorithms

First, the paper gives a stability study for the random linear equation $X_{n+1}=(I-A_{n})x_n$. It is shown that for a quite general class of random matrices $\{A_n\}$ of interest, the stability of such a vector equation can be guaranteed by that of a corresponding scalar linear equation, for which various results are given without requiring stationary or mixing conditions. Then, these results are applied to the main topic of the paper, i.e., to the estimation of time varying parameters in linear stochastic systems, giving a unified stability condition for various tracking algorithms including the standard Kalman filter, least mean squares, and least squares with forgetting factor.

Adaptive Lexicographic Optimization in Multi-Class M/GI/1 Queues

We consider a multi-class M/GI/1 system, in which an average response time objective is associated with each class. The performance of each class is measured by the ratio of the average response time over the corresponding value of the objective. To achieve fairness in service allocation it is required to find a policy that lexicographically minimizes the vector of performance ratios arranged in nonincreasing order. We provide such a policy that is adaptive, uses only knowledge of arrival and departure instants, and is thus easy to implement. We also consider a variant of this policy which adapts faster to changes in the statistical parameters of the model. Both policies are analyzed via associated stochastic recursions using techniques of stochastic approximation.

Recursive Stochastic Algorithms for Global Optimization in $\mathbb{R}^d $

An algorithm of the form $X_{k + 1} = X_k - a_k (\nabla U(X_k ) + \xi _k ) + b_k W_k $, where $U( \cdot )$ is a smooth function on $\mathbb{R}^d $, $\{ \xi _k \} $ is a sequence of $\mathbb{R}^d $-valued random variables, $\{ W_k \} $ is a sequence of independent standard d-dimensional Gaussian random variables, $a_k = {A / k}$ and $b_k = {{\sqrt B } / {\sqrt {k\log \log k} }}$ for k large, is considered. An algorithm of this type arises by adding slowly decreasing white Gaussian noise to a stochastic gradient algorithm. It is shown, under suitable conditions on $U( \cdot )$, $\{ \xi _k \} $, A, and B, that $X_k $ converges in probability to the set of global minima of $U( \cdot )$. No prior information is assumed as to what bounded region contains a global minimum. The analysis is based on the asymptotic behavior of the related diffusion process $dY(t) = - \nabla U(Y(t))dt + c(t)dW(t)$, where $W( \cdot )$ is a standard d-dimensional Wiener process and $c(t) = {{\sqrt C } / {\sqrt {\log t} }}$ for t large.

Performance gradient estimation for the very large finite Markov chains

Using an embedded Markov chain, the steady-state performance gradient estimation for very large Markov chains is decomposed into two smaller problems, one solved by the stochastic recursive (SR) method and the other by the likelihood ratio (LR) method. The combination, named SR-LR, can have several magnitudes of lower space requirement than the SR and several magnitudes of lower time consumptions than the LR. Both the LR and the SR are the extreme cases of the combined algorithm. The authors demonstrate the SR-LR methods on Markov chains with hundreds of millions of states, which are created using queuing networks. >

The complexity of a simple stochastic OR-tree model in which “directional search” is bad

A simple stochastic model of OR-trees is investigated in which the directional search algorithm has a very high average run time. The optimal algorithms just make the opposite of directional search: they spread their questions over the tree as balanced as possible. This is the first time that in a nontrivial stochastic recursion tree model the optimality of a nondirectional algorithm with respect to the average run time has been proved.