Stochastic Recursions Research Articles

Perfect sampling of stochastic matching models with reneging

Abstract In this paper, we introduce a slight variation of the dominated-coupling-from-the-past (DCFTP) algorithm of Kendall, for bounded Markov chains. It is based on the control of a (typically non-monotonic) stochastic recursion by another (typically monotonic) one. We show that this algorithm is particularly suitable for stochastic matching models with bounded patience, a class of models for which the steady-state distribution of the system is in general unknown in closed form. We first show that the Markov chain of this model can easily be controlled by an infinite-server queue. We then investigate the particular case where patience times are deterministic, and this control argument may fail. In that case we resort to an ad-hoc technique that can also be seen as a control (this time, by the arrival sequence). We then compare this algorithm to the primitive coupling-from-the-past (CFTP) algorithm and to control by an infinite-server queue, and show how our perfect simulation results can be used to estimate and compare, for instance, the loss probabilities of various systems in equilibrium.

Impact of COVID-19 on Fractal Capital Market Recursion

We present the optimization of the current prices function of the Mexican Stock Exchange index and we will focus our attention on aspects of maximizing stock market margins with limits in the bookkeeping operation. We ourselves will model the possible stochastic recursion scenarios in "n" as a fractal annihilation factor within this stock market. Finally we will obtain the expected delta of the price range and its Hamiltonian to minimize the operational risk of the capital market with the presence of COVID19 in Mexico.

Dynamic Programming and a Verification Theorem for the Recursive Stochastic Control Problem of Jump-Diffusion Models With Random Coefficients

In this article, we consider the stochastic optimal control problem for (forward) stochastic differential equations (SDEs) with jump diffusions and random coefficients under a recursive-type objective functional captured by a backward SDE (BSDE). Due to the jump-diffusion process with random coefficients in both the constraint (forward SDE) and the recursive BSDE objective functional, the associated Hamilton–Jacobi–Bellman equation (HJBE) is an integro-type second-order nonlinear stochastic PDE driven by both Brownian motion and (compensated) Poisson process, which we call the integro-type stochastic HJBE (ISHJBE) with jump diffusions. We first prove the dynamic programming principle for the value function using the backward semigroup associated with the recursive objective functional and the precise estimates of BSDEs, by which the continuity of the value function is also shown. Then we establish a verification theorem, which provides a sufficient condition of optimality and characterizes the value function using the (stochastic) solution of the ISHJBE with jump diffusions. Under suitable assumptions, we show the existence and uniqueness of the weak solution to the ISHJBE via the Sobolev space technique. Finally, we apply the verification theorem to the general indefinite linear-quadratic problem and the utility maximization problem; for both problems, explicit optimal solutions are characterized by solving the corresponding ISHJBE with jump diffusions.

Stochastic recursions on directed random graphs

For a vertex-weighted directed graph G(Vn,En;An) on the vertices Vn={1,2,…,n}, we study the distribution of a Markov chain {R(k):k≥0} on Rn such that the ith component of R(k), denoted Ri(k), corresponds to the value of the process on vertex i at time k. We focus on processes {R(k):k≥0} where the value of Ri(k+1) depends only on the values {Rj(k):j→i} of its inbound neighbors, and possibly on vertex attributes. We then show that, provided G(Vn,En;An) converges in the local weak sense to a marked Galton–Watson process, the dynamics of the process for a uniformly chosen vertex in Vn can be coupled, for any fixed k, to a process {R0̸(r):0≤r≤k} constructed on the limiting marked Galton–Watson tree. Moreover, we derive sufficient conditions under which R0̸(k) converges, as k→∞, to a random variable R∗ that can be characterized in terms of the attracting endogenous solution to a branching distributional fixed-point equation. Our framework can also be applied to processes {R(k):k≥0} whose only source of randomness comes from the realization of the graph G(Vn,En;An).

A Modified Method of Successive Approximations for Stochastic Recursive Optimal Control Problems

Based on the stochastic maximum principle for the partially coupled forward-backward stochastic control system (FBSCS for short), a modified method of successive approximations (MSA for short) is established for stochastic recursive optimal control problems. The second-order adjoint processes are introduced in the augmented Hamiltonian minimization step since the control domain is not necessarily convex. Thanks to the theory of bounded mean oscillation martingales (BMO martingales for short), we give a delicate proof of the error estimate and then prove the convergence of the modified MSA algorithm. In a special case, we obtain a logarithmic convergence rate. When the control domain is convex and compact, a sufficient condition which makes the control returned from the MSA algorithm be a near-optimal control is given for a class of linear FBSCSs.

Queueing and risk models with dependencies

This paper analyzes various stochastic recursions that arise in queueing and insurance risk models with a ‘semi-linear’ dependence structure. For example, an interarrival time depends on the workload, or the capital, immediately after the previous arrival; or the service time of a customer depends on her waiting time. In each case, we derive and solve a fixed-point equation for the Laplace–Stieltjes transform of a key performance measure of the model, like waiting time or ruin time.

Recursive Stochastic Parameter Identification Based on Data Assimilation and its Experimental Verification in Planar 3-link Manipulator

Recursive Stochastic Parameter Identification Based on Data Assimilation and its Experimental Verification in Planar 3-link Manipulator

Histopathological image-based breast cancer detection employing 3D-convolutional neural network feature extraction and Stochastic Diffusion Kernel Recursive Neural Networks classification

ABSTRACT Among the various cancer categories, breast cancer diagnoses are commonly found in many women, and automatic classification of breast cancer images is a crucial task using computer-aided analysis. This paper proposed a method for the detection of histopathological image-based breast cancer and the main motive of the technique is to employ feature extraction and classification based on DL techniques. The input image is filtered using edge smoothening for noise removal and image resizing on the pre-processing method and used 3D-Convolutional Neural network (3D_CNN) for extracting the features of the image process. Extracted deep features have been given as input for the classification process using Stochastic Diffusion Kernel Recursive Neural Networks (SDKRNN). The suggested model combines the sturdiness of convolutional and kernel blocks and demonstrates performance stability when compared to existing techniques. The experimental results are assessed using a variety of performance metrics and compared to current breast cancer detection methodologies. The result demonstrated the experimental evaluation analyzed for various datasets in terms of accuracy, precision, F-1 score, specificity, and AUC of 98%, 93.8%, 89%, 80%, and 70% and it determined the outperforms in the detection of breast cancer.

A multiplicative version of the Lindley recursion

This paper presents an analysis of the stochastic recursion W_{i+1} = [V_iW_i+Y_i]^+ that can be interpreted as an autoregressive process of order 1, reflected at 0. We start our exposition by a discussion of the model’s stability condition. Writing Y_i=B_i-A_i, for independent sequences of nonnegative i.i.d. random variables {A_i}_{iin {mathbb N}_0} and {B_i}_{iin {mathbb N}_0}, and assuming {V_i}_{iin {mathbb N}_0} is an i.i.d. sequence as well (independent of {A_i}_{iin {mathbb N}_0} and {B_i}_{iin {mathbb N}_0}), we then consider three special cases (i) V_i equals a positive value a with certain probability pin (0,1) and is negative otherwise, and both A_i and B_i have a rational LST, (ii) V_i attains negative values only and B_i has a rational LST, (iii) V_i is uniformly distributed on [0, 1], and A_i is exponentially distributed. In all three cases, we derive transient and stationary results, where the transient results are in terms of the transform at a geometrically distributed epoch.

Stochastic Recursive Inclusions in Two Timescales with Nonadditive Iterate-Dependent Markov Noise

In this paper, we study the asymptotic behavior of a stochastic approximation scheme on two timescales with set-valued drift functions and in the presence of nonadditive iterate-dependent Markov noise. We show that the recursion on each timescale tracks the flow of a differential inclusion obtained by averaging the set-valued drift function in the recursion with respect to a set of measures accounting for both averaging with respect to the stationary distributions of the Markov noise terms and the interdependence between the two recursions on different timescales. The framework studied in this paper builds on a recent work by Ramaswamy and Bhatnagar, by allowing for the presence of nonadditive iterate-dependent Markov noise. As an application, we consider the problem of computing the optimum in a constrained convex optimization problem, where the objective function and the constraints are averaged with respect to the stationary distribution of an underlying Markov chain. Further, the proposed scheme neither requires the differentiability of the objective function nor the knowledge of the averaging measure.

Maximum Principle for Stochastic Recursive Optimal Control Problem under Model Uncertainty

In this paper, we consider a stochastic recursive optimal control problem under model uncertainty. In this framework, the cost function is described by solutions of a family of backward stochastic differential equations with uncertainty parameter $\theta$, which is used to represent different market conditions. With the help of linearization techniques and weak convergence methods, we derive the corresponding stochastic maximum principle. Moreover, a linear quadratic robust control problem is also studied.

Some Limit Properties of Markov Chains Induced by Recursive Stochastic Algorithms

Recursive stochastic algorithms have gained significant attention in the recent past due to data-driven applications. Examples include stochastic gradient descent for solving large-scale optimizati...

Stochastic recursions: Between Kesten’s and Grincevičius–Grey’s assumptions

We study the stochastic recursion Xn=Ψn(Xn−1), where (Ψn)n≥1 is a sequence of i.i.d. random Lipschitz mappings close to the random affine transformation x↦Ax+B. We describe the tail behaviour of the stationary solution X under the assumption that there exists α>0 such that E|A|α=1 and the tail of B is regularly varying with index −α<0. We also find the second order asymptotics of the tail of X when Ψ(x)=Ax+B.

Coupling in the queue with impatience: case of several servers

We present the explicit construction of a stable queue with several servers and impatient customers, under stationary ergodic assumptions. Using a stochastic comparison of the (multivariate) workload sequence with two monotonic stochastic recursions, we propose a sufficient condition of existence of a unique stationary state of the system using Renovation theory. Whenever this condition is relaxed we use extension techniques to prove the existence of a stationary state in some cases.

Power Conditioning With Intelligent Control Using a Novel Recursive Stochastic Optimization

This paper presents a novel smart-grid power-conditioning system with intelligent control for solving a critical issue affecting the performance and life of three-phase asynchronous machines that are highly sensitive to voltage and phase drifts power disturbances of the electric grid. The present work of controlling the power supply of an electric-grid is based on an original power conditioning control algorithm; named recursive stochastic optimization (RSO) algorithm. An asynchronous machine steady-state model is derived based on symmetrical components analysis, permitting the evaluation of the effect of both voltage and phase unbalance using voltage and current unbalance factors. The three-phase power supply is conveyed through a power conditioning and intelligent control system, driven by a digital signal processor (DSP) controller, which measures the input currents, estimates the unbalance factors, and searches for voltage levels that minimize the electric-grid unbalance using the RSO, in order to act on a feedback power-electronics stage. The RSO algorithm shows excellent results along with very fast response time of the order of 20 ms allowing real-time control. Results obtained are validated with a Simulink dynamic model. The intelligent control system architecture based on TMS320C6748 DSP board with signal acquisition and control microcontroller is proposed.

Limit theorems for affine Markov walks conditioned to stay positive

Consider the real Markov walk $S_n = X_1+ \dots+ X_n$ with increments $\left(X_n\right)_{n\geqslant 1}$ defined by a stochastic recursion starting at $X_0=x$. For a starting point $y>0$ denote by $\tau_y$ the exit time of the process $\left( y+S_n \right)_{n\geqslant 1}$ from the positive part of the real line. We investigate the asymptotic behaviour of the probability of the event $\tau_y \geqslant n$ and of the conditional law of $y+S_n$ given $\tau_y \geqslant n$ as $n \to +\infty$.

On Sampling Rates in Simulation-Based Recursions

We consider the context of “simulation-based recursions,” that is, recursions that involve quantities needing to be estimated using a stochastic simulation. Examples include stochastic adaptations of fixed-point and gradient descent recursions obtained by replacing function and derivative values appearing within the recursion by their Monte Carlo counterparts. The primary motivating settings are simulation optimization and stochastic root finding problems, where the low point and the zero of a function are sought, respectively, with only Monte Carlo estimates of the functions appearing within the problem. We ask how much Monte Carlo sampling needs to be performed within simulation-based recursions in order that the resulting iterates remain consistent and, more importantly, efficient, where “efficient” implies convergence at the fastest possible rate. Answering these questions involves trading off two types of error inherent in the iterates: the deterministic error due to recursion and the “stochastic” error due to sampling. As we demonstrate through a characterization of the relationship between sample sizing and convergence rates, efficiency and consistency are intimately coupled with the speed of the underlying recursion, with faster recursions yielding a wider regime of “optimal” sampling rates. The implications of our results for practical implementation are immediate since they provide specific guidance on optimal simulation expenditure within a variety of stochastic recursions.

Stochastic stability of monotone economies in regenerative environments

We introduce and analyze a new class of monotone stochastic recursions in a regenerative environment which is essentially broader than that of Markov chains. We prove stability theorems and apply our results to three canonical models in recursive economics, generalizing some known stability results to the cases when driving sequences are not independent and identically distributed.

A Generalization of the Borkar-Meyn Theorem for Stochastic Recursive Inclusions

In this paper, the stability theorem of Borkar and Meyn is extended to include the case when the mean field is a set-valued map. Two different sets of sufficient conditions are presented that guarantee the “stability and convergence” of stochastic recursive inclusions. Our work builds on the works of Benaïm, Hofbauer and Sorin as well as Borkar and Meyn. As a corollary to one of the main theorems, a natural generalization of the Borkar and Meyn theorem follows. In addition, the original theorem of Borkar and Meyn is shown to hold under slightly relaxed assumptions. As an application to one of the main theorems, we discuss a solution to the “approximate drift problem.” Finally, we analyze the stochastic gradient algorithm with “constant-error gradient estimators” as yet another application of our main result.

Continuous modal parameter identification of cable-stayed bridges based on a novel improved ensemble empirical mode decomposition

The objective of this paper was to perform an effective and meticulous continuous modal parameter identification. Since the data obtained from the cable-stayed bridge was non-linear and time varying, and also there exists a phenomenon of mode mixing in the current decomposition techniques, such as empirical mode decomposition (EMD), which further complicates the extraction of accurate structural information, therefore, a novel improved Ensemble EMD method was proposed. This method can effectively deal with the non-linear and time varying structural behaviour and eliminate the phenomenon of mode mixing effectively, specially for cable-stayed bridges, because in this method the added white noise was selected by a pre-defined process and also the intrinsic mode function (IMF) selection was made self-adaptively, then finally Pareto technique was adopted to reconstruct the IMF. After the signal decomposition and reconstruction, Recursive Stochastic Subspace Identification was employed to carry out the continuous modal parameter identification. Sutong Yangtze Bridge, a long-span cable-stayed bridge, with main span of 1088 m was taken as a case study and the proposed method was applied. The result showed that the proposed method was effective in attaining its goals and can endows better results in real life bridge health monitoring.