Markovian Noise Research Articles

The right time to sell a stock whose price is driven by Markovian noise

We consider the problem of finding the optimal time to sell a stock, subject to a fixed sales cost and an exponential discounting rate ρ. We assume that the price of the stock fluctuates according to the equation dYt=Yt(μ dt+σξ(t) dt), where (ξ(t)) is an alternating Markov renewal process with values in {±1}, with an exponential renewal time. We determine the critical value of ρ under which the value function is finite. We examine the validity of the “principle of smooth fit” and use this to give a complete and essentially explicit solution to the problem, which exhibits a surprisingly rich structure. The corresponding result when the stock price evolves according to the Black and Scholes model is obtained as a limit case.

Estimating unobservable signal by Markovian noise induction

A sub threshold signal is transmitted through a channel and may be detected when some noise - with known structure and proportional to some level - is added to the data. There is an optimal noise level, called of stochastic resonance, that corresponds to the minimum variance of the estimators in the problem of recovering unobservable signals. For several noise structures it has been shown the evidence of stochastic resonance effect. Here we study the case when the noise is a Markovian process. We propose consistent estimators of the sub threshold signal and we solve further a problem of hypotheses testing. We also discuss evidence of stochastic resonance for both estimation and hypotheses testing problems via examples.

Drift by dichotomous Markov noise.

We derive explicit results for the asymptotic probability density and drift velocity in systems driven by dichotomous Markov noise, including the situation in which the asymptotic dynamics crosses unstable fixed points. The results are illustrated on the problem of the rocking ratchet.

Mechanism of hypersensitive transport in tilted sharp ratchets.

The noise-flatness-induced hypersensitive transport of overdamped Brownian particles in a tilted ratchet system driven by multiplicative nonequilibrium three-level Markovian noise and additive white noise is considered. At low temperatures, the enhancement of current is very sensitive to the applied small static tilting force. It is established that the enhancement of mobility depends nonmonotonically on the parameters (flatness, correlation time) of multiplicative noise. The optimal values of noise parameters maximizing the mobility are found.

Two-dimensional spectroscopy for a two-dimensional rotator coupled to a Gaussian–Markovian noise bath

The dynamics of a system in the condensed phase are more clearly characterized by multitime correlation functions of physical observables than by two-time ones. We investigate a two-dimensional motion of a rigid rotator coupled to a Gaussian–Markovian harmonic oscillator bath to probe this issue. The analytical expression of a four-time correlation function of a dipole that is the observable of two-dimensional microwave or far-infrared spectroscopy is obtained from a generating functional approach. The spectra in the absence of damping are discrete and reveal transitions between eigenstates of the angular momentum quantized due to the cyclic boundary condition. For a weakly damped case, the two-dimensional spectrum predicts three echolike peaks corresponding to transition processes between the rotational energy levels, which cannot be observed in one-dimensional (linear-absorption) spectroscopy related to the two-time correlation function of the dipole [J. Phys. Soc. Jpn. 71, 2414 (2002)]. The two-dimensional spectra are more sensitive to the noise effects than the one-dimensional spectra. It is because the effects of the initial thermal distribution determine the profile of the continuous line shape in one-dimensional spectroscopy, while such thermal effects are canceled through the higher-order optical transition process in two-dimensional spectroscopy. If the rotator system is strongly coupled to the colored noise bath, the system exhibits one overdamped and other oscillatory motions. We observe peaks arising from interaction between these two modes in the two-dimensional spectra, which are difficult to distinguish in one-dimensional spectra.

On the uniform ergodicity of Markov processes of order 2

We study the uniform ergodicity of Markov processes (Zn, n ≥ 1) of order 2 with a general state space (Z, 𝒵). Markov processes of order higher than 1 were defined in the literature long ago, but scarcely treated in detail. We take as the basis for our considerations the natural transition probability Q of such a process. A Markov process of order 2 is transformed into one of order 1 by combining two consecutive variables Z2n–1 and Z2n into one variable Yn with values in the Cartesian product space (Z × Z, 𝒵 ⊗ 𝒵). Thus, a Markov process (Yn, n ≥ 1) of order 1 with transition probability R is generated. Uniform ergodicity for the process (Zn, n ≥ 1) is defined in terms of the same property for (Yn, n ≥ 1). We give some conditions on the transition probability Q which transfer to R and thus ensure the uniform ergodicity of (Zn, n ≥ 1). We apply the general results to study the uniform ergodicity of Markov processes of order 2 which arise in some nonlinear time series models and as sequences of smoothed values in sequential smoothing procedures of Markovian observations. As for the time series models, Markovian noise sequences are covered.

On the uniform ergodicity of Markov processes of order 2

We study the uniform ergodicity of Markov processes (Z n , n ≥ 1) of order 2 with a general state space (Z, 𝒵). Markov processes of order higher than 1 were defined in the literature long ago, but scarcely treated in detail. We take as the basis for our considerations the natural transition probability Q of such a process. A Markov process of order 2 is transformed into one of order 1 by combining two consecutive variables Z 2n–1 and Z 2n into one variable Y n with values in the Cartesian product space (Z × Z, 𝒵 ⊗ 𝒵). Thus, a Markov process (Y n , n ≥ 1) of order 1 with transition probability R is generated. Uniform ergodicity for the process (Z n , n ≥ 1) is defined in terms of the same property for (Y n , n ≥ 1). We give some conditions on the transition probability Q which transfer to R and thus ensure the uniform ergodicity of (Z n , n ≥ 1). We apply the general results to study the uniform ergodicity of Markov processes of order 2 which arise in some nonlinear time series models and as sequences of smoothed values in sequential smoothing procedures of Markovian observations. As for the time series models, Markovian noise sequences are covered.

Linear stochastic approximation driven by slowly varying Markov chains

We study a linear stochastic approximation algorithm that arises in the context of reinforcement learning. The algorithm employs a decreasing step-size, and is driven by Markov noise with time-varying statistics. We show that under suitable conditions, the algorithm can track the changes in the statistics of the Markov noise, as long as these changes are slower than the rate at which the step-size of the algorithm goes to zero.

Three and four current reversals versus temperature in correlation ratchets with a simple sawtooth potential.

Transport of Brownian particles in a simple sawtooth potential subjected to both unbiased thermal and nonequilibrium symmetric three-level Markovian noise is considered. The effects of three and four current reversals as a function of temperature are established in such correlation ratchets. The parameter space coordinates of the fixed points associated with these current reversals and the necessary and sufficient conditions for the existence of the current reversals are found.

Moment evolution and level-crossing statistics in dichotomous and multilevel flows with time-dependent control parameters.

We study the dynamics of the first two moments and of threshold crossings by the stochastic trajectory in dichotomous diffusion x=xi(t), where xi(t) is a dichotomous Markov process. The transition rate of the latter is regarded as a control parameter and allowed to have specified time variations. The stabilizing or destabilizing effect of this variation is demonstrated, and qualitative changes in the statistical properties of the system are shown to occur. The analysis is then extended to linear dichotomous flow, and to a generalization of dichotomous diffusion in which x is driven by a multilevel Markov noise.

Trichotomous-noise-induced catastrophic shifts in symbiotic ecosystems.

An N-species Lotka-Volterra stochastic model of a symbiotic ecological system with the Verhulst self-regulation mechanism is considered. The effect of fluctuating environment on the carrying capacity of a population is modeled as the colored three-level Markovian (trichotomous) noise. In the framework of the mean-field theory an explicit self-consistency equation for stationary states is presented. Stability and instability conditions and colored-noise-induced discontinuous transitions (catastrophic shifts) in the model are investigated. In some cases the mean field exhibits hysteresis as a function of the noise parameters. It is shown that the occurrence of catastrophic shifts can be controlled by noise parameters, such as correlation time, amplitude, and flatness. The dependence of the critical coupling strengths on the noise parameters is found and illustrated by phase diagrams. Implications of the results on some modifications of the model are discussed.

Solvability of the master equation for dichotomous flow.

We consider the one-dimensional stochastic flow x=f(x)+g(x)xi(t), where xi(t) is a dichotomous Markov noise, and use a simple procedure to identify the conditions under which the integro-differential equation satisfied by the total probability density P(x,t) of the driven variable can be reduced to a differential equation of finite order. This generalizes the enumeration of the "solvable" cases.

Correlation ratchets: four current reversals and disjunct "windows".

Multinoise correlation ratchets with a simple sawtooth potential are considered. It is proved that in the case of symmetric nonequilibrium three-level Markovian noise the direction and value of the induced current can be controlled by thermal noise. Moreover, it is established that four current reversals (CRs) occur and that for the CRs there exist characteristic disjunct "windows" in temperature and switching rate as control parameters. The necessary and sufficient conditions for the existence of the above effects are given and can be used in particle separation techniques.

Characteristic functional approach to multiplicative fractal noise with application to environmental fluctuations in nonlinear chemical systems far from equilibrium:: A temporal analog of Anderson localization

We compute the characteristic functional of a nonlinear transformation of a set of random variables. The equation is applied to the study of fractal stochastic processes with multiplicative noise. For illustration we investigate two types of chemical systems subjected to environmental fluctuations. We analyze the interaction between the intrinsic chemical fluctuations and environmental noise and evaluate the stochastic properties of the system in the presence of multiplicative noise. For an autocatalytic reaction subjected to external noise and with Gaussian and generally non-Markovian multiplicative external noise the averaged joint probability density of concentrations displays long tails of the negative power law type, which are independent of the detailed properties of the noise. We also study the influence of multiplicative noise on a nonlinear chemical system with a stable limit cycle. In the presence of multiplicative noise the limit cycle collapses and the average trajectory in the composition space becomes a spiral. This effect is a temporal analog of Anderson localization. We evaluate the damping factors and localization times for different types of noise ranging from noise with infinite memory (static disorder) and noise with long memory (dynamic disorder with fractal noise) to Markovian noise (dynamic disorder with short memory).

Pattern-dependent noise prediction in signal-dependent noise

Maximum and near-maximum likelihood sequence detectors in signal-dependent noise are discussed. It is shown that the linear prediction viewpoint allows a very simple derivation of the branch metric expression that has previously been shown as optimum for signal-dependent Markov noise. The resulting detector architecture is viewed as a noise predictive maximum likelihood detector that operates on an expanded trellis and relies on computation of branch-specific, pattern-dependent noise predictor taps and predictor error variances. Comparison is made on the performance of various low-complexity structures using the positional-jitter/width-variation model for transition noise. It is shown that when medium noise dominates, a reasonably low complexity detector that incorporates pattern-dependent noise prediction achieves a significant signal-to-noise ratio gain relative to the extended class 4 partial response maximum likelihood detector. Soft-output detectors as well as the use of soft decision feedback are discussed in the context of signal-dependent noise.

Constructive role of temperature in ratchets driven by trichotomous noise.

The dynamics of an overdamped Brownian particle in a piecewise linear spatially periodic potential subjected to both thermal and colored symmetric three-level Markovian (trichotomous) noises is investigated. In the case of large flatness, the exact formula for the stationary current is presented. The dependence of the current on the system parameters is analyzed and the conditions for the occurrence of current reversals are found. It is shown that the direction and value of the current can be controlled by a thermal noise. Asymptotic formulas for the current for various limits of the noise parameters are calculated and compared with the results of other authors. For small noise amplitudes, it is demonstrated that the temperature at which the current is maximized is proportional to the height of the potential barrier, being a slowly varying function of the other system parameters.

The capacity-cost function of discrete additive noise channels with and without feedback

We consider modulo-q additive noise channels, where the noise process is a stationary irreducible and aperiodic Markov chain of order k. We begin by investigating the capacity-cost function (C(/spl beta/)) of such additive-noise channels without feedback. We establish a tight upper bound to (C(/spl beta/)) which holds for general (not necessarily Markovian) stationary q-ary noise processes. This bound constitutes the counterpart of the Wyner-Ziv lower bound to the rate-distortion function of stationary sources with memory. We also provide two simple lower bounds to C(/spl beta/) which along with the upper bound can be easily calculated using the Blahut algorithm for the computation of channel capacity. Numerical results indicate that these bounds form a tight envelope on C(/spl beta/). We next examine the effect of output feedback on the capacity-cost function of these channels and establish a lower bound to the capacity-cost function with feedback (C/sub FB/(/spl beta/)). We show (both analytically and numerically) that for a particular feedback encoding strategy and a class of Markov noise sources, the lower bound to C/sub FB/(/spl beta/) is strictly greater than C(/spl beta/). This demonstrates that feedback can increase the capacity-cost function of discrete channels with memory.

The Viterbi algorithm and Markov noise memory

This work designs sequence detectors for channels with intersymbol interference (ISI) and correlated (and/or signal-dependent) noise. We describe three major contributions. (i) First, by modeling the noise as a finite-order Markov process, we derive the optimal maximum-likelihood sequence detector (MLSD) and the optimal maximum a posteriori (MAP) sequence detector extending to the correlated noise case the Viterbi algorithm. We show that, when the signal-dependent noise is conditionally Gauss-Markov, the branch metrics in the MLSD are computed from the conditional second-order noise statistics. We evaluate the branch metrics using a bank of finite impulse response (FIR) filters. (ii) Second, we characterize the error performance of the MLSD and MAP sequence detector. The error analysis of these detectors is complicated by the correlation asymmetry of the channel noise. We derive upper and lower bounds and computationally efficient approximations to these bounds based on the banded structure of the inverses of Gauss-Markov covariance matrices. An experimental study shows the tightness of these bounds. (iii) Finally, we derive several classes of suboptimal sequence detectors, and demonstrate how these and others available in the literature relate to the MLSD. We compare their error rate performance and their relative computational complexity, and show how the structure of the MLSD and the performance evaluation guide us in choosing a best compromise between several types of suboptimal sequence detectors.

Photocount distribution from the coherent signal recorded by a detector with variable efficiency against the background of noise

We consider the problem of photocount distribution of an optical photodetector which records the coherent radiation field β(t) against the background of normal Markov noise α(t). We generalize the well-known Mandel formula describing the probability P(n) of detecting n photocounts over the time interval (0, T) to the case where the quantum efficiency e(t) of the detector depends on time. The influence of the variable efficiency e(t) on the photocount distribution P(n) is analyzed. The results are compared with the case where e(t)≡1.

Trichotomous noise-induced transitions.

A nonlinear one-dimensional process driven by a multiplicative exponentially correlated three-level Markovian noise (trichotomous noise) is considered. An explicit second-order linear ordinary differential equation for the stationary probability density distribution is obtained for the process. In the case of a linear process with an additive trichotomous noise the exact formula for the steady-state distribution is obtained. The well-known dichotomous noise can be regarded as a special case of the trichotomous noise. As a rule, the system variable has three specific values where the probability density distribution can be singular. For the case of the Hongler model the dependence of the behavior of the stationary probability density on the noise parameters is investigated in detail and illustrated by a phase diagram. Applications to the Gompertz and Verhulst models are also discussed.