Switching Diffusions Research Articles

A numerical method for ergodic optimal control of switching diffusions with reflection

The ergodic or long run average cost control problem for diffusions is one of the classical problems of stochastic control that still eludes a completely satisfactory treatment. This is certainly true for the setting in which the systems to be controlled is modeled by the solution of a switching stochastic differential equation with reflection (SSDER). In this paper, we advance in this rather difficult problem by setting forth preliminary results for the unidimensional case. Besides carving out a numerical method, our treatment of the ergodic control in this scenario straddles issues of existence and uniqueness of solution of the SSDER and a verification theorem for the associated HJB equation. We conclude by illustrating the effectiveness of the method considering the control of energy consumption in a large parallel processing computer system composed of one queue and several processing stations.

Discounted Risk-Sensitive Optimal Control of Switching Diffusions: Viscosity Solution and Numerical Approximation

This work considers the infinite horizon discounted risk-sensitive optimal control problem for the switching diffusions with a compact control space and controlled through the drift; thus, the the generator of the switching diffusions also depends on the controls. Note that the running cost of interest can be unbounded, so a decent estimation on the value function is obtained, under suitable conditions. To solve such a risk-sensitive optimal control problem, we adopt the viscosity solution methods and propose a numerical approximation scheme. We can verify that the value function of the optimal control problem solves the optimality equation as the unique viscosity solution. The optimality equation is also called the Hamilton–Jacobi–Bellman (HJB) equation, which is a second-order partial differential equation (PDE). Since, the explicit solutions to such PDEs are usually difficult to obtain, the finite difference approximation scheme is derived to approximate the value function. As a byproduct, the ϵ-optimal control of finite difference type is also obtained.

Constrained stochastic differential games with Markovian switchings and additive structure: The total expected payoff

The main objective of this work is to give conditions for the existence of Nash equilibria for a nonzero-sum constrained stochastic differential game with additive structure and Markovian switchings. In this type of game, each player is interested only in maximizing their finite-horizon total payoff when an additional cost function of the same type is required to be dominated above by another function (in particular, by a constant). The dynamic system for this game is controlled by two players and evolves according to a Markov-modulated diffusion (also known as switching diffusions or piecewise diffusion or diffusion with Markovian switchings). Given that, each player has to solve an optimization problem with constraints. The existence of a Nash equilibrium is thus proved using the Lagrange multipliers approach combined with standard dynamic programming arguments. The Lagrange approach allows the transformation of a constrained game into an unconstrained game. Therefore, this work gives conditions under which a Nash equilibrium for the unconstrained stochastic differential game is also a Nash equilibrium for the corresponding nonzero-sum constrained stochastic differential game. The theory developed here is illustrated by a pollution accumulation problem with two players. Therein, the evolution is governed by a linear stochastic differential equation with Markovian switching, and the decay pollution rate depends on a Markov chain.

Linear quadratic leader–follower stochastic differential games for mean-field switching diffusions

In this paper, we consider a linear quadratic (LQ) leader–follower stochastic differential game for regime switching diffusions with mean-field interactions. One of the salient features of this paper is that conditional mean-field terms are included in the state equation and cost functionals. Based on stochastic maximum principles (SMPs), the follower’s problem and the leader’s problem are solved sequentially and an open-loop Stackelberg equilibrium is obtained. Further, with the help of the so-called four-step scheme, the corresponding Hamiltonian systems for the two players are decoupled and then the open-loop Stackelberg equilibrium admits a state feedback representation if some new-type Riccati equations are solvable.

On exponential contraction and expansion of Markovian switching diffusions

This paper is concerned with the exponential contraction and expansion of regime-switching diffusions. The focus is on the behavior of the distance from one solution to another solution, rather than with respect to some equilibrium point. The moment and almost sure exponential contraction and expansion are investigated. General criteria and easily verifiable conditions for the moment and almost sure exponential contraction and expansion are obtained. The contribution of the Markovian switching to the exponential contraction and expansion is revealed. Several examples together with numerical experiments are provided for demonstration. The convergence of contracting observers with noisy measurements under Markovian switching and the connection between exponential contraction and global attractivity of stochastic population systems are presented.

Exact Optimal Stopping for Multidimensional Linear Switching Diffusions

The paper studies a class of multidimensional optimal stopping problems with infinite horizon for linear switching diffusions. There are two main novelties in the optimal problems considered: The underlying stochastic process has discontinuous paths, and the cost function is not necessarily integrable on the entire time horizon, where the latter is often a key assumption in classical optimal stopping theory for diffusions. Under relatively mild conditions, we show, for the class of multidimensional optimal stopping problems under consideration, that the first entry time of the stopping region is an optimal stopping time. Further, we prove that the corresponding optimal stopping boundaries can be represented as the unique solution to a nonlinear integral equation. We conclude with an application of our results to the problem of quickest real-time detection of a Markovian drift. Funding: P. Ernst received financial support from the Army Research Office [Grant ARO-YIP-71636-MA], the National Science Foundation [Grant DMS-1811936], and the Office of Naval Research [Grants N00014-18-1-2192 and N00014-21-1-2672].

Synchronization of hybrid switching diffusions delayed networks via stochastic event-triggered control

In this paper, the synchronization problem of stochastic complex networks with time delays and hybrid switching diffusions (SCNTH) is concerned based on event-triggered control. Therein, a new class of event-triggered function is proposed for the control design. Particularly, different from the existing work, the triggered instant generated by event-triggered control in this paper is a stochastic sequence instead of a number sequence to be more realistic for stochastic systems, which is a breakthrough. Furthermore, some sufficient conditions are derived to guarantee asymptotical synchronization in mean square, exponential synchronization in mean square and almost surely exponential synchronization of SCNTH based on sampled-data control, event-driven control theory and stability analysis. Meanwhile, the Zeno phenomenon can be avoided. Then, the synchronization of single-link robot arms is investigated in detail as a practical application of the obtained results. Ultimately, a numerical example is given for demonstration.

Stabilization of stochastic coupled systems with Lévy noise and regime switching diffusions via intermittent control with a time delay

In this paper, the stabilization problem of stochastic Lévy noise coupled systems with time delays and regime switching diffusions (SLCSTR) is researched via intermittent control with a time delay (ICTD). It is worth pointing out that both Lévy noise and regime switching diffusions are considered into coupled systems. Furthermore, different from previous intermittent control based on current state, ICTD combining the benefits of aperiodically intermittent control and delay feedback control is based on past state. Meanwhile, the stabilization problem of SLCSTR is investigated via delay feedback control. Then, by Lyapunov method, graph theory and some techniques of inequalities, some stabilization criteria of SLCSTR in the sense of exponential stability in mean square are obtained. In addition, we investigate the stabilization problem of stochastic delayed coupled oscillators with Lévy noise and regime switching diffusions as a practical application of our theoretical results. Finally, numerical simulations are given for illustration.

Optimal control of multiple Markov-switching stochastic systems with numerical applications

In this article the authors set up an optimal control framework for a hybrid stochastic system with dual or multiple Markov switching diffusion processes, while Markov chains governing these switching diffusions are not identical as assumed in the existing literature. For this purpose, the authors first combine two or more separate Markov chains into one compound Markov chain within continuous time context and give out concrete procedures about how to estimate corresponding generator matrix of the newly combined Markov chain via a modified EM algorithm, then the HJB equations for the optimal control problem with synthetic Markov switching diffusion are stated. As an application and illustration, the authors lay out an adapted numerical solution scheme for the problem about optimal consumption–investment decisions under multiple cyclical or switching markets. Relevant economic analysis and interpretations are provided as results.

On ergodic control problem for viscous Hamilton–Jacobi equations for weakly coupled elliptic systems

In this article, we study ergodic problems in the whole space RN for weakly coupled systems of viscous Hamilton-Jacobi equations with coercive right-hand sides. The Hamiltonians are assumed to have a fairly general structure, and the switching rates need not be constant. We prove the existence of a critical value ▪ such that the ergodic eigenvalue problem has a solution for every ▪ and no solution for ▪. Moreover, the existence and uniqueness of non-negative solutions corresponding to the value ▪ are also established. We also exhibit the implication of these results to the ergodic optimal control problems of controlled switching diffusions.

A survey of numerical solutions for stochastic control problems: Some recent progress

<p style='text-indent:20px;'>This paper presents a survey on some of the recent progress on numerical solutions for controlled switching diffusions. We begin by recalling the basics of switching diffusions and controlled switching diffusions. We then present regular controls and singular controls. The main objective of this paper is to provide a survey on some recent advances on Markov chain approximation methods for solving stochastic control problems numerically. A number of applications in insurance, mathematical biology, epidemiology, and economics are presented. Several numerical examples are provided for demonstration.</p>

Explicit criteria for moment exponential stability and instability of switching diffusions with Lévy noise

This paper is devoted to moment exponential stability of a class of Markovian switching diffusions with Lévy noise. Our objective is to find verifiable conditions for moment exponential stability and instability. By employing the ireducibility of the Markovian switching process and the Fredholm alternative, we derive explicit criteria for moment exponential stability and instability for linear and general nonlinear systems. Explicit contribution of the Markovian switching to the stability and instability is revealed. Using the criteria developed, we can design feedback controls or perturbations to stabilise a given diffusion with Markovian switching and Lévy noise. Several examples are provided to demonstrate our findings.

Controlled Switching Diffusions Under Ambiguity: The Average Criterion

This paper concerns controlled switching diffusions. In particular, we consider that the drift coefficient of the diffusion process depends on an unknown (and possibly nonobservable) parameter. For giving solution to our control problem, we formulate it as a game against nature, where the ambiguity is represented by nature that chooses values of the unknown parameter through actions so playing the role as an opposite player of the controller. Our objective is to give conditions to characterize the ergodic optimality and guarantee the existence of optimal policies for the central controller. Finally, we provide two examples to illustrate our results.

Distributions of Functionals of Switching Diffusions with Jumps

The paper deals with methods for calculating distributions of functionals of switching diffusions with jumps. The switching between two collections of diffusion coefficients occurs at the Poisson time moments which are independent of the initial diffusions. At the same moments, the diffusion can have jumps. The process controlling the switching is determined by a Markov chain.

Razumikhin method to stability of delay coupled systems with hybrid switching diffusions

Delay coupled systems have received a great deal of attention in recent years. However, delay coupled systems with hybrid switching diffusions (DCSHSD) have rarely been discussed and are still important. Thus, this paper is devoted to studying the stability of DCSHSD based on Razumikhin method. Then, combining Lyapunov method with graph theory, pth moment exponential stability of DCSHSD is studied and three kinds of stability criteria, including the Razumikhin-type theorem, the Lyapunov-type theorem and a coefficients-type theorem, are obtained. Wherein, an upper bound of Lyapunov exponent is estimated. In particular, as a practical application of our theoretical results, stability of delay coupled oscillators with hybrid switching diffusions is studied. Finally, a numerical example is provided to demonstrate the validity and feasibility of the theoretical results.

Joint Distributions of Functionals of the Telegraph Process and Switching Diffusions

The paper deals with methods of computation of joint distributions of functionals of the telegraph process and switching diffusions. A switching between two collections of diffusion coefficients happens at Poisson time moments that are independent of the initial diffusions. It is also possible to consider more general switching diffusions when the choice is performed from three or more collections of diffusion coefficients.

A stochastic maximum principle for switching diffusions using conditional mean-fields with applications to control problems

This paper obtains a maximum principle for switching diffusions with mean-field interactions. The motivation stems from a wide range of applications for networked control systems in which large-scale systems are encountered and mean-field interactions are involved. Because of the complexity due to the switching, little has been done for the associate control problems with mean-field interactions. The main ingredient of this work is the use of conditional mean-fields, which is distinct from the existing literature. Using the maximum principle, optimal controls of linear quadratic Gaussian controls with mean-field interactions for switching diffusions are carried out. Numerical examples are also provided for demonstration.

Lyapunov criteria for the Feller–Dynkin property of martingale problems

We give necessary and sufficient criteria for the Feller–Dynkin property of solutions to martingale problems in terms of Lyapunov functions. Moreover, we derive a Khasminskii-type integral test for the Feller–Dynkin property of multidimensional diffusions with random switching. For one dimensional switching diffusions with state-independent switching, we provide an integral-test for the Feller–Dynkin property.

Properties of switching jump diffusions: Maximum principles and Harnack inequalities

This work examines a class of switching jump diffusion processes. The main effort is devoted to proving the maximum principle and obtaining the Harnack inequalities. Compared with the diffusions and switching diffusions, the associated operators for switching jump diffusions are non-local, resulting in more difficulty in treating such systems. Our study is carried out by taking into consideration of the interplay of stochastic processes and the associated systems of integro-differential equations.

Time-averaged mean square displacement for switching diffusion.

We consider a classic two-state switching diffusion model from a single-particle tracking perspective. The mean and the variance of the time-averaged mean square displacement (TAMSD) are computed exactly. When the measurement time (i.e., the trajectory duration) is comparable to or smaller than the mean residence times in each state, the ergodicity breaking parameter is shown to take arbitrarily large values, suggesting an apparent weak ergodicity breaking for this ergodic model. In this regime, individual random trajectories are not representative while the related TAMSD curves exhibit a broad spread, in agreement with experimental observations in living cells and complex fluids. Switching diffusions can thus present, in some cases, an ergodic alternative to commonly used and inherently non-ergodic continuous-time random walks that capture similar features.