Nonlinear Markov Research Articles

Advanced traveller information systems under recurrent traffic conditions: Network equilibrium and stability

In this paper the stability of traffic equilibrium is analysed by using a framework where advanced traveller information systems (ATIS) are explicitly modelled. The role played by information in traffic networks is discussed, with particular reference to the day-to-day dynamics of the traffic network and to system stability at equilibrium. The perspective adopted is that of transportation planning under recurrent network conditions. The network is considered to be in equilibrium, viewed as a fixed-point state of a day-to-day deterministic process, here modelled as a time-discrete non-linear Markovian dynamic system. In discussing the effects generated by the introduction of ATIS, the paper examines: changes in the fixed point(s) with respect to the absence of ATIS, how the theoretical conditions for fixed-point existence and uniqueness are affected, and the impact on the stability properties and the stability region at equilibrium. Most of the analyses are carried out with explicit theoretical considerations. Moreover, a toy network is also employed to explore numerically the effects of removing some assumptions concerning the accuracy of ATIS.

Theoretical search for solutions to minimize negative influence of segregation in mixing of particulate solids

The objective of this study is to examine several optimization problems in the batch mixing of segregating particulate solids that can be set up and solved using Markov chain models. To improve the adequacy of such models and exclude some physical contradictions that arise in the linear form, a non-linear Markov chain model for the mixing of segregating components is proposed. Optimal solutions are obtained by controlling the particle flow outside the mixing operating volume while the components are being loaded, modifying particle circulation inside the mixing zone during the process, and by structuring the load in the mixing zone. Solutions are found that not only reduce the negative influence of segregation, but also exclude it altogether. The gain resulting from optimization grows with the rate of segregation. The optimal solutions presented here can be used to improve the design of mixers.

Delay chemical master equation: direct and closed-form solutions

The stochastic simulation algorithm (SSA) describes the time evolution of a discrete nonlinear Markov process. This stochastic process has a probability density function that is the solution of a differential equation, commonly known as the chemical master equation (CME) or forward-Kolmogorov equation. In the same way that the CME gives rise to the SSA, and trajectories of the latter are exact with respect to the former, trajectories obtained from a delay SSA are exact representations of the underlying delay CME (DCME). However, in contrast to the CME, no closed-form solutions have so far been derived for any kind of DCME. In this paper, we describe for the first time direct and closed solutions of the DCME for simple reaction schemes, such as a single-delayed unimolecular reaction as well as chemical reactions for transcription and translation with delayed mRNA maturation. We also discuss the conditions that have to be met such that such solutions can be derived.

Energy-to-peak filtering for T–S fuzzy systems with Markovian jumping: The finite-time case

This paper studies the robust finite-time energy-to-peak filtering problem for a class of uncertain nonlinear Markovian jumping systems (MJSs) with energy constraint external disturbances. The Takagi–Sugeno (T–S) fuzzy model is employed to represent the nonlinear MJSs. By selecting the appropriate Lyapunov–Krasovskii functional, a sufficient condition is derived such that the filtering error dynamic fuzzy MJSs are finite-time stable and have a prescribed level of L2–L∞ disturbance attenuation in a finite time-interval. The sufficient condition on the existence of robust finite-time energy-to-peak fuzzy filtering criterion is formulated in the form of linear matrix inequalities and the designed finite-time energy-to-peak fuzzy filter is described as an optimization one. A numerical example is given to illustrate the effectiveness of the proposed design approach.

Long Time Results for a Weakly Interacting Particle System in Discrete Time

We study long time behavior of a discrete time weakly interacting particle system, and the corresponding nonlinear Markov process in , described in terms of a general stochastic evolution equation. In a setting where the state space of the particles is compact such questions have been studied in previous works, however, for the case of an unbounded state space very few results are available. Under suitable assumptions on the problem data we study several time asymptotic properties of the N-particle system and the associated nonlinear Markov chain. In particular, we show that the evolution equation for the law of the nonlinear Markov chain has a unique fixed point and starting from an arbitrary initial condition convergence to the fixed point occurs at an exponential rate. The empirical measure μNn of the N-particles at time n is shown to converge to the law μn of the nonlinear Markov process at time n, in the Wasserstein-1 distance, in L1, as N → ∞, uniformly in n. Several consequences of this uniform convergence are studied, including the interchangeability of the limits n → ∞ and N → ∞ and the propagation of chaos property at n = ∞. Rate of convergence of μNn to μn is studied by establishing uniform in time polynomial and exponential probability concentration estimates.

Invariant Measures of Genetic Recombination Processes

We construct a non-linear Markov process connected with a biological model of a bacterial genome recombination. The description of invariant measures of this process gives us the solution of one problem in elementary probability theory.

Constrained MPC design of nonlinear Markov jump system with nonhomogeneous process

In this paper, a differential-inclusion-based MPC scheme is developed for the controller design for a discrete time nonlinear Markov jump system with nonhomogeneous transition probability. By adopting a differential-inclusion-based convex model predictive control mechanism, the nonlinear Markov jump system with nonhomogeneous transition probability is enclosed by a set of linear Markov jump systems. In this way, the controller design for the nonlinear Markov jump system can be solved via solving a set of linear Markov jump systems. Two numerical examples with different weighting parameters R are presented to illustrate the applicability of the results obtained.

Non-fragile finite-time filter design for time-delayed Markovian jumping systems via T–S fuzzy model approach

In this paper, the non-fragile finite-time filtering problem is studied for a class of nonlinear Markovian jumping systems (NMJSs) with time delays and uncertainties. To design the mode-dependent non-fragile state filter, Takagi–Sugeno (T–S) fuzzy models are employed to represent the time-delayed and uncertain NMJSs. Then, based on the Lyapunov–Krasovskii functional, a sufficient condition is derived for the existence of a desired non-fragile \(H_{\infty }\) filter which also guarantees the finite-time boundness of the filtering error dynamical NMJSs. By this criterion, the approach to designing a non-fragile fuzzy filter is developed in terms of linear matrix inequalities. Finally, a numerical simulation is provided to illustrate the performance of the proposed method.

PI tracking control for nonlinear time-varying delay Markov jump systems

This paper considers robust stochastic stability and PI tracking control problem for Markov jump systems with both input delay and an unknown nonlinear function. Based on the traditional PI control strategy, a new controller design scheme is proposed for nonlinear time-delay Markov jump systems which can realize multiple control objectives including robust stochastic stability and tracking performance. By using the Lyapunov stability theory and LMI algorithms, a sufficient condition for the solution to robust stochastic stability and tracking control problem is obtained. Then, the desired controller with PI structure is designed, which ensures the resulting closed-loop system is robust stochastically stable and the system state has favorable tracking performance. Finally, a numerical example is provided to illustrate the effectiveness of the proposed results.

Limits of relative entropies associated with weakly interacting particle systems

The limits of scaled relative entropies between probability distributions associated with N-particle weakly interacting Markov processes are considered. The convergence of such scaled relative entropies is established in various settings. The analysis is motivated by the role relative entropy plays as a Lyapunov function for the (linear) Kolmogorov forward equation associated with an ergodic Markov process, and Lyapunov function properties of these scaling limits with respect to nonlinear finite-state Markov processes are studied in the companion paper [6].

Limits of relative entropies associated with weakly interacting particle systems

The limits of scaled relative entropies between probability distributions associated with N-particle weakly interacting Markov processes are considered. The convergence of such scaled relative entropies is established in various settings. The analysis is motivated by the role relative entropy plays as a Lyapunov function for the (linear) Kolmogorov forward equation associated with an ergodic Markov process, and Lyapunov function properties of these scaling limits with respect to nonlinear finite-state Markov processes are studied in the companion paper [6].

Generalized Mass Action Law and Thermodynamics of Nonlinear Markov Processes

The nonlinear Markov processes are measure-valued dynamical systems which preserve positivity. They can be represented as the law of large numbers limits of general Markov models of interacting particles. In physics, the kinetic equations allow Lyapunov functionals (entropy, free energy, etc.). This may be considered as a sort of inheritance of the Lyapunov functionals from the microscopic master equations. We study nonlinear Markov processes that inherit thermodynamic properties from the microscopic linear Markov processes. We develop the thermodynamics of nonlinear Markov processes and analyze the asymptotic assumption, which are sufficient for this inheritance.

Local stability of Kolmogorov forward equations for finite state nonlinear Markov processes

The focus of this work is on local stability of a class of nonlinear ordinary differential equations (ODE) that describe limits of empirical measures associated with finite-state weakly interacting N-particle systems. Local Lyapunov functions are identified for several classes of such ODE, including those associated with systems with slow adaptation and Gibbs systems. Using results from [5] and large deviations heuristics, a partial differential equation (PDE) associated with the nonlinear ODE is introduced and it is shown that positive definite subsolutions of this PDE serve as local Lyapunov functions for the ODE. This PDE characterization is used to construct explicit Lyapunov functions for a broad class of models called locally Gibbs systems. This class of models is significantly larger than the family of Gibbs systems and several examples of such systems are presented, including models with nearest neighbor jumps and models with simultaneous jumps that arise in applications.

An epsilon-Nash equilibrium for non-linear Markov games of mean-field-type on finite spaces

We investigate mean field games from the point of view of a large number of indistinguishable players which eventually converges to in- finity. The players are weakly coupled via their empirical measure. The dynamics of the individual players is governed by pure jump type propagators over a finite space. Investigations are conducted in the framework of non-linear Markov processes. We show that the individual optimal strategy results from a consistent coupling of an optimal control problem with a forward non-autonomous dynamics. In the limit as the number N of players goes to infinity this leads to a jump-type analog of the well-known non-linear McKean-Vlasov dynamics. The case where one player has an individual preference different from the ones of the remaining players is also covered. The two results combined reveal a 1 N -Nash Equilibrium for the approximating system of N players.

Optimal Control for Unknown Discrete-Time Nonlinear Markov Jump Systems Using Adaptive Dynamic Programming

In this paper, we develop and analyze an optimal control method for a class of discrete-time nonlinear Markov jump systems (MJSs) with unknown system dynamics. Specifically, an identifier is established for the unknown systems to approximate system states, and an optimal control approach for nonlinear MJSs is developed to solve the Hamilton-Jacobi-Bellman equation based on the adaptive dynamic programming technique. We also develop detailed stability analysis of the control approach, including the convergence of the performance index function for nonlinear MJSs and the existence of the corresponding admissible control. Neural network techniques are used to approximate the proposed performance index function and the control law. To demonstrate the effectiveness of our approach, three simulation studies, one linear case, one nonlinear case, and one single link robot arm case, are used to validate the performance of the proposed optimal control method.

Robust Filtering for Nonlinear Nonhomogeneous Markov Jump Systems by Fuzzy Approximation Approach.

This paper addresses the problem of robust fuzzy L2-L∞ filtering for a class of uncertain nonlinear discrete-time Markov jump systems (MJSs) with nonhomogeneous jump processes. The Takagi-Sugeno fuzzy model is employed to represent such nonlinear nonhomogeneous MJS with norm-bounded parameter uncertainties. In order to decrease conservation, a polytope Lyapunov function which evolves as a convex function is employed, and then, under the designed mode-dependent and variation-dependent fuzzy filter which includes the membership functions, a sufficient condition is presented to ensure that the filtering error dynamic system is stochastically stable and that it has a prescribed L2-L∞ performance index. Two simulated examples are given to demonstrate the effectiveness and advantages of the proposed techniques.

Infinite horizon [formula omitted] control for nonlinear stochastic Markov jump systems with [formula omitted]-dependent noise via fuzzy approach

In this paper, the H∞ control problem is studied for nonlinear stochastic Markov jump systems with state, control and external disturbance-dependent noise ((x,u,v)-dependent noise for short). A sufficient condition is derived for the infinite horizon H∞ control of such systems in terms of a set of coupled second-order Hamilton–Jacobi inequalities (HJIs). In general, it is difficult to solve these coupled HJIs. By using fuzzy approach, the infinite horizon H∞ control design for nonlinear stochastic Markov jump systems is developed via solving a set of linear matrix inequalities (LMIs) instead of HJIs. Two numerical examples are presented to illustrate the effectiveness of the proposed design methods.

A neural network based online learning and control approach for Markov jump systems

In this paper, we propose an optimal online control method for discrete-time nonlinear Markov jump systems (MJSs). The Markov chain and the weighted sum technique are introduced to convert the Markov jumping problem into an optimal control problem. We then use adaptive dynamic programming (ADP) to accomplish online learning and control with specific learning algorithm and detailed stability analysis, including the convergence of the performance index function sequence and the existence of the corresponding admissible control input. Neural networks are applied to implement this ADP approach and online learning method is used to tune the weights of the critic and the action networks. Two different numerical examples are given to demonstrate the effectiveness of the proposed method.

Lyapunov functionals for boundary-driven nonlinear drift–diffusion equations

We exhibit a large class of Lyapunov functionals for nonlinear drift–diffusion equations with non-homogeneous Dirichlet boundary conditions. These are generalizations of large deviation functionals for underlying stochastic many-particle systems, the zero-range process and the Ginzburg–Landau dynamics, which we describe briefly. We prove, as an application, linear inequalities between such an entropy-like functional and its entropy production functional for the boundary-driven porous medium equation in a bounded domain with positive Dirichlet conditions: this implies exponential rates of relaxation related to the first Dirichlet eigenvalue of the domain. We also derive Lyapunov functions for systems of nonlinear diffusion equations, and for nonlinear Markov processes with non-reversible stationary measures.

Impact of meteorological factors on the spatiotemporal patterns of dengue fever incidence

Dengue fever is one of the most widespread vector-borne diseases and has caused more than 50million infections annually over the world. For the purposes of disease prevention and climate change health impact assessment, it is crucial to understand the weather-disease associations for dengue fever. This study investigated the nonlinear delayed impact of meteorological conditions on the spatiotemporal variations of dengue fever in southern Taiwan during 1998–2011. We present a novel integration of a distributed lag nonlinear model and Markov random fields to assess the nonlinear lagged effects of weather variables on temporal dynamics of dengue fever and to account for the geographical heterogeneity. This study identified the most significant meteorological measures to dengue fever variations, i.e., weekly minimum temperature, and the weekly maximum 24-hour rainfall, by obtaining the relative risk (RR) with respect to disease counts and a continuous 20-week lagged time. Results show that RR increased as minimum temperature increased, especially for the lagged period 5–18weeks, and also suggest that the time to high disease risks can be decreased. Once the occurrence of maximum 24-hour rainfall is >50mm, an associated increased RR lasted for up to 15weeks. A temporary one-month decrease in the RR of dengue fever is noted following the extreme rain. In addition, the elevated incidence risk is identified in highly populated areas. Our results highlight the high nonlinearity of temporal lagged effects and magnitudes of temperature and rainfall on dengue fever epidemics. The results can be a practical reference for the early warning of dengue fever.