Discrete-time Homogeneous Markov Chain Research Articles

A Novel Secured Multi-Access Edge Computing based VANET with Neuro fuzzy systems based Blockchain Framework

In vehicle ad-hoc networks, the progression of wireless communication technology to 6G, overcomes storage, processing, privacy, and power limits to create an efficient and intelligent next generation transportation system. Vehicular ad hoc network may now offer remarkable availability, reliability, and throughput using 6G technology. However, the VANET system’s data should be protected. This paper proposes an effective batch authentication and key exchange technique to avoid contact with hostile vehicle users. Moreover, three types of systems are proposed: PKI, ID-based, and MAC-based. The neuro-fuzzy inference technique was used to predict VANET security ratings. The Homogeneous Discrete-Time Markov Chain model is used to secure data transit. Additionally, this research examined the work from a blockchain perspective combined with MEC. There are 3 level to comprise the architecture: perception, edge computing, and services. Throughout the blockchain transmission process, the first layer make certain the security of VANET data. The perception layer makes use of edge computing and cloud services on the edge. The service layer protects data by using both traditional cloud storage and blockchain technology. The lowest layer of the system architecture is dedicated to the throughput and quality of service requirements of MEC users. The primary challenge is achieving consensus across blockchain nodes while maintaining the MEC system’s and blockchain’s performance. To simulate the joint optimization problem, a Markov decision process with reward function is utilized. The simulation results are conferred to illustrate the validity of study assertions.

Set controllability of Markov jump switching Boolean control networks and its applications

In this article, a novel method is proposed for investigating the set controllability of Markov jump switching Boolean control networks (MJSBCNs). Specifically, the switching signal is described as a discrete-time homogeneous Markov chain. By resorting to the expectation and switching indicator function, an expectation system is constructed. Based on the expectation system, a novel verifiable condition is established for solving the set reachability of MJSBCNs. With the newly obtained results on set reachability, a necessary and sufficient condition is further derived for the set controllability of MJSBCNs. The obtained results are applied to Boolean control networks with Markov jump time delays. Examples are demonstrated to justify the theoretical results.

On longest consecutive patterns in Markov chains

ABSTRACT Consider a discrete-time homogeneous Markov chain with initial state i. We study the distribution of , the length of the longest consecutive visits of this chain to state j until time n. We provide two limiting theorems for and establish asymptotics for the moment generating function of . We conclude by closing the open problem raised by the authors of [T. Konstantopoulos, Z. Liu, and X. Yang, Laplace transform asymptotics and large deviation principles for longest success runs in Bernoulli trials, J. Appl. Probab. 53 (2016), pp. 747–764.] by providing two large deviation principles for .

An Analysis of a Storage System for a Wind Farm with Ramp-Rate Limitation

This paper provides evidence on how the variability of the power produced by a wind farm and its revenue are affected by implementing a ramp-rate limitation strategy and by adding a storage device to the system. The wind farm receives penalties whenever the ramp-rate limitations are not respected and may be supported by batteries to avoid this scenario. In this paper, we model the battery usage as a discrete time homogeneous Markov chain with rewards thanks to which it is possible to simulate the state of the charge of the battery and to calculate the amount of penalties suffered by the wind farm during any period. An application is performed considering the power produced by a hypothetical wind turbine located in Sardinia (Italy) using real wind speed data and electricity prices from a period of 10 years. We applied the concept of ramp-rate limitation on our hourly dataset, studying several limitation scenarios and battery capacities.

Context-Aware Quantification for VANET Security: A Markov Chain-Based Scheme

Recently, the quantification of VANET security has drawn significant attention due to the lack of standard computational metrics. The salient features of VANET, such as highly dynamic connections, sensitive information sharing, and unreliable fading channels, make the security quantification challenging. Accurate measurement for VANET security depends on the sufficient understanding of “context”, or making sense of the states, environment, or situation. This article proposes a context-aware security quantification scheme for VANET based on the Markov chain. Firstly, a Homogeneous Continuous-Time Markov Chain (HCTMC)-based security state model is designed for VANET. The value of each state of the HCTMC is determined with a value function that incorporates the security strength of transmitted data, dynamic and randomness of the vehicular channel, and transmission delay of the current situated environment of VANETs. Finally, the state transition matrix is derived based on the Homogeneous Discrete-Time Markov Chain (HDTMC) and Homogeneous Poisson Process (HPP). Simulation results show that the security quantification method enables the VANET’s system to adopt context-aware defense strategies according to the situated environment.

Covariance-based least-squares filtering algorithm under Markovian measurement delays

ABSTRACTThis paper addresses the least-squares linear filtering problem of signals from measurements which may be randomly delayed by one or two sampling times. The delays are modelled by a homogeneous discrete-time Markov chain to capture the dependence between them. Assuming that the evolution equation generating the signal is not available and that only the first- and second-order moments of the processes involved in the observation model are known, a recursive filtering algorithm is derived using an innovation approach. Recursive formulas for the filtering error variances are also obtained to measure the precision of the proposed estimators.

Markov Chain Model with Catastrophe to Determine Mean Time to Default of Credit Risky Assets

This article deals with the problem of probabilistic prediction of the time distance to default for a firm. To model the credit risk, the dynamics of an asset is described as a function of a homogeneous discrete time Markov chain subject to a catastrophe, the default. The behaviour of the Markov chain is investigated and the mean time to the default is expressed in a closed form. The methodology to estimate the parameters is given. Numerical results are provided to illustrate the applicability of the proposed model on real data and their analysis is discussed.

A portfolio optimisation model for credit risky bonds with Markov model credit rating dynamics

In this paper, a credit risk optimisation model for the portfolio of credit risky bonds with l∞-norm risk measure is proposed. The proposed model is formulated as a linear programming problem which makes it computationally efficient for the portfolio of large size. The rates of returns of the bonds are the input parameters of the proposed model and of the other portfolio optimisation models which are considered for the comparison with the former. The complete approach of generating rate of returns of the bonds, given their initial credit ratings and transition probability matrices, is presented. The time homogeneous discrete time Markov chain model is assumed for the credit rating dynamics of bonds. With extensive numerical illustrations, the proposed approach of obtaining rate of returns of the bonds is demonstrated. Furthermore, comparison of the proposed credit risk optimisation model with other inline existing portfolio optimisation models is performed.

Quantized fuzzy passification for nonlinear systems with Markov-based transmission delays

In this paper, the quantized fuzzy passification issue for nonlinear systems subject to random transmission delays is studied. A discrete-time homogeneous Markov chain, whose state transition probability matrix is partially unknown, is employed to model the random transmission delays. And a logarithmic quantizer is introduced to overcome the channel capacity constraint. A sufficient condition is derived by constructing a Lyapunov function, which depends both on the fuzzy basis and the transmission delay, to guarantee that the augmented closed-loop system is mean-square exponentially stable and passive in the sense of expectation. Then, the controller is designed with the linearization of this condition. Finally, the effectiveness of the proposed design technique for nonlinear systems is demonstrated through a numerical example.

Reliable observer-based H∞ control for discrete-time fuzzy systems with time-varying delays and stochastic actuator faults via scaled small gain theorem

Abstract This paper is concerned with the reliable observer-based H ∞ control problem for discrete-time Takagi–Sugeno (T–S) fuzzy systems with time-varying delay and stochastic actuator faults based on input–output approach. A discrete-time homogeneous Markov chain is used to represent the stochastic behavior of actuator faults. First, the discrete-time T–S fuzzy system is transformed into the form of interconnection of two subsystems by employing a new model transformation for the delayed state variables. Furthermore, a sufficient condition on discrete-time T–S fuzzy systems with time-varying delay and actuator faults, which guarantees the corresponding closed-loop system to be stochastically stable and preserves a guaranteed H ∞ performance, is derived by employing the scaled small gain theorem. Meanwhile, the solvability condition for the reliable observer-based H ∞ control is also established, by which the reliable H ∞ fuzzy controller can be solved as linear matrix inequalities (LMIs). Finally, a numerical example is provided to demonstrate the effectiveness of the proposed approach.

Fault detection for singular networked control systems with randomly occurring delays and partially known distribution transmission delays

In this paper, the fault detection problem is discussed for a class of discrete-time singular networked control systems with randomly occurring delays and network-induced delays. The considered randomly occurring delays include the multiple state delays and the infinite-distributed delay. Both of them are assumed to occur in random ways, which are governed by two independent stochastic variables satisfying the Bernoulli binary distribution. Network-induced delays are modeled as a discrete-time homogeneous Markov chain, and the transition probabilities of the Markov chain are partially known. A stochastic Lyapunov functional is constructed to design a full-order fault detection filter guaranteeing that the fault detection dynamics is stochastically admissible with a prescribed [Formula: see text] performance index. Hence, the fault detection filter design problem is converted into a convex optimization problem in terms of a set of linear matrix inequalities. If these linear matrix inequalities are feasible, the corresponding parameters of the designed fault detection filter are determined. Finally, a numerical example is addressed to demonstrate the effectiveness of the developed method.

Mixed H∞ and passivity based state estimation for fuzzy neural networks with Markovian-type estimator gain change

This paper is concerned with the mixed H∞ and passivity based state estimation for a class of discrete-time fuzzy neural networks with the estimator gain change, where a discrete-time homogeneous Markov chain taking value in a finite set Γ={0,1} is introduced to model this phenomenon. Based on the Markovian system approach and linear matrix inequality technique, a new sufficient condition has been derived such that the estimation error system is exponentially stable in the mean square sense and achieves a prescribed mixed H∞ and passivity performance level. The estimator parameter is then determined by solving a set of linear matrix inequalities (LMIs). A numerical example is presented to show the effectiveness of the proposed design method.

Fault detection for discrete-time networked nonlinear systems with incomplete measurements

This article addresses the problem of fault detection (FD) for discrete-time networked systems with global Lipschitz nonlinearities and incomplete measurements, including time delays, packet dropouts and signal quantisation. By utilising a discrete-time homogeneous Markov chain, an improved model which considers packet dropout compensation has been proposed to describe the above network-induced phenomena. We aim to design a mode-dependent fault detection filter (FDF) such that the FD system is asymptotically mean-square stable and satisfies a prescribed attenuation level. The addressed FD problem is then converted into an auxiliary H ∞ filtering problem of Markov jump system with time-varying delay. A sufficient condition for the existence of the FDF is derived in terms of certain linear matrix inequalities (LMIs). When these LMIs are feasible, the explicit expression of the desired FDF can also be characterised. A numerical example is exploited to show the effectiveness of the results obtained.

Reliable H∞ filtering for discrete-time singular systems with randomly occurring delays and sensor failures

In this study, the reliable H ∞ filtering problem is studied for discrete-time singular systems with randomly occurring delays and sensor failures. Two stochastic variables, that are mutually independent but obey the Bernoulli distribution, are introduced to govern the random occurrences of the discrete-time-varying delay and the infinite-distributed delay. The failures of sensors are quantified by a stochastic variable taking values in a given interval. A discrete-time homogeneous Markov chain is used to represent the stochastic behaviour of sensor failures. The main purpose of the addressed reliable H ∞ filtering problem is to design a reliable mode-dependent filter such that the filtering error dynamics is not only stochastically admissible but also achieves a prescribed H ∞ performance level. A sufficient condition is first established for the existence of the desired filter, and then, the corresponding solvability condition for the desired filter gains is established. The case of Markov chain with partially unknown transition probabilities is also considered. A numerical example is provided to illustrate the effectiveness of the proposed method.

Reliable $H_\infty$ Control for Discrete-Time Fuzzy Systems With Infinite-Distributed Delay

In this paper, the problem of reliable <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$H_\infty$</tex></formula> control is investigated for discrete-time Takagi–Sugeno (T–S) fuzzy systems with infinite-distributed delay and actuator faults. A discrete-time homogeneous Markov chain is used to represent the stochastic behavior of actuator faults. In terms of a stochastic fuzzy Lyapunov functional, a sufficient condition is proposed to ensure that the resultant closed-loop system is exponentially stable in the mean-square sense with an <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$H_\infty$</tex></formula> performance index. Based on the derived condition, the reliable <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$H_\infty$</tex></formula> control problem is solved, and an explicit expression of the desired controller is also given. The case of no failure in the actuator is also considered. A numerical example is given to demonstrate that our results are effective and less conservative.

A Numerical Algorithm on the Computation of the Stationary Distribution of a Discrete Time Homogenous Finite Markov Chain

The transition matrix, which characterizes a discrete time homogeneous Markov chain, is a stochastic matrix. A stochastic matrix is a special nonnegative matrix with each row summing up to 1. In this paper, we focus on the computation of the stationary distribution of a transition matrix from the viewpoint of the Perron vector of a nonnegative matrix, based on which an algorithm for the stationary distribution is proposed. The algorithm can also be used to compute the Perron root and the corresponding Perron vector of any nonnegative irreducible matrix. Furthermore, a numerical example is given to demonstrate the validity of the algorithm.

The optimal search for a Markovian target when the search path is constrained: the infinite-horizon case

A target moves among a finite number of cells according to a discrete-time homogeneous Markov chain. The searcher is subject to constraints on the search path, i.e., the cells available for search in the current epoch is a function of the cell searched in the previous epoch. The aim is to identify a search policy that maximizes the infinite-horizon total expected reward earned. We show the following structural results under the assumption that the target transition matrix is ergodic: 1) the optimal search policy is stationary; and 2) there exists /spl epsi/-optimal stationary policies which may be constructed by the standard value iteration algorithm in finite time. These results are obtained by showing that the dynamic programming operator associated with the search problem is an m-stage contraction mapping on a suitably defined space. An upper bound of m and the coefficient of contraction /spl alpha/ is given in terms of the transition matrix and other variables pertaining to the search problem. These bounds on m and /spl alpha/ may be used to derive bounds on suboptimal search polices constructed.

Markovian bounds on functions of finite Markov chains

In this paper, we obtain Markovian bounds on a function of a homogeneous discrete time Markov chain. For deriving such bounds, we use well-known results on stochastic majorization of Markov chains and the Rogers–Pitman lumpability criterion. The proposed method of comparison between functions of Markov chains is not equivalent to generalized coupling method of Markov chains, although we obtain same kind of majorization. We derive necessary and sufficient conditions for existence of our Markovian bounds. We also discuss the choice of the geometric invariant related to the lumpability condition that we use.

Markovian bounds on functions of finite Markov chains

In this paper, we obtain Markovian bounds on a function of a homogeneous discrete time Markov chain. For deriving such bounds, we use well known results on stochastic majorization of Markov chains and the Rogers-Pitman's lumpability criterion. The proposed method of comparison between functions of Markov chains is not equivalent to generalized coupling method of Markov chains although we obtain same kind of majorization. We derive necessary and sufficient conditions for existence of our Markovian bounds. We also discuss the choice of the geometric invariant related to the lumpability condition that we use.

Reduction techniques for discrete-time Markov chains on totally ordered state space using stochastic comparisons

We propose in this paper two methods to compute Markovian bounds for monotone functions of a discrete time homogeneous Markov chain evolving in a totally ordered state space. The main interest of such methods is to propose algorithms to simplify analysis of transient characteristics such as the output process of a queue, or sojourn time in a subset of states. Construction of bounds are based on two kinds of results: well-known results on stochastic comparison between Markov chains with the same state space; and the fact that in some cases a function of Markov chain is again a homogeneous Markov chain but with smaller state space. Indeed, computation of bounds uses knowledge on the whole initial model. However, only part of this data is necessary at each step of the algorithms.