Discrete-time Finite-state Markov Research Articles

Coupling compartmental models with Markov chains and measure evolution equations to capture virus mutability

The COVID-19 pandemic lit a fire under researchers who have subsequently raced to build models which capture various physical aspects of both the biology of the virus and its mobility throughout the human population. These models could include characteristics such as different genders, ages, frequency of interactions, mutation of virus, etc. Here, we propose two mathematical formulations to include virus mutation dynamics. The first uses a compartmental epidemiological model coupled with a discrete-time finite-state Markov chain. If one includes a nonlinear dependence of the transition matrix on current infected, the model is able to reproduce pandemic waves due to different variants. The second approach expands such an idea to a continuous state-space leveraging a combination of ordinary differential equations with an evolution equation for measure. This approach allows to include reinfections with partial immunity with respect to variants genetically similar to that of first infection.

Comprehensive Analysis and Optimization of Reliable Viterbi Decoder Circuits Implemented in Modular VLSI Design Logic Styles

The Viterbi Algorithm is a recursive optimal solution for estimating the most likely state sequence of discrete-time finite-state Markov process and Hidden Markov Models (HMM) observed in memoryless noise. The Viterbi algorithm is extensively used for decoding convolutional codes, in the constraint length k that encompasses its use in digital communications specifically in satellite and cellular communications. Storage devices to speed up access, speech synthesis and recognition technologies use the Viterbi algorithm or its variants. In this paper low-power, high-speed and reduced transistor count Viterbi decoding circuits with enhanced error detection capabilities are designed and implemented with signature-based error detection schemes in three design logic styles primarily Conventional CMOS, Hybrid logic and GDI. The significance of the work is the upshot of realizing low latency and low power dissipation with high reliability in the iterative process of finding the least path metric by superseding the subtractor in CSA & PCSA circuits with an optimized comparator. When evaluated against the Traditional/Benchmark CSA & PCSA circuits, the Conventional CMOS design approach attains low power consumption and high accuracy with a reduction in average power dissipation by 4.69% and 3.83% and an improvement in delay performance by 7.89% and 3.79% respectively, with a tradeoff for high area utilization. Whereas, the GDI design approach results in an extreme reduction of transistor count by 71.52% and 74.94% with a weaker logic swing for CSA & PCSA units respectively, complimented by an increase in power dissipation (approximately multiplied by a factor of 5) and deterioration in delay performance by one order of magnitude. The Hybrid logic stages CSA & PCSA units that are 32.52% and 9.27% faster and achieve optimization in area utilization by 48.68% and 51.09% respectively, at the expense of elevated power dissipation by one order of magnitude. All the circuits were designed and simulated using GPDK 90 nm technology libraries on Cadence Design Suite 6.1.6 platform at 27 °C temperature on 1.2 V supply-rail and SPICE codes were generated as well.

A stochastic model of a production-inventory system with consideration of production disruption

In this paper, we develop a stochastic production-inventory model that faces random production disruption. In each fixed time planning horizon, the inventory position is affected by production disruption. The production disruption time and the disruption recovery time in a single planning horizon are stochastic. The process of the inventory levels at the beginning of the time horizon is formulated by using a finite state discrete-time Markov chain. We derive expressions of the transition probability and the long-run average cost. The optimum solution is obtained through numerical experiments. The result shows that under the circumstance of production time constraint, the incorporation of safety stock in a disruption prone production-inventory system helps to minimise the long run average cost. Finally, sensitivity analysis has been given to demonstrate the usefulness of the model. This model assists decision makers to not only determine the optimal safety stock quantity but also reveal distinct effect and/or joint impacts of individual cost parameters, disruption probability on the decision regarding selection of safety stock.

Validating User Flows to Protect Software Defined Network Environments

Software Defined Network is a promising network paradigm which has led to several security threats in SDN applications that involve user flows, switches, and controllers in the network. Threats as spoofing, tampering, information disclosure, Denial of Service, flow table overloading, and so on have been addressed by many researchers. In this paper, we present novel SDN design to solve three security threats: flow table overloading is solved by constructing a star topology-based architecture, unsupervised hashing method mitigates link spoofing attack, and fuzzy classifier combined with L1-ELM running on a neural network for isolating anomaly packets from normal packets. For effective flow migration Discrete-Time Finite-State Markov Chain model is applied. Extensive simulations using OMNeT++ demonstrate the performance of our proposed approach, which is better at preserving holding time than are other state-of-the-art works from the literature.

Novel advancements in the Markov chain stock model: analysis and inference

In this paper we propose further advancements in the Markov chain stock model. First, we provide a formula for the second order moment of the fundamental price process with transversality conditions that avoid the presence of speculative bubbles. Second, we assume that the process of the dividend growth is governed by a finite state discrete time Markov chain and, under this hypothesis, we are able to compute the moments of the price process. We impose assumptions on the dividend growth process that guarantee finiteness of price and risk and the fulfilment of the transversality conditions. Subsequently, we develop non parametric statistical techniques for the inferential analysis of the model. We propose estimators of price, risk and forecasted prices and for each estimator we demonstrate that they are strongly consistent and that properly centralized and normalized they converge in distribution to normal random variables, then we give also the interval estimators. An application that demonstrate the practical implementation of methods and results to real dividend data concludes the paper.

Delay and energy consumption analysis of priority guaranteed MAC protocol for wireless body area networks

Wireless body area networks are captivating growing interest because of their suitability for wide range of applications. However, network lifetime is one of the most prominent barriers in deploying these networks for most applications. Moreover, most of these applications have stringent QoS requirements such as delay and throughput. In this paper, the modified superframe structure of IEEE 802.15.4 based MAC protocol is proposed which addresses the aforementioned problems and improves the energy consumption efficiency. Moreover, priority guaranteed CSMA/CA mechanism is used where different priorities are assigned to body nodes by adjusting the data type and size. In order to save energy, a wake-up radio based mechanism to control sleep and active modes of body sensors are used. Furthermore, a discrete time finite state Markov model to find the node states is used. Analytical expressions are derived to model and analyze the behavior of average energy consumption, throughput, packet drop probability, and average delay during normal and emergency data. Extensive simulations are conducted for analysis and validation of the proposed mechanism. Results show that the average energy consumption and delay are relatively higher during emergency data transmission with acknowledgment mode due to data collision and retransmission.

Modeling and Analysis of Link Duration in Vehicular Ad Hoc Networks Under Different Fading Channel Conditions

Modeling and analysis of communication link duration in vehicular ad hoc networks (VANETs) is crucial as it directly affects many performance metrics such as packet delivery ratio, throughput and end-to-end delay. In this paper, we investigate the properties of communication link duration in one-dimensional VANETs for different fading channel conditions. We employ a stochastic microscopic mobility model that takes into account time dependence of the inter-vehicle distance. A discrete-time finite-state Markov chain with state dependent transition probabilities is used to model the inter-vehicle distance. The presence of fading channel conditions will introduce fluctuations in the received signal power. Even when the inter-vehicle distance between a given pair of vehicles is less than their transmission range, poor channel conditions may result in link failure. A modified distance transition probability matrix is derived to describe the combined effects of relative velocity among a pair of vehicles, link failure due to outage, and inter-vehicle distance. The probability distribution of the communication link duration is derived from the modified distance transition probability matrix. The analytical results obtained from the model are validated by extensive simulation results.

Model Predictive Control for Discrete-Time Systems with Random Delay and Randomly Occurring Nonlinearity

In this paper, the model predictive control problem is investigated for a class of discrete-time systems with random delay and randomly occurring nonlinearity. The randomly occurring nonlinearity, which describes the phenomena of a class of nonlinear disturbances occurring in a random way, is modeled according to a Bernoulli distributed white sequence with a known conditional probability. Moreover, the random delay is governed by a discrete-time finite-state Markov chain. The approach of delay fractioning is applied to the controller synthesis. It is shown that the proposed model predictive controller guarantees the stochastic stability of the closed-loop system. Finally, a numerical simulation is given to illustrate the effectiveness of the proposed method.

Stochastic Analysis of a Single-Hop Communication Link in Vehicular Ad Hoc Networks

A vehicular ad hoc network (VANET) is a promising addition to our future intelligent transportation systems, which supports various safety and infotainment applications. The high node mobility and frequent topology changes in VANETs impose new challenges in maintaining a long-lasting connection between network nodes. As a result, the lifetime of communication links is a crucial issue in VANET development and operation. This paper presents a probabilistic analysis of the communication link in VANETs for three vehicle density ranges. First, we present the stationary distribution of the communication link length using mesoscopic mobility models. Second, we propose a stochastic microscopic mobility model that captures time variations of intervehicle distances (distance headways). A discrete-time finite-state Markov chain with state-dependent transition probabilities is proposed to model the distance headway. Third, the proposed stochastic microscopic model and first passage time analysis are used to derive the probability distribution of the communication link lifetime. Numerical results are presented to evaluate the proposed model, which demonstrate a close agreement between analytical and simulation results.

Probability-based constrained MPC for structured uncertain systems with state and random input delays

This article is concerned with probability-based constrained model predictive control (MPC) for systems with both structured uncertainties and time delays, where a random input delay and multiple fixed state delays are included. The process of input delay is governed by a discrete-time finite-state Markov chain. By invoking an appropriate augmented state, the system is transformed into a standard structured uncertain time-delay Markov jump linear system (MJLS). For the resulting system, a multi-step feedback control law is utilised to minimise an upper bound on the expected value of performance objective. The proposed design has been proved to stabilise the closed-loop system in the mean square sense and to guarantee constraints on control inputs and system states. Finally, a numerical example is given to illustrate the proposed results.

On the Queue Dynamics of Multiuser Multichannel Cognitive Radio Networks

For protocol and system design in cognitive radio networks (CRNs), it is essential to identify which system settings or environmental conditions have great impacts on the quality-of-service (QoS) performance for secondary users (SUs) and how they influence it, which are still open issues. In this paper, an analytical framework to quantify the queue dynamics of a multi-SU multichannel CRN is developed. In the analytical framework, we consider the important lower-layer mechanisms and settings, including spectrum sensing errors, medium-access control (MAC) protocols, link adaptation technologies such as adaptive modulation and coding (AMC) and automatic repeat request (ARQ), and limited buffer size. By modeling the queue dynamics as a discrete-time finite-state Markov chain (FSMC), we derive the analytical expressions of the average queue length, packet-dropping rate, and packet-collision rate. Based on these expressions, the QoS metrics including average queueing delay, packet-loss rate, and effective throughput are obtained. Simulation and numerical results are presented to verify the accuracy of the proposed analytical framework and investigate the QoS performance of SUs.

Exact Solutions for Rate and Synchrony in Recurrent Networks of Coincidence Detectors

We provide analytical solutions for mean firing rates and cross-correlations of coincidence detector neurons in recurrent networks with excitatory or inhibitory connectivity, with rate-modulated steady-state spiking inputs. We use discrete-time finite-state Markov chains to represent network state transition probabilities, which are subsequently used to derive exact analytical solutions for mean firing rates and cross-correlations. As illustrated in several examples, the method can be used for modeling cortical microcircuits and clarifying single-neuron and population coding mechanisms. We also demonstrate that increasing firing rates do not necessarily translate into increasing cross-correlations, though our results do support the contention that firing rates and cross-correlations are likely to be coupled. Our analytical solutions underscore the complexity of the relationship between firing rates and cross-correlations.

On the relation between reversibility and monotonicity of fluctuation spectra for discrete time finite state Markov chains

For the irreducible aperiodic discrete time finite state Markov chain, (i) if it is reversible, we provide a necessary and sufficient condition for all fluctuation spectra to be monotonic on [ 0 , π ] and (ii) if it is irreversible, then there exist some nonmonotonic fluctuation spectra.

Adaptive Importance Sampling Technique for Markov Chains Using Stochastic Approximation

For a discrete-time finite-state Markov chain, we develop an adaptive importance sampling scheme to estimate the expected total cost before hitting a set of terminal states. This scheme updates the change of measure at every transition using constant or decreasing step-size stochastic approximation. The updates are shown to concentrate asymptotically in a neighborhood of the desired zero-variance estimator. Through simulation experiments on simple Markovian queues, we observe that the proposed technique performs very well in estimating performance measures related to rare events associated with queue lengths exceeding prescribed thresholds. We include performance comparisons of the proposed algorithm with existing adaptive importance sampling algorithms on some examples. We also discuss the extension of the technique to estimate the infinite horizon expected discounted cost and the expected average cost.

Approximations of general discrete time queues by discrete time queues with arrivals modulated by finite chains

Recently, Asmussen and Koole (Journal of Applied Probability 30, pp. 365–372) showed that any discrete or continuous time marked point process can be approximated by a sequence of arrival streams modulated by finite state continuous time Markov chains. If the original process is customer (time) stationary then so are the approximating processes. Also, the moments in the stationary case converge. For discrete marked point processes we construct a sequence of discrete processes modulated by discrete time finite state Markov chains. All the above features of approximating sequences of Asmussen and Koole continue to hold. For discrete arrival sequences (to a queue) which are modulated by a countable state Markov chain we form a different sequence of approximating arrival streams by which, unlike in the Asmussen and Koole case, even the stationary moments of waiting times can be approximated. Explicit constructions for the output process of a queue and the total input process of a discrete time Jackson network with these characteristics are obtained.

Approximations of general discrete time queues by discrete time queues with arrivals modulated by finite chains

Recently, Asmussen and Koole (Journal of Applied Probability30, pp. 365–372) showed that any discrete or continuous time marked point process can be approximated by a sequence of arrival streams modulated by finite state continuous time Markov chains. If the original process is customer (time) stationary then so are the approximating processes. Also, the moments in the stationary case converge. For discrete marked point processes we construct a sequence of discrete processes modulated by discrete time finite state Markov chains. All the above features of approximating sequences of Asmussen and Koole continue to hold. For discrete arrival sequences (to a queue) which are modulated by a countable state Markov chain we form a different sequence of approximating arrival streams by which, unlike in the Asmussen and Koole case, even the stationary moments of waiting times can be approximated. Explicit constructions for the output process of a queue and the total input process of a discrete time Jackson network with these characteristics are obtained.

A filtered EM algorithm for joint hidden Markov model and sinusoidal parameter estimation

Derives finite-dimensional discrete-time filters for estimating the parameters of discrete-time finite-state Markov chains imbedded in a mixture of Gaussian white noise and deterministic signals of known functional form with unknown parameters. The filters that are derived estimate quantities used in the expectation-maximization (EM) algorithm for maximum likelihood (ML) estimation of the Markov chain parameters (transition probabilities and state levels) as well as the parameters of the deterministic interference. Two types of deterministic signals are considered: periodic or almost periodic signals with unknown frequency components, amplitudes, and phases, and polynomial drift in the states of the Markov process with the coefficients of the polynomial unknown. The filter-based EM algorithm has negligible memory requirements. In comparison, implementing the EM algorithm using smoothed variables (forward-backward variables) requires memory proportional to the number of observations. In addition, the filters are suitable for multiprocessor implementation unlike the forward-backward algorithm.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Spectral decomposition approach for transient analysis of multi-server discrete-time queues

Discrete time queueing systems are extensively used in modeling the ATM environment at the cell level. This paper examines the transient behavior of a discrete time FCFS multiple server queue. For the arrival process we assume a general discrete time Markovian batch arrival process, where the batch size distribution of the arrivals in successive slots is governed by a finite state discrete time Markov chain. We show that once the spectral decomposition of the “probability generating matrix” of the arrival process is obtained, the complete solution in the transform domain may be given. Using the complex analysis technique and Cauchy's integral formula, we present an efficient numerical method for the numerical calculation of a few performance measures of interest, namely transient probability of queue being empty, and the mean of the queue length distribution. We also generalize our numerical method in the situations where the superposition of a number of independent arrival sources are fed to the queue. It is shown that the numerical complexity of obtaining the transient solution in this case can be substantially reduced by using the approach based on the Kronecker product. The finite buffer case is also analyzed.

Hidden Markov model signal processing in presence of unknown deterministic interferences

Expectation maximization algorithms are used to extract discrete-time finite-state Markov signals imbedded in a mixture of Gaussian white-noise and deterministic signals of known functional form with unknown parameters. Maximum-likelihood estimates of the Markov state levels, state estimates, transition possibilities, and the parameters of the deterministic signals are obtained. Two types of deterministic signals are considered: periodic, or almost periodic signals with unknown frequency components, amplitudes, and phases; and polynomial drift in the states of the Markov process with the coefficients of the polynomial unknown. The techniques and supporting theory appear more elegant and powerful than ad hoc heuristic alternatives. An illustrative application to extracting ionic channel currents in cell membranes in the presence of white Gaussian noise and AC hum is included. >

Contributions to the application of the Viterbi algorithm

The Viterbi algorithm is an efficient technique to estimate the state sequence of a discrete-time finite-state Markov process in the presence of memoryless noise. This work sets up a relationship to a general class of linear and nonlinear fast algorithms such as FFT, FWT, and optimal sorting. The performance of a Viterbi detector is a function of the minimum distance between signals in the observation space of the estimated Markov process. It is shown that this distance may efficiently be calculated with dynamic programming using a slightly modified Viterbi algorithm of an increased basis.