Multi-dimensional Markov Chain Research Articles

Analysis of the finite source MAP/PH/N retrial G-queue operating in a random environment

Finite source retrial G-queues are good mathematical models of communication systems and networks, so their investigation is important for theory and applications. In this paper, we analyze the MAP/PH/N retrial queue with finite number of sources and MAP arrivals of negative customers operating in a finite state Markovian random environment. The arrival of a negative customer with equal probability goes to any busy server to remove the customer being in service. The multi-dimensional Markov chain describing the behavior of the system is investigated. The algorithms for calculating the stationary state probabilities are elaborated. Main performance measures are obtained. Illustrative numerical examples are presented.

A Markov chain model of the evolution of complex neuronal network structures in the presence of plasticity

We consider neuronal network models with plasticity and randomness; we show that complicated global structures can evolve even in the presence of simple local update rules. Our models are discrete in time and our analysis uses tools from the theory of Markov chains. Specifically, we propose a discrete-time model of the evolution of a neuronal network capable of learning with multiple input neurons (whose spiking consists of Poisson processes), one output integrate-and-fire neuron, and a network with conductances between them. Along with the input firing, the stochasticity of the output neuron's state is modulated by the conductances in the network, that evolve according to synaptic plasticity in the form of long-term potentiation. The aggregate system evolves according to discrete-state, multi-dimensional Markov chain. We demonstrate that the network is capable of rich properties (e.g. bifurcation, various forms of stability, etc) that depend on maximum possible values of conductance, the rates of the Poisson processes, and the number of levels in the integrate-and-fire output neuron model. Most importantly, we believe that this approach provides a well-positioned balance between neuro-biological relevance and ability to be analyzed theoretically. In the future, we plan to apply principles from Lyapunov exponents (for sensitivity to initial conditions) and information theory (using the KL divergence for model order reduction and employing feedback information theoretic principles using the directed information to elucidate information dynamics) so that the key emerging principles underlying the complex behaviors can be illustrated in a provably good manner.

Nonlinear filtering of statistically connected video sequences based on hidden Markov chains

A method for the synthesis of algorithms for nonlinear filtering of statistically connected (correlated) video sequences of digital halftone images distorted by a white Gaussian noise is proposed. The method is based on the use of hidden Markov chains as a mathematical model for video sequences of digital halftone images, which, in limited observation intervals, can be represented by multidimensional Markov chains.

Collaboration of discrete-time Markov chains: Tensor and product form

We consider first discrete-time Markov chains (DTMCs) in competition over a set of resources. We build a multidimensional Markov chain based on the Cartesian product of the state space and on competition rules between the chains. We then generalize this approach to DTMCs with collaboration between components. The competition and the collaboration between chains simply assume that when a resource is owned by a component (or when a component is in a specific subset of states) it affects the transition probabilities of the other components of the chain. We prove that under some competition rules the steady-state distribution of the multidimensional chain has a product form. This work extends Boucherie’s theory based on continuous-time chains. The proof relies on algebraic properties of the generalized tensor product defined by Plateau and Stewart.

Performance analysis for hierarchical resource allocation in multiplexed mobile packet data networks

The general packet radio service (GPRS) can be thought of as an overlay network above a Global Systems for Mobile (GSM) network. In cooperation with the GSM system, GPRS supports both voice and packet data services. Obviously, combining voice and data traffic implies the nature of contending with finite radio resources. The paper focuses on voice/data integrated GPRS downlink traffic and proposes two hierarchical resource allocation strategies by considering time slots and radio blocks simultaneously with two different TDMA frame configurations. The radio block based resource allocation can characterize the multiplexing scheme of the practical GPRS operation. The performance of the proposed strategies were evaluated analytically by multidimensional Markov chain and verified through intensive computer simulations. Based on both the analytical and simulation results, it showed that the proposed allocation strategies can offer high utilization of transmission resource, as well as low blocking probability to GPRS users. The results of the study provide not only a practical thinking for implementation but also a guideline for analysis.

Call Admission Control with Load-Balancing Capability in Integrated Cellular/WLAN Networks

We propose a call admission control scheme in cellular and wireless local area networks (WLANs) integration: integrated service-based admission control with load-balancing capability (ISACL). The novel aspects of the ISACL scheme include that load transfer in the cellular/WLAN overlapping areas is allowed for the admission of originating data calls from the area with cellular access only and vertical handoff requests to the cellular network. Packet-level quality of service (QoS) constraints in the WLANs and other-cell interference in the code division multiple access (CDMA) cellular network are taken into account to derive the WLANs and cellular capacity. We model the integrated networks using a multi-dimensional Markov chain and the important performance measures are derived for effective optimization of the admission parameters. The analytical model is validated by a computer simulation. The variation of admission parameters with traffic load and the dependence of resource utilization on admission parameters are investigated. It is shown that optimal balancing of the traffic load between the cellular network and WLANs results in the maximum resource utilization. Numerical results demonstrate that substantial performance improvements can be achieved by applying the proposed ISACL scheme.

A novel call admission control scheme in cellular/WLAN integration and performance analysis

In order to achieve the Quality of Service (QoS) provisioning and efficient resource utilization in cellular network and Wireless Local Area Network (WLAN) integration, an Integrated Service-Based Call Admission Control (ISB-CAC) scheme is proposed in this paper. The integrated network is modeled by using multi-dimensional Markov chains. The numerical analysis is presented to evaluate the important performance measures such as the blocking probability of originating calls, the dropping probability, and the average transfer time, etc. The steady-state probabilities of the multi-dimensional Markov chains are obtained by using an iterative approach, and the CAC parameters are optimally designed. The analytical model is validated by the computer simulation. It is shown that compared with the conventional WLAN-First Call Admission Control (WF-CAC) scheme, the proposed ISB-CAC scheme not only provides better QoS for mobile users but also utilizes the bandwidth resources more efficiently.

Modeling and Analysis of Hybrid Cellular/WLAN Systems with Integrated Service-Based Vertical Handoff Schemes

We propose two vertical handoff schemes for cellular network and wireless local area network (WLAN) integration: integrated service-based handoff (ISH) and integrated service-based handoff with queue capabilities (ISHQ). Compared with existing handoff schemes in integrated cellular/WLAN networks, the proposed schemes consider a more comprehensive set of system characteristics such as different features of voice and data services, dynamic information about the admitted calls, user mobility and vertical handoffs in two directions. The code division multiple access (CDMA) cellular network and IEEE 802.11e WLAN are taken into account in the proposed schemes. We model the integrated networks by using multi-dimensional Markov chains and the major performance measures are derived for voice and data services. The important system parameters such as thresholds to prioritize handoff voice calls and queue sizes are optimized. Numerical results demonstrate that the proposed ISHQ scheme can maximize the utilization of overall bandwidth resources with the best quality of service (QoS) provisioning for voice and data services.

Performance analysis of multihop cellular network with fixed channel assignment

Multihop cellular network (MCN) has been proposed to incorporate the flexibility of ad hoc networks into traditional single-hop cellular networks (SCNs). The performance analysis of MCN through analytical models is not trivial because the classic Erlang B formula no longer applies to MCN where multihop transmission is allowed. In this paper, we first propose a clustered MCN (cMCN) architecture with the use of dedicated information ports (DIPs), which are deployed wireless ports functioning as central controllers for multihop users. The proposed cMCN can be considered as a complement of the existing cellular network. Then, we study the feasibility of modeling time division multiple access (TDMA)-based cMCN with fixed channel assignment (FCA) scheme for uplink transmission. An exact multi-dimensional Markov chain model to analyze the performance of cMCN with FCA is developed. Furthermore, an approximated model which results in reduced complexity is also presented. The analytical results from both models are matched with the simulation results closely. The results show that cMCN with the proposed FCA scheme can reduce the call blocking probability significantly as compared to SCNs with either the conventional FCA or a dynamic channel assignment (DCA) scheme.

A multi-dimensional Markov chain and the Meixner ensemble

We show that the transition probability of the Markov chain (G(i,1),...,G(i,n))i≥1, where the G(i,j)’s are certain directed last-passage times, is given by a determinant of a special form. An analogous formula has recently been obtained by Warren in a Brownian motion model. Furthermore we demonstrate that this formula leads to the Meixner ensemble when we compute the distribution function for G(m,n). We also obtain the Fredholm determinant representation of this distribution, where the kernel has a double contour integral representation.

Efficient Methods for Performance Evaluations of Call Admission Control Schemes in Multi-Service Cellular Networks

Many dynamic call admission control (CAC) schemes have been proposed in the literature for adaptive reservations in cellular networks. Efficient application of these schemes requires reliable and up-to-date feedback of system performance to the CAC mechanism. However, exact analyses of these schemes in real time using multi-dimensional Markov chain models are challenging due to the need to solve large sets of flow equations. One dimensional Markov chain models have been widely used to derive performance metrics such as call blocking probabilities of multiple traffic classes assuming that all classes of calls have equal capacity requirements and exponentially distributed channel holding times with equal mean values. These assumptions need to be relaxed for a more general evaluation of CAC performance in multi-service cellular networks. In this paper we classify CAC schemes according to their Markov chain models into two categories: symmetric and asymmetric, and develop computationally efficient analytical methods to compute call blocking probabilities of various traffic classes for several widely known CAC schemes under relaxed assumptions. We obtain a product form solution to evaluate symmetric schemes and propose a novel performance evaluation approximation method with low computational cost for asymmetric schemes. Numerical results demonstrate the accuracy and efficiency of the proposed method.

An optimal critical level policy for inventory systems with two demand classes

AbstractTraditional inventory systems treat all demands of a given item equally. This approach is optimal if the penalty costs of all customers are the same, but it is not optimal if the penalty costs are different for different customer classes. Then, demands of customers with high penalty costs must be filled before demands of customers with low penalty costs. A commonly used inventory policy for dealing with demands with different penalty costs is the critical level inventory policy. Under this policy demands with low penalty costs are filled as long as inventory is above a certain critical level. If the inventory reaches the critical level, only demands with high penalty costs are filled and demands with low penalty costs are backordered. In this article, we consider a critical level policy for a periodic review inventory system with two demand classes. Because traditional approaches cannot be used to find the optimal parameters of the policy, we use a multidimensional Markov chain to model the inventory system. We use a sample path approach to prove several properties of this inventory system. Although the cost function is not convex, we can build on these properties to develop an optimization approach that finds the optimal solution. We also present some numerical results. © 2008 Wiley Periodicals, Inc. Naval Research Logistics, 2008

Toward accurate modeling of the IEEE 802.11e EDCA under finite load and error-prone channel

In this paper we study the performance of IEEE 802.11e enhanced distributed channel access (EDCA) priority schemes under finite load and error-prone channel. We introduce a multi-dimensional Markov Chain model that includes all the mandatory differentiation mechanisms of the standard: QoS parameters, CWMIN, CWMAX arbitration inter-frame space (AIFS), and the virtual collision handler. The model faithfully represents the functionality of the EDCA access mechanisms, including lesser known details of the standard such as the management of the backoff counter which is technically different from the one used in the legacy DCF. We study the priority schemes under both finite load and saturation conditions. Our analysis also takes into consideration channel conditions.

Random Access with Multi-Packet Reception

The queuing performance of a finite-user slotted Aloha type random access protocol with multi-packet reception capability at the base station is considered. The system evolution under this protocol can be described by a multi-dimensional Markov chain, which is not amenable to analysis due to complex interactions between queues of different users. A novel "user-centric" modeling and analysis approach is proposed, which focuses on the evolution of the one-dimensional Markov chain associated with the queue of a typical user in the system, assuming that all other users are in "steady state". While the proposed approach is applicable quite generally, this paper addresses the special case of identical users with Bernoulli packet arrivals in order to obtain insightful analytical expressions for various performance metrics of interest (throughput, average delay etc.). Both static and fading wireless channels are considered, under appropriate assumptions on the underlying power control mechanisms. Sufficient stability conditions as well as asymptotic performance measures in the infinite power and infinite user regime are derived for both cases. The fundamental throughput versus power tradeoff inherent in the random access protocol is established. The theoretical results derived in the paper are validated via simulations.

An algebraic condition for product form in stochastic automata networks without synchronizations

We consider Stochastic Automata Networks (SANs) in continuous time and we prove a sufficient condition for the steady-state distribution to have product form. We consider synchronization-free SANs in which the transitions of one automaton may depend upon the states of the other automata. This model can represent efficiently multidimensional Markov chains whose transitions are limited to one component but whose rates may depend on the state of the chain. The sufficient condition we obtain is quite simple and our theorem generalizes former results on SANs as well as results on modulated Markovian queues, such as Boucherie’s theory on competing Markov chain, on reversible queues considered by Kelly and on modulated Jackson queueing networks studied by Zhu. The sufficient condition and the proof are purely algebraic and are based on the intersection of kernels for a certain set of matrices.

Markov Chains with Hybrid Repeating Rows - Upper-Hessenberg, Quasi-Toeplitz Structure of the Block Transition Probability Matrix

In this paper we consider discrete-time multidimensional Markov chains having a block transition probability matrix which is the sum of a matrix with repeating block rows and a matrix of upper-Hessenberg, quasi-Toeplitz structure. We derive sufficient conditions for the existence of the stationary distribution, and outline two algorithms for calculating the stationary distribution.

Markov Chains with Hybrid Repeating Rows - Upper-Hessenberg, Quasi-Toeplitz Structure of the Block Transition Probability Matrix

In this paper we consider discrete-time multidimensional Markov chains having a block transition probability matrix which is the sum of a matrix with repeating block rows and a matrix of upper-Hessenberg, quasi-Toeplitz structure. We derive sufficient conditions for the existence of the stationary distribution, and outline two algorithms for calculating the stationary distribution.

A single-chain-based multidimensional Markov chain model for subsurface characterization

Multidimensional Markov chain models in geosciences were often built on multiple chains, one in each direction, and assumed these 1-D chains to be independent of each other. Thus, unwanted transitions (i.e., transitions of multiple chains to the same location with unequal states) inevitably occur and have to be excluded in estimating the states at unobserved locations. This consequently may result in unreliable estimates, such as underestimation of small classes (i.e., classes with smaller than average areas) in simulated realizations. This paper presents a single-chain-based multidimensional Markov chain model for estimation (i.e., prediction and conditional stochastic simulation) of spatial distribution of subsurface formations with borehole data. The model assumes that a single Markov chain moves in a lattice space, interacting with its nearest known neighbors through different transition probability rules in different cardinal directions. The conditional probability distribution of the Markov chain at the location to be estimated is formulated in an explicit form by following the Bayes’ Theorem and the conditional independence of sparse data in cardinal directions. Since no unwanted transitions are involved, the model can estimate all classes fairly. Transiogram models (i.e., 1-D continuous Markov transition probability diagrams) are used to provide transition probability input with needed lags to generalize the model. Therefore, conditional simulation can be conducted directly and efficiently. The model provides an alternative for heterogeneity characterization of subsurface formations.

Asymmetric FCA for Downlink and Uplink Transmission in Clustered Multihop Cellular Network

A clustered multihop cellular network (CMCN) with virtual cells has been proposed to achieve the characteristics of macrocell/microcell hierarchically overlaid architecture by applying clustering techniques. As a complement to the traditional cellular networks, CMCN is able to incorporate the flexibility of ad hoc networks by allowing multihop transmission. In this paper, we first propose to use dedicated information ports (DIPs) as clusterheads for CMCN; then we analyze the performance of fixed channel assignment (FCA) for downlink transmission in CMCN. Two multi-dimensional Markov chain models are developed to study the call blocking probability. Due to the nature of multihop transmission in CMCN, channel assignment for uplink and downlink transmission is different and unbalanced. We then propose an asymmetric FCA (AFCA) for uplink and downlink transmission in CMCN. By making use of the proposed AFCA for uplink and downlink transmission, we can reduce the call blocking probability significantly as compared with the FCA for traditional single-hop cellular networks. The salient contribution is that the proposed CMCN with AFCA scheme can increase the spectrum efficiency and the system capacity by introducing the structure of CMCN with DIPs for virtual microcells.

A Generalized Channel Preemption Model for Multiclass Traffic in Mobile Wireless Networks

This paper presents a generalized channel preemption model (GCPM) for multiclass traffic in mobile wireless networks. In the proposed model, a mobile call is identified by four parameters - call type, traffic class, channel requirement, and preemption ratio - in which different classes of traffic can have different priority levels. To effectively reduce dropping probability, high-priority handoff calls are allowed to fully or partially preempt low-priority ongoing calls when the mobile network becomes congested. A fully preempted call can be completely disconnected, whereas a partially preempted call can still communicate with certain degree of quality-of-service degradation. We introduce an analytical model with multidimensional Markov chains. In the model, a cost function, which is expressed as a weighted sum of the ongoing calls to be preempted and the bandwidth reduction rate in the preempted calls, is defined to select candidate calls from a certain traffic class. The statistical result from the model allows us to tune the parameters of the GCPM to support high-priority handoff calls and therefore offers new insights in understanding the tradeoffs in supporting multiclass traffic.