Two-dimensional Markov Chain Research Articles

Dynamic Framing and Power Allocation for Real-Time Wireless Networks with Variable-Length Coding: A Tandem Queue Approach

Ensuring high reliability and low latency poses challenges for numerous applications that require rigid performance guarantees, such as industrial automation and autonomous vehicles. Our research primarily concentrates on addressing the real-time requirements of ultra-reliable low-latency communication (URLLC). Specifically, we tackle the challenge of hard delay constraints in real-time transmission systems, overcoming this obstacle through a finite blocklength coding scheme. In the physical layer, we encode randomly arriving packets using a variable-length coding scheme and transmit the encoded symbols by truncated channel inversion over parallel channels. In the network layer, we model the encoding and transmission processes as tandem queues. These queues backlog the data bits waiting to be encoded and the encoded symbols to be transmitted, respectively. This way, we represent the system as a two-dimensional Markov chain. By focusing on instances when the symbol queue is empty, we simplify the Markov chain into a one-dimensional Markov chain, with the packet queue being the system state. This approach allows us to analytically express power consumption and formulate a power minimization problem under hard delay constraints. Finally, we propose a heuristic algorithm to solve the problem and provide an extensive evaluation of the trade-offs between the hard delay constraint and power consumption.

Analysis of a Two-Stage Tandem Queuing System with Priority and Clearing Service in the Second Stage

This paper considers a two-stage tandem queuing system with ordinary customers and priority customers. Upon arrival, ordinary customers are individually served in the first stage, then move to the second stage and receive clearing service. Priority customers can bypass the first stage and proceed directly to the second stage for clearing service. The second stage has N service seats. All customers currently in the second stage are served simultaneously (i.e., clearing service). Once there are N customers in the second stage, the first stage will be blocked, and newly arriving priority customers will balk and leave without joining. We first formulate a two-dimensional Markov chain to analyze this queuing system and derive the stability condition. Subsequently, the stationary distribution of the system is derived using the matrix-analytic method and spectral expansion technique. Furthermore, analytical expressions for the mean queue length, mean sojourn time, and other performance measures are presented. Finally, some numerical examples are provided to illustrate the effects of various parameters, offering valuable insights for designing such two-stage tandem queuing systems.

A Novel Analytical Model for the IEEE 802.11p/bd Medium Access Control, with Consideration of the Capture Effect in the Internet of Vehicles

The traditional vehicular ad hoc network (VANET), which is evolving into the internet of vehicles (IoV), has drawn great attention for its enormous potential in road safety improvement, traffic management, infotainment service support, and even autonomous driving. IEEE 802.11p, as the vital standard for wireless access in vehicular environments, has been released for more than one decade and its evolution, IEEE 802.11bd, has also been released for a few months. Since the analytical models for the IEEE 802.11p/bd medium access control (MAC) play important roles in terms of performance evaluation and MAC protocol optimization, a lot of analytical models have been proposed. However, the existing analytical models are still not accurate as a result of ignoring some important factors of the MAC itself and real communication scenarios. Motivated by this, a novel analytical model is proposed, based on a novel two-dimensional (2-D) Markov chain model. In contrast to the existing studies, all the important factors are considered in this proposed model, such as the backoff freezing mechanism, retry limit, post-backoff states, differentiated packet arrival probabilities for empty buffer queue, and queue model of packets in the buffer. In addition, the influence of the capture effect under a Nakagami-m fading channel has also been considered. Then, the expressions of successful transmission, collided transmission, normalized unsaturated throughput, and average packet delay are all meticulously derived, respectively. At last, the accuracy of the proposed analytical model is verified via the simulation results, which show that it is more accurate than the existing analytical models.

Controlled two-dimensional Markov chains between two absorbing barriers

Let ( X n , Y n ) be a two-dimensional discrete-time controlled Markov chain. The problem of optimally controlling the chain until the sum X n + Y n is equal to either of two constants is considered. The aim is to leave the continuation region as soon as possible. The optimal control is found in two particular cases by solving the partial difference equation satisfied by the value function.

Modelling of wind and photovoltaic power output considering dynamic spatio-temporal correlation

The simulation technology of wind and solar power output can provide data support for the planning of new energy stations and the optimization and scheduling of power systems. In order to solve the problem that the existing output models can not accurately describe the dynamic spatio-temporal dependence between wind and solar output, a dynamic spatiotemporal correlation model of wind and solar output based on Markov stochastic process and Copula function theory is proposed. Firstly, wind power and photovoltaic output are regarded as a stochastic process, and the time autocorrelation models of wind power and photovoltaic output are constructed based on a one-dimensional Markov chain and hybrid Copula function. Next, two one-dimensional Markov chains were coupled to form a two-dimensional Markov chain, and a spatial correlation model between wind and solar output was constructed using the dynamic SJC Copula function. Take the measured data of adjacent wind farms and photovoltaic power stations in Hami, Xinjiang as an example for simulation. The simulation results show that the proposed model can effectively reflect the spatio-temporal correlation of the original data and reflect the dynamic changes in the correlation between wind and solar energy.

Pairwise Markov Models and Hybrid Segmentation Approach

The article studies segmentation problem (also known as classification problem) with pairwise Markov models (PMMs). A PMM is a process where the observation process and underlying state sequence form a two-dimensional Markov chain, it is a natural generalization of a hidden Markov model. To demonstrate the richness of the class of PMMs, we examine closer a few examples of rather different types of PMMs: a model for two related Markov chains, a model that allows to model an inhomogeneous Markov chain as a conditional marginal process of a homogeneous PMM, and a semi-Markov model. The segmentation problem assumes that one of the marginal processes is observed and the other one is not, the problem is to estimate the unobserved state path given the observations. The standard state path estimators often used are the so-called Viterbi path (a sequence with maximum state path probability given the observations) or the pointwise maximum a posteriori (PMAP) path (a sequence that maximizes the conditional state probability for given observations pointwise). Both these estimators have their limitations, therefore we derive formulas for calculating the so-called hybrid path estimators which interpolate between the PMAP and Viterbi path. We apply the introduced algorithms to the studied models in order to demonstrate the properties of different segmentation methods, and to illustrate large variation in behaviour of different segmentation methods in different PMMs. The studied examples show that a segmentation method should always be chosen with care by taking into account the purpose of modelling and the particular model of interest.

Single-Server Queuing-Inventory Systems with Negative Customers and Catastrophes in the Warehouse

In this paper, we studied single-server models of queuing-inventory systems (QIS) with catastrophes in the warehouse part and negative customers (n-customers) in service facility. Consumer customers (c-customers) that arrived to buy inventory can be queued in an infinite buffer. Under catastrophes, all inventory of the system is destroyed but customers in the system (on server or in buffer) are still waiting for replenishment of stocks. Upon arrival of n-customer one c-customer is pushed out, if any. One of two replenishment policies (RP) can be used in the system: either (s,S) or randomized. In the investigated QISs, a hybrid service scheme was used: if upon arrival of the c-customer, the inventory level is zero, then according to the Bernoulli scheme, this customer is either lost (lost sale scheme) or joining the queue (backorder scheme). Mathematical models of the investigated QISs were constructed as two-dimensional Markov chains (2D MC). Ergodicity conditions of the investigated QISs were obtained, and the matrix-analytic method (MAM) was used to calculate the steady-state probabilities of the constructed 2D MCs. Formulas for performance measures were found and the results of numerical experiments are presented.

Local Viterbi property in decoding

Abstract The article studies the decoding problem (also known as the classification or the segmentation problem) with pairwise Markov models (PMMs). A PMM is a process where the observation process and the underlying state sequence form a two-dimensional Markov chain, a natural generalization of hidden Markov model. The standard solutions to the decoding problem are the so-called Viterbi path—a sequence with maximum state path probability given the observations—or the pointwise maximum a posteriori (PMAP) path that maximizes the expected number of correctly classified entries. When the goal is to simultaneously maximize both criterions—conditional probability (corresponding to Viterbi path) and pointwise conditional probability (corresponding to PMAP path)—then they are combined into one single criterion via the regularization parameter $C$. The main objective of the article is to study the behaviour of the solution—called the hybrid path—as $C$ grows. Increasing $C$ increases the conditional probability of the hybrid path and when $C$ is big enough then every hybrid path is a Viterbi path. We show that hybrid paths also approach the Viterbi path locally: we define $m$-locally Viterbi paths and show that the hybrid path is $m$-locally Viterbi whenever $C$ is big enough. This all might lead to an impression that when $C$ is relatively big then any hybrid path that is not yet Viterbi differs from the Viterbi path by a few single entries only. We argue that this intuition is wrong, because when unique and $m$-locally Viterbi, then different hybrid paths differ by at least $m$ entries. Thus, when $C$ increases then the different hybrid paths tend to differ from each other by larger and larger intervals. Hence the hybrid paths might offer a variety of rather different solutions to the decoding problem.

Modeling Value of Information in remote sensing from correlated sources

This paper investigates data correlation in remote sensing networks and how it can be characterized through diverse models quantifying the Value of Information (VoI), a metric that describes how informative the data transmitted by the sensors are. For each sensor, the VoI evaluations comprise the average node-specific Age of Information (AoI), the average cost spent for sending updates, and the AoI of neighbor nodes, assumed to be correlated sources of information and therefore benefiting the VoI of other sensors nearby. We discuss how this metric can be tracked through a two-dimensional Markov chain, but we also show how this representation can be simplified by including the impact of neighbor nodes within the transition probabilities, so as to obtain a simpler model that gives the same insight in terms of VoI evaluations.

Analytical Description of the Diffusion in a Cellular Automaton with the Margolus Neighbourhood in Terms of the Two-Dimensional Markov Chain

The one-parameter two-dimensional cellular automaton with the Margolus neighbourhood is analyzed based on considering the projection of the stochastic movements of a single particle. Introducing the auxiliary random variable associated with the direction of the movement, we reduce the problem under consideration to the study of a two-dimensional Markov chain. The master equation for the probability distribution is derived and solved exactly using the probability-generating function method. The probability distribution is expressed analytically in terms of Jacobi polynomials. The moments of the obtained solution allowed us to derive the exact analytical formula for the parametric dependence of the diffusion coefficient in the two-dimensional cellular automaton with the Margolus neighbourhood. Our analytic results agree with earlier empirical results of other authors and refine them. The results are of interest for the modelling two-dimensional diffusion using cellular automata especially for the multicomponent problem.

Analysis on D2D Heterogeneous Networks with State-Dependent Priority燭raffic

In this work, we consider the performance analysis of state dependent priority traffic and scheduling in device to device (D2D) heterogeneous networks. There are two priority transmission types of data in wireless communication, such as video or telephone, which always meet the requirements of high priority (HP) data transmission first. If there is a large amount of low priority (LP) data, there will be a large amount of LP data that cannot be sent. This situation will cause excessive delay of LP data and packet dropping probability. In order to solve this problem, the data transmission process of high priority queue and low priority queue is studied. Considering the priority jump strategy to the priority queuing model, the queuing process with two priority data is modeled as a two-dimensional Markov chain. A state dependent priority jump queuing strategy is proposed, which can improve the discarding performance of low priority data. The quasi birth and death process method (QBD) and fixed point iteration method are used to solve the causality, and the steady-state probability distribution is further obtained.Then, performance parameters such as average queue length, average throughput, average delay and packet dropping probability for both high and low priority data can be expressed. The simulation results verify the correctness of the theoretical derivation. Meanwhile, the proposed priority jump queuing strategy can significantly improve the drop performance of low-priority data.

Impacts of Sensing Energy and Data Availability on Throughput of Energy Harvesting Cognitive Radio Networks

This paper aims to investigate the impacts of sensing energy and data availability on the secondary throughput of energy harvesting cognitive radio networks (EH-CRNs), where secondary transmitters (STs) are powered by the energy harvested from primary radio frequency (RF) signals. To reach the aim, we consider two extreme cases by employing two data arrival processes: heavy data arrival and general data arrival. In the heavy data arrival case, we consider STs always have data to transmit, and study the impacts of sensing energy on the secondary throughput. In the general data arrival case, we consider the data arrives with a certain probability, and study the impacts of data availability (i.e., data arrival probability and data buffer capacity) on the secondary throughput. Moreover, to compare the secondary throughput under non-cooperative spectrum sensing (NCSS) and cooperative spectrum sensing (CSS), we further study two CRN scenarios for each data arrival case: <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Scenario 1</i> with one pair of secondary users (SUs) conducting NCSS, and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Scenario 2</i> with multiple pairs of SUs conducting CSS. As the harvest-then-transmit protocol is employed by STs, it is essential to balance energy harvesting and data transmission to achieve high secondary throughput. We thus utilize an energy threshold approach to determine the actions of each ST. Specifically, under the heavy data arrival, we derive the average secondary throughput for each scenario by taking the sensing energy as an energy unit and modeling the energy states of each ST as a Markov chain. Under the general data arrival, we derive the secondary throughput for each scenario by modeling the energy and data states of each ST as a two-dimensional Markov chain. Simulation results show that: (1) both the secondary throughput and the optimal energy threshold decrease with the sensing energy; (2) the secondary throughput increases with the data availability, while the optimal energy threshold decreases with the data availability; (3) from the normalized secondary throughput perspective, CSS is not always better than NCSS.

Analytic and Computational Analysis of GI/Ma,b/c Queueing System

Bulk-service queueing systems have been widely applied in many areas in real life. While single-server queueing systems work in some cases, multi-servers can efficiently handle most complex applications. Bulk-service, multi-server queueing systems (compared to well-developed single-server queueing systems) are more complex and harder to deal with, especially when the inter-arrival time distributions are arbitrary. This paper deals with analytic and computational analyses of queue-length distributions for a complex bulk-service, multi-server queueing system GI/Ma,b/c, wherein inter-arrival times follow an arbitrary distribution, a is the quorum, and b is the capacity of each server; service times follow exponential distributions. The introduction of quorum a further increases the complexity of the model. In view of this, a two-dimensional Markov chain has to be involved. Currently, it appears that this system has not been addressed so far. An elegant analytic closed-form solution and an efficient algorithm to obtain the queue-length distributions at three different epochs, i.e., pre-arrival epoch (p.a.e.), random epoch (r.e.), and post-departure epoch (p.d.e.) are presented, when the servers are in busy and idle states, respectively.

QoS based enhanced lossless contention MAC protocol for ultraviolet network.

Ultraviolet (UV) communication, with its excellent characteristics of non-line-of-sight propagation, high confidentiality, and flexible networking, has gradually gained traction in the field of research. The UV media access control (MAC) protocol is relatively lacking at present, and the UV lossless contention MAC (UVLLC-MAC) protocol designed by our team earlier is limited by the predictability of communication network topology; hence, it is not applicable to multinode and multi-data networks. In order to be applicable to multinode network and satisfy the quality of service (QoS) for various data, an enhanced UVLLC-MAC (eUVLLC-MAC) protocol is proposed based on the binary exponential backoff (BEB) and UVLLC protocols. A two-dimensional Markov chain is established to analyze the network performance, and the throughput and delay expressions of the UV network are derived systematically. Subsequently, the effects of reset probability, maximum backoff order, and priority types on network performance are analyzed, and the protocol parameters are optimized to obtain the best network performance. Finally, it is observed that compared with the BEB and UVLLC-MAC protocols, the network under the eUVLLC-MAC protocol can obtain better performance in throughput, packet loss rate and delay comprehensively, which demonstrates the effectiveness of the proposed protocol. Additionally, the eUVLLC-MAC protocol can provide guidance for multinode and multi-data UV networking.

Sustainability-oriented maintenance management of highway bridge networks based on Q-learning

Current bridge management systems provide maintenance strategies that balance costs and structural safety of bridge networks. However, they are cost-oriented rather than sustainability-oriented. Moreover, the existing literature lacks research on reducing the environmental impacts of a bridge network during operation and maintenance periods. This paper develops a condition-based maintenance approach for highway bridges, aiming to minimize the total carbon emissions of bridge networks subject to maintenance budget constraints. A two-dimensional Markov chain model is applied to predict the deterioration processes of bridges, and a Q-learning algorithm is proposed to determine the maintenance strategy for a single bridge. Then, integer programming optimizes the budget allocation process for a bridge network. The proposed method is demonstrated using the concrete bridges in New York state. Sensitivity analyses indicate the impacts of budget levels on maintenance plans and total carbon emissions. 10% drop in budget level results in 30 additional bridge repairs delayed as well as 39 tonne extra annual carbon emissions. The proposed optimization framework would contribute to intelligent infrastructure asset management and sustainable society.

Retrial Queuing-Inventory Systems with Delayed Feedback and Instantaneous Damaging of Items

This paper studies a Markov model of a queuing-inventory system with primary, retrial, and feedback customers. Primary customers form a Poisson flow, and if an inventory level is positive upon their arrival, they instantly receive the items. If the inventory level is equal to zero upon arrival of a primary customer, then this customer, according to the Bernoulli scheme, either leaves the system or goes into an infinite buffer to repeat their request in the future. The rate of retrial customers is constant, and if the inventory level is zero upon arrival of a retrial customer, then this customer, according to the Bernoulli scheme, either leaves orbit or remains in orbit to repeat its request in the future. According to the Bernoulli scheme, each served primary or retrial customer either leaves the system or feedbacks into orbit to repeat their request. Destructive customers that form a Poisson flow cause damage to items. Unlike primary, retrial, and feedback customers, destructive customers do not require items, since, upon arrival of such customers, the inventory level instantly decreases by one. The system adopted one of two replenishment policies: (s, Q) or (s, S). In both policies, the lead time is a random variable that has an exponential distribution. It is shown that the mathematical model of the system under study was a two-dimensional Markov chain with an infinite state space. Algorithms for calculating the elements of the generating matrices of the constructed chains were developed, and the ergodicity conditions for both policies were found. To calculate the steady-state probabilities, a matrix-geometric method was used. Formulas were found for calculating the main performance measures of the system. The results of the numerical experiments, including the minimization of the total cost, are demonstrated.

The Optimal Packets Scheduling for Buffer-Aid Energy Harvesting RSUs in Cooperative Vehicle Infrastructure System

In this paper, we investigate the optimal packet scheduling for RSU-to-vehicle downlink communication in Cooperative Vehicle Infrastructure System (CVIS). The RSU is powered by a capacity-limited battery storing harvested energy with reliable energy source as backup, which meets requirement of renewable energy and ensures sustainability of communication system. We aim to find the optimal packet scheduling policy that minimizes the data queuing delay under a given the reliable power constraint and downlink availability rate. According to probabilistic scheduling policy, we adopt data buffer queue and harvesting energy queue as the state-space, and determine the transition action by the joint state of packet arrivals and channel state to establish a two-dimensional Markov chain. Based on this, the optimization problem is formulated. By solving the problem and analysing its solution, we derive the optimal transmission parameters and the optimal scheduling policy which turns out to be threshold-based. Simulations are demonstrated to verify the accuracy of the theoretical derivation results.

Limit theorems and structural properties of the cat-and-mouse Markov chain and its generalisations

AbstractWe revisit the so-called cat-and-mouse Markov chain, studied earlier by Litvak and Robert (2012). This is a two-dimensional Markov chain on the lattice $\mathbb{Z}^2$ , where the first component (the cat) is a simple random walk and the second component (the mouse) changes when the components meet. We obtain new results for two generalisations of the model. First, in the two-dimensional case we consider far more general jump distributions for the components and obtain a scaling limit for the second component. When we let the first component be a simple random walk again, we further generalise the jump distribution of the second component. Secondly, we consider chains of three and more dimensions, where we investigate structural properties of the model and find a limiting law for the last component.

Algorithms for Queueing Systems with Reneging and Priorities Modeled as Quasi-Birth-Death Processes

We consider a Markovian multiserver queueing system with two customer classes, preemptive priorities, and reneging. We formulate this system as an infinite level-dependent quasi-birth-death process (LDQBD). We introduce an algorithm that endogenously truncates the level and calculates lower and upper bounds on stationary probabilities of this LDQBD such that the gap between the bounds can be any desired amount. Our algorithm can be applied to any LDQBD for which the rate matrices become elementwise nonincreasing above some level. This appears to be the first algorithm that provides bounds on stationary probabilities for an infinite-level LDQBD. To obtain these bounds, the algorithm first obtains lower and upper bounds on the rate matrices of the LDQBD using a novel method, which can be applied to any LDQBD. We then extend this algorithm to approximate performance measures of the system of interest and calculate exact lower and upper bounds for those that can be expressed as probabilities, such as the probability that an incoming low-priority customer will wait. We generate a wide range of instances with up to 100 servers and compare the solution times and accuracy of our algorithm with two existing algorithms. These numerical experiments indicate that our algorithm is faster than the other two algorithms for a given accuracy requirement. We investigate the impact of changing service rates on the proportion of low-priority customers served and their wait time, and we demonstrate how ignoring one of these measures can possibly mislead decision makers. Summary of Contribution: We contribute to operations research by modeling a practically important queueing system and developing an algorithm to accurately compute performance measures for that system. We also contribute to computer science by providing error and complexity analysis for the algorithm to solve a broad class of two-dimensional Markov chains with infinite state space.

Low-Latency and Energy-Efficient Wireless Communications With Energy Harvesting

Energy harvesting (EH) aided communications hold a great potential in the design of green communication systems for their high energy efficiency. However, the random power supply due to EH may cause an intolerable delay in data transmission. To overcome this, a Reliable Energy Source (RES) is desired to provide transmission power when the large delay is induced. In this paper, we study the delay-optimal scheduling policy for EH aided communications with the constraint of average power provided by RES. More specifically, the delay-minimal scheduling is obtained through the two-dimensional Markov chain modeling and linear programming (LP) formulation. To further reduce the computational complexity, we present a value iteration algorithm, based on which we not only reveal a threshold-based structure of the delay-optimal scheduling policy for EH-aided communications with large-capacity batteries, but also conceive a low complexity policy that is asymptotically optimal. For EH-aided communications with finite-capacity batteries, we present a unified framework based on large deviation theory. The non-asymptotic framework demonstrates that the delay-power tradeoff curve of the low complexity scheduling policy is capable of converging to that of the delay-optimal policy exponentially as the capacity of the battery increases.