Theory Of Continuous-time Markov Chains Research Articles

Blocking Rate Analysis in Heterogeneous Multicell Cognitive-Radio 5G Networks

Cognitive radio (CR) technology is being recognized as one of the main enabling technologies for supporting massive high-speed connectivity in multicell fifth-generation (5G) networks by allowing dynamic spectrum access policies according to predetermined etiquette. Under this paradigm, the available spectrum is opportunistically allocated to the different cells. A key metric that is heavily employed in designing efficient CR-based multicell spectrum-allocation mechanisms is the blocking rate. An efficient spectrum-allocation mechanism aims to adaptively allocate channels to the different cells such that the overall blocking rate is minimized. In this article, we use the theory of continuous-time Markov chains to derive a closed-form expression for the blocking rate in multicell CR-based 5G networks. In our derivations, we build two detailed mathematical models describing the frequency-spectrum usage of both primary and secondary users. We combine the two models in a hierarchical fashion. As opposed to related models in the literature which suffer high complexity, the proposed hierarchical structure yields simplicity and tractability. Furthermore, while existing research works overlook spectrum heterogeneity, the proposed model addresses this important factor. Simulation results demonstrate high accuracy in evaluating the blocking rate across the different cells in the network.

Optimal management of stochastic invasion in a metapopulation with Allee effects

Invasive species account for incalculable damages worldwide, in both ecological and bioeconomic terms. The question of how a network of invasive populations can be optimally managed is one that deserves further exploration. A study accounting for partial observability and imperfect detection, in particular, could yield useful insights into species eradication efforts. Here, we generalized a simple model system that we developed in previous work. This model consists of three interacting populations with underlying strong Allee effects and stochastic dynamics, inhabiting distinct locations connected by dispersal, which can generate bistability. To explore the stochastic dynamics, we formulated an individual-based modeling approach. Next, using the theory of continuous-time Markov chains, we approximated the original high-dimensional model by a Markov chain with eight states, with each state corresponding to a combination of population thresholds. We then used the reduced model as the core for a powerful decision-making tool, referred to as a Partially Observable Markov Decision Process (POMDP). Analysis of this POMDP indicates when the system results in optimal management outcomes.

Availability-Aware Channel Allocation for Multi-Cell Cognitive Radio 5G Networks

In this paper, we propose a channel allocation algorithm for heterogeneous multi-cell cognitive radio 5G networks. The proposed algorithm considers channel-availability and channel-demand heterogeneity across CR cells. We use the theory of continuous-time Markov chains to develop mathematical expressions for channel availability and channel demand. Both expressions are incorporated in a binary integer program to find the optimal channel allocation. Further, we equip the proposed channel allocation algorithm with a novel interference domain identification scheme to detect interference range overlaps between cognitive radio cells. This scheme is essential for channel allocation, as it plays an important role in avoiding inter-cell interference. Our simulation results show significant performance gains compared to existing algorithms overlooking channel-availability. Furthermore, we use the theory of Dantzig-Wolfe decomposition to develop a framework for transforming centralized channel allocation algorithms into distributed ones. Through simulation experiments, we demonstrate the efficacy of the proposed framework.

Highly Reliable Transmission and Channel Assignment for CR-IoT Networks

In this article, we propose a highly reliable transmission strategy based on exploiting channel diversity for Internet of Things (IoT) networks equipped with cognitive radio (CR) capabilities (referred to as CR-IoT networks). Specifically, we investigate the channel-assignment problem with the objective of minimizing the number of assigned channels for each CR transmission subject to link reliability, data rate, and success probability constraints. We use the theory of continuous-time Markov chains to derive mathematical expressions for channel availability and reliability. These expressions are employed in a binary linear program model to formulate the channel assignment optimization problem, which is suboptimally solved using a polynomial-time sequential fixing procedure. Theoretical performance analysis of the proposed channel assignment scheme is derived in terms of a lower bound on the achievable probability of successful packet transmission. Through simulation experiments, we demonstrate significant performance enhancement compared to existing reliability-unaware CR channel assignment algorithms.

Rewriting theory for the life sciences: A unifying theory of CTMC semantics

The Kappa biochemistry and the MØD organic chemistry frameworks are amongst the most intensely developed applications of rewriting-based methods in the life sciences to date. A typical feature of these types of rewriting theories is the necessity to implement certain structural constraints on the objects to be rewritten (a protein is empirically found to have a certain signature of sites, a carbon atom can form at most four bonds, ...). In this paper, we contribute a number of original developments that permit to implement a universal theory of continuous-time Markov chains (CTMCs) for stochastic rewriting systems. Our core mathematical concepts are a novel rule algebra construction for the relevant setting of rewriting rules with conditions, both in Double- and in Sesqui-Pushout semantics, augmented by a suitable stochastic mechanics formalism extension that permits to derive dynamical evolution equations for pattern-counting statistics. A second main contribution of our paper is a novel framework of restricted rewriting theories, which comprises a rule-algebra calculus under the restriction to so-called constraint-preserving completions of application conditions (for rules considered to act only upon objects of the underlying category satisfying a globally fixed set of structural constraints). This novel framework in turn renders a faithful encoding of bio- and organo-chemical rewriting in the sense of Kappa and MØD possible, which allows us to derive a rewriting-based formulation of reaction systems including a full-fledged CTMC semantics as instances of our universal CTMC framework. While offering an interesting new perspective and conceptual simplification of this semantics in the setting of Kappa, both the formal encoding and the CTMC semantics of organo-chemical reaction systems as motivated by the MØD framework are the first such results of their kind.

Exact analysis of summary statistics for continuous-time discrete-state Markov processes on networks using graph-automorphism lumping

We propose a unified framework to represent a wide range of continuous-time discrete-state Markov processes on networks, and show how many network dynamics models in the literature can be represented in this unified framework. We show how a particular sub-set of these models, referred to here as single-vertex-transition (SVT) processes, lead to the analysis of quasi-birth-and-death (QBD) processes in the theory of continuous-time Markov chains. We illustrate how to analyse a number of summary statistics for these processes, such as absorption probabilities and first-passage times. We extend the graph-automorphism lumping approach [Kiss, Miller, Simon, Mathematics of Epidemics on Networks, 2017; Simon, Taylor, Kiss, J. Math. Bio. 62(4), 2011], by providing a matrix-oriented representation of this technique, and show how it can be applied to a very wide range of dynamical processes on networks. This approach can be used not only to solve the master equation of the system, but also to analyse the summary statistics of interest. We also show the interplay between the graph-automorphism lumping approach and the QBD structures when dealing with SVT processes. Finally, we illustrate our theoretical results with examples from the areas of opinion dynamics and mathematical epidemiology.

Justification of the architecture of an interactive information model of the process of space-mission vehicle launch preparation

The article presents a solution of the problem of substantiating the architecture of an interactive information model for the launch preparation of an integrated launch vehicle (ILV). The relevance of the research is due to the complexity and high dynamics of the ILV launch preparation process, as well as the latest achievements in the field of information modeling and decision support theory. The aim of the research is to raise the situational awareness of the decision maker in managing the process of preparing ILV for launch. The process of formation of situational awareness as an integral part of the decision-making process is analyzed. On the basis of this analysis a model of an indicator of the effectiveness of situational awareness formation is proposed for the first time. A model for evaluating the effectiveness of the process under study is developed on the basis of the mathematical apparatus of the theory of continuous-time Markov chains. The results of numerical experiments confirming the adequacy of the model and making it possible to state the quantitative requirements for the process of formation of situational awareness, ensuring its maximum efficiency, are presented. The tasks of the information system aimed at satisfying the requirements are stated, and the architecture of the information model of the ILV launch preparation process is developed. The research results can be used to justify the quantitative requirements for the developed information models of the launch situation at the stage of development of technical specifications.

A theoretical inquiry into aspects of the structure and the management of ecological–economic systems

We use the theory of continuous-time Markov chains (CTMCs) to analyze hitherto unstudied questions about jointly determined ecological–economic systems. Two specific questions are examined. First, on the methodological front, we show how the theory of CTMCs can be used to effectively model dynamic and stochastic ecological–economic systems. Then, given recent concern about the sustainability of desirable states and lock-in into undesirable states, we partition the state space of our stylized ecological–economic system into good and bad states, and demonstrate the formal relationship between these two sets of states. Second, we discuss a way of looking at the task of managing ecological–economic systems that captures this formal link between the good and the bad states, and has other desirable properties.

Properties of Statistical Equilibrium Equations: Positivity and Uniqueness

While linear statistical equilibrium equations play an important role in the description of nonequilibrium processes in astrophysics, some of their basic mathematical properties, such as uniqueness and positivity, have not been fully explored. In this paper these properties are related to concepts of connectivity from the theory of continuous-time Markov chains. For the irreducible case (in which every state is connected to every other state, either directly or through intermediate states), the solution is shown to be positive and unique when one positive normalization condition is provided. It is then shown how a general linear statistical equilibrium problem can be reduced by dividing the system into inessential and essential states and then partitioning the latter into separate irreducible subproblems. It is shown that: (1) The inessential states all have zero populations. If a positive normalization condition is imposed separately on each irreducible subproblem, then (2) the essential states all have positive populations and (3) the overall solution is unique.

On the Chapman—Kolmogorov equation

In the form considered in this paper, the Chapman-Kolmogorov equation connects a doubly infinite collection of functions, and imposes complex constraints on each of them. The main theorems characterize the functions which satisfy these constraints, and generalize known results in the theory of continuous-time Markov chains.