Open Markov Research Articles

Qualitative reachability for open interval Markov chains.

Interval Markov chains extend classical Markov chains with the possibility to describe transition probabilities using intervals, rather than exact values. While the standard formulation of interval Markov chains features closed intervals, previous work has considered open interval Markov chains, in which the intervals can also be open or half-open. In this article we focus on qualitative reachability problems for open interval Markov chains, which consider whether the optimal (maximum or minimum) probability with which a certain set of states can be reached is equal to 0 or 1. We present polynomial-time algorithms for these problems for both of the standard semantics of interval Markov chains. Our methods do not rely on the closure of open intervals, in contrast to previous approaches for open interval Markov chains, and can address situations in which probability 0 or 1 can be attained not exactly but arbitrarily closely.

Investigation of the limits of the secular and Markov approximations in the dissipative dynamics of laser-aligned molecules

We point out the breakdown of two approximations widely used to describe decoherence in open quantum systems, the secular and Markov approximations. We probe their limits by studying the influence of pressure on the alignment revivals (echoes) created in properly chosen gas mixtures (HCl and CO2, pure and diluted in He) by one (two) intense and short laser pulse(s). Experiments, as well as predictions using molecular dynamics simulations, consistently demonstrate in some of the aforementioned systems the break-down of these approximations at very short times (<15 ps) after the laser kick(s).

Service Chain Caching and Workload Scheduling in Mobile Edge Computing

The computing and storage resources of an edge node in a mobile edge computing (MEC) system are limited. One edge computing node cannot cache all the services supported by the entire MEC system. A service chain caching and scheduling strategy should be designed to improve the service processing capacities of the edge computing nodes under high computing load conditions. This article combines the open Markov queuing network for the first time to model the process of edge node analysis and processing of computationally intensive tasks as a mixed integer nonlinear programming problem. Moreover, a mathematical model for the service chains is established. We analyze the average service response time during task execution to optimize the performance of service chain caching and scheduling. Experiments show that the outer approximation (OA) algorithm obtains better results than MindtPy and Multistart. The average response time obtained by the OA algorithm is 2.2% and 6.9% lower than those obtained by the MindtPy and Multistart algorithms, respectively. The OA algorithm using this mathematical model can effectively reduce the average service response time and improve the resource utilization of the edge server.

Weighted Bayesian Belief Network: A Computational Intelligence Approach for Predictive Modeling in Clinical Datasets.

There are growing concerns about the mortality due to Breast cancer many of which often result from delayed detection and treatment. So an effective computational approach is needed to develop a predictive model which will help patients and physicians to manage the situation timely. This study presented a Weighted Bayesian Belief Network (WBBN) modeling for breast cancer prediction using the UCI breast cancer dataset. New automated ranking method was used to assign proper weights to attribute value pair based on their impact on causing the disease. Association between attributes was generated using weighted association rule mining between two attributes, multiattributes, and with class labels to generate rules. Weighted Bayesian confidence and weighted Bayesian lift measures were used to produce strong rules to build the model. To build WBBN, the Open Markov tool was used for structure and parametric learning using generated strong rules. The model was trained using 70% records and tested on 30% records with a threshold value of minimum support = 36% and confidence = 70% which produced results with an accuracy of 97.18%. Experimental results show that WBBN achieved better results in most cases compared to other predictive models. The study would contribute to the fight against breast cancer and the quality of treatment.

Protection Degree and Migration in the Stochastic SIRS Model: A Queueing System Perspective

With the prevalence of COVID-19, the modeling of epidemic propagation and its analyses have played a significant role in controlling epidemics. However, individual behaviors, in particular the self-protection and migration, which have a strong influence on epidemic propagation, were always neglected in previous studies. In this paper, we mainly propose two models from the individual and population perspectives. In the first individual model, we introduce the individual protection degree that effectively suppresses the epidemic level as a stochastic variable to the SIRS model. In the alternative population model, an open Markov queueing network is constructed to investigate the individual number of each epidemic state, and we present an evolving population network via the migration of people. Besides, stochastic methods are applied to analyze both models. In various simulations, the infected probability, the number of individuals in each state and its limited distribution are demonstrated.

Open Markov Type Population Models: From Discrete to Continuous Time

We address the problem of finding a natural continuous time Markov type process—in open populations—that best captures the information provided by an open Markov chain in discrete time which is usually the sole possible observation from data. Given the open discrete time Markov chain, we single out two main approaches: In the first one, we consider a calibration procedure of a continuous time Markov process using a transition matrix of a discrete time Markov chain and we show that, when the discrete time transition matrix is embeddable in a continuous time one, the calibration problem has optimal solutions. In the second approach, we consider semi-Markov processes—and open Markov schemes—and we propose a direct extension from the discrete time theory to the continuous time one by using a known structure representation result for semi-Markov processes that decomposes the process as a sum of terms given by the products of the random variables of a discrete time Markov chain by time functions built from an adequate increasing sequence of stopping times.

Open Markov Chains: Cumulant Dynamics, Fluctuations and Correlations.

In this work we propose a model for open Markov chains that can be interpreted as a system of non-interacting particles evolving according to the rules of a Markov chain. The number of particles in the system is not constant, because we allow the particles to arrive or leave the state space according to prescribed protocols. We describe this system by looking at the population of particles on every state by establishing the rules of time-evolution of the distribution of particles. We show that it is possible to describe the distribution of particles over the state space through the corresponding moment generating function. This description is given through the dynamics ruling the behavior of such a moment generating function and we prove that the system is able to attain the stationarity under some conditions. We also show that it is possible to describe the dynamics of the two first cumulants of the distribution of particles, which in some way is a simpler technique to obtain useful information of the open Markov chain for practical purposes. Finally we also study the behavior of the time-dependent correlation functions of the number of particles present in the system. We give some simple examples of open chains that either, can be fully described through the moment generating function or partially described through the exact solution of the cumulant dynamics.

Projecting Long-term Health and Economic Burden of COPD in the United States

In the United States, COPD is a leading cause of mortality, with a substantial societal health and economic burden. With anticipated population growth, it is important for various stakeholders to have an estimate for the projected burden of disease. The goal of this study was to model the 20-year health and economic burden of COPD, from 2019 to 2038, in the United States. Using country-specific data from published literature and publicly available datasets, a dynamic open cohort Markov model was developed in a probabilistic Monte Carlo simulation. Population growth was modeled across different subgroups of age, sex, and smoking. The COPD prevalence rates were calibrated for different subgroups, and distributions of severity grades were modeled based on smoking status. Direct costs, indirect absenteeism costs, losses of quality-adjusted life years (QALYs), and number of exacerbations and deaths associated with COPD were projected. The 20-year discounted direct medical costs attributable to COPD were estimated to be $800.90 billion (95%credible interval [CrI], 565.29 billion-1,081.29 billion), with an expected $337.13 billion in male subjects and $463.77 billion in female subjects. The 20-year discounted indirect absenteeism costs were projected to be $101.30 billion (70.82 billion-137.41 billion). The 20-year losses of QALYs, number of exacerbations, and number of deaths associated with COPD were 45.38 million (8.63 million-112.07 million), 315.08 million (228.59 million-425.33 million), and 9.42 million (8.93 million-9.93 million), respectively. The proportion of disease burden attributable to continued smoking was 34%in direct medical costs, 35%in indirect absenteeism costs, and 37%in losses of QALYs over 20 years. This study projects the substantial burden of COPD that the American society is expected to incur with current patterns for treatments and smoking rates. Mitigating such burden requires targeted budget appropriations and cost-effective interventions.

Encounter Delivery Strategy for Strongly Relational Relay Nodes in MSN

MSN has the characteristics of social network, and the nodes under intellectual consideration show social selfishness, which is characterized by kinship index. We propose an encounter delivery strategy based on strong-relationship relay nodes. Firstly, in the hierarchical communities and clusters based on the kinship index, the active strong relationship node set is obtained through the open strong relationship priority algorithm and Markov prediction. Then establish a kinship trust model, conduct behavior detection on active strong relationship nodes to obtain a comprehensive kinship trust value, and then screen out some of the “false kinship” nodes to screen out a set of trusted active strong relationship nodes. Finally, according to the kinship indication with the destination node, the node’s kinship movement behavior is predicted, and the candidate relay node set is selected. Simulation experiment results show that this strategy can improve the reliability of message transmission, reduce network delay, and improve the efficiency of data packet transmission to a certain extent.

Finding Expected Revenue of Open HM-Network with Limited Waiting Time and Unreliable Queueing Systems

The object of research in this paper is an open Markov queueing network with revenues, limited number of customers, random waiting time for them in the queue, and unreliable queueing systems. The service times of customers in systems and the recovery times of lines in them have an exponential distribution with parameters depending on the number of customers in the systems and the number of service lines in service, and the waiting times in queues and service hours are also distributed according to an exponential law distribution, the parameters of which depend on the number of customers in systems. Using an asymptotic analysis with a large number of customers, the total expected revenue of such network was found. To do this, first derived and solved the differential equation in partial derivatives of the density distribution of revenue, and then received and decided to ordinary differential equation for the expected revenue of the network. A model example is calculated.

Анализ ожидаемых доходов в открытых марковских сетях с различными особенностями

Анализ ожидаемых доходов в открытых марковских сетях с различными особенностями

From ODE to Open Markov Chains, via SDE: an application to models for infections in individuals and populations

Abstract We present a methodology to connect an ordinary differential equation (ODE) model of interacting entities at the individual level, to an open Markov chain (OMC) model of a population of such individuals, via a stochastic differential equation (SDE) intermediate model. The ODE model here presented is formulated as a dynamic change between two regimes; one regime is of mean reverting type and the other is of inverse logistic type. For the general purpose of defining an OMC model for a population of individuals, we associate an Ito processes, in the form of SDE to ODE system of equations, by means of the addition of Gaussian noise terms which may be thought to model non essential characteristics of the phenomena with small and undifferentiated influences. The next step consists on discretizing the SDE and using the discretized trajectories computed by simulation to define transitions of a finite valued Markov chain; for that, the state space of the Ito processes is partitioned according to some rule. For the example proposed for illustration, the state space of the ODE system referred – corresponding to a model of a viral infection – is partitioned into six infection classes determined by some of the critical points of the ODE system; we detail the evolution of some infected population in these infection classes.

Potential impact of diabetes prevention on mortality and future burden of dementia and disability: a modelling study

Aims/hypothesisDiabetes is associated with an increased risk of dementia. We estimated the potential impact of trends in diabetes prevalence upon mortality and the future burden of dementia and disability in England and Wales.MethodsWe used a probabilistic multi-state, open cohort Markov model to integrate observed trends in diabetes, cardiovascular disease and dementia to forecast the occurrence of disability and dementia up to the year 2060. Model input data were taken from the English Longitudinal Study of Ageing, Office for National Statistics vital data and published effect estimates for health-state transition probabilities. The baseline scenario corresponded to recent trends in obesity: a 26% increase in the number of people with diabetes by 2060. This scenario was evaluated against three alternative projected trends in diabetes: increases of 49%, 20% and 7%.ResultsOur results suggest that changes in the trend in diabetes prevalence will lead to changes in mortality and incidence of dementia and disability, which will become visible after 10–15 years. If the relative prevalence of diabetes increases 49% by 2060, expected additional deaths would be approximately 255,000 (95% uncertainty interval [UI] 236,000–272,200), with 85,900 (71,500–101,600) cumulative additional cases of dementia and 104,900 (85,900–125,400) additional cases of disability. With a smaller relative increase in diabetes prevalence (7% increase by 2060), we estimated 222,200 (205,700–237,300) fewer deaths, and 77,000 (64,300–90,800) and 93,300 (76,700–111,400) fewer additional cases of dementia and disability, respectively, than the baseline case of a 26% increase in diabetes.Conclusions/interpretationReducing the burden of diabetes could result in substantial reductions in the incidence of dementia and disability over the medium to long term.

Cheating as a dynamic marketing strategy in monopoly, cartel and duopoly

In 2015 it was discovered that Volkswagen had manipulated the exhaust emissions of its (diesel) cars. Since then, numerous other automotive car manufacturers were strongly suspected to violate against the same emission standards. This paper investigates how and why firms (monopoly, cartel and duopoly) engage in cheating, more precisely, promising attributes that are actually not part of the product. Firms make claims in order to better market their product but risk damaging their future reputation. The upshot of the paper is the stark difference between open loop and Markov perfect oligopolistic equilibrium outcomes. More precisely, the latter mitigates cheating substantially even below the levels attained by monopolies and cartels (unless consumers have a very short memory), which is contrary to the outcome in the limiting static version of the game. Therefore, revealing the true state (e.g., by mandating strict inspections) could force firms to use this information and play in Markov instead of open loop strategies.

On the General Definition of the Production of Entropy in Open Markov Quantum Systems

We discuss the question of the general definition of the production of entropy per unit time for a quantum system governed by the Lindblad equation. The difficulty is as follows: in order to determine the total production of entropy, one must know the entropy flow from the system into the environment. This requires additional information on the environment and on its interaction with the system. The Lindblad equation for the reduced density matrix of the system does not contain such information. Therefore, the following question arises: What minimum additional information about the environment must be added to the Lindblad equation in order to find the flow of entropy into the environment and the total production of entropy? To answer this question, we use the concept of a complementary quantum channel known from the the quantum information theory. We also prove a theorem on the nonnegativity of production of entropy, and, under certain assumptions, the adiabatic and nonadiabatic contribution to it.

FINDING NONSTATIONARY STATE PROBABILITIES OF OPEN MARKOV NETWORKS WITH MULTIPLE CLASSES OF CUSTOMERS AND VARIOUS FEATURES

This paper discusses a system of difference-differential equations (DDE) that is satisfied by the time-dependent state probabilities of open Markov queueing networks with various features. The number of network states in this case and the number of equations in this system is infinite. Flows of customers arriving at the network are a simple and independent, the time of customer services is exponentially distributed. The intensities of transitions between the network states are deterministic functions depending on its states.To solve the system of DDE, we propose a modified method of successive approximations, combined with the method of series. The convergence of successive approximations with time to a stationary probability distribution, the form of which is indicated in the paper has been proved. The sequence of approximations converges to a unique solution of the system of equations. Any successive approximation can be represented as a convergent power series with an infinite radius of convergence, the coefficients of which satisfy recurrence relations, which is convenient for calculations on a computer. Examples of the analysis of Markov G-networks with various features have been presented.

Calculating Expected Incomes in Open Markov Networks with Requests of Different Classes and Different Peculiarities

A system of difference-differential equations for the expected incomes of open Markov queueing networks with different peculiarities is considered. The number of network states and also the number of equations in this system are both infinite. The incoming flows of requests are elementary and independent while their service times have exponential distributions. The incomes from transitions between different states of the network are deterministic functions that depend on its states; the incomes gained by the queuing server systems per unit time under the invariable states also depend on these states only. The system of the difference-differential equations is solved using the modified method of successive approximations combined with the series method. An example of a Markov G-network with signals and the group elimination of positive requests is studied. As demonstrated below, the expected incomes can be increasing and decreasing time-varying functions; can take positive and negative values.

Large deviations for dynamical fluctuations of open Markov processes, with application to random cascades on trees

The large deviations at ‘Level 2.5 in time’ for time-dependent ensemble-empirical-observables, introduced by Maes et al (2008 Markov Proc. Relat. Fields 14 445) for the case of independent Markov jump processes, are extended to the case of open Markov processes with reservoirs: explicit formulas are given for the joint probability of empirical occupation numbers and empirical flows, both for discrete-time dynamics and for continuous-time jump dynamics, with possibly time-dependent dynamical rules and/or time-dependent driving of the reservoirs. This general formalism is then applied to random cascades on trees, where particles are injected at the root via a ‘source reservoir’, while the particles are removed at the leaves of the last generation of the tree via ‘sink reservoirs’.

Application possibilities of LBN for civil engineering issues

Bayesian Networks (BN) are efficient to represent knowledge and for the reasoning in uncertainty. However the classic BN requires manual definition of the network structure by an expert, who also defines the values entered into the conditional probability tables. In practice, it can be time-consuming, hence the article proposes the use of Learning Bayesian Networks (LBN). The aim of the study is not only to present LBN, which can be helpful in civil engineering problems, but also to analyze and evaluate the potential of a selected software. Based on a real example the functional values of the Open Markov, Hugin and AgenaRisk applications were compared.

Открытые марковские сети массового обслуживания с контрольными очередями и карантинным узлом

Открытые марковские сети массового обслуживания с контрольными очередями и карантинным узлом