Stationary Distribution Of Markov Chain Research Articles

Efficient Computation of Optimal Thresholds in Cloud Auto-scaling Systems

We consider a horizontal and dynamic auto-scaling technique in a cloud system where virtual machines hosted on a physical node are turned on and off to minimise energy consumption while meeting performance requirements. Finding cloud management policies that adapt the system to the load is not straightforward, and we consider here that virtual machines are turned on and off depending on queue load thresholds. We want to compute the optimal threshold values that minimize consumption costs and penalty costs (when performance requirements are not met). To solve this problem, we propose several optimisation methods, based on two different mathematical approaches. The first one is based on queueing theory and uses local search heuristics coupled with the stationary distributions of Markov chains. The second approach tackles the problem using Markov Decision Process (MDP) in which we assume that the policy is of a special multi-threshold type called hysteresis. We improve the heuristics of the former approach with the aggregation of Markov chains and queues approximation techniques. We assess the benefit of threshold-aware algorithms for solving MDPs. Then we carry out theoretical analyzes of the two approaches. We also compare them numerically and we show that all of the presented MDP algorithms strongly outperform the local search heuristics. Finally, we propose a cost model for a real scenario of a cloud system to apply our optimisation algorithms and to show their practical relevance. The major scientific contribution of the article is a set of fast (almost in real time) load-based threshold computation methods that can be used by a cloud provider to optimize its financial costs.

Long time behaviour for population model by [formula omitted]-stable processes with Markov switching

In this paper, we focus on extinction and ergodicity for the n-dimensional population model driven by α-stable processes with Markov switching. Some sufficient conditions for extinction or ergodicity are given. Our conditions disclose the parameter α and stationary distribution of Markov chain to the impact on the extinction or ergodicity.

APPLICATION OF POWER NUMERICAL METHOD FOR THE STATIONARY DISTRIBUTION OF MARKOV CHAIN

The evolution of this model is represented by transitions from one state to the next. Also, the physical or mathematical behavior of this system can also be illustrated by identifying all of the possible states and explaining how it transitions between them. The iterative solution approaches for the stationary distribution of Markov chains, which begin with an initial estimate of the solution vector and it becomes closer and closer to the true solution with each iteration are investigated. Our goal is to compute solutions of stationary distribution of Markov chain by utilizing the power iterative method which leaves the transition matrices unchanged and saves time by considering the discretization effect, and the convergency. Matrices operations such as multiplication with one or more vectors, lower, diagonal and upper concepts of matrix, with the help of several existing Markov chain laws, theorems, formulas, and the normalization principle are applied. For the illustrative examples, the stationary distribution vectors and table of convergence are obtained.

Application of Jacobi and Gauss–Seidel Numerical Iterative Solution Methods for the Stationary Distribution of Markov Chain

The Physical or Mathematical behaviour of this model may be represented by describing all the different states it may occupy and by indicating how it moves among these states. In this study, the stationary distribution of Markov chains was solved using iterative methods that begin with an initial estimate of the solution vector and then modified it in a way that brings it closer and closer to the real solution with each step or iteration. These methods also involved matrix operations like multiplication with one or more vectors, which preserves the transition matrices while speeding up the process. We computed the solutions using Jacobi iterative method and Gauss-Seidel iterative method in order to shed more light on the solutions of stationary distribution in Markov chain. This was done with the aid of several already-existing laws, theorems, and formulas of Markov chain and the application of normalization principle and matrix operations such as lower, upper, and diagonal matrices. The stationary distribution vector’s 𝜋𝑖,𝑖=1,2,…,4 are obtained for the illustrative example one as 𝜋(3) = (0.078125,0.109375,0.21875,0.59375) as well as the four eigenvalues of the matrix as 𝜆1=1.0, 𝜆2=−0.7718, 𝜆3,4=−0.1141±0.5576𝑖 using Jacobi iterative technique, and for illustrative example two using Gauss-Siedel method as 𝝅(𝟑) = (0.090909, 0.181818, 0.363636, 0.363636). The research shown that Gauss Siedel method converged faster than Jacobi method.

F-TPE: Flexible Thumbnail-Preserving Encryption Based on Multi-Pixel Sum-Preserving Encryption

People are accustomed to utilizing mobile phones to capture images and uploading them to the cloud due to various incomparable advantages such as saving the storage space on the device. In this context, privacy concerns are raised since images may contain some sensitive information. Encrypting images by traditional schemes can alleviate privacy leak, but the usability is often deprived since it is tough to browse them on the cloud. Recently, Tajik <i>et al.</i> proposed an ideal <i>thumbnail-preserving encryption</i> (TPE) scheme to achieve the balance of usability and privacy by leveraging a two-pixel substitution encryption method while the Markov chain is utilized to prove the security. However, the connectivity of Markov chain in this scheme is weak since the length of pixel groups in the encryption process is only two, leading to a long mixing time of achieving the stationary distribution of Markov chain. To this end, we firstly propose a method of <i>multi-pixel sum-preserving encryption</i> (MP-SPE) that realizes the encipherment of vectors of arbitrary length. Then, with the help of MP-SPE, a novel flexible ideal TPE scheme (F-TPE) is designed and the connection of Markov chain is improved. The experiments have demonstrated that the proposed scheme can effectively attain the balance between usability and privacy. In addition, F-TPE takes much less time in encrypting compared with the existing work.

Assessing the impact of promotion algorithms in social networks on user popularity

In the modern world, social networks are one of the main tools for interaction between people and are becoming more and more popular. Therefore, it is important to understand how the algorithms of social networks and the structural and dynamic aspects of user behavior affect their popularity. The study is based on graph theory and the Frobenius – Perron theory of non-negative matrix decomposition using the stationary distribution of Markov chains. As a result, the behavior of the model over a long period of time is shown and appropriate conclusions are made about the distribution and popularity of users in social networks.

On the Application of Successive Over-relaxation Algorithmic and Block Numerical Iterative Solutions for the Stationary Distribution in Markov Chain

The evolution of a system is represented by transitions from one state to the next, and the system's physical or mathematical behavior can also be depicted by defining all of the numerous states it can be in and demonstrating how it moves between them. In this study, the iterative solution methods for the stationary distribution of Markov chains were investigated, which start with an initial estimate of the solution vector and then alter it in such a way that it gets closer and closer to the genuine solution with each step or iteration., and also involved matrices operations such as multiplication with one or more vectors, which leaves the transition matrices unchanged and saves time. Our goal is to use Successive Overrelaxation Algorithmic and Block Numerical Iterative Solution Methods to compute the solutions. With the help of some existing Markov chain laws, theorems, and formulas, the normalization principle and matric operations such as lower, upper, and diagonal matrices are used. The stationary distribution vector’s π^((k+1) )={■(π_1^((k+1) )&amp;π_2^((k+1) )&amp;■(π_3^((k+1) )&amp;π_4^((k+1) )&amp;π_5^((k+1) ) )),k=0,1,2,…,n} are obtained for the illustrative examples, taken the initial stationary solution to be π^((0) )=(■(0.2&amp;0.2&amp;■(0.2&amp;0.2&amp;0.2)))^T and it was observed that all subsequent iterations yield exactly the same result as π^((1) ), and this shows that, the block iterative method requires only a single iteration to obtain the solution to full machine precision.

An Algebraic Approach to Product-form Stationary Distributions for Some Reaction Networks

Exact results for product-form stationary distributions of Markov chains are of interest in different fields. In stochastic reaction networks (CRNs), stationary distributions are mostly known in special cases where they are of product-form. However, there is no full characterization of the classes of networks whose stationary distributions have product-form. We develop an algebraic approach to product-form stationary distributions in the framework of CRNs. Under certain hypotheses on linearity and decomposition of the state space for conservative CRNs, this gives sufficient and necessary algebraic conditions for product-form stationary distributions. Correspondingly, we obtain a semialgebraic subset of the parameter space that captures rates where, under the corresponding hypotheses, CRNs have product-form. We employ the developed theory to CRNs and some models of statistical mechanics, besides sketching the pertinence in other models from applied probability.

Mean square exponential stability of discrete-time Markov switched stochastic neural networks with partially unstable subsystems and mixed delays

In this paper, we study the mean square exponential stability of discrete-time stochastic neural networks with partially unstable subsystems and mixed delays. The mixed delays under consideration involve discrete delay and distributed delay. Moreover, the discrete delay term satisfies the Bernoulli distribution. Different from the deterministic switching, we consider Markov switching and our system has partially unstable subsystems. By constructing a novel Lyapunov–Krasovskii functional and using the stationary distribution of Markov chain, we give sufficient conditions for the mean square exponential stability of the suggested system. Finally, two numerical examples are given to check the theory results.

Markov chain to analyze web usability of a university website using eye tracking data

AbstractWeb usability is a crucial feature of a website, allowing users to easily find information in a short time. Eye tracking data registered during the execution of tasks allow to measure web usability in a more objective way compared to questionnaires. In this work, we evaluated the web usability of the website of the University of Cagliari through the analysis of eye tracking data with qualitative and quantitative methods. Performances of two groups of students (i.e., high school and university students) across 10 different tasks were compared in terms of time to completion, number of fixations and difficulty ratio. Transitions between different areas of interest (AOI) were analyzed in the two groups using Markov chain. For the majority of tasks, we did not observe significant differences in the performances of the two groups, suggesting that the information needed to complete the tasks could easily be retrieved by students with little previous experience in using the website. For a specific task, high school students showed a worse performance based on the number of fixations and a different Markov chain stationary distribution compared to university students. These results allowed to highlight elements of the pages that can be modified to improve web usability.

Sensitivity of the stationary distributions of denumerable Markov chains

In this note we present new perturbation bounds for the stationary distribution of Markov chains with denumerable state space. Bounds are obtained using the Dobrushin’s coefficient δ(P) and the ergodic coefficient α(P). Bounds are shown to be relevant when α(P)<δ(P). In addition we present bounds only involving the coefficient α(P).

Link prediction of time-evolving network based on node ranking

Many real-world networks belong to the kind that evolves over time. So it is very meaningful and challenging to predict whether the link will occur in the network of future time. In this paper, both time-evolving scale-free (SF) network and real-world dynamic network are taken into consideration first and then two kinds of methods are respectively proposed for link prediction. Different from many existing similarity-based dynamic network link prediction methods, many of which adopt node-pair similarity such as common neighbors (CN), Adamic–Adar (AA), and so on, we measure the similarity between nodes from a new perspective. With further research into node ranking, some eigenvector-based methods, such as PageRank (PR), Cumulative Nomination (CuN) and so on, can compute the values of node importance which can be regarded as the stationary distribution of Markov chain for all nodes iteratively. Therefore, from a statistical point of view, the importance of a node is like the probability of attracting other nodes to connect with it and the derivative value of a node pair is like the probability of attracting each other. These node-ranking-based approaches are very novel in the field of link prediction in that few researches have paid enough attention to them before. In addition, an adaptively time series forecasting method is proposed in this paper, and it uses the historical similarity series to predict the future similarity between each node pair adaptively. Experimental results show that our proposed algorithms can predict the future links not only for the growing SF network but also for the dynamic networks in the real-world.

Stein’s method for stationary distributions of Markov chains and application to Ising models

We develop a new technique, based on Stein’s method, for comparing two stationary distributions of irreducible Markov chains whose update rules are close in a certain sense. We apply this technique to compare Ising models on $d$-regular expander graphs to the Curie–Weiss model (complete graph) in terms of pairwise correlations and more generally $k$th order moments. Concretely, we show that $d$-regular Ramanujan graphs approximate the $k$th order moments of the Curie–Weiss model to within average error $k/\sqrt{d}$ (averaged over size $k$ subsets), independent of graph size. The result applies even in the low-temperature regime; we also derive simpler approximation results for functionals of Ising models that hold only at high temperatures.

Learning Policies for Markov Decision Processes From Data

We consider the problem of learning a policy for a Markov decision process consistent with data captured on the state-action pairs followed by the policy. We parameterize the policy using features associated with the state-action pairs. The features can be handcrafted or defined using kernel functions in a reproducing kernel Hilbert space. In either case, the set of features can be large and only a small, unknown subset may need to be used to fit a specific policy to the data. The parameters of such a policy are recovered using $\ell _1$ -regularized logistic regression. We establish bounds on the difference between the average reward of the estimated and the unknown original policies (regret) in terms of the generalization error and the ergodic coefficient of the underlying Markov chain. To that end, we combine sample complexity theory and sensitivity analysis of the stationary distribution of Markov chains. Our analysis suggests that to achieve regret within order $O(\sqrt{\epsilon })$ , it suffices to use training sample size of the order of $\Omega (\log n \cdot \text{poly}(1/\epsilon))$ , where $n$ is the number of the features. We demonstrate the effectiveness of our method on a synthetic robot navigation example.

EasyLB: Adaptive Load Balancing Based on Flowlet Switching for Wireless Sensor Networks.

Load balancing is effective in reducing network congestion and improving network throughput in wireless sensor networks (WSNs). Due to the fluctuation of wireless channels, traditional schemes achieving load balancing in WSNs need to maintain global or local congestion information, which turn out to be complicated to implement. In this paper, we design a flowlet switching based load balancing scheme, called EasyLB, by extending OpenFlow protocol. Flowlet switching is efficient to achieve adaptive load balancing in WSNs. Nevertheless, one tricky problem lies in determining the flowlet timeout value, . Setting it too small would risk reordering issue, while setting it too large would reduce flowlet opportunities. By formulating the timeout setting problem with a stationary distribution of Markov chain, we give a theoretical reference for setting an appropriate timeout value in flowlet switching based load balancing scheme. Moreover, non-equal probability path selection and multiple parallel load balancing paths are considered in timeout setting problem. Experimental results show that, by setting timeout value following the preceding theoretical reference, EasyLB is adaptive to wireless channel condition change and achieves fast convergence of load balancing after link failures.

Taylor series expansion approach for epistemic uncertainty propagation in queueing‐inventory models

In this paper, we provide an uncertainty analysis for queueing‐inventory models, by extending the multivariate Taylor series expansion methodology to such stochastic models. Specifically, we derive a closed‐form expressions for the higher‐order sensitivity of discrete‐time Markov chain stationary distribution with respect to multiple parameter. We establish efficient bound on the remainder term corresponding to the multivariate Taylor series. Additionally, we estimate different quantities of interest of the output measures of the studied queueing‐inventory model. Using the copulas theory, we also include the effect of the dependence structure of parameters. The efficacy of the proposed method is shown with several numerical examples and obtained numerical results are compared with those of Monte Carlo simulation.

Algebraic multigrid for directed graph Laplacian linear systems (NS‐LAMG)

SummaryWe propose nonsymmetric lean algebraic multigrid (NS‐LAMG), a new algebraic multigrid algorithm for directed graph Laplacian systems that combines ideas from undirected graph Laplacian multigrid solvers and multigrid algorithms for Markov chain stationary distribution systems. Low‐degree elimination, proposed in LAMG for undirected graphs, is generalized to directed graphs and is a key component of NS‐LAMG. In the setup phase, we propose a simple stationary‐aggregation multigrid algorithms for Markov chain stationary distribution systems solver enhanced by low‐degree elimination to find the right null‐space vector that is used for the intergrid transfer operators. Numerical results show that low‐degree elimination improves performance and that NS‐LAMG outperforms generalized minimal residual method with restart and stable bi‐conjugate gradient method for real‐world, directed graph Laplacian linear systems.

Block-accelerated aggregation multigrid for Markov chains with application to PageRank problems

Recently, the adaptive algebraic aggregation multigrid method has been proposed for computing stationary distributions of Markov chains. This method updates aggregates on every iterative cycle to keep high accuracies of coarse-level corrections. Accordingly, its fast convergence rate is well guaranteed, but often a large proportion of time is cost by aggregation processes. In this paper, we show that the aggregates on each level in this method can be utilized to transfer the probability equation of that level into a block linear system. Then we propose a Block–Jacobi relaxation that deals with the block system on each level to smooth error. Some theoretical analysis of this technique is presented, meanwhile it is also adapted to solve PageRank problems. The purpose of this technique is to accelerate the adaptive aggregation multigrid method and its variants for solving Markov chains and PageRank problems. It also attempts to shed some light on new solutions for making aggregation processes more cost-effective for aggregation multigrid methods. Numerical experiments are presented to illustrate the effectiveness of this technique.

Stability and Criticality Analysis for Integer Linear Programs With Markovian Problem Data

This paper presents the stability and criticality analysis of integer linear programs with respect to perturbations in stochastic data given as Markov chains. These perturbations affect the initial distribution, the transition matrix, or the stationary distribution of Markov chains. Stability analysis is concerned with obtaining the set of all perturbations for which a solution remains optimal. This paper gives expressions for stability regions for perturbations in the initial distribution, the transition matrix, the stationary distribution, and the product of elements of the transition matrix and the stationary distribution. Furthermore, criticality measures that describe the sensitivity of the objective function with respect to an element of the problem data are derived. Stability regions that preserve the stochasticity of the problem data are given. Finally, stability regions for perturbations of elements of the transition matrix, given that the problem is not linear in the initial distribution or the transition matrix, are obtained using a small perturbation analysis. The results are applied to sensor placement problems and numerical examples are given.

A site entropy rate and degree centrality based algorithm for image co-segmentation

In this paper, we propose a graph based algorithm that efficiently segments common objects from multiple images. We first generate a number of object proposals from each image. Then, an undirected graph is constructed based on proposal similarities and co-saliency maps. Two different methods are followed to extract the proposals containing common objects. They are: (1) degree centrality of nodes obtained after graph thresholding and (2) site entropy rate of nodes calculated on the stationary distribution of Markov chain constructed on the graph. Finally, we obtain the co-segmented image region by selecting the more salient of the object proposals obtained by the two methods, for each image. Multiple instances of the common object are also segmented efficiently. The proposed method has been compared with many existing co-segmentation methods on three standard co-segmentation datasets. Experimental results show its effectiveness in co-segmentation, with larger IoU values as compared to other co-segmentation methods.