Transition Rate Matrix Research Articles

A traverse algorithm approach to stochastic stability analysis of Markovian jump systems with unknown and uncertain transition rates

This paper intents to investigate the problem of mean-square stability analysis of Markovian jump systems with generally unknown and uncertain transition rates. Different from pervious works that the transition rates from one mode to others may be partially unknown or uncertain, in this note, the case that the transition rates from one mode to others are totally unknown will be investigated. By means of transition rate estimation, two ways are provided to tackle with the totally unknown case. In general, five cases in the transition rates matrix are studied for the mean-square stability analysis, which almost have covered all types of generally unknown and uncertain transition rates. Simultaneously, corresponding conditions for checking the mean-square stability of the considered Markovian jump systems are developed for the five studied cases. Finally, numerical examples are provided to verify the effectiveness of the proposed results.

Absorption kinetics of vacancies by cavities in aluminum: Numerical characterization of sink strengths and first-passage statistics through Krylov subspace projection and eigenvalue deflation

Modeling the microstructural evolution of metal and alloys, specifically under irradiation, is essential to predict the aging properties of materials. Many models are based on a transition rate matrix describing the jump frequencies of defects and involve a master equation governing the time evolution of a state probability vector. Here, we present non-stochastic numerical techniques to characterize the motion of individual defects migrating over long distances prior to recombining or being absorbed by another defect, resorting to the theory of absorbing Markov chains. These important events are fully determined by their first-passage time distribution to distant locations, no-passage distribution, and walker fluxes to the sinks. We show that these functions can be efficiently computed using a method combining Krylov subspace projection and eigenvalue deflation. For a model system describing the absorption of a vacancy by a cavity in aluminum, the use of a small Krylov subspace deflated by the unique eigenmode corresponding to the quasi-stationary distribution is sufficient to capture the kinetics of the defect absorption faithfully. This method can be used in kinetic Monte Carlo simulations to perform stochastic non-local moves or in cluster dynamics simulations to compute sink strengths.

Performance analysis and system optimization of an energy-saving mechanism in cloud computing with correlated traffic

<p style='text-indent:20px;'>Energy consumption is becoming a significant part of overall operational cost in cloud data centers. For the purpose of satisfying the Service Level Agreement (SLA) of cloud users while enhancing the energy efficiency in cloud computing systems, in this paper we propose an energy-saving mechanism with a sleep mode. Taking into consideration the traffic's correlation and the stochastical behavior of data arrival requests in a random cloud environment with the proposed energy-saving mechanism, we model the system as a MAP/M/<inline-formula><tex-math id="M1">\begin{document}$ N $\end{document}</tex-math></inline-formula>/<inline-formula><tex-math id="M2">\begin{document}$ N $\end{document}</tex-math></inline-formula>+<inline-formula><tex-math id="M3">\begin{document}$ K $\end{document}</tex-math></inline-formula> queue with a synchronous multi-vacation. Then, we present a theoretical basis for analyzing and evaluating the system performance by taking a state transition rate matrix in the steady state. Next, we investigate the change trends for the energy saving rate of the system and the average latency of tasks by carrying out numerical experiments. Moreover, we give a</p>

Reliability modelling for linear and circular [formula omitted]-out-of-[formula omitted]: F systems with shared components

Reliability analysis for k-out-of-n systems has received much attention in the field of reliability due to its practical importance. In this work, models for linear and circular k-out-of-n: F systems with shared components, consisting of m subsystems, are discussed. The finite Markov chain imbedding approach cannot be used directly to derive reliability of the considered systems because of the common area existing between adjacent subsystems. For this reason, we first derive the probability that the number of failed components is a given value in the m common areas. Then, all disjoint cases, each of which is regarded as a series system consisting of m subsystems with different initial state probability distributions, are obtained, with which the nith step transition rate matrix is calculated by using the Markov chain technique, for i=1,2,…,m. Finally, the reliability of the system is obtained by summing the reliabilities of all cases along with the finite Markov chain imbedding approach. Some numerical examples are presented to illustrate the efficiency of the proposed reliability evaluation method.

Aperiodic sampled-data controller design for switched Itô stochastic Markovian jump systems

In this paper, aperiodic sampled-data control for a class of Itô stochastic Markovian jump systems with switching transition rate matrix containing partially known elements is investigated. By introducing a loop-based Lyapunov functional, mean square exponential stability criteria are presented and mode-dependent aperiodic sampled-data controllers are designed. Due to the constraint on the upper bound of the sampling interval is no greater than the dwell time, the transition rate matrix may switch in a sampling interval. The issue concerning the asynchronization between the sampled-data controller mode and the system mode is hence considered. A practical RLC electrical circuit example is presented to illustrate the effectiveness of the method.

NB-Cache: Non-Blocking In-Network Caching for High-Performance Content Routers

Information-Centric Networking (ICN) provides scalable and efficient content distribution at the Internet scale due to in-network caching and native multicast. To support these features, a content router needs high performance at its data plane, which consists of three forwarding steps: checking the Content Store (CS), then the Pending Interest Table (PIT), and finally the Forwarding Information Base (FIB). In this work, we build an analytical model of the router and identify that CS is the actual bottleneck. Then, we propose a novel mechanism called “NB-Cache” to address CS’s performance issue from a network-wide point of view. In NB-Cache, when packets arrive at a router whose CS is fully loaded, instead of being blocked and waiting for the CS, these packets are forwarded to the next-hop router, whose CS may not be fully loaded. This approach essentially utilizes Content Stores of all the routers along the forwarding path in parallel rather than checking each CS sequentially. NB-Cache follows a design pattern of on-demand load balancing and can be formulated into a non-trivial N-queue bypass model. We use the Markov chain to establish its theoretical base and find an algorithm for automated transition rate matrix generation. Experiments show significant improvement of data plane performance: 70% reduction in round-trip time (RTT) and 130% increase in throughput. NB-Cache decouples the fast packet forwarding from the slower content retrieval thus substantially reducing CS’s heavy dependency on fast but expensive memory.

Time-dependent reliability analysis for repairable consecutive-k-out-of-n:F system

In a repairable consecutive system, after the system operates for a certain time, some components may fail, some failed components may be repaired and the state of the system may change. The models developed in the existing literature usually assume that the state of the system varies over time depending on the values of n and k and the state of the system is known. Since the system reliability will vary over time, it is of great interest to analyse the time-dependent system reliability. In this paper, we develop a novel and simple method that utilizes the eigenvalues of the transition rate matrix of the system for the computation of time-dependent system reliability when the system state is known. In addition, the transition performance probabilities of the system from a known state to the possible states are also analysed. Computational results are presented to illustrate the applicability and accuracy of the proposed method.

Exact analysis of production lines with Coxian-2-distributed processing times and parallel machines

This study considers production lines where each station has parallel machines with 2-phase Coxian processing times. The setting in this paper is designed specifically for scrutiny of the replenishment of raw materials and finished goods inventories with intermediate buffers in-between stations. Each buffer has a limit of capacity. Raw material supply and demand for finished goods are generated according to independent stationary Poisson processes. Coxian-2 processing times can be utilized to model failure-prone machines with exponential service times, times to failure, and repair times. The second phase of Coxian-2 can also be considered as a rework operation visited with a predefined probability. We model the line as a continuous-time Markov chain and propose recursive algorithms to generate the transition rate matrix. Although the general recursive form is specific to 3-station 4-buffer lines, routines for calculating the number of states and generating the states work for any M−station (M + 1)-buffer systems. The developed model allows obtaining steady-state distribution and performance metrics such as throughput, the average number of items in the buffers, and average system cost consisting of production, holding, and shortage costs. Furthermore, we enrich our study with numerical experiments and analyze the impacts of buffer capacities, processing rates of the machines and the number of parallel machines on the system performance. Moreover, the exact analysis provided in this paper can also be used as the decomposition block for the performance analysis of longer lines.

Condition-based maintenance planning for multi-state systems under time-varying environmental conditions

Systems commonly operate under time-varying environmental condition (EC). Since the EC can affect the degradation process of a system, maintenance planning can be quite challenging. This study focuses on a case where the evolution processes of the system degradation and the EC are governed by Markov processes, for which the transition rate matrix of degradation states varies with the EC. To maintain the system, an inspection/maintenance policy is adopted, where a maintenance action is carried out immediately when a failure occurs during an inspection interval or upon an inspection epoch when the system’s degradation state reaches a certain threshold. Imperfect maintenance (IM) and replacement are considered in this study, where a certain number of consecutive IM actions can be carried out before each replacement. We first derive the long-run average cost based on the semi-regenerative properties of the system, and then jointly determine the inspection interval, preventive maintenance threshold and number of IM actions before a replacement by minimizing the long-run average cost. A numerical study is conducted to demonstrate the proposed maintenance policy.

Fuzzy-model-based tracking control of Markov jump nonlinear systems with incomplete mode information

This paper considers the tracking control problem for nonlinear Markov jump systems based on T–S fuzzy model approach with incomplete mode information. It is assumed that the mode transition rate matrix is not a priori knowledge and only partial information is available. Moreover, the mode where the system stays when operating is not fully accessible to the designed controller. In this incomplete mode information scenario, a hidden Markov model based mechanism is modified to simulate the mode deficiency mapping. The incomplete transition rate matrix is well defined in the form of a polynomial. Based on this, by constructing a polynomially parameter-dependent Lyapunov matrices and linear matrix techniques, sufficient conditions are established to ensure the stochastic stability and a prescribed tracking performance. The controller design scheme are presented by solving a series of LMIs. Examples are given in the end to illustrate the effectiveness of our proposed results.

On scale functions for Lévy processes with negative phase-type jumps

We provide a novel expression of the scale function for a Lévy process with negative phase-type jumps. It is in terms of a certain transition rate matrix which is explicit up to a single positive number. A monotone iterative scheme for the calculation of the latter is presented and it is shown that the error decays exponentially fast. Our numerical examples suggest that this algorithm allows us to employ phase-type distributions with hundreds of phases, which is problematic when using the known formula for the scale function in terms of roots. Extensions to other distributions, such as matrix-exponential and infinite-dimensional phase-type, can be anticipated.

Optimal inventory control policies of a two-stage push–pull production inventory system with lost sales under stochastic production, transportation, and external demand

An analytical model based on Markov processes is proposed for the analysis of a linear, horizontally integrated, two stages, push–pull inventory system. Uncertainty about both supply and demand is taken into consideration. Exponentially distributed lead times, compound Poisson external demand and lost sales are assumed. An algorithm that creates the infinitesimal generator matrix of the system is developed and an exact numerical solution of the system performance measures is also provided. The proposed model can be either used to evaluate what if scenarios exploring the behavior of the system or to optimize performance measures of the considered system. As an example, the model is used to analyze and get insights of the behavior of a supply–demand balanced system.

Asymptotical Stability and Stabilization of Continuous-Time Probabilistic Logic Networks

Discrete-time probabilistic logic networks (DT-PLNs), of which probabilistic Boolean networks (PBNs) are a special type, are an important qualitative model for gene regulatory networks (GRNs). Although a DT-PLN can predict the long-term behavior of a GRN, using it to describe the transient kinetics at the microtimescale level remains inconvenient. In this article, we investigate the problems associated with the stability and stabilization of continuous-time probabilistic logic networks (CT-PLNs). First, we demonstrate that the concept of finite-time stability for DT-PLNs cannot be extended to CT-PLNs owing to the nonsingularity of transitional probability matrices. Thus, we introduce the concept of asymptotical stability, which is defined as the convergence in distribution of the network state. Second, by developing the theory of invariant subsets for CT-PLNs, a necessary and sufficient condition for asymptotical stability with respect to a subset is proposed, which is expressed in terms of the transition rate matrix of probability. Third, for a CT-PLN with input nodes, termed a continuous-time probabilistic logic control network, we discuss the subsets that are invariant under piece-wise constant input. Based thereupon, we propose a necessary and sufficient condition under which asymptotically stabilizing sampled-data feedback exists. A method for designing sampled-data feedback is proposed. Last, Monte Carlo simulation algorithms are proposed to efficiently simulate a CT-PLN in the time domain. Examples are provided to demonstrate the methods proposed in this article.

On reliability analysis of a load-sharing k-out-of-n: G system with interacting Markov subsystems

Dependency plays a key role in the design stage of multi-component manufacturing systems. To capture stochastic dependencies between subsystems, this paper considers a load-sharing -out-of-: G system which contains non-identical multi-state subsystems. The state evolution of each subsystem follows a continuous-time homogeneous Markov chain. Most existing research on -out-of-: G systems focuses on independent subsystems. However, a subsystem failure usually leads to a higher failure rate for each surviving subsystem within the system. This paper aims to study the interaction among subsystems and analyse the system reliability performance. The transition rate matrix for a given subsystem is assumed to depend on the total number of the remaining surviving subsystems, which can be represented as the sum of a baseline transition rate matrix and a stochastic dependency matrix. This paper proves that the transition rate matrix of the system is the generalised Kronecker sum of transition rate matrices of subsystems, and develops an explicit method to calculate the generalised Kronecker sum. The theory of aggregated stochastic processes is employed to obtain closed-form formulas for reliability indexes. A case study of multi-engine aircraft systems is provided where numerical examples are given to illustrate the developed model and obtained results.

AN IMPROVEMENT OF MARKOVIAN INTEGRATION BY PARTS FORMULA AND APPLICATION TO SENSITIVITY COMPUTATION

This paper establishes a new version of integration by parts formula of Markov chains for sensitivity computation, under much lower restrictions than the existing researches. Our approach is more fundamental and applicable without using Girsanov theorem or Malliavin calculus as did by past papers. Numerically, we apply this formula to compute sensitivity regarding the transition rate matrix and compare with a recent research by an IPA (infinitesimal perturbation analysis) method and other approaches.

Fault Estimation for Continuous-Time Markovian Jump Systems: An Auxiliary System Approach

This paper addresses the fault estimation problem for continuous-time Markovian jump systems (MJSs). Because the system is stochastic but the estimated fault is deterministic, it is not easy to study the fault estimation problem directly. To overcome this contradiction, an auxiliary system approach is proposed, where a deterministic intermediate variable is constructed by taking expectation on some stochastic variables and the error state and fault are also estimated. Moreover, based on some novel enlarging techniques, more general estimation situations about partially unknown transition rate matrix and semi-Markovian switching are firstly considered. Linear matrix inequality (LMI) conditions are given to guarantee the error states uniformly bounded. Two numerical examples are presented to illustrate the utility and advantage of established methods.

Taboo rate and hitting time distribution of continuous-time reversible Markov chains

The taboo rate is first defined, which satisfies with the Chapman–Kolmogorov equation. Then the differentials of hitting time distribution are expressed by many different taboo rates, which deeply reveal the intrinsic relationship between the transition rate matrix and the hitting time distribution in continuous-time reversible Markov chains. As an example, the explicit expressions of the differentials of the hitting time distribution at a single state are provided for the birth and death chain, hence the transition rate matrix can be identified. Such differentials improve the theory of statistical identification of continuous-time reversible Markov chains.

Two-commodity queueing-inventory system with two classes of customers

We analyze a two-commodity queueing-inventory system with an individual ordering policy. The maximum storage capacity for the ith commodity is $$S_i$$ $$(i=1,2)$$ . The reorder level for ith commodity is fixed as $$s_i$$ and whenever the inventory level of ith commodity falls on $$s_i$$ an order for $$Q_i\,(=S_i-s_i)$$ items ( $$i=1,2$$ ) is placed for that commodity irrespective of the inventory level of the other commodity. There are two types of customers, which are classified as a priority (Type-1) and ordinary (Type-2) customers. Priority customers demand only commodity-1, whereas ordinary customers demand only commodity-2. Each customer class arrives according to an independent Poisson process with different rates. Service time for each customer class is also independent of each other and follows an exponential distribution. Type-1 customers have non-preemptive priority over Type-2 customers. When the server is idle and there are both types of customers, then the service is offered to Type-1 customers. Type-2 customers have got service when there are no Type-1 customers waiting. It is assumed that waiting space for the priority customers is finite whereas there is no queue capacity for ordinary customers. The system is formulated by a five-dimensional continuous-time Markov chain. The structure of the infinitesimal generator matrix is shown to be of the QBD type. Steady-state distribution is obtained using the matrix-geometric method. The system load is formulated in a closed-form. A comprehensive numerical study is performed on the performance measures. Finally, an optimization study is presented.

Improved Stabilization Results for Markovian Switching CVNNs with Partly Unknown Transition Rates

In this paper, stochastic stability and stabilization problems are investigated for the Markovian switching complex-valued neural networks with mixed delays, where the transition rates (TRs) of the Markov chain are partly unknown, which might reflect more the realistic dynamical behaviors of the neural networks. On the basis of the Lyapunov stability theory and the stochastic analysis method as well as the properties of the TR matrix, several mode-dependent criteria are derived to guarantee the considered complex-valued network to be globally asymptotically stable in mean-square sense. Then, by proposing an appropriate mode-dependent controller, stabilization conditions in terms of matrix inequalities are derived to guarantee the closed-loop system to be stochastically mean-square stable. Finally, two simulation examples are presented to illustrate the effectiveness of the proposed theoretical results.

On eigenvalue bounds for the finite-state birth-death process intensity matrix

The paper sets forth a novel eigenvalue interlacing property across the finite-state birth-death process intensity matrix and two clearly identified submatrices as an extension of Cauchy’s interlace theorem for Hermitian matrix eigenvalues. A supplemental proof involving an examination of probabilities acquired from specific movements across states and a derivation of a form for the eigenpolynomial of the matrix through convolution and Laplace transform is then presented towards uncovering a similar characteristic for the general Markov chain transition rate matrix. Consequently, the proposition generates bounds for each eigenvalue of the original matrix, easing numerical computation. To conclude, the applicability of the property to some real square matrices upon transformation is explored.