State Space Truncation Research Articles

Dimensioning the pending interest table in content-centric networks

In chunk-based interest-driven content-centric networks, each interest packet forwarded upstream by a node on a face implies the return of at most one data chunk to the node from that face shortly after. As a consequence, the congestion of the downstream transmission buffer in the data path of the node is highly correlated with the occupancy of the pending interest table (PIT). Therefore a systematic study and analysis of the PIT occupancy are of paramount importance to understanding congestion in CCN. In particular, in this paper, we propose an analytical model to estimate the PIT occupancy distribution via a continuous-time Markov chain (CTMC) model that considers the effects of interest blocking, interest timeout and retries. To validate our model and assumptions, we invoke simulation with realistic traffic streams and show how the filtering effects of caching and interest aggregation make the Markov assumption reasonable in the nodes that are the most susceptible to experiencing congestion. To solve the model numerically, we use two alternative approximations, state space truncation and state aggregation, and then give some numerical results to demonstrate the accuracy of our approximations.

Efficient and accurate Lyapunov function-based truncation technique for multi-dimensional Markov chains with applications to discriminatory processor sharing and priority queues

Online service providers aim to satisfy the tail performance requirements of customers through Service-Level Objectives (SLOs). One approach to ensure tail performance requirements is to model the service as a Markov chain and obtain its steady-state probability distribution. However, obtaining the distribution can be challenging, if not impossible, for certain types of Markov chains, such as those with multi-dimensional or infinite state-space and state-dependent transitions. Examples include M/M/1 with Discriminatory Processor Sharing (DPS) and preemptive M/M/c with multiple priority classes and customer abandonment.To address this fundamental problem, we propose a Lyapunov function-based state-space truncation technique that leverages moments or bounds on moments of the state variables. This technique allows us to obtain tight truncation bounds while ensuring arbitrary probability mass guarantees for the truncated chain. We highlight the efficacy of our technique for multi-dimensional DPS and M/M/c priority queue with abandonment, demonstrating a substantial reduction in state space (up to 74%) compared to existing approaches. Additionally, we present three practical use cases that highlight the applicability of our truncation technique by analyzing the performance of the DPS system.

Facilitating Numerical Solutions of Inhomogeneous Continuous Time Markov Chains Using Ergodicity Bounds Obtained with Logarithmic Norm Method

The problem considered is the computation of the (limiting) time-dependent performance characteristics of one-dimensional continuous-time Markov chains with discrete state space and time varying intensities. Numerical solution techniques can benefit from methods providing ergodicity bounds because the latter can indicate how to choose the position and the length of the “distant time interval” (in the periodic case) on which the solution has to be computed. They can also be helpful whenever the state space truncation is required. In this paper one such analytic method—the logarithmic norm method—is being reviewed. Its applicability is shown within the queueing theory context with three examples: the classical time-varying M/M/2 queue; the time-varying single-server Markovian system with bulk arrivals, queue skipping policy and catastrophes; and the time-varying Markovian bulk-arrival and bulk-service system with state-dependent control. In each case it is shown whether and how the bounds on the rate of convergence can be obtained. Numerical examples are provided.

Slack reactants: A state-space truncation framework to estimate quantitative behavior of the chemical master equation.

State space truncation methods are widely used to approximate solutions of the chemical master equation. While most methods of this kind focus on truncating the state space directly, in this work, we propose modifying the underlying chemical reaction network by introducing slack reactants that indirectly truncate the state space. More specifically, slack reactants introduce an expanded chemical reaction network and impose a truncation scheme based on desired mass conservation laws. This network structure also allows us to prove inheritance of special properties of the original model, such as irreducibility and complex balancing. We use the network structure imposed by slack reactants to prove the convergence of the stationary distribution and first arrival times. We then provide examples comparing our method with the stationary finite state projection and finite buffer methods. Our slack reactant system appears to be more robust than some competing methods with respect to calculating first arrival times.

Stochastic model order reduction in randomly parametered linear dynamical systems

• Stochastic model order reduction for randomly parametered linear dynamical structures. • Reduction in state-space dimension using SEREP. • Development of surrogate model in the reduced state space using PCE. • Significant savings in computational costs for reliability analysis of large systems. • Proposed methodology enables using existing commercial FE softwares. This study focuses on the development of reduced order models for stochastic analysis of complex large ordered linear dynamical systems with parametric uncertainties, with an aim to reduce the computational costs without compromising on the accuracy of the solution. Here, a twin approach to model order reduction is adopted. A reduction in the state space dimension is first achieved through system equivalent reduction expansion process which involves linear transformations that couple the effects of state space truncation in conjunction with normal mode approximations. These developments are subsequently extended to the stochastic case by projecting the uncertain parameters into the Hilbert subspace and obtaining a solution of the random eigenvalue problem using polynomial chaos expansion. Reduction in the stochastic dimension is achieved by retaining only the dominant stochastic modes in the basis space. The proposed developments enable building surrogate models for complex large ordered stochastically parametered dynamical systems which lead to accurate predictions at significantly reduced computational costs.

State Space Truncation with Quantified Errors for Accurate Solutions to Discrete Chemical Master Equation.

The discrete chemical master equation (dCME) provides a general framework for studying stochasticity in mesoscopic reaction networks. Since its direct solution rapidly becomes intractable due to the increasing size of the state space, truncation of the state space is necessary for solving most dCMEs. It is therefore important to assess the consequences of state space truncations so errors can be quantified and minimized. Here we describe a novel method for state space truncation. By partitioning a reaction network into multiple molecular equivalence groups (MEGs), we truncate the state space by limiting the total molecular copy numbers in each MEG. We further describe a theoretical framework for analysis of the truncation error in the steady-state probability landscape using reflecting boundaries. By aggregating the state space based on the usage of a MEG and constructing an aggregated Markov process, we show that the truncation error of a MEG can be asymptotically bounded by the probability of states on the reflecting boundary of the MEG. Furthermore, truncating states of an arbitrary MEG will not undermine the estimated error of truncating any other MEGs. We then provide an overall error estimate for networks with multiple MEGs. To rapidly determine the appropriate size of an arbitrary MEG, we also introduce an a priori method to estimate the upper bound of its truncation error. This a priori estimate can be rapidly computed from reaction rates of the network, without the need of costly trial solutions of the dCME. As examples, we show results of applying our methods to the four stochastic networks of (1) the birth and death model, (2) the single gene expression model, (3) the genetic toggle switch model, and (4) the phage lambda bistable epigenetic switch model. We demonstrate how truncation errors and steady-state probability landscapes can be computed using different sizes of the MEG(s) and how the results validate our theories. Overall, the novel state space truncation and error analysis methods developed here can be used to ensure accurate direct solutions to the dCME for a large number of stochastic networks.

Strong Truncation Approximation in Tandem Queues with Blocking

Markov models are frequently used for performance modeling. However most models do not have closed form solutions, and numerical solutions are often not feasible due to the large or even infinite state space of models of practical interest. For that, the state‐space truncation is often demanded for computation of this kind of models. In this paper, we use the strong stability approach to establish analytic error bounds for the truncation of a tandem queue with blocking. Numerical examples are carried out to illustrate the quality of the obtained error bounds.

Error bounds for state space truncation of finite Jackson networks

A computational and an analytic error bound are derived for the truncation of finite Jackson networks. Numerical support is provided for the special application of a cellular mobile communication network.

Enhanced Dynamic Programming Algorithms for Series Line Optimization

Dynamic programming value iteration is made more efficient on a five-machine unreliable series line by characterizing the transient and insensitive states. Holding costs are minimized subject to a service level constraint in a make-to-stock system with backordering. State-space truncations are chosen by checking the recurrent class in previous runs. An approximate model is developed that reduces the number of machine states. Monotone control theory is used to restrict the search for a control switching surface. Numerical optimal policies are compared with the heuristic control point policy and several characteristics of optimal policies are identified.

JOIN MINIMUM COST QUEUE FOR MULTICLASS CUSTOMERS: STABILITY AND PERFORMANCE BOUNDS

We consider a system of K parallel queues providing different grades of service through each of the queues and serving a multiclass customer population. Service differentiation is achieved by specifying different join prices to the queues. Customers of class j define a cost function ψij(ci,xi) for taking service from queue i when the join price for queue i is ci and congestion in queue i is xi and join the queue that minimizes ψij(·,·). Such a queuing system will be called the “join minimum cost queue” (JMCQ) and is a generalization of the join shortest queue (JSQ) system. Non-work-conserving (called Paris Metro pricing system) and work-conserving (called the Tirupati system) versions of the JMCQ are analyzed when the cost to an arrival of joining a queue is a convex combination of the join price for that queue and the expected waiting time in that queue at the arrival epoch. Our main results are for a two-queue system.We obtain stability conditions and performance bounds. To obtain the lower and upper performance bounds, we propose two quasi-birth–death (QBD) processes that are derived from the original systems by suitably truncating the state space. The state space truncation in the non-work-conserving JMCQ follows the method of van Houtum and colleagues. We then show that this method is not applicable to the work-conserving JMCQ and provide sample-path-based proofs to show that the number in each queue is bounded by the number in the corresponding queues of these QBD processes. These sample-path proof techniques might also be of independent interest. We then show that the performance measures like mean queue length and revenue rate of the system are also bounded by the corresponding quantities of these QBD processes. Numerical examples show that these bounds are fairly tight. Finally, we generalize some of these results to systems with more queues.

State-space truncation methods for parallel model reduction of large-scale systems

We discuss a parallel library of efficient algorithms for model reduction of large-scale systems with state-space dimension up to O (10 4). We survey the numerical algorithms underlying the implementation of the chosen model reduction methods. The approach considered here is based on state-space truncation of the system matrices and includes absolute and relative error methods for both stable and unstable systems. In contrast to serial implementations of these methods, we employ Newton-type iterative algorithms for the solution of the major computational tasks. Experimental results report the numerical accuracy and the parallel performance of our approach on a cluster of Intel Pentium II processors.

Parallel algorithms for model reduction of discrete-time systems

Computing reduced-order models of controlled dynamical systems is of fundamental importance in many analysis and synthesis problems in systems and control theory. Algorithmic aspects of model reduction methods based on state-space truncation for linear discrete-time systems are addressed here. In contrast to the often-used approach of applying methods for continuous-time systems to discrete-time models employing a bilinear transformation, we devise special algorithms for discrete-time systems. Usually, this is more reliable and efficient. All methods discussed require in an initial stage the computation of the Gramians of the system. Using an accelerated fixed-point iteration for computing the full-rank factors of the Gramians yields some favorable computational aspects, particularly for non-minimal systems. The computations only require efficient implementations of basic linear algebra operations readily available on modern computer architectures. We discuss aspects of the parallel implementation of these methods and show the performance and scalability on distributed memory computers. Our approach enables users to deal with very complex systems using relatively cheap infrastructure, as, for example, a local PC or workstation network.

The Differences between the Optimal Perturbations of Coupled Models of ENSO

The optimal perturbations (singular vectors) of a dynamical coupled model, a hybrid coupled model, and a linear inverse model of ENSO are compared. The hybrid coupled model consists of a dynamical ocean model and a statistical atmospheric model. The dynamical ocean model is identical to that used in the dynamical coupled model, and the atmospheric model is a statistical model derived from long time series of the dynamical coupled model. The linear inverse model was also derived from long time series from the dynamical coupled model. Thus all three coupled models are very closely related and all produce similar ENSO oscillations. The dynamical model and hybrid model also possess similar levels of hindcast skill. However, the optimal perturbations of the tangent linear versions of each model are not the same. The hybrid and linear inverse models are unable to recover the SST structure of the optimal perturbations of the dynamical model. The SST structure of the dynamical coupled model is a result of nonnormality introduced by latent heating of the atmosphere by deep convection over the west Pacific warm pool. It is demonstrated that standard statistical techniques remove the effects of the latent heating on the nonnormality of the hybrid and linear inverse models essentially rendering them more normal than their dynamical model counterpart. When the statistical components of the hybrid coupled model and the linear inverse models were recomputed using SST anomalies that are appropriately scaled by the standard deviation of SST variability, nonnormality was reintroduced into these models and they recovered the optimal perturbation structure of the dynamical model. Even though the hybrid and linear inverse model with scaled SSTs can recover the large-scale features of the correct optimal structure, state space truncation means that the dynamics of the resulting optimal perturbations is not the same as that governing optimal perturbation growth in the dynamical model. The consequences of these results for observed estimates of optimal perturbations for ENSO are discussed.

Analytic comparison results for communication networks

Queueing networks are known to provide a useful modelling and evaluation tool in computer and telecommunications. Unfortunately, realistic features like finite capacities, link failures, dynamic routing and non-exponentiality usually prohibit analytic solutions. Numerical and approximate computations as well as simplifications and performance bounds for queueing networks therefore become of practical interest. These are usually based on some form of a system modification or rather a comparison of a system under different conditions. This tutorial will outline and survey a technique to conclude comparison results and error bounds for comparing performance measures of a system under different circumstances. Most notably, this includes: • perturbations • system comparisons • state space truncations or extensions • modifications to obtain simple performance bounds The advantages and disadvantages of this technique, which is based on recursive or dynamic Markov reward structures, compared to the standard stochastic comparison or sample path approach will also be outlined. To illustrate and support the results a number of practical queueing network applications will be presented. These applications cannot be solved analytically, but simple performance bounds will be provided. The applications include simple expressions for: • Jackson networks subject to breakdowns • Jackson networks with large finite buffers • a performability front-end database system • a communications example with alternate routing • a circuit-switch telecommunications model with link failures

Design of a reduced order H ∞ robust controller for an expendable launch vehicle in the presence of structured and unstructured parameter uncertainty

In order to improve the robust stability and robust performance of an expendable launch vehicle control system in which a PID controller was employed, an H ∞ optimal robust controller has been successfully designed. The only weakpoint is the higher order of the H ∞ controller as compared with that of the PID controller. When the system model will involve the flexibility of the first and second transverse bending modes and liquid fuel sloshing, the order of the full order H ∞ optimal robust controller will reach 26. No doubt, this will create many difficulties for implementation. In this paper the design of the reduced order H ∞ robust controller will be studied. The goal of the design not only requires the reduced order controller to be robust against the unstructured and structured parameter uncertainty, but also requires the controller to have a high accuracy and an optimal exogenous disturbance attenuation. After analysis, design and simulations, a 7th order (reduced order) H ∞ robust controller has been synthesized. Two model reduction methods have been considered for finding the reduced plant design model: the State Space Truncation and Balanced Truncation. The design validity is certified by simulations of the control process for an expendable launch vehicle. The simulation results indicate that both the robust stability and the robust performance of the 7th order H ∞ robust controller are all better than that of the classical PID controller.

The Computation of Average Optimal Policies in Denumerable State Markov Decision Chains

This paper studies the expected average cost control problem for discrete-time Markov decision processes with denumerably infinite state spaces. A sequence of finite state space truncations is defined such that the average costs and average optimal policies in the sequence converge to the optimal average cost and an optimal policy in the original process. The theory is illustrated with several examples from the control of discrete-time queueing systems. Numerical results are discussed.

The Computation of Average Optimal Policies in Denumerable State Markov Decision Chains

This paper studies the expected average cost control problem for discrete-time Markov decision processes with denumerably infinite state spaces. A sequence of finite state space truncations is defined such that the average costs and average optimal policies in the sequence converge to the optimal average cost and an optimal policy in the original process. The theory is illustrated with several examples from the control of discrete-time queueing systems. Numerical results are discussed.

State-dependent signalling in queueing networks

It has recently been shown that networks of queues with state-dependent movement of negative customers, and with state-independent triggering of customer movement have product-form equilibrium distributions. Triggers and negative customers are entities which, when arriving to a queue, force a single customer to be routed through the network or leave the network respectively. They are ‘signals' which affect/control network behaviour. The provision of state-dependent intensities introduces queues other than single-server queues into the network.This paper considers networks with state-dependent intensities in which signals can be either a trigger or a batch of negative customers (the batch size being determined by an arbitrary probability distribution). It is shown that such networks still have a product-form equilibrium distribution. Natural methods for state space truncation and for the inclusion of multiple customer types in the network can be viewed as special cases of this state dependence. A further generalisation allows for the possibility of signals building up at nodes.

State-dependent signalling in queueing networks

It has recently been shown that networks of queues with state-dependent movement of negative customers, and with state-independent triggering of customer movement have product-form equilibrium distributions. Triggers and negative customers are entities which, when arriving to a queue, force a single customer to be routed through the network or leave the network respectively. They are ‘signals' which affect/control network behaviour. The provision of state-dependent intensities introduces queues other than single-server queues into the network. This paper considers networks with state-dependent intensities in which signals can be either a trigger or a batch of negative customers (the batch size being determined by an arbitrary probability distribution). It is shown that such networks still have a product-form equilibrium distribution. Natural methods for state space truncation and for the inclusion of multiple customer types in the network can be viewed as special cases of this state dependence. A further generalisation allows for the possibility of signals building up at nodes.

Truncation of Markov Chains with Applications to Queueing

State-space truncation is frequently demanded for computation of large or infinite Markov chains. Conditions are given that guarantee an error bound or rate of convergence. Roughly, these conditions apply either when probabilities of large states are sufficiently small, or when transition probabilities (rates) for state increases become small in sufficiently large states. The verification of these conditions is based on establishing bounds for bias terms of reward structures. The conditions and their verification are illustrated by two nonproduct form queueing examples: an overflow model and a tandem queue with blocking. A concrete truncation and explicit error bound are obtained. Some numerical support is provided.