Multivariate Markov Processes Research Articles

Bi-level bayesian control scheme for fault detection under partial observations

Due to limitations of sensor techniques and system structures, the internal states of most complex devices are unobservable. Only partial observations can be monitored at discrete time epochs, which poses great challenges for fault detection and prediction. To facilitate the development of advanced approaches, this paper proposes a bi-level Bayesian control scheme for detecting faults of complex devices under partial observations. The deterioration of an operational system is modeled as a hidden three-state multivariate Markov process, in which the parameter estimates are obtained by an expectation maximization (EM) algorithm. Based on the multivariate Markov process, a bi-level Bayesian control scheme is presented to monitor the Bayesian posterior probability of the system in a warning state and uses two levels of sampling frequency to dynamically detect impending failures. The decision variables of the bi-level fault detection scheme are optimized and solved in a semi-Markov decision process (SMDP) framework. The particularity of this work is that two phases of system deterioration are considered into detection scheme design to avoid unnecessary false alarms. Using partially observed multivariate data from discrete monitoring, the proposed approach is validated to be capable of detecting the early fault occurrences. By comparing with other advanced methods, the superiority of the proposed model is demonstrated.

Statistical Causality for Multivariate Nonlinear Time Series via Gaussian Process Models

The ability to test for statistical causality in linear and nonlinear contexts, in stationary or non-stationary settings, and to identify whether statistical causality influences trend of volatility forms a particularly important class of problems to explore in multi-modal and multivariate processes. In this paper, we develop novel testing frameworks for statistical causality in general classes of multivariate nonlinear time series models. Our framework accommodates flexible features where causality may be present in either: trend, volatility or both structural components of the general multivariate Markov processes under study. In addition, we accommodate the added possibilities of flexible structural features such as long memory and persistence in the multivariate processes when applying our semi-parametric approach to causality detection. We design a calibration procedure and formal testing procedure to detect these relationships through classes of Gaussian process models. We provide a generic framework which can be applied to a wide range of problems, including partially observed generalised diffusions or general multivariate linear or nonlinear time series models. We demonstrate several illustrative examples of features that are easily testable under our framework to study the properties of the inference procedure developed including the power of the test, sensitivity and robustness. We then illustrate our method on an interesting real data example from commodity modelling.

Statistical Causality for Multivariate Non-Linear Time Series via Gaussian Processes

The ability to test for statistical causality in linear and non-linear contexts, in stationary or non-stationary settings and to identify whether statistical causality influences trend of volatility forms a piratically important class of problems to explore in multi-modal and multivariate processes. In this paper we develop novel testing frameworks for statistical causality in general classes of multi-variate non-linear time-series models. Our framework accommodates flexible features where causality may be present in either: trend, volatility or both structural components of the general multivariate Markov processes under study. In addition, we accommodate the added possibilities of flexible structural features such as long memory and persistence in the multivariate processes when applying our semi-parametric approach to causality detection. We design a calibration procedure and formal testing procedure to detect these relationships through classes of Gaussian process models. We provide a generic framework which can be applied to a wide range of problems, including partially observed generalised diffusions or general multivariate linear or non-linear time series models. We develop several illustrative examples of features that are easily testable under our framework to study the properties of the inference procedure developed including power of the test, sensitivity and robustness. We then illustrate our method on an interesting real data example from commodity modelling.

On the equilibrium personnel structure in the presence of vertical and horizontal mobility via multivariate Markov chains

The present paper employs the Multivariate Homogeneous Markov System (MHMS) in the context of Markov manpower planning modelling. The system is regulated by an embedded multivariate Markov process that allows us to distinguish employees’ mobility patterns that take place either within or among the existing divisions (departments) of an organization. The motivation behind this step arises from the generalization of univariate Markov manpower planning models in which the organization is considered a single (probably hierarchical) group and from the fact that departmental mobility is actually common in most realistic establishments. The first part of the paper presents the functional relations of the MHMS governing intra/inter-departmental transitions. Using these functional forms, we proceed by studying the system’s equilibrium behaviour. This asymptotic analysis reveals the inherent tendencies of the system to reach the limiting structures of specific forms and properties under conditions imposed in the long run.

HEAT KERNEL MODELS FOR ASSET PRICING

A heat kernel approach is proposed for the development of a novel method for asset pricing over a finite time horizon. We work in an incomplete market setting and assume the existence of a pricing kernel that determines the prices of financial instruments. The pricing kernel is modeled by a weighted heat kernel driven by a multivariate Markov process. The heat kernel is chosen so as to provide enough freedom to ensure that the resulting model can be calibrated to appropriate data, e.g. to the initial term structure of bond prices. A class of models is presented for which the prices of bonds, caplets, and swaptions can be computed in closed form. The dynamical equations for the price processes are derived, and explicit formulae are obtained for the short rate of interest, the risk premium, and for the stochastic volatility of prices. Several of the closed-form models presented are driven by combinations of Markovian jump processes with different probability laws. Such models provide a basis for consistent applications in various market sectors, including equity markets, fixed-income markets, commodity markets, and insurance. The flexible multidimensional and multivariate structure on which the resulting price models are based lends itself well to the modeling of dependence across asset classes. As an illustration, the impact of spiraling debt, a typical feature of a financial crisis, is modeled explicitly, and the contagion effects can be readily observed in the dynamics of the associated asset returns.

MULTIVARIATE SUBORDINATION OF MARKOV PROCESSES WITH FINANCIAL APPLICATIONS

This paper develops the procedure of multivariate subordination for a collection of independent Markov processes with killing. Starting from d independent Markov processes with killing and an independent d‐dimensional time change , we construct a new process by time, changing each of the Markov processes with a coordinate . When is a d‐dimensional Lévy subordinator, the time changed process is a time‐homogeneous Markov process with state‐dependent jumps and killing in the product of the state spaces of . The dependence among jumps of its components is governed by the d‐dimensional Lévy measure of the subordinator. When is a d‐dimensional additive subordinator, Y is a time‐inhomogeneous Markov process. When with forming a multivariate Markov process, is a Markov process, where each plays a role of stochastic volatility of . This construction provides a rich modeling architecture for building multivariate models in finance with time‐ and state‐dependent jumps, stochastic volatility, and killing (default). The semigroup theory provides powerful analytical and computational tools for securities pricing in this framework. To illustrate, the paper considers applications to multiname unified credit‐equity models and correlated commodity models.

Constructing Markov Processes with Dependent Jumps by Multivariate Subordination: Applications to Multi-Name Credit-Equity Modeling

This paper develops the procedure of multivariate subordination for a collection of independent Markov processes with killing. Starting from d independent Markov processes Xi with killing and an independent d-dimensional time change T, we construct a new process by time changing each of the Markov processes Xi with a coordinate Ti. When T is a d-dimensional Levy subordinator, the time changed process Yi:=Xi(Ti(t)) is a time-homogeneous Markov process with state-dependent jumps and killing in the product of the state spaces of Xi. The dependence among jumps of its components is governed by the d-dimensional Levy measure of the subordinator. When T is a d-dimensional additive subordinator, Y is a time-inhomogeneous Markov process. When Ti= ∫0tVsi ds with Vi forming a multi-variate Markov process, (Yi,Vi) is a Markov process, where each Vi plays a role of stochastic volatility of Yi. This construction provides a rich modeling architecture for building multivariate models in finance with time-dependent and state-dependent jumps, stochastic volatility, and killing (default). The semigroup theory provides powerful analytical and computational tools for securities pricing in this framework. To illustrate, the paper considers applications to multi-name unified credit-equity models and correlated commodity models.

Heat Kernel Framework for Asset Pricing in Finite Time

A heat kernel approach is proposed for the development of a general, flexible, and mathematically tractable asset pricing framework in finite time. The pricing kernel, giving rise to the price system in an incomplete market, is modelled by weighted heat kernels which are driven by multivariate Markov processes and which provide enough degrees of freedom in order to calibrate to relevant data, e.g. to the term structure of bond prices. It is shown how, for a class of models, the prices of bonds, caplets, and swaptions can be computed in closed form. The dynamical equations for the price processes are derived, and explicit formulae are obtained for the short rate of interest, the risk premium, and for the stochastic volatility of prices. Several of the closed-form asset price models presented in this paper are driven by combinations of Markovian jump processes with different probability laws. Such models provide a rich basis for consistent applications in several sectors of a financial market including equity, fixed-income, commodities, and insurance. The flexible, multidimensional and multivariate structure, on which the asset price models are constructed, lends itself well to the transparent modelling of dependence across asset classes. As an illustration, the impact on prices by spiralling debt, a typical feature of a financial crisis, is modelled explicitly, and contagion effects are readily observed in the dynamics of asset returns.

Optimal Scaling of the Random Walk Metropolis: General Criteria for the 0.234 Acceptance Rule

Scaling of proposals for Metropolis algorithms is an important practical problem in Markov chain Monte Carlo implementation. Analyses of the random walk Metropolis for high-dimensional targets with specific functional forms have shown that in many cases the optimal scaling is achieved when the acceptance rate is approximately 0.234, but that there are exceptions. We present a general set of sufficient conditions which are invariant to orthonormal transformation of the coordinate axes and which ensure that the limiting optimal acceptance rate is 0.234. The criteria are shown to hold for the joint distribution of successive elements of a stationary pth-order multivariate Markov process.

Optimal Scaling of the Random Walk Metropolis: General Criteria for the 0.234 Acceptance Rule

Scaling of proposals for Metropolis algorithms is an important practical problem in Markov chain Monte Carlo implementation. Analyses of the random walk Metropolis for high-dimensional targets with specific functional forms have shown that in many cases the optimal scaling is achieved when the acceptance rate is approximately 0.234, but that there are exceptions. We present a general set of sufficient conditions which are invariant to orthonormal transformation of the coordinate axes and which ensure that the limiting optimal acceptance rate is 0.234. The criteria are shown to hold for the joint distribution of successive elements of a stationary pth-order multivariate Markov process.

Archimedean survival processes

Archimedean copulas are popular in the world of multivariate modelling as a result of their breadth, tractability, and flexibility. McNeil and Nešlehová (2009) [12] showed that the class of Archimedean copulas coincides with the class of positive multivariate ℓ1-norm symmetric distributions. Building upon their results, we introduce a class of multivariate Markov processes that we call ‘Archimedean survival processes’ (ASPs). An ASP is defined over a finite time interval, is equivalent in law to a vector of independent gamma processes, and its terminal value has an Archimedean survival copula. There exists a bijection from the class of ASPs to the class of Archimedean copulas. We provide various characterisations of ASPs, and a generalisation.

Deterministic and stochastic naive T cell population dynamics: symmetric and asymmetric cell division

There is growing evidence that asymmetric cell division has a key role in fate decision during T cell responses, however it is unknown if this process also has a role in the normal homeostasis of the naive T cell population. In order to explore asymmetric cell division, we develop a mathematical model in which naive T cells exist in one of two states: a resting and a cycling state. We consider three variants of the model, differing in the state of daughter cells produced by cell division, and study the steady states and time evolution of the populations in each case. We begin with a deterministic model: two coupled ordinary differential equations for the two cell populations (resting and cycling). We then introduce a stochastic generalization of the model and show, by means of the appropriate moment generating function, that the equations of the deterministic model are those of the mean sizes of the populations. The stochastic model allows us to explore questions the deterministic model fails to address, such as extinction of the population and expected lifetimes of a naive T cell clone, when the number of cells is small. Finally, we consider a multi-variate Markov process, in which each cell is classified according to its generation, or number of divisions, up to a maximum number of generations, consistent with the detection limit of fluorescent labelling techniques. Immunologically, this is a challenging question to address, and our results indicate that the mathematical models developed allow us to discern between symmetric and asymmetric division scenarios. These results can be, in principle, experimentally tested and we provide a brief description of the experimental procedure that will allow to determine the relative role of symmetric versus asymmetric cell division in naive T cell homeostasis, without directly observing this process in vivo.

Study of dependence for some stochastic processes: Symbolic Markov copulae

We study dependence between components of multivariate (nice Feller) Markov processes: what conditions need to be satisfied by a multivariate Markov process so that its components are Markovian with respect to the filtration of the entire process and such that they follow prescribed laws? To answer this question, we introduce a symbolic approach, which is rooted in the concept of pseudo-differential operator (PDO). We investigate connections between dependence, in the sense described above, and the PDOs. In particular, we study the problem of constructing a multivariate nice Feller process with given marginal laws in terms of symbols of the related PDOs. This approach leads to relatively simple conditions, which provide solutions to this problem.

Stochastic Modeling of Naïve T Cell Homeostasis for Competing Clonotypes via the Master Equation

Stochastic models for competing clonotypes of T cells by multivariate, continuous-time, discrete state, Markov processes have recently been proposed in the literature. A stochastic modeling framework is important because of rare events associated with small populations of some critical cell types. Usually, computational methods for these problems employ a trajectory-based approach, based on Monte Carlo simulation. This is partly because the complementary, probability-density function (PDF) approaches can be expensive, but here we describe some efficient PDF approaches by directly solving the governing equations, known as the Master Equation. These computations are made very efficient through an approximation of the state space by projections and through the use of Krylov subspace methods when evolving the matrix exponential. These computational methods allow us to explore the evolution of the PDFs associated with these stochastic models, and bimodal distributions arise. Both experimental and theoretical investigations have emphasized the need to take into account effects due to aging. Thus time-dependent propensities naturally arise in immunological processes. Incorporating time-dependent propensities into the framework of the Master Equation significantly complicates the corresponding computational methods, but here we describe an efficient approach via Magnus formulas. Although this contribution focuses on the example of competing clonotypes, the general principles are relevant to multivariate Markov processes and provide fundamental techniques for computational immunology.

Stochastic simulation of calcium microdomains in the vicinity of an L-type calcium channel

We study numerically the local dynamics of the intracellular calcium concentration in the vicinity of a voltage- and calcium-dependent plasma membrane L-type calcium channel. To account for the low number of Ca(2+) ions and buffer molecules present in sub-femtoliter volumes, we use an exact stochastic simulation algorithm including diffusion. We present a novel, unified simulation method that implements reaction-diffusion events of Ca(2+) ions and buffer molecules, stochastic ion channel gating and channel conductance as a multivariate Markov process. For fixed-voltage dynamics, e.g. under voltage-clamp conditions, it is shown that voltage-sensitive channel-gating steps can be incorporated exactly. We compare multi- and single-voxel geometries and show that the single-voxel approach leads to almost identical first- and second-order moments, at much lower computation time. Numerical examples illustrate the variability in local Ca(2+) fluctuations as induced by bursts of channel openings in response to membrane depolarisations. Finally, by introducing calmodulin as a link, it is shown how this variability is passed on to downstream signalling pathways. The method may prove useful to study calcium microdomains and calcium-regulated processes triggered by membrane depolarisations as evoked by, e.g., viral channel-forming proteins during virus-host cell interactions.

Vector equilibrium correction models with non‐linear discontinuous adjustments

Summary Cointegration is studied for a non-linear autoregressive process characterized by discontinuous and regime-dependent equilibrium or error correction. Here the disequilibrium, as measured by the norm of linear ‘stable’ or cointegrating relations, determines the regime and hence the equilibrium correction of the process. Importantly, switching between regimes is thereby allowed to be caused endogenously. The transition function may be either observable as in, e.g. threshold processes, or unobservable when transition probabilities are specified as in, e.g. autoregressive conditional root processes. Conditions for stationarity, geometric ergodicity as well as existence of moments are derived using a general multivariate Markov process. From these conditions it is shown that imposing parametric restrictions on only one of the regimes of the non-linear vector autoregression is sufficient to ensure higher-order moments and linear cointegrating relations which are geometrically ergodic and hence also stationary. Additionally, estimation is considered when the cointegrating relations are known and asymptotic theory is provided for this case. Based on many existing empirical analyses of, e.g. real exchange rates and interest rates spreads, the proposed dynamics appears to be desirable. This is also reflected in the included analysis of the German term structure where empirical evidence is found for discontinuous threshold error correction as opposed to classic linear error correction.

Multivariate Markov processes for stochastic systems with delays: application to the stochastic Gompertz model with delay.

Using the method of steps, we describe stochastic processes with delays in terms of Markov diffusion processes. Thus, multivariate Langevin equations and Fokker-Planck equations are derived for stochastic delay differential equations. Natural, periodic, and reflective boundary conditions are discussed. Both Ito and Stratonovich calculus are used. In particular, our Fokker-Planck approach recovers the generalized delay Fokker-Planck equation proposed by Guillouzic et al. The results obtained are applied to a model for population growth: the Gompertz model with delay and multiplicative white noise.

Markovian processes, two-sided autoregressions and finite-sample inference for stationary and nonstationary autoregressive processes

In this paper, we develop finite-sample inference procedures for stationary and non-stationary autoregressive (AR) models. The method is based on special properties of Markov processes and a split-sample technique. The results on Markovian processes (intercalary independence and truncation) only require the existence of conditional densities. They are proved for possibly nonstationary and/or non-Gaussian multivariate Markov processes. In the context of a linear regression model with AR(1) errors, we show how these results can be used to simplify the distributional properties of the model by conditioning a subset of the data on the remaining observations. This transformation leads to a new model which has the form of a two-sided autoregression to which standard classical linear regression inference techniques can be applied. We show how to derive tests and confidence sets for the mean and/or autoregressive parameters of the model. We also develop a test on the order of an autoregression. We show that a combination of subsample-based inferences can improve the performance of the procedure. An application to U.S. domestic investment data illustrates the method.

Stochastic modelling of household behaviour before, during and after a consumer price promotion

AbstractA stochastic model is developed to study household behaviour with regard to purchase quantity, brand choice and purchase timing before, during and after a price change and a price promotion such as price‐offs and price‐cuts. The basic assumption of the model is that price promotion and levels of consumer inventory influence a household's purchase‐timing and brand‐switching decisions. The model incorporates market segments and brand switching on aggregated demand for the brands by the use of multivariate Markov processes. A transient stochastic model is employed to analyse the dynamic process of household behaviour before, during, and after a price promotion. The interpurchase time that is derived from the model does not require any assumptions and is not independently, identically distributed. An empirical analysis using the Information Resources Incorporated cracker data indicated that (1) price promotion does affect household purchase of the brand and (2) households with larger family size tend to purchase promoted items. We conjecture that households with larger family size take advantage of the lower price of the promoted brands while smaller households tend to remain loyal to one brand.

Analysis of effective service time with age dependent interruptions and its application to optimal rollback policy for database management

A stochastic model is developed describing a service system subject to inhomogeneous Poisson interruptions with age dependent interruption periods. By studying the probabilistic flow of the underlying multivariate Markov process, the Laplace transform of the effective service time is explicitly obtained. For general renewal interruptions, only the expected effective service time is derived. As an application, an optimal checkpoint policy is examined for database management. It is shown that an optimal policy maximizing the ergodic availability of the database is to implement a checkpoint as soon as the cumulative uptime of the database reaches a prespecified constantk*. A computational procedure is then developed for findingk* and numerical results are exhibited.