Underlying Markov Process Research Articles

Markov properties in presence of measurement noise

Recently, several powerful tools for the reconstruction of stochastic differential equations from measured data sets have been proposed [e.g., Siegert, Phys. Lett. A 243, 275 (1998); Hurn, J. Time Series Anal. 24, 45 (2003)]. Efficient application of the methods, however, generally requires Markov properties to be fulfilled. This constraint typically seems to be violated on small scales, which frequently is attributed to physical effects. On the other hand, measurement noise such as uncorrelated measurement and discretization errors has large impacts on the statistics of measurements on small scales. We demonstrate that the presence of measurement noise, likewise, spoils Markov properties of an underlying Markov process. This fact is promising for the further development of techniques for the reconstruction of stochastic processes from measured data, since limitations at small scales might stem from artificial noise sources rather than from intrinsic properties of the dynamics of the underlying process. Measurement noise, however, can be controlled much better than the intrinsic dynamics of the underlying process.

A risk model with paying dividends and random environment

We consider a discrete time risk model where dividends are paid to insureds and the claim size has a discrete phase-type distribution, but the claim sizes vary according to an underlying Markov process called an environment process. In addition, the probability of paying the next dividend is affected by the current state of the underlying Markov process. We provide explicit expressions for the ruin probability and the deficit distribution at ruin by extracting a QBD (quasi-birth-and-death) structure in the model and then analyzing the QBD process. Numerical examples are also given.

Maximization of the portfolio growth rate under fixed and proportional transaction costs

This paper considers a discrete-time Markovian model of asset prices with economic factors and transaction costs with proportional and fixed terms. Existence of optimal strategies maximizing average growth rate of portfolio is proved in the case of complete and partial observation of the process modelling the economic factors. The proof is based on a modification of the vanishing discount approach. The main difficulty is the discontinuity of the controlled transition operator of the underlying Markov process.

Analysis of a Fork/Join Station With Inputs from a Finite Population: Sub-Network With Multi-Server Stations

This paper presents an exact analysis of a fork/join station with inputs from stations composed of multiple exponential servers. The queue length process at the input buffers is analyzed exactly in terms of the underlying Markov process. The semi-Markov kernel characterizing the departure process is analyzed to derive expressions for the marginal and joint distributions of inter-departure times from the fork/join station. These analyses are used to study the effect of inputs from multiple servers on key performance measures at a fork/join station. Comparison studies show that replacing the multiple servers by a single server with equivalent capacity could result insignificantly different estimates of throughput, synchronization delays and queue lengths at the individual buffers. These insights could lead to better representation of synchronization constraints in closed queuing network models of computer networks, fabrication/assembly systems and material control strategies for manufacturing systems.

Graphical Models for Composable Finite Markov Processes

Abstract. Composable Markov processes were introduced by Schweder (1970) in order to capture the idea that a process can be composed of different components where some of these only depend on a subset of the other components. Here we propose a graphical representation of this kind of dependence which has been called ‘local dependence’. It is shown that the graph allows to read off further independencies characterizing the underlying Markov process. Also, some standard methods for inference are adapted to exploit the graphical representation, e.g. for testing local independence.

American Options with Stopping Time Constraints

This paper concerns the pricing of American options with stochastic stopping time constraints expressed in terms of the states of a Markov process. Following the ideas of Menaldi, Robin and Sun [21] we transform the constrained into an unconstrained optimal stopping problem. The transformation replaces the original payoff by the value of a generalized barrier option. We suggest a new Monte Carlo method to numerically calculate the option value also for multidimensional Markov processes. Because of presence of stopping time constraints the classical Longstaff-Schwartz least-square Monte Carlo algorithm or its extension introduced in [7] cannot be directly applied. We adapt the Longstaff-Schwartz algorithm to solve the stochastic Cauchy-Dirichlet problem related to the valuation problem of the barrier option along a set of simulated trajectories of the underlying Markov process.

Propagation through conical crossings: An asymptotic semigroup

AbstractWe consider the standard model problem for a conical intersection of electronic surfaces in molecular dynamics. Our main result is the construction of a semi‐group in order to approximate the Wigner function associated with the solution of the Schrödinger equation at leading order in the semiclassical parameter. The semigroup stems from an underlying Markov process that combines deterministic transport along classical trajectories within the electronic surfaces and random jumps between the surfaces near the crossing. Our semigroup can be viewed as a rigorous mathematical counterpart of so‐called trajectory surface hopping algorithms, which are of major importance in molecular simulations in chemical physics. The key point of our analysis, the incorporation of the nonadiabatic transitions, is based on the Landau‐Zener type formula of Fermanian‐Kammerer and Gérard10 for the propagation of two‐scale Wigner measures through conical crossings. © 2005 Wiley Periodicals, Inc.

A Spectral Method for a Nonpreemptive Priority BMAP/G/1 QUEUE

In this paper we consider a nonpreemptive priority queue with two priority classes of customers. Customers arrive according to a batch Markovian arrival process (BMAP). In order to calculate the boundary vectors we propose a spectral method based on zeros of the determinant of a matrix function and the corresponding eigenvectors. It is proved that there are M zeros in a set Ω, where M is the size of the state space of the underlying Markov process. The zeros are calculated by the Durand-Kerner method, and the stationary joint probability of the numbers of customers of classes 1 and 2 at departures is derived by the inversion of the two-dimensional Fourier transform. For a numerical example, the stationary probability is calculated.

A New Policy Evaluation Algorithm for Markov Decision Processes with Quasi Birth-Death Structure

ABSTRACT This paper describes a new algorithm for policy evaluation for Markov decision processes (MDP) that possess a quasi birth-death structure. The proposed algorithm is based on matrix analytic methods which use probabilistic concepts associated with restricting the underlying Markov process to certain state subsets. A telecommunications application example shows that the method offers significant computational reduction compared to a standard MDP policy evaluation approach.

Estimation of Continuous-Time Markov Processes Sampled at Random Time Intervals

We introduce a family of generalized-method-of-moments estimators of the parameters of a continuous-time Markov process observed at random time intervals. The results include strong consistency, asymptotic normality, and a characterization of standard errors. Sampling is at an arrival intensity that is allowed to depend on the underlying Markov process and on the parameter vector to be estimated. We focus on financial applications, including tick-based sampling, allowing for jump diffusions, regime-switching diffusions, and reflected diffusions.

Product form solution for g-networks with dependent service

We consider a G-network with Poisson flow of positive customers. Each positive customer entering the network is characterized by a set of stochastic parameters: customer route, the length of customer route, customer volume and his service length at each route stage as well. The following node types are considered: (0) an exponential node with cn servers, infinite buffer and FIFO discipline; (1) an infinite-server node; (2) a single-server node with infinite buffer and LIFO PR discipline; (3) a single-server node with infinite buffer and PS discipline. Negative customers arriving at each node also form a Poisson flow. A negative customer entering a node with k customers in service, with probability 1/k chooses one of served positive customer as a target. Then, if the node is of a type 0 the negative customer immediately kills (displaces from the network) the target customer, and if the node is of types 1-3 the negative customer with given probability depending on parameters of the target customer route kills this customer and with complementary probability he quits the network with no service. A product form for the stationary probabilities of underlying Markov process is obtained.

On the Positive Correlation between Income Inequality and Unemployment

Several empirical studies in the literature have documented the existence of a positive correlation between income inequalitiy and unemployment. I provide a theoretical framework under which this correlation can be better understood. The analysis is based on a dynamic job search under uncertainty. I start by proving the uniqueness of a stationary distribution of wages in the economy. Drawing upon this distribution, I provide a general expression for the Gini coefficient of income inequality. The expression has the advantage of not requiring a particular specification of the distribution of wage offers. Next, I show how the Gini coefficient varies as a function of the parameters of the model, and how it can be expected to be positively correlated with the rate of unemployment. Two examples are offered. The first, of a technical nature, to show that the convergence of the measures implied by the underlying Markov process can fail in some cases. The second, to provide a quantitative assessment of the model and of the mechanism linking unemployment and inequality.

Optimal Momentum Hedging via Hypoelliptic Reduced Monge--Ampère PDE

The celebrated optimal portfolio theory of Merton was successfully extended by the author to assets that do not obey Log-Normal price dynamics in [S. Stojanovic, Computational Financial Mathematics Using Mathematica®: Optimal Trading in Stocks and Options, Birkhäuser Boston, Boston, 2003]. Namely, a general one-factor model was solved and applied in the case of appreciation-rate reversing market dynamics. Here, we extend a general methodology to solve the stochastic control problem of optimal portfolio hedging under momentum market dynamics: the corresponding HJB PDE is transformed into the associated Monge--Ampère PDE, which is, utilizing the special structure of the problem, further reduced to a lower-dimensional Monge--Ampère PDE, which is then finally solved numerically. The present problem, in addition to being a two-factor model, has a substantive difficulty due to the degeneracy of the underlying Markov process, yielding the hypoellipticity of its infinitesimal generator, and the corresponding degeneracy of all the fully nonlinear PDEs derived. Furthermore, we solve the problem of optimal hedging and pricing of European and American options in momentum markets, derive a hypoelliptic Black--Scholes PDE/obstacle problem, and introduce a notion of options trading opportunity.

Irreversible Markov processes for phylogenetic models

AbstractA frequently used model in phylogenetics uses a discrete Markov process to model molecular drift as a cause of evolutionary diversification. Predictions are often based on eigenvalue computations which, we argue, only make sense if the process is reversible. Since there is no evidence (to our knowledge) to support the reversible nature of the process, a re‐examination is made of the model and necessary algorithms with emphasis on irreversible processes. The paper includes careful discussion of the underlying Markov processes, careful distinction between the properties of reversible and irreversible processes (including earlier widely accepted analysis) all in the language of linear algebra, a new least squares approach to data adjustments, and numerical examples. Copyright © 2003 John Wiley & Sons, Ltd.

The Dynamics of Trustworthiness Among the Few

Conventional stochastic models of evolutionary processes with infinitely many agents are deterministic models in disguise. Only finite population models become truly stochastic. Therefore this paper focuses on an indirect evolutionary model of pair wise interaction in a pool of three (corresponding to analysing oligopolies in terms of duopoly markets). The outcomes of the process over the long haul are characterized by the stationary distribution of the underlying Markov process. Our example indicates that intermediate cases cannot be seen as convex combinations of the two polar non-stochastic cases of two or infinitely many individuals. JEL Classification Numbers: C72, C73.

Weighted Markov chains and graphic state nodes for information retrieval

AbstractDecision‐making in uncertain environments, such as data mining, involves a computer user navigating through multiple steps, from initial submission of a query through evaluating retrieval results, determining degrees of acceptability of the results, and advancing to a terminal state of evaluating where the interaction is successful or not. This paper describes iterative information seeking (IS) as a Markov process during which users advance through states of “nodes”. Nodes are graphic objects on a computer screen that represent both the state of the system and the group of users' or an individual user's degree of confidence in an individual node. After examining nodes to establish a confidence level, the system records the decision as weights affecting the probability of the transition paths between nodes. By training the system in this way, the model incorporates into the underlying Markov process users' decisions as a means to reduce uncertainty. The Markov chain becomes a weighted one whereby the IS makes justified suggestions.

Performance modelling of hierarchical cellular networks using PEPA

We present a performance evaluation study of hierarchical cellular networks using Performance Evaluation Process Algebra (PEPA). These networks constitute a new application area for this process algebra formalism. We show that this formalism can easily be used to model such systems. We also show that the strong equivalence aggregation technique behind PEPA allows a significant reduction of the state space of the underlying Markov process. Using the resulting model, we derive performance criteria such as new call blocking and dropping handover probabilities.

The autocorrelation function for spectral determinants of quantum graphs

The paper considers the spectral determinant of quantum graph families with chaotic classical limit and no symmetries. The secular coefficients of the spectral determinant are found to follow distributions with zero mean and variance approaching a constant in the limit of large network size. This constant is in general different from the random matrix result and depends on the classical limit. A closed expression for this system dependent constant is given here explicitly in terms of the spectrum of an underlying Markov process.

Language-based Performance Prediction for Distributed and Mobile Systems

We present a framework for performance prediction of distributed and mobile systems. We rely on process calculi and their structural operational semantics. The dynamic behaviour is described through transition systems whose transitions are labelled by encodings of their proofs that we then map into stochastic processes. We enhance related works by allowing general continuous distributions resorting to a notion of enabling between transitions. We also discuss how the number of resources available affects the overall model. Finally, we introduce a notion of bisimulation that takes stochastic information into account and prove it to be a congruence. When only exponential distributions are of interest our equivalence induces a lumpable partition on the underlying Markov process.

Bond pricing in a hidden Markov model of the short rate

Abstract. We consider a diffusion type model for the short rate, where the drift and diffusion parameters are modulated by an underlying Markov process. The underlying Markov process is assumed to have a stochastic differential driven by Wiener processes and a marked point process. The model for the short rate thus falls within the category of hidden Markov models. For this model we look at the bond pricing problem. In order to obtain more concrete results we introduce the notion of a semi-affine term structure and give sufficient conditions for the existence of such a term structure. For a special case, when the underlying process is a Markov chain with only two states, we obtain a closed form expression for bond prices. Furthermore we consider the pricing problem when the modulating process can not be directly observed. It turns out that pricing in this context may be viewed as a filtering problem.