Time-homogeneous Markov Process Research Articles

Adaptive Output Regulation of Heterogeneous Multiagent Systems Under Markovian Switching Topologies.

This paper investigates the adaptive output regulation problem for heterogeneous linear multiagent systems under randomly switching communication topologies. The switching mechanism is governed by a time-homogeneous Markov process, whose states correspond to all possible communication topologies among agents. A novel distributed adaptive cooperative controller is presented, where the dynamic compensators are utilized to estimate the exogenous signal for all the agents in mean square sense. The distributed control law is based upon the local information of agents, without using the global information of the communication topologies. Finally, illustrative examples are put forward to demonstrate the effectiveness of the given control scheme.

A unified approach for drawdown (drawup) of time-homogeneous Markov processes

AbstractDrawdown (respectively, drawup) of a stochastic process, also referred as the reflected process at its supremum (respectively, infimum), has wide applications in many areas including financial risk management, actuarial mathematics, and statistics. In this paper, for general time-homogeneous Markov processes, we study the joint law of the first passage time of the drawdown (respectively, drawup) process, its overshoot, and the maximum of the underlying process at this first passage time. By using short-time pathwise analysis, under some mild regularity conditions, the joint law of the three drawdown quantities is shown to be the unique solution to an integral equation which is expressed in terms of fundamental two-sided exit quantities of the underlying process. Explicit forms for this joint law are found when the Markov process has only one-sided jumps or is a Lévy process (possibly with two-sided jumps). The proposed methodology provides a unified approach to study various drawdown quantities for the general class of time-homogeneous Markov processes.

Comparison of time-inhomogeneous Markov processes

AbstractComparison results are given for time-inhomogeneous Markov processes with respect to function classes with induced stochastic orderings. The main result states the comparison of two processes, provided that the comparability of their infinitesimal generators as well as an invariance property of one process is assumed. The corresponding proof is based on a representation result for the solutions of inhomogeneous evolution problems in Banach spaces, which extends previously known results from the literature. Based on this representation, an ordering result for Markov processes induced by bounded and unbounded function classes is established. We give various applications to time-inhomogeneous diffusions, to processes with independent increments, and to Lévy-driven diffusion processes.

Global convergence of discrete-time inhomogeneous Markov processes from dynamical systems perspective

Given the continuous real-valued objective function f and the discrete time inhomogeneous Markov process Xt defined by the recursive equation of the form Xt+1=Tt(Xt,Yt), where Yt is an independent sequence, we target the problem of finding conditions under which the Xt converges towards the set of global minimums of f. Our methodology is based on the Lyapunov function technique and extends the previous results to cover the case in which the sequence f(Xt) is not assumed to be a supermartingale. We provide a general convergence theorem. An application example is presented: the general result is applied to the Simulated Annealing algorithm.

Additive subordination and its applications in finance

This paper studies additive subordination, which we show is a useful technique for constructing time-inhomogeneous Markov processes with analytical tractability. This technique is a natural generalization of Bochner’s subordination that has proved to be extremely useful in financial modeling. Probabilistically, Bochner’s subordination corresponds to a stochastic time change with respect to an independent Levy subordinator, while in additive subordination, the Levy subordinator is replaced by an additive one. We generalize the classical Phillips theorem for Bochner’s subordination to the additive subordination case, based on which we provide Markov and semimartingale characterizations for a rich class of jump-diffusions and pure jump processes obtained from diffusions through additive subordination, and obtain spectral decomposition for them. To illustrate the usefulness of additive subordination, we develop an analytically tractable cross-commodity model for spread option valuation that is able to calibrate the implied volatility surface of each commodity. Moreover, our model can generate implied correlation patterns that are consistent with market observations and economic intuitions.

Approximation of symmetrizations by Markov processes

Under continuity and recurrence assumptions, we prove that the iteration of successive partial symmetrizations that form a time-homogeneous Markov process, converges to a symmetrization. We cover several settings, including the approximation of the spherical nonincreasing rearrangement by Steiner symmetrizations, polarizations and cap symmetrizations. A key tool in our analysis is a quantitative measure of the asymmetry.

Perturbation analysis of inhomogeneous finite Markov chains

Abstract In this paper we provide a perturbation analysis of finite time-inhomogeneous Markov processes. We derive closed-form representations for the derivative of the transition probability at time t, with t > 0. Elaborating on this result, we derive simple gradient estimators for transient performance characteristics either taken at some fixed point in time t, or for the integrated performance over a time interval [0 , t]. Bounds for transient performance sensitivities are presented as well. Eventually, we identify a structural property of the derivative of the generator matrix of a Markov chain that leads to a significant simplification of the estimators.

On convergence properties of infinitesimal generators of scaled multitype CBI processes∗

It is a common method for proving the weak convergence of a sequence of time-homogeneous Markov processes toward a time-homogeneous Markov process first to show the convergence of the corresponding infinitesimal generators and then to check some additional conditions. The aim of the present paper is to investigate convergence properties of discrete infinitesimal generators of appropriately scaled random step functions formed from a multitype continuous state and continuous time branching process with immigration. We also present a convergence result for usual infinitesimal generators of the branching processes in question appropriately normalized.

Limits for density dependent time inhomogeneous Markov processes

A new functional law of large numbers to approximate a time inhomogeneous Markov process that is only density dependent in the limit as an index parameter goes to infinity is developed. This extends previous results by other authors to a broader class of Markov processes while relaxing some of the conditions required for those results to hold. This result is applied to a stochastic metapopulation model that accounts for spatial structure as well as within patch dynamics with the novel addition of time dependent dynamics. The resulting nonautonomous differential equation is analysed to provide conditions for extinction and persistence for a number of examples. This condition shows that the migration of a species will positively impact the reproduction in less populated areas while negatively impacting densely populated areas.

Additive Subordination and Its Applications in Finance

Motivated by the need to model time-dependent behavior, this paper studies additive subordination, which we show is a useful technique for constructing time-inhomogeneous Markov processes with analytical tractability. This technique is a natural generalization of Bochner's subordination, which has proven to be extremely useful in financial modeling. Probabilistically, Bochner's subordination corresponds to a stochastic time change with respect to an independent Levy subordinator, while in additive subordination, the Levy subordinator is replaced by an additive one. We generalize the classical Phillips Theorem for Bochner's subordination to the additive subordination case, based on which we provide Markov and semimartingale characterizations for a rich class of jump-diffusions and pure jump processes obtained from diffusions through additive subordination, and study equivalent measure changes for them. We also derive the spectral decomposition of the propagator obtained from applying additive subordination to a strongly continuous semigroup of symmetric contractions on a Hilbert space, and provide sufficient conditions for an eigenfunction expansion to exist and converge uniformly on compacts. To illustrate the usefulness of additive subordination, we develop the first analytically tractable model for crack spread option valuation in the literature that is able to calibrate the implied volatility surface of each commodity. Moreover, our model can generate implied correlation patterns that are consistent with market observations and economic intuitions.

Moment growth bounds on continuous time Markov processes on non-negative integer lattices

We consider time homogeneous Markov processes in continuous time with state space Z + N \mathbb {Z}_{+}^{N} and provide two sufficient conditions and one necessary condition for the existence of moments E ( ‖ X ( t ) ‖ r ) E(\|X(t)\|^r) of all orders r ∈ N r \in \mathbb {N} for all t ≥ 0 t \geq 0 . The sufficient conditions also guarantee an exponential in time growth bound for the moments. The class of processes studied has finitely many state independent jumpsize vectors ν 1 , … , ν M \nu _1,\dots ,\nu _M . This class of processes arises naturally in many applications such as stochastic models of chemical kinetics, population dynamics and epidemiology for example. We also provide a necessary and sufficient condition for stoichiometric boundedness of species in terms of ν j \nu _j .

Double Layer Gateway Model for Connection Between Production Network and Enterprise Network

Integration of enterprise informational systems such as ERP to production networks becomes deeper. Double-layer gateway model presented in this paper could reduce potential risks of enterprise systems integration. The scope of this paper is creating generic model which should be safer than standard models. Model of safety solution for gateway is suggested. Several models including presented by CISCO were analyzed. Models were assessed as time homogeneous discrete state Markov process solved by Kolmogorov equation. Enterprise network security, human factor, vulnerabilities in PLCs and network equipment were taken into account. Dangerous of disregarding safety efforts during production network integration is shown. Model presented in this paper was developed for pharmacy industry but it can be used in other production sectors with high mistake risk as well. Solution presented in this paper is more than 200 times secure than very common solution without safety efforts and 19 times safer than solution presented by major company.

Fault detection for networked control systems with quantization and Markovian packet dropouts

In this paper, the problem of fault detection is investigated for networked control systems with signal quantization and random packet dropouts. In the study, the packet dropouts are modeled by a time-homogeneous Markov process. A residual generator is constructed, and the corresponding fault detection problem is converted into an H∞ filtering problem. A sufficient condition for the design of fault detection is derived, which makes the resulting residual system to be stochastically stable with a prescribed H∞ performance level. Finally, a numerical example is given to illustrate the effectiveness and efficiency of the proposed design method.

Optimal Observation Times for Multistate Markov Models—Applications to Pneumococcal Colonization Studies

SummaryApplications of finite state Markov transition models are numerous and the problem of estimating transition rates of such processes has been considered in many fields of science. Because these processes cannot always be followed in continuous time, the investigators often confront the question of when to measure the state of the process. The estimation of transition rates then needs to be based on a sequence of discrete time data, and the variance and estimability of the estimators greatly depend on the time spacings between consecutive observations. We study optimal time spacings for a sequence of discrete time observations to estimate the transition rates of a time homogeneous multistate Markov process. For comparative studies, optimal time spacings to estimate rate ratios are considered. Optimality criteria are formulated through the minimization of the variances of the parameter estimators of interest and are investigated assuming a stationary initial distribution. For practical purposes, we propose a simple approximation for the optimal time spacing and study the limits for its applicability. The work is motivated by studies of colonization with Streptococcus pneumoniae.

Discrete Time Homogeneous Markov Processes for the Study of the Basic Risk Processes

In this paper Markov models useful for following the time evolution of the aggregate claim amount and the claim number in the homogeneous time environment are presented. More precisely the homogeneous Markov reward processes in both discounted and not discounted cases are applied to solve the aggregate claim amount and the claim number processes respectively. In the last section the application of the proposed models is presented. Two different real-world databases are mixed for the construction of input data.

Exchangeable Markov Processes on $$[k]^{\mathbb N}$$ [ k ] N with Cadlag Sample Paths

Any exchangeable, time-homogeneous Markov processes on \([k]^{\mathbb {N}}\) with cadlag sample paths projects to a Markov process on the simplex whose sample paths are cadlag and of locally bounded variation. Furthermore, any such process has a de Finetti-type description as a mixture of independent, identically distributed copies of time-inhomogeneous Markov processes on \([k]\). In the Feller case, these time-inhomogeneous Markov processes have a relatively simple structure; however, in the non-Feller case, a greater variety of behaviors is possible since the transition law of the underlying Markov process on \([k]^{\mathbb N}\) can depend in a nontrivial way on its exchangeable \(\sigma \)-algebra.

MULTIVARIATE SUBORDINATION OF MARKOV PROCESSES WITH FINANCIAL APPLICATIONS

This paper develops the procedure of multivariate subordination for a collection of independent Markov processes with killing. Starting from d independent Markov processes with killing and an independent d‐dimensional time change , we construct a new process by time, changing each of the Markov processes with a coordinate . When is a d‐dimensional Lévy subordinator, the time changed process is a time‐homogeneous Markov process with state‐dependent jumps and killing in the product of the state spaces of . The dependence among jumps of its components is governed by the d‐dimensional Lévy measure of the subordinator. When is a d‐dimensional additive subordinator, Y is a time‐inhomogeneous Markov process. When with forming a multivariate Markov process, is a Markov process, where each plays a role of stochastic volatility of . This construction provides a rich modeling architecture for building multivariate models in finance with time‐ and state‐dependent jumps, stochastic volatility, and killing (default). The semigroup theory provides powerful analytical and computational tools for securities pricing in this framework. To illustrate, the paper considers applications to multiname unified credit‐equity models and correlated commodity models.

Constructing Markov Processes with Dependent Jumps by Multivariate Subordination: Applications to Multi-Name Credit-Equity Modeling

This paper develops the procedure of multivariate subordination for a collection of independent Markov processes with killing. Starting from d independent Markov processes Xi with killing and an independent d-dimensional time change T, we construct a new process by time changing each of the Markov processes Xi with a coordinate Ti. When T is a d-dimensional Levy subordinator, the time changed process Yi:=Xi(Ti(t)) is a time-homogeneous Markov process with state-dependent jumps and killing in the product of the state spaces of Xi. The dependence among jumps of its components is governed by the d-dimensional Levy measure of the subordinator. When T is a d-dimensional additive subordinator, Y is a time-inhomogeneous Markov process. When Ti= ∫0tVsi ds with Vi forming a multi-variate Markov process, (Yi,Vi) is a Markov process, where each Vi plays a role of stochastic volatility of Yi. This construction provides a rich modeling architecture for building multivariate models in finance with time-dependent and state-dependent jumps, stochastic volatility, and killing (default). The semigroup theory provides powerful analytical and computational tools for securities pricing in this framework. To illustrate, the paper considers applications to multi-name unified credit-equity models and correlated commodity models.

Bayesian Consistency for Markov Models

We consider sufficient conditions for Bayesian consistency of the transition density of time homogeneous Markov processes. To date, this remains somewhat of an open problem, due to the lack of suitable metrics with which to work. Standard metrics seem inadequate, even for simple autoregressive models. Current results derive from generalizations of the i.i.d. case and additionally require some non-trivial model assumptions. We propose suitable neighborhoods with which to work and derive sufficient conditions for posterior consistency which can be applied in general settings. We illustrate the applicability of our result with some examples; in particular, we apply our result to a general family of nonparametric time series models.

Invariant measures for hybrid stochastic systems

In this paper, we seek to understand the behavior of dynamical systems that are perturbed by a parameter that changes discretely in time. If we impose certain conditions, we can study certain embedded systems within a hybrid system as time-homogeneous Markov processes. In particular, we prove the existence of invariant measures for each embedded system and relate the invariant measures for the various systems through the flow. We calculate these invariant measures explicitly in several illustrative examples.