Finite Number Of States Research Articles

Path integral and instanton calculus for stochastic jump processes with a finite number of states

We develop a method to analyze stochastic jump processes with a finite number of states ${0,1,...,N}$. It is known that the system can be analyzed using su(2) Lie algebra and the spin coherent state path integral. First, we analyze a linear susceptible-infected-susceptible epidemic model by using the Lie algebraic method and path integral. Then, we apply the instanton approximation to more general models where stochastic noise can induce bimodal distributions. By using density variables, we show that instanton solution can be interpreted as a solution of the equation of motion of the deterministic system corresponding to the stochastic system. It is also shown that instanton approximation correctly reproduces the noise-induced bimodality of the system.

An Exact Discretization of a Lax Equation for Shock Clustering and Burgers Turbulence I: Dynamical Aspects and Exact Solvability

We consider a finite dimensional system which arises as an exact discretization of the Lax equation for shock clustering, which describes the evolution of the generator of a Markov process with a finite number of states. The spectral curve of the system is a nodal curve which is fully reducible. In this work, we will show that the flow is conjugate to a straight line motion, and we will also show that the equation is exactly solvable. En route, we will establish a dictionary between an open, dense set of the lower triangular infinitesimal generator matrices and an associated set of algebro-geometric data.

Marginal cost pricing for system optimal traffic assignment with recourse under supply-side uncertainty

Transportation networks are often subject to fluctuations in supply-side parameters such as capacity and free-flow travel time due to factors such as incidents, poor weather, and bottlenecks. In such scenarios, assuming that network arcs exist in a finite number of states with different delay functions with different probabilities, a marginal cost pricing scheme that leads to a socially optimal outcome is proposed. The suggested framework makes the behavioral assumption that travelers do not just choose paths but follow routing policies that respond to en route information. Specifically, it is assumed that travelers are fully-rational and that they compute the optimal online shortest path assuming full-reset. However, such policies may involve cycling, which is unrealistic in practice. Hence, a network transformation that helps restrict cycles up to a certain length is devised and the problem is reformulated as a convex optimization problem with symmetric delay functions. The results of numerical tests on the Sioux Falls test network are presented using the Frank–Wolfe algorithm.

On approximate predictability of metric systems

In this paper we introduce and characterize the notion of approximate predictability for the general class of metric systems, which are a powerful modeling framework to deal with complex heterogeneous systems such as hybrid systems. Approximate predictability corresponds to the possibility of predicting the occurrence of specific states belonging to a particular subset of interest, in advance with respect to their occurrence, on the basis of observations corrupted by measurement errors. We establish a relation between approximate predictability of a given metric system and approximate predictability of a metric system that approximately simulates the given one. This relation allows checking approximate predictability of a system with an infinite number of states, provided that one is able to construct a metric system with a finite number of states and inputs, approximating the original one in the sense of approximate simulation. The analysis of approximate predictability of Piecewise Affine (PWA) systems is carried out as an application of the proposed approach.

КОЛМОГОРОВСКИЕ МАТРИЦЫ И НЕПРЕРЫВНЫЕ МАРКОВСКИЕ ЦЕПИ С КОНЕЧНЫМ ЧИСЛОМ СОСТОЯНИЙ

In terms of ergodicity of averaged systems with constant coefficients (and Kolmogorov matrix), the signs of ergodicity of continuous Markov chains with a finite number of States with periodic and almost periodic coefficients are indicated.

THE CONTINUOUS-TIME TIME BERNULLI PROCESS

A random Bernoulli process with continuous time and a finite number of states (random events) is proposed. The process is obtained by two mutually complementary methods - directly from the Poisson process with an intensity parameter that depends on time and methods of queuing theory, from a queuing system with two parameters. In the first case, the process was formalized on a probability space with measure, as a measurable function of time. The intensity of the Poisson process was considered as a measure. The Bernoulli process for each fixed time was obtained as a conditional distribution from a suitable Poisson distribution. The parameter of the Poisson distribution was determined from the differential equation, in the formulation of which the approximation of the Bernoulli formula by the Poisson formula was essentially used. In the second method, standard methods of queuing theory were used. A two- parameter queuing model was formulated in which for all customer flows the time between occurrence of neighboring customers was a random value satisfying the exponential law. The model was formalized by a system of differential equations, whose analytical solution represented the continuous-time Bernoulli process. In finding solutions, the method of generating functions was used. It is of interest to derive the Bernoulli process both from the probability space constructed for the Poisson process and from the queuing theory model. The authors believe that the proposed process can be generalized to a wider class of functions than that used in the work, down to measurable ones. The possibilities for the practical application of the continuous-time Bernoulli process will undoubtedly be expanded, since its discrete analog is well known in many fields of science and technology.

Development and evaluation of a continuous-time Markov chain model for detecting and handling data currency declines

Data currency declines, caused by recorded data values becoming outdated, can damage the usability and accountability of data resources. Detecting and updating outdated values may improve data currency and reduce the associated damage, but such efforts may be costly and cannot always be justified. This study models currency decline scenarios using a continuous-time Markov chain stochastic process with a finite number of states, each reflecting a valid data value. The model considers state transition probabilities, transition time distributions, and the tradeoff between the damage associated with outdated data and the cost of reacquisition. The proposed formulation permits the currency level to be estimated without having to rely on a baseline for comparison, as well as the prediction of future currency declines, assessment of their accumulated damage, and optimization of the timing of cost-effective data auditing and reacquisition. The study introduces a comprehensive evaluation of the proposed model, using a large real-world dataset relating to the handling of insurance claims over multiple time periods. The evaluation results highlight the applicability of the model, and its potential contribution to proactive data quality management and cost-effective handling of currency declines.

Inverse problem for multi-body interaction of nonlinear waves

The inverse problem is studied in multi-body systems with nonlinear dynamics representing, e.g., phase-locked wave systems, standard multimode and random lasers. Using a general model for four-body interacting complex-valued variables we test two methods based on pseudolikelihood, respectively with regularization and with decimation, to determine the coupling constants from sets of measured configurations. We test statistical inference predictions for increasing number of sampled configurations and for an externally tunable temperature-like parameter mimicing real data noise and helping minimization procedures. Analyzed models with phasors and rotors are generalizations of problems of real-valued spherical problems (e.g., density fluctuations), discrete spins (Ising and vectorial Potts) or finite number of states (standard Potts): inference methods presented here can, then, be straightforward applied to a large class of inverse problems. The high versatility of the exposed techniques also concerns the number of expected interactions: results are presented for different graph topologies, ranging from sparse to dense graphs.

On Convergence Rate Estimates for Some Birth and Death Processes

Homogeneous birth and death processes with a finite number of states are studied. We analyze the slowest and fastest rates of convergence to the limit mode. Estimates of these bounds for some classes of mean-field models are obtained. The asymptotics of the convergence rate for some models of chemical kinetics is studied in the case where the number of system states tends to infinity.

Condition-based joint maintenance optimization for a large-scale system with homogeneous units

ABSTRACTA joint maintenance policy that simultaneously repairs multiple units is useful for large-scale systems where the setup cost to initiate the maintenance is generally higher than the repair costs. This study proposes a new method for scheduling maintenance activities in a large-scale system with homogeneous units that degrade over time. Specifically, we consider the maintenance type that renews all units at each maintenance activity, which is practically applicable for systems where the units need to be regularly maintained. To make the analysis computationally tractable, we discretize the health condition of each unit into a finite number of states. The proposed optimization formulation triggers the maintenance activity based on the fraction of units at each degradation state. Based on relevant asymptotic theories, we analytically obtain the optimal threshold in the fraction of units at each state that minimizes the long-run average maintenance cost. Our implementation results with a wide range of parameter settings show that the proposed maintenance strategy is more cost-effective than alternative strategies.

Application of Hidden Markov Models in Biomolecular Simulations.

Hidden Markov models (HMMs) provide a framework to analyze large trajectories of biomolecular simulation datasets. HMMs decompose the conformational space of a biological molecule into finite number of states that interconvert among each other with certain rates. HMMs simplify long timescale trajectories for human comprehension, and allow comparison of simulations with experimental data. In this chapter, we provide an overview of building HMMs for analyzing bimolecular simulation datasets. We demonstrate the procedure for building a Hidden Markov model for Met-enkephalin peptide simulation dataset and compare the timescales of the process.

Intimate Partner Violence: A Stochastic Model.

Intimate partner violence (IPV) has been a well-studied problem in the past psychological literature, especially through its classical methodology such as qualitative, quantitative and mixed methods. This article introduces two basic stochastic models as an alternative approach to simulate the short and long-term dynamics of a couple at risk of IPV. In both models, the members of the couple may assume a finite number of states, updating them in a probabilistic way at discrete time steps. After defining the transition probabilities, we first analyze the evolution of the couple in isolation and then we consider the case in which the individuals modify their behavior depending on the perceived violence from other couples in their environment or based on the perceived informal social support. While high perceived violence in other couples may converge toward the own presence of IPV by means a gender-specific transmission, the gender differences fade-out in the case of received informal social support. Despite the simplicity of the two stochastic models, they generate results which compare well with past experimental studies about IPV and they give important practical implications for prevention intervention in this field.

Time complexity of algorithms of full analysis of dynamical systems with discrete time, defined on a finite set of states

For discrete dynamical systems with a finite number of states, we obtain order of time complexity ascension of algorithms of their full analysis.

Trading VIX futures under mean reversion with regime switching

This paper studies the optimal VIX futures trading problems under a regime-switching model. We consider the VIX as mean reversion dynamics with dependence on the regime that switches among a finite number of states. For the trading strategies, we analyze the timings and sequences of the investor’s market participation, which leads to several corresponding coupled system of variational inequalities. The numerical approach is developed to solve these optimal double stopping problems by using projected-successive-over-relaxation (PSOR) method with Crank–Nicolson scheme. We illustrate the optimal boundaries via numerical examples of two-state Markov chain model. In particular, we examine the impacts of transaction costs and regime-switching timings on the VIX futures trading strategies.

Optimal control problem regularization for the Markov process with finite number of states and constraints

The optimal control problem is considered for a system given by the Markov chain with integral constraints. It is shown that the solution to the optimal control problem on the set of all predictable controls satisfies Markov property. This optimal Markov control can be obtained as a solution of the corresponding dual problem (in case if the regularity condition holds) or (in other case) by means of proposed regularization method. The problems arising due to the system nonregularity along with the way to cope with those problems are illustrated by an example of optimal control problem for a single channel queueing system.

Factorization of chiral string amplitudes

We re-examine a closed-string model defined by altering the boundary conditions for one handedness of two-dimensional propagators in otherwise-standard string theory. We evaluate the amplitudes using Kawai-Lewellen-Tye factorization into open-string amplitudes. The only modification to standard string theory is effectively that the spacetime Minkowski metric changes overall sign in one open-string factor. This cancels all but a finite number of states: As found in earlier approaches, with enough supersymmetry (e.g., type II) the tree amplitudes reproduce those of the massless truncation of ordinary string theory. However, we now find for the other cases that additional fields, formerly thought to be auxiliary, describe new spin-2 states at the two adjacent mass levels (tachyonic and tardyonic). The tachyon is always a ghost, but can be avoided in the heterotic case.

Predictions in an illness-death model

Multi-state models allow subjects to move among a finite number of states during a follow-up period. Most often, the objects of study are the transition intensities. The impact of covariates on them can also be studied by specifying regression models. Thus, estimation in multi-state models is usually focused on the transition intensities (or the cumulative transition intensities) and on the regression parameters. However, from a clinical or epidemiological point of view, other quantities could provide additional information and may be more relevant to answer practical questions. For example, given a set of covariates for a subject, it may be of interest to estimate the probability to experience a future event or the expected time without any event. To address these kinds of issues, we need to estimate quantities such as transition probabilities, cumulative probabilities and life expectancies. The purpose of this paper is to review a large number of these quantities in an illness-death model which is perhaps the most common multi-state model in the medical literature, and to propose a way to estimate them in addition to the transition intensities and the regression parameters. An illustration is given using interval-censored data from a large cohort study on cognitive ageing.

Universal bounds on current fluctuations.

For current fluctuations in nonequilibrium steady states of Markovian processes, we derive four different universal bounds valid beyond the Gaussian regime. Different variants of these bounds apply to either the entropy change or any individual current, e.g., the rate of substrate consumption in a chemical reaction or the electron current in an electronic device. The bounds vary with respect to their degree of universality and tightness. A universal parabolic bound on the generating function of an arbitrary current depends solely on the average entropy production. A second, stronger bound requires knowledge both of the thermodynamic forces that drive the system and of the topology of the network of states. These two bounds are conjectures based on extensive numerics. An exponential bound that depends only on the average entropy production and the average number of transitions per time is rigorously proved. This bound has no obvious relation to the parabolic bound but it is typically tighter further away from equilibrium. An asymptotic bound that depends on the specific transition rates and becomes tight for large fluctuations is also derived. This bound allows for the prediction of the asymptotic growth of the generating function. Even though our results are restricted to networks with a finite number of states, we show that the parabolic bound is also valid for three paradigmatic examples of driven diffusive systems for which the generating function can be calculated using the additivity principle. Our bounds provide a general class of constraints for nonequilibrium systems.

Trading VIX Futures Under Mean Reversion with Regime Switching

This paper studies the optimal VIX futures trading problems under a regime-switching model. We consider the VIX as mean reversion dynamics with dependence on the regime that switches among a finite number of states. For the trading strategies, we analyze the timings and sequences of the investor’s market participation, which leads to several corresponding coupled system of variational inequalities. The numerical approach is developed to solve these optimal double stopping problems by using projected-successive-over-relaxation (PSOR) method with Crank-Nicolson scheme. We illustrate the optimal boundaries via numerical examples of two-state Markov chain model. In particular, we examine the impacts of transaction costs and regime-switching timings on the VIX futures trading strategies.

Stratification, target set reachability and incremental enlargement principle

This paper presents a brief exposition of a version of the concept of stratification, call it CST for short. In our approach to stratification, CST is a computational system in which the objects of computation are strata of data. Usually, the strata are nested or stacked with nested strata centering on a target set, T. CST has a potential for significant applications in planning, robotics, optimal control, pursuit, multiobjective optimization, exploration, search and other fields. Very simple, familiar examples of stratification are dictionaries, directories and catalogues. A multi-layer perceptron may be viewed as a system with a stratified structure. In spirit, CST has similarity to dynamic programing (DP), but it is much easier to understand and much easier to implement. An interesting question which relates to neuroscience is: Does the human brain employ stratification to store information? It would be natural to represent a concept such as chair, as a collection of strata with one or more strata representing a type of chair.Underlining our approach is a model, call it FSM. FSM is a discrete-time, discrete-state dynamical system which has a finite number of states. The importance of FSM as a model derives from the fact that through the use of granulation and/or quantization almost any kind of system can be approximated to by a finite state system. A concept which plays an important role in our approach is that of target set reachability. Reachability involves moving (transitioning) FSM from a state w to a state in target state, T, in a minimum number of steps. To this end, the state space, W, is stratified through the use of what is refer as the incremental enlargement principle. It should also be noted that the concept reachability is related to the concept of accessibility in modal logic.