Stochastic Markov Chains Research Articles

Large deviations and conditioning for chaotic non-invertible deterministic maps: analysis via the forward deterministic dynamics and the backward stochastic dynamics

The large deviation properties of trajectory observables for chaotic non-invertible deterministic maps as studied recently by Smith (2022 Phys. Rev. E 106 L042202) and by Gutierrez et al (2023 arXiv:2304.13754) are revisited in order to analyze in detail the similarities and the differences with the case of stochastic Markov chains. More concretely, we focus on the simplest example displaying the two essential properties of local stretching and global folding, namely the doubling map xt+1=2xt[mod1] on the real-space interval x∈[0,1[ that can also be analyzed via the decomposition x=∑l=1+∞σl2l into binary coefficients σl=0,1 . The large deviation properties of trajectory observables can be studied either via deformations of the forward deterministic dynamics or via deformations of the backward stochastic dynamics. Our main conclusions concerning the construction of the corresponding Doob canonical conditioned processes are: (i) non-trivial conditioned dynamics can be constructed only in the backward stochastic perspective where the reweighting of existing transitions is possible, and not in the forward deterministic perspective; (ii) the corresponding conditioned steady state is not smooth on the real-space interval x∈[0,1[ and can be better characterized in the binary space σl=1,2,..,+∞ . As a consequence, the backward stochastic dynamics in the binary space are also the most appropriate framework to analyze higher levels of large deviations, and we obtain the explicit large deviations at level 2 for the probability of the empirical density of long backward trajectories.

Stochastic Optimization of Microgrid Operation With Renewable Generation and Energy Storages

The operation of energy storage systems (ESSs) in power systems where variable renewable energy sources (VRESs) and ESSs must contribute to securing the supply, can be considered as an arbitrage against scarcity. The value of using stored energy instantly must be balanced against its potential future value and future risk of scarcity. This paper proposes a multi-stage stochastic programming model for the operation of microgrids with VRESs, ESSs and thermal generators that is divided into a short- and a long-term model. The short-term model utilizes information from forecasts updated every six hours, while the long-term model considers the value of stored energy beyond the forecast horizon. The model is solved using stochastic dual dynamic programming and Markov chains, and the results show that the significance of accounting for short- and long-term uncertainty increases for systems with a high degree of variable renewable generation and ESSs and limited dispatchable generation capacity.

Spectral Statistics of Non-Hermitian Matrices and Dissipative Quantum Chaos.

We propose a measure, which we call the dissipative spectral form factor (DSFF), to characterize the spectral statistics of non-Hermitian (and nonunitary) matrices. We show that DSFF successfully diagnoses dissipative quantum chaos and reveals correlations between real and imaginary parts of the complex eigenvalues up to arbitrary energy scale (and timescale). Specifically, we provide the exact solution of DSFF for the complex Ginibre ensemble (GinUE) and for a Poissonian random spectrum (Poisson) as minimal models of dissipative quantum chaotic and integrable systems, respectively. For dissipative quantum chaotic systems, we show that the DSFF exhibits an exact rotational symmetry in its complex time argument τ. Analogous to the spectral form factor (SFF) behavior for Gaussian unitary ensemble, the DSFF for GinUE shows a "dip-ramp-plateau" behavior in |τ|: the DSFF initially decreases, increases at intermediate timescales, and saturates after a generalized Heisenberg time, which scales as the inverse mean level spacing. Remarkably, for large matrix size, the "ramp" of the DSFF for GinUE increases quadratically in |τ|, in contrast to the linear ramp in the SFF for Hermitian ensembles. For dissipative quantum integrable systems, we show that the DSFF takes a constant value, except for a region in complex time whose size and behavior depend on the eigenvalue density. Numerically, we verify the above claims and additionally show that the DSFF for real and quaternion real Ginibre ensembles coincides with the GinUE behavior, except for a region in the complex time plane of measure zero in the limit of large matrix size. As a physical example, we consider the quantum kicked top model with dissipation and show that it falls under the Ginibre universality class and Poisson as the "kick" is switched on or off. Lastly, we study spectral statistics of ensembles of random classical stochastic matrices or Markov chains and show that these models again fall under the Ginibre universality class.

IMPROVABILITY OF THE FABRICATION LINE IN A SHIPYARD

The ship production process is a complex manufacturing system involving numerous working stations mutually interconnected by transport devices and buffers. Such a production system can be efficiently modeled using the stochastic system approach and Markov chains. Once formulated, the mathematical model enables analysis of the governing production system properties like the production rate, work-in-process, and probabilities of machine blockage and starvation that govern the production system bottleneck identification and its continuous improvement. Although the continuous improvement of the production system is a well-known issue, it is usually based on managerial intuition or more complex discrete event simulation yielding sub-optimal results. Therefore, a semi-analytical procedure for the improvability analysis using the Markov chain framework is presented in this paper in the case of the shipyard’s fabrication lines. Potential benefits for the shipyards are pointed out as the main gain of the improvability analysis.

An example to teach Markov chains and Hardy–Weinberg equilibrium through Mendel’s laws

ABSTRACT Graduates in biology or genetics learn Mendel’s laws and Hardy–Weinberg equilibrium as students, and they know, use, and sometimes teach these concepts. However, it is unusual to learn about stochastic processes during the graduate studies of these topics, although the applications of Markov chains in the fields of genetics or biology are increasing. In this paper, and from Mendel’s Laws, we are going to obtain an example of the Hardy–Weinberg Equilibrium that can be used to teach it. In the demonstration, stochastic processes theory and Markov chains will be used, in order to develop the transition matrix. In this way, we obtain an example that can be used to illustrate an explanation that involves, through Mendel’s laws, Hardy–Weinberg equilibrium for degree students, and stochastic processes for advanced students in biology or genetic sciences (predoctoral students at university). In this example, we are going to use the Scientific Workplace version 6.0 software.

Stochastic modelling and availability analysis of repairable system of a milk processing plant

This paper describes the performance evaluation of pasteurising system of a milk processing plant in terms of its operational availability. For the stochastic modelling and analysis Markov chains has been applied. Particle swarm optimisation (PSO) technique is implemented for optimising the results obtained. The optimum combinations of failure and repair rates for various subsystems can be useful in decision making such as maintenance priorities, spare parts and repairmen, etc. Based on this, a decision support system (DSS) has been developed which indicates the most critical components of the system that need the utmost care while setting maintenance priorities.

Bridging the gap between constant step size stochastic gradient descent and Markov chains

We consider the minimization of a strongly convex objective function given access to unbiased estimates of its gradient through stochastic gradient descent (SGD) with constant step size. While the detailed analysis was only performed for quadratic functions, we provide an explicit asymptotic expansion of the moments of the averaged SGD iterates that outlines the dependence on initial conditions, the effect of noise and the step size, as well as the lack of convergence in the general (nonquadratic) case. For this analysis we bring tools from Markov chain theory into the analysis of stochastic gradient. We then show that Richardson–Romberg extrapolation may be used to get closer to the global optimum, and we show empirical improvements of the new extrapolation scheme.

Risk Minimization for Insurance Products via F-Doubly Stochastic Markov Chains

We study risk-minimization for a large class of insurance contracts. Given that the individual progress in time of visiting an insurance policy’s states follows an F -doubly stochastic Markov chain, we describe different state-dependent types of insurance benefits. These cover single payments at maturity, annuity-type payments and payments at the time of a transition. Based on the intensity of the F -doubly stochastic Markov chain, we provide the Galtchouk-Kunita-Watanabe decomposition for a general insurance contract and specify risk-minimizing strategies in a Brownian financial market setting. The results are further illustrated explicitly within an affine structure for the intensity.

Integrability of a deterministic cellular automaton driven by stochastic boundaries

We propose an interacting many-body space–time-discrete Markov chain model, which is composed of an integrable deterministic and reversible cellular automaton (rule 54 of Bobenko et al 1993 Commun. Math. Phys. 158 127) on a finite one-dimensional lattice , and local stochastic Markov chains at the two lattice boundaries which provide chemical baths for absorbing or emitting the solitons. Ergodicity and mixing of this many-body Markov chain is proven for generic values of bath parameters, implying the existence of a unique nonequilibrium steady state. The latter is constructed exactly and explicitly in terms of a particularly simple form of matrix product ansatz which is termed a patch ansatz. This gives rise to an explicit computation of observables and k-point correlations in the steady state as well as the construction of a nontrivial set of local conservation laws. The feasibility of an exact solution for the full spectrum and eigenvectors (decay modes) of the Markov matrix is suggested as well. We conjecture that our ideas can pave the road towards a theory of integrability of boundary driven classical deterministic lattice systems.

H∞ output tracking control over networked control systems with Markovian jumping parameters

Summary The problem of H∞ output tracking control over networked control systems (NCSs) with communication limits and environmental disturbances is studied in this paper. A wide range of time-varying stochastic problem arising in networked tracking control system is reduced to a standard convex optimization problem involving linear matrix inequalities (LMIs). The closed-loop hybrid NCS is modeled as a Markov jump linear system in which random time delays and packet dropouts are described as two stochastic Markov chains. Gridding approach is introduced to guarantee the finite value of the sequences of transmission delays from sensor to actuator. Sufficient conditions for the stochastic stabilization of the hybrid NCS tracking system are derived by the LMI-based approach through the computation of the optimal H∞ performance. The mode-dependent robust H∞ output tracking controller is obtained by the optimal iteration method. Numerical examples are given to demonstrate the effectiveness of the proposed robust output tracking controller for NCS. Copyright © 2016 John Wiley & Sons, Ltd.

Labor market segmentation in a developing country

This article discusses regional labor segmentation in Colombia using stochastic Markov chains. We discuss the evolution of the wages for university graduates in Colombian urban areas. The results show that there was regional labor market segmentation in wages for university graduates and corroborate the existence of structural changes in wages during the 1990s.

Modeling regular occupancy in commercial buildings using stochastic models

Buildings account for some 40% of the world's total energy usage. One factor that could help to improve energy efficiency in buildings is more accurate modeling of occupancy. In this paper, we propose two novel stochastic inhomogeneous Markov chains to model building occupancy under two scenarios of multi-occupant single-zone (MOSZ) and multi-occupant multi-zone (MOMZ) respectively. In the MOSZ scenario, instead of using occupancy (i.e. the number of occupants in a zone) as the state, we define the state of the inhomogeneous Markov chain as the increment of occupancy in the zone. In the MOMZ scenario, by taking into account interactions among zones, we propose another inhomogeneous Markov chain whose state is a vector in which each component represents the increment of occupancy in each zone. In this way, we can significantly simplify the calculation of transition probability matrix which is a key parameter in Markov chain models. Several simulations with real data have been conducted to evaluate the performance of the proposed models under the two scenarios. To quantify the performance of the proposed models, five variables related to occupancy properties and two evaluation criteria are defined. The results show that our proposed approaches have superior performance over existing ones.

Hybrid Simulations of Heterogeneous Biochemical Models in SBML

Models of biochemical systems presented as a set of formal reaction rules can be interpreted in different formalisms, most notably as either deterministic Ordinary Differential Equations, stochastic continuous-time Markov Chains, Petri nets, or Boolean transition systems. While the formal composition of reaction systems can be syntactically defined as the (multiset) union of the reactions, the composition and simulation of models in different formalisms remain a largely open issue. In this article, we show that the combination of reaction rules and events, as already present in SBML, can be used in a nonstandard way to define stochastic and Boolean simulators and give meaning to the hybrid composition and simulation of heterogeneous models of biochemical processes. In particular, we show how two SBML reaction models can be composed into one hybrid continuous--stochastic SBML model through a high-level interface for composing reaction models and specifying their interpretation. Furthermore, we describe dynamic strategies for automatically partitioning reactions with stochastic or continuous interpretations according to dynamic criteria. The performances are then compared to static partitioning. The proposed approach is illustrated and evaluated on several examples, including the reconstructions of the hybrid model of the mammalian cell cycle regulation of Singhania et al. as the composition of a Boolean model of cell cycle phase transitions with a continuous model of cyclin activation, the hybrid stochastic--continuous models of bacteriophage T7 infection of Alfonsi et al., and the bacteriophage λ model of Goutsias, showing the gain in both accuracy and simulation time of the dynamic partitioning strategy.

Intensity-based premium evaluation for unemployment insurance products

We present a flexible premium determination method for insurance products, in particular, for unemployment insurance products. The price is determined with the real-world pricing formula and under the assumption that the employment–unemployment progress of an insured person follows an F-doubly stochastic Markov chain. The stochastic intensity processes are estimated for the German labor market, using Cox’s proportional hazards model with time-dependent covariates on a sample of integrated labor market biographies. The estimation procedure is based on a counting process framework with stochastic compensators, which we show to be naturally connected to the class of F-doubly stochastic Markov chains. Based on the statistical analysis, the prices are computed using Monte Carlo simulations.

Analysis of basketball game states and transition probabilities using the Markov chains

The abstract system of a basketball game has been established in the paper. Parts of the game are marked with the common characteristics; they are repeated, therefore, they can be denoted with the category: game states. The presented model enables the recognition and analysis of interaction between the set of system states. The discretization of the continuous course or flow of a basketball game and the definition of equivalence among game states have given the prerequisites for the determination of transition probabilities between system states. Discrete stochastic processes and Markov chains were used for events modeling and transition probabilities calculation between the states. The matrix of transition probabilities has been structured between particular states of the Markov chain. The proposed model differentiates game states within four phases of game flow and enables the prediction of the future states.

Labor Market Segmentation in Colombia Using Markov Chains

This article discusses regional labor segmentation using stochastic Markov chains. We present a formal model and derive preliminary results using the evolution of wages in Colombian urban areas. The results show that there was regional labor market segmentation in wages for university graduates during the period from 1993 to 2000, and corroborate the existence of structural changes in wages during the nineteen nineties.

A Class of $\mathbb{F}$-Doubly Stochastic Markov Chains

We define a new class of processes, very useful in applications, $\mathbf{F}$-doubly stochastic Markov chains which contains among others Markov chains. This class is fully characterized by some martingale properties, and one of them is new even in the case of Markov chains. Moreover a predictable representation theorem holds and doubly stochastic property is preserved under natural change of measure.

Stochastic processes in science, engineering, and finance

Probability Theory Basics of Stochastic Processes Random Point Processes Markov Chains in Discrete Time Markov Chains in Continuous Time Martingales Brownian Motion

Why do complications of gastrointestinal disease and procedures come as serial rather than singular events?

It often appears as if complications arising from gastrointestinal disease and interventional procedures come in 'series' and 'cluster' in individual patients. The aim of this study was to analyse the clustering of complications in terms of stochastic modelling and Markov chains. A patient with gastrointestinal disease is simulated to move along either a 'bad track' associated with multiple consecutive complications or a 'good track' free of complications, the transitions within each track and between the two tracks being governed by probability values. Because the occurrence of a single complication increases the risk of further complications, subjects who encounter their first complication are more likely to experience a second, third or even more complications, before they leave the 'bad track' and their personal chain of cascading complications becomes discontinued. The model of a Markov chain explains why the overall number of complications in the total population of patients with digestive disease remains low even if individual patients are expected to encounter more than a single complication. Published complication rates for gastrointestinal procedures underestimate the true risk to the patient, because they do not consider the added vulnerability to cascading complications after the first event.

From error probability to information theoretic (multi-modal) signal processing

We propose an information theoretic model that unifies a wide range of existing information theoretic signal processing algorithms in a compact mathematical framework. It is mainly based on stochastic processes, Markov chains and error probabilities. The proposed framework will allow us to discuss revealing analogies and differences between several well-known algorithms and to propose interesting extensions resulting directly from our formalism. We will then describe how the theory can be applied to the rapidly emerging field of multi-modal signal processing: we will show how our framework can be efficiently used for multi-modal medical image processing and for joint analysis of multi-media sequences (audio and video).