Stationary Probability Distribution Research Articles

Partial observability and management of ecological systems.

The actual state of ecological systems is rarely known with certainty, but management actions must often be taken regardless of imperfect measurement (partial observability). Because of the difficulties in accounting for partial observability, it is usually treated in an ad hoc fashion, or simply ignored altogether. Yet incorporating partial observability into decision processes lends a realism that has the potential to improve ecological outcomes significantly. We review frameworks for dealing with partial observability, focusing specifically on dynamic ecological systems with Markovian transitions, i.e., transitions among system states that are influenced by the current system state and management action over time. Fully observable states are represented in an observable Markov decision process (MDP), whereas obscure or hidden states are represented in a partially observable process (POMDP). POMDPs can be seen as a natural extension of observable MDPs. Management under partial observability generalizes the situation for complete observability, by recognizing uncertainty about the system's state and incorporating sequential observations associated with, but not the same as, the states themselves. Decisions that otherwise would depend on the actual state must be based instead on state probability distributions (“belief states”). Partial observability requires adaptation of the entire decision process, including the use of belief states and Bayesian updates, valuation that includes expectations over observations, and optimal strategy that identifies actions for belief states over a continuous belief space. We compare MDPs and POMDPs and highlight POMDP applications to some common ecological problems. We clarify the structure and operations, approaches for finding solutions, and analytic challenges of POMDPs for practicing ecologists. Both observable and partially observable MDPs can use an inductive approach to identify optimal strategies and values, with a considerable increase in mathematical complexity with POMDPs. Better understanding of POMDPs can help decision makers manage imperfectly measured ecological systems more effectively.

Hierarchical Symbol Transition Entropy: A Novel Feature Extractor for Machinery Health Monitoring

This article develops a novel collaborative health monitoring framework based on hierarchical symbol transition entropy (HSTE) and 2-D-extreme learning machine (2-D-ELM) without fusion. In the proposed framework, a novel metric called symbol transition entropy (STE) is first presented to evaluate the dynamical complexity of time series through multistep transition and the joint probability distribution of the symbol state and its transition state. Compared with the existing entropy algorithms, STE has better robustness and captures more detailed dynamical changes. Subsequently, a new feature representation method called HSTE is proposed by combining STE with the hierarchical analysis. The two-order tensor features can be constructed for multichannel data by stacking HSTE values extracted from each single-channel data. Finally, 2-D-ELM is incorporated to identify the extracted two-order tensor features without vectorization. The feasibility of the proposed schemes is verified through simulation and experimental studies, and the final results confirm that the developed schemes have better performance than the existing entropy-based collaborative fault diagnosis methods.

Sizing of Energy Storage for Grid Inertial Support in Presence of Renewable Energy

Penetration of renewable energy resources (RERs) in the power grid continues to increase as we strive toward a greener environment for the future. While they have many advantages, most RERs possess little or no rotational kinetic energy, thereby threatening the frequency stability of future power grids. Energy storage systems (ESSs) can be used to mitigate this problem, as they are capable of providing virtual inertia to the system. This paper proposes a novel analytical approach for sizing ESSs to provide inertial support to the grid and maintain frequency stability in presence of RERs. This method analytically estimates the total inertia of the system using probability distributions of the outage states of conventional generators and RERs. RERs are modeled as multi-state units, considering the variability of their output power due to changes in weather conditions and also the forced outages of their equipment. The ESS is utilized to provide virtual inertia to the grid in the form of active power once the estimated system inertia falls below the minimum required during a disturbance. The efficacy of the proposed approach is tested on the IEEE 39 bus system and the results obtained by the analytical approach are validated using Monte Carlo simulation.

Cost Optimization of an Unreliable server queue with two stage service process under hybrid vacation policy

In this investigation, an unreliable server Markovian queueing model is developed for a service system by considering two stage service process and hybrid vacation policy. By including the features of combination of working vacation (WV) and complete vacation (CV), the steady state probability distribution of the queue size of two stage service model via matrix geometric approach has been established. The cost function has been formulated to evaluate the optimal values of the decision variables of the service system. Particle swarm optimization (PSO) and Artificial bee colony (ABC) optimization algorithms are employed to compute the optimal service rates at optimum cost. To validate the model, numerical illustrations along with sensitivity analysis have been provided.

Fock-space approach to stochastic susceptible-infected-recovered models

We investigate the stochastic susceptible-infected-recovered (SIR) model of infectious disease dynamics in the Fock-space approach. In contrast to conventional SIR models based on ordinary differential equationsfor the subpopulation sizes of S, I, and R individuals, the stochastic SIR model is driven by a master equationgoverning the transition probabilities among the system's states defined by SIR occupation numbers. In the Fock-space approach the master equationis recast in the form of a real-valued Schrödinger-type equationwith a second quantization Hamiltonian-like operator describing the infection and recovery processes. We find exact analytic expressions for the Hamiltonian eigenvalues for any population size N. We present small- and large-N results for the average numbers of SIR individuals and basic reproduction number. For small N we also obtain the probability distributions of SIR states, epidemic sizes and durations, which cannot be found from deterministic SIR models. Our Fock-space approach to stochastic SIR models introduces a powerful set of tools to calculate central quantities of epidemic processes, especially for relatively small populations where statistical fluctuations not captured by conventional deterministic SIR models play a crucial role.

Characterizing outbreak vulnerability in a stochastic SIS model with an external disease reservoir.

In this article, we take a mathematical approach to the study of population-level disease spread, performing a quantitative and qualitative investigation of an SISκ model which is a susceptible-infectious-susceptible (SIS) model with exposure to an external disease reservoir. The external reservoir is non-dynamic, and exposure from the external reservoir is assumed to be proportional to the size of the susceptible population. The full stochastic system is modelled using a master equation formalism. A constant population size assumption allows us to solve for the stationary probability distribution, which is then used to investigate the predicted disease prevalence under a variety of conditions. By using this approach, we quantify outbreak vulnerability by performing the sensitivity analysis of disease prevalence to changing population characteristics. In addition, the shape of the probability density function is used to understand where, in parameter space, there is a transition from disease free, to disease present, and to a disease endemic system state. Finally, we use Kullback-Leibler divergence to compare our semi-analytical results for the SISκ model with more complex susceptible-infectious-recovered (SIR) and susceptible-exposed-infectious-recovered (SEIR) models.

Stochastic Modeling for the Analysis and Forecasting of Stock Market Trend using Hidden Markov Model

The HMM is generally applied to forecast the hidden system of observation data. In this paper, we deal with the development of HMM for a proper understanding of finance variables in the stock market. Formulation of relationships between and within both the changing share values of Housing Development Finance Corporation Bank Limited (HDFC Bank Ltd) as visible/observed states influenced by the indicators of S&P Bombay Stock Exchange Sensitive Index (Sensex) as invisible/influencing states. Stochastic modeling with hidden Markov models is carried out for exploring various parameters of the model. Mathematical derivations for all the required statistical measures are obtained using the method of moments for the proposed probability distribution. Deducing mathematical formulation of initial probability vector, transition and observed probability matrices were carried out with the empirical data sets. Probability distribution for visible states of various lengths is obtained. It is observed from the empirically analysis that there is the maximum likelihood of rising the share prices of HDFC bank in consecutive two days. Furthermore, an attempt is made to estimate the long-run steady-state behavior of both the SENSEX and HDFC Bank share prices. The share value of HDFC bank will be on rising state from the 19th day onwards and it may be recommended for good investment choice for the long run. The findings of these studies will be valid for effective decision-making in portfolio management.

Non-classical light state transfer in su(2) resonator networks

We use a normal mode approach to show full and partial state transfer in a class of coupled resonator networks with underlying su(2) symmetry that includes the so-called J_{x} photonic lattice. Our approach defines an auxiliary Hermitian coupling matrix describing the network that yields the normal modes of the system and its time evolution in terms of orthogonal polynomials. Our results provide insight on the full quantum state reconstruction time in a general su(2) network of any size and the full quantum transfer time in the J_{x} network of size 4 n + 1 with n=1,2,3,ldots For any other network sizes, the Fock state probability distribution of the initial state is conserved but the amplitudes suffer a phase shift proportional to pi /2 that results in partial quantum state transfer.

Worldwide bilateral geopolitical interactions network inferred from national disciplinary profiles

A disciplinary profile of a country is defined as the versor whose components are the number of papers produced in a given discipline divided by the overall production of the country. Starting from the Essential Science Indicators (ESI) schema of classification of subject areas, we obtained the yearly disciplinary profiles of a worldwide graph, where on each node sits a country, in the two time intervals 1988--1988 and 1992--2017, the fall of the Berlin Wall being the watershed. We analyze the empirical pairwise cross-correlation matrices of the time series of disciplinary profiles. The contrast with random matrix theory proves that, beyond measurement noise, the empirical cross-correlation matrices bring genuine information. Arising from the Shannon theorem as the least-structured model consistent with the measured pairwise correlations, the stationary probability distribution of disciplinary profiles can be described by a Boltzmann distribution related to a generalized ${n}_{d}$-dimensional Heisenberg model. The set of network interactions of the Heisenberg model has been inferred and to it, two clusterization methods, hierarchical clustering, and principal component analysis have been applied. This allows obtaining a characterization of the worldwide bilateral interactions based on physical modeling. A simple geopolitical analysis reveals the consistency of the results obtained and gives a boost to a deeper historical analysis. In order to obtain the optimal set of pairwise interactions, we used a pseudolikelihood approach. We analytically computed the pseudolikelihood and its gradient. The analytical computations deserve interest in whatever inference Bayesian problem involving an ${n}_{d}$-dimensional Heisenberg model.

Markov chains generated by convolutions of orthogonality measures

About two dozens of exactly solvable Markov chains on one-dimensional finite and semi-infinite integer lattices are constructed in terms of convolutions of orthogonality measures of the Krawtchouk, Hahn, Meixner, Charlier, q-Hahn, q-Meixner and little q-Jacobi polynomials. By construction, the stationary probability distributions, the complete sets of eigenvalues and eigenvectors are provided by the polynomials and the orthogonality measures. An interesting property possessed by these stationary probability distributions, called ‘convolutional self-similarity,’ is demonstrated.

Selective maintenance of multi‐state systems with the repairperson fatigue effect and stochastic break duration

AbstractSystems in the industry are often required to execute a sequence of missions, and the state of the multi‐state systems may deteriorate with the increasing of running time. To improve the system reliability of the next mission, selective maintenance is applied in the limited break. Due to the fatigue effect, the work rate of the repairperson decays with time. However, the existing selective maintenance studies do not consider the impact of fatigue effect on work rate, which leads to underestimating the duration of maintenance actions and overestimating the system reliability. To overcome the above problems, a new selective maintenance model for multi‐state systems is developed to maximize the system reliability of the next mission. First, the decay process of the work rate of a repairperson is described as a piecewise continuous decreasing exponential function of working time. Next, a homogenous discrete‐state continuous‐time Markov process is used to describe the performance capacity degradation of components. Considering the stochastic break duration, based on the state probability distribution of each component obtained by the Markov model, the universal generation function (UGF) method is applied to evaluate the multi‐state system reliability. Then, taking the selected maintenance actions and maintenance priority as decision variables, the resulting selective maintenance optimization problem is solved by a tailored genetic algorithm. Finally, two computational experiments are carried out to demonstrate the effectiveness of the proposed method.

An MPPCA-based approach for anomaly detection of structures under multiple operational conditions and missing data

Structural anomaly detection based on the structural health monitoring (SHM) data has attracted significant attention owing to its important role in the early warning of structural damage to existing civil structures. Data-driven approaches, where damage-sensitive features are extracted directly from the SHM data using statistical pattern recognition (SPR) techniques without physical models of structures, have been widely studied. Principal component analysis (PCA) and probabilistic PCA (PPCA) are powerful and efficient SPR methods for linear or weakly nonlinear cases. However, some special structures may be subjected to multiple operational conditions, wherein structural configurations such as geometry and mass distribution may change due to the movement of parts or the whole structure, as in retractable roof structures. These changes may give erroneous results in the SPR of the SHM data and eventually in the anomaly detection by a single PCA or PPCA model. This paper presents an improved approach using a mixture of probabilistic principal component analysis (MPPCA) for the anomaly detection of structures under multiple operational conditions with missing measurement data. First, the baseline MPPCA model was constructed for stress data collected under healthy conditions, where the estimation of the MPPCA parameters was reformulated for the missing data cases. Second, three anomaly statistics were presented for newly monitored incomplete data to detect and localize structural anomalies. The probability distributions of the anomaly statistics were estimated to obtain thresholds for outlier detection. Finally, the effectiveness of the MPPCA-based method was investigated by applying the method to the anomaly detection of a retractable roof structure with numerically simulated and real monitored stress data.

The role of harvesting in population control in the presence of correlated noise sources

In this paper, we compare the role of constant and Michaelis-Menten type harvesting in single species population control in presence of correlated noise sources. Our aim is to determine a statistical correlation between the noises in the model parameters and population size. The characteristic time scale over which the population size of prey and predator varies are very much different, this fact justifies to consider a single species model as the predator population can be chosen to be constant within the generation time of prey. Steady state probability distributions and stationary potentials of the population are determined by Fokker-Planck equations. Stochastic bifurcation analysis and mean first passage times have been computed. Noise induced critical transitions are observed depending on the strength of the noises. The extinction possibility of population in stochastic control with Michaelis-Menten type harvesting is higher than the constant harvesting. One of our key findings is the transition from bistable to tristable state for weak noises. Another valuable finding is the noise enhanced stability phenomenon for negatively correlated noises. Constant rate of harvesting is better in deterministic case whereas Michaelis-Menten type harvesting is better in stochastic case to reduce population size. The obtained results may be helpful to understand toxin producing phytoplankton blooms and its control in marine ecosystem. Moreover, the study can be useful to explain wild prey population outbreak or extinction in forest.

On the problem of the switching of the fast server

A Markovian model of a system with two heterogeneous servers is investigated, in which the slow server is always on, and the fast server can turn on when the job queue length reaches a certain value. Models with unlimited and limited size of the buffer of waiting jobs are considered. A numerical method for calculating the probability distribution of states and characteristics of the systems under study is developed. The results of numerical experiments are given.

A queuing-inventory model for manufacturing systems with fluid-type inventory

This study considered a make-to-order manufacturing system combined with a fluid-type inventory. In this system, customers arrive at the system to place an order for a customized product. A machine manufactures products, thereby handling orders based on the first-come-first-served principle. The system has a preprocessing facility that converts the raw materials into a ready-to-use inventory. The inventory is stored in finite-capacity storage and consumed to manufacture finished products. The preprocessing facility replenishes the inventory based on the well-known (s,S)-policy with the setup time. This study proposes a new queuing-inventory model to analyze this composite system. The stationary joint probability distribution for the unmet demands and inventory level is presented in closed form. Next, a cost model is constructed, followed by numerical examples.

Steady-state analysis of the stochastic Beverton-Holt growth model driven by correlated colored noises

In this paper, we investigate a single population growth model with Beverton-Holt function, which is driven by cross-correlation between multiplicative and additive colored noises. Firstly, using approximate Fokker-Planck equation, the stationary probability distribution of the model is obtained, and then its the shape structure is discussed in detail. In addition, the influence of noise characteristics on mean, variance and skewness is studied numerically. Finally, an explicit expression of the mean first passage time is given by using the steepest-descent approximation. It is found that: (i) the P-bifurcation occurs when the two noises are positively correlated or zero correlated, but not in the case of negative correlation; (ii) a strong negative correlation degree and correlation time can promote population growth, while the strong positive correlation degree plays an opposite role; (iii) the noise enhanced stability induced by multiplicative noise is different from that induced by the additive one.

Fermionic quantum field theories as probabilistic cellular automata

A class of fermionic quantum field theories with interactions is shown to be equivalent to probabilistic cellular automata, namely cellular automata with a probability distribution for the initial states. Probabilistic cellular automata on a one-dimensional lattice are equivalent to two-dimensional quantum field theories for fermions. They can be viewed as generalized Ising models on a square lattice and therefore as classical statistical systems. As quantum field theories they are quantum systems. Thus quantum mechanics emerges from classical statistics. As an explicit example for an interacting fermionic quantum field theory we describe a type of discretized Thirring model as a cellular automaton. The updating rule of the automaton is encoded in the step evolution operator that can be expressed in terms of fermionic annihilation and creation operators. The complex structure of quantum mechanics is associated to particle-hole transformations. The naive continuum limit exhibits Lorentz symmetry. We exploit the equivalence to quantum field theory in order to show how quantum concepts as wave functions, density matrix, noncommuting operators for observables and similarity transformations are convenient and useful concepts for the description of probabilistic cellular automata.

SEMI-MARKOV RELIABILITY MODELS

Traditional technologies for reliability analysis of semi-Markov systems are limited to obtaining a stationary state probability distribution. However, when solving practical control problems in such systems, the study of transient processes is of considerable interest. This implies the subject of research - the analysis of the laws of distribution of the system states probabilities. The goal of the work is to obtain the desired distribution at any time. The complexity of the problem solving is determined by the need to obtain a result for arbitrary distribution laws of the duration of the system's stay in each state before leaving. An easy-to-implement method for the analysis of semi-Markov reliability models has been suggested. The method is based on the possibility of approximating probability-theoretic descriptions of failure and recovery flows in the system using the Erlang distribution laws of the proper order. The developed computational scheme uses the most important property of Erlang flows, which are formed as a result of sieving the simplest Poisson flow. In this case, the semi-Markov model is reduced to the Markov one, which radically simplifies the analysis of real systems.

Analysis of semi-Markov systems with fuzzy initial data

In real operating conditions of complex systems, random changes in their possible states occur in the course of their operation. The traditional approach to describing such systems uses Markov models. However, the real non-deterministic mechanism that controls the duration of the system's stay in each of its possible states predetermines the insufficient adequacy of the models obtained in this case. This circumstance makes it expedient to consider models that are more general than Markov ones. In addition, when choosing such models, one should take into account the fundamental often manifested feature of the statistical material actually used in the processing of an array of observations, their small sample. All this, taken together, makes it relevant to study the possibility of developing less demanding, tolerant models of the behavior of complex systems. A method for the analysis of systems described under conditions of initial data uncertainty by semi-Markov models is proposed. The main approaches to the description of this uncertainty are considered: probabilistic, fuzzy, and bi-fuzzy. A procedure has been developed for determining the membership functions of fuzzy numbers based on the results of real data processing. Next, the following tasks are solved sequentially. First, the vector of stationary state probabilities of the Markov chain embedded in the semi-Markov process is found. Then, a set of expected values for the duration of the system's stay in each state before leaving it is determined, after which the required probability distribution of the system states is calculated. The proposed method has been developed to solve the problem in the case when the parameters of the membership functions of fuzzy initial data cannot be clearly estimated under conditions of a small sample

ON THE APPLICATION OF MATRIX SCALING AND POWERING METHODS OF SMALL STATE SPACES FOR SOLVING TRANSIENT DISTRIBUTION IN MARKOV CHAIN

The iterative solution methods for transient distribution in Markov chain is the computation of state probability distributions at an arbitrary point of time, which in the case of a discrete-time Markov chain means, finding the distribution at some arbitrary time step denoted , a row vector whose component is the probability that the Markov chain is in state at time step . In this study, the solutions of transient distribution in Markov chain using matrix scaling and powering methods for small state spaces which produce a significantly more accurate response in less time for some types of situations and also, tries to get to the end result as quickly as possible has been investigated, in order to provide some insight into the solutions of transient distribution of Markov chain. Our goal is to compute solutions and algorithms for tiny state spaces utilizing matrix scaling and powering approaches, which begin with an initial estimate of the solution vector and then comes closer and closer to the true solution with each step or iteration. With the help of several existing Markov chain laws, theorems, and formulas, matrices operations such as multiplication with one or more vectors, Padé variant of the matrix-powering and scaling technique are used. While the algorithms are explained, the transient distribution vector’s Padé approximants , and a backward error analysis of the Padé approximation are obtained for certain illustrative examples..