Inhomogeneous Markov Research Articles

Probabilistic forecasting of industrial electricity load with regime switching behavior

This paper suggests a novel inhomogeneous Markov switching approach for the probabilistic forecasting of industrial companies’ electricity loads, for which the load switches at random times between production and standby regimes. The model that we propose describes the transitions between the regimes using a hidden Markov chain with time-varying transition probabilities that depend on calendar variables. We model the demand during the production regime using an autoregressive moving-average (ARMA) process with seasonal patterns, whereas we use a much simpler model for the standby regime in order to reduce the complexity. The maximum likelihood estimation of the parameters is implemented using a differential evolution algorithm. Using the continuous ranked probability score (CRPS) to evaluate the goodness-of-fit of our model for probabilistic forecasting, it is shown that this model often outperforms classical additive time series models, as well as homogeneous Markov switching models. We also propose a simple procedure for classifying load profiles into those with and without regime-switching behaviors.

Strong Laws of Large Numbers and the Asymptotic Equipartition Property for the Asymptotic N-Branch Markov Chains Indexed by a Cayley Tree

We introduce a concept of asymptotic N-branch Markov chains indexed by a Cayley tree. Then we study the strong law of large numbers and the asymptotic equipartition property for the introduced Markov chains with a finite state space, which generalizes a class of results obtained for inhomogeneous Markov chains indexed by a Cayley tree and nonsymmetric Markov chains indexed by a binary tree.

Accelerated Gibbs sampling of normal distributions using matrix splittings and polynomials

Standard Gibbs sampling applied to a multivariate normal distribution with a specified precision matrix is equivalent in fundamental ways to the Gauss–Seidel iterative solution of linear equations in the precision matrix. Specifically, the iteration operators, the conditions under which convergence occurs, and geometric convergence factors (and rates) are identical. These results hold for arbitrary matrix splittings from classical iterative methods in numerical linear algebra giving easy access to mature results in that field, including existing convergence results for antithetic-variable Gibbs sampling, REGS sampling, and generalizations. Hence, efficient deterministic stationary relaxation schemes lead to efficient generalizations of Gibbs sampling. The technique of polynomial acceleration that significantly improves the convergence rate of an iterative solver derived from a symmetric matrix splitting may be applied to accelerate the equivalent generalized Gibbs sampler. Identicality of error polynomials guarantees convergence of the inhomogeneous Markov chain, while equality of convergence factors ensures that the optimal solver leads to the optimal sampler. Numerical examples are presented, including a Chebyshev accelerated SSOR Gibbs sampler applied to a stylized demonstration of low-level Bayesian image reconstruction in a large 3-dimensional linear inverse problem.

Time inhomogeneous Stochastic Differential Equations involving the local time of the unknown process, and associated parabolic operators

In this paper we study time inhomogeneous versions of one-dimensional Stochastic Differential Equations (SDE) involving the Local Time of the unknown process on curves. After proving existence and uniqueness for these SDEs under mild assumptions, we explore their link with Parabolic Differential Equations (PDE) with transmission conditions. We study the regularity of solutions of such PDEs and ensure the validity of a Feynman–Kac representation formula. These results are then used to characterize the solutions of these SDEs as time inhomogeneous Markov Feller processes.

Ergodicity of inhomogeneous Markov chains through asymptotic pseudotrajectories

In this work, we consider an inhomogeneous (discrete time) Markov chain and are interested in its long time behavior. We provide sufficient conditions to ensure that some of its asymptotic properties can be related to the ones of a homogeneous (continuous time) Markov process. Renowned examples such as a bandit algorithms, weighted random walks or decreasing step Euler schemes are included in our framework. Our results are related to functional limit theorems, but the approach differs from the standard "Tightness/Identification" argument; our method is unified and based on the notion of pseudotrajectories on the space of probability measures.

Environmental Sensors-Based Occupancy Estimation in Buildings via IHMM-MLR

Occupancy estimation in buildings can benefit various applications such as heating, ventilation, and air-conditioning control, space monitoring, and emergency evacuation. Due to the consideration of temporal dependency in occupancy data, hidden Markov model (HMM) has been shown to be effective in occupancy estimation. However, the conventional HMM that assumes invariant temporal dependency of occupancy dynamics for different time instances is unrealistic. Moreover, the performance of the conventional HMM that utilizes mixture of Gaussian for emission probability in terms of continuous observations can be easily affected by the noise in sensory data. To address these problems, in this paper, we propose a new architecture, i.e., inhomogeneous hidden Markov model with multinomial logistic regression (IHMM-MLR), for building occupancy estimation using nonintrusive environmental sensors. Instead of using the time-invariant transition probability matrix, we apply a time-dependent (inhomogeneous) transition probability matrix which can capture the temporal dependency for different time instances. Meanwhile, we employ an efficient probabilistic model, i.e., MLR, for emission probability. Online and offline occupancy estimation schemes are presented for real-time and accurate long-term applications respectively. Real experiments have indicated the effectiveness of our proposed approach.

Probabilistic and Distributed Control of a Large-Scale Swarm of Autonomous Agents

We present a distributed control algorithm simultaneously solving both the stochastic target assignment and optimal motion control for large-scale swarms to achieve complex formation shapes. Our probabilistic swarm guidance using inhomogeneous Markov chains (PSG-IMC) algorithm adopts a Eulerian density-control framework, under which the physical space is partitioned into multiple bins and the swarm's density distribution over each bin is controlled in a probabilistic fashion to efficiently handle loss or the addition of agents. We assume that the number of agents is much larger than the number of bins and that each agent knows in which bin it is located, the desired formation shape, and the objective function and motion constraints. PSG-IMC determines the bin-to-bin transition probabilities of each agent using a time IMC. These time-varying Markov matrices are computed by each agent in real time using the feedback from the current swarm distribution, which is estimated in a distributed manner. The PSG-IMC algorithm minimizes the expected cost of transitions per time instant that are required to achieve and maintain the desired formation shape, even if agents are added to or removed from the swarm. PSG-IMC scales well with a large number of agents and complex formation shapes and can also be adapted for area exploration applications. We demonstrate the effectiveness of this proposed swarm guidance algorithm by using numerical simulations and hardware experiments with multiple quadrotors.

An Entropy Rate Theorem for a Hidden Inhomogeneous Markov Chain

Objective: The main object of our study is to extend some entropy rate theorems to a Hidden Inhomogeneous Markov Chain (HIMC) and establish an entropy rate theorem under some mild conditions. Introduction: A hidden inhomogeneous Markov chain contains two different stochastic processes; one is an inhomogeneous Markov chain whose states are hidden and the other is a stochastic process whose states are observable. Materials and Methods: The proof of theorem requires some ergodic properties of an inhomogeneous Markov chain, and the flexible application of the properties of norm and the bounded conditions of series are also indispensable. Results: This paper presents an entropy rate theorem for an HIMC under some mild conditions and two corollaries for a hidden Markov chain and an inhomogeneous Markov chain. Conclusion: Under some mild conditions, the entropy rates of an inhomogeneous Markov chains, a hidden Markov chain and an HIMC are similar and easy to calculate.

Short term predictions of occupancy in commercial buildings—Performance analysis for stochastic models and machine learning approaches

Real-time occupancy predictions are essential components for the smart buildings in the imminent future. The occupancy information, such as the presence states and the occupants’ number, allows a robust control of the indoor environment to enhance the building energy performances. With many current studies focusing on the commercial building occupancy, most researchers modeled either the occupancy presence or the occupants’ number without evaluating the model potentials on both of them. This study focuses on 1) providing a unique data set containing the occupancy for the offices located in the U.S with difference pattern varieties, 2) proposing two methods, then comparing them with four existing methods, and 3) both presence of occupancy and occupancy number are predicted and tested using the approaches proposed in this study. In detail, the paper develops a new moving-window inhomogeneous Markov model based on change point analysis. A hierarchical probability sampling model is modified based on existed models. They are additional compared to well-known models from previous researchers. The study further explores and evaluates the predictive power of the models by various temporal scenarios, including 15-min ahead, 30-min ahead, 1-h ahead, and 24-h ahead forecasts. The final results show that the proposed Markov model outperforms the other methods with a max 22% difference in terms of presence forecasts for 15-min, 30min and 1-h ahead. The proposed Markov model also outperforms other models in occupancy number prediction for all forecast windows with 0.34 RMSE and 0.23 MAE error respectively. However, there is not much performance difference between models for 24-h ahead predictions of occupancy presence forecast.

Asymptotic expansions for solutions of parabolic systems associated with multi-scale switching diffusions

This work develops asymptotic expansions of systems of partial differential equations associated with multi-scale switching diffusions. The switching process is modeled by using an inhomogeneous continuous-time Markov chain. In the model, there are two small parameters e and δ. The first one highlights the fast switching, whereas the other delineates the slow diffusion. Assuming that e and δ are related in that e = δ γ , our results reveal that different values of γ lead to different behaviors of the underlying systems, resulting in different asymptotic expansions. Although our motivation comes from stochastic problems, the approach is mainly analytic and is constructive. The asymptotic series are rigorously justified with error bounds provided. An example is provided to demonstrate the results.

A new modeling approach for short-term prediction of occupancy in residential buildings

Occupancy models are necessary towards design and operation of smart buildings. Developing an appropriate algorithm to predict occupancy presence will allow a better control and optimization of the whole building energy consumption. However, most previous studies of development of such model only focus on commercial buildings. The occupancy model of residential houses are usually based on Time User Survey data. This study focuses on providing a unique data set of four residential houses collected from occupancy sensors. A new inhomogeneous Markov model for occupancy presence prediction is proposed and compared to commonly used models such as Probability Sampling, Artificial Neural Network, and Support Vector Regression. Training periods for the presence prediction are optimized based on change-point analysis of historical data. The study further explores and evaluates the predictive capability of the models by various temporal scenarios, including 15-min ahead, 30-min ahead, 1-hour ahead, and 24-hour ahead forecasts. The spatial-level comparison is additionally conducted by evaluating the prediction accuracy at both room-level and house-level. The final results show that the proposed Markov model outperforms the other methods in terms of an average 5% correctness with 11% maximum difference in 15-min ahead forecast of the occupancy presence. However, there is not much differences observed for 24-hour ahead forecasts.

Probabilistic Forecasting of Electricity Load with Inhomogeneous Markov Switching Models

In this paper we suggest a novel inhomogeneous Markov switching approach for probabilistic forecasting of electricity load of industrial companies, for which the load switches at random times between a production and a standby regime. The model we propose describes the transitions between the regimes by a hidden Markov chain with time-varying transition probabilities depending on calendar variables. The demand during the production regime is modeled by an ARMA process with seasonal patterns, whereas we use a much simpler model for the standby regime to reduce complexity. The maximum likelihood estimation of the parameters is implemented with a Differential Evolution algorithm. Using the continuous ranked probability score (CRPS) to evaluate the goodness of fit of our model for probabilistic forecasting it is shown that this model often outperforms classical additive time series models as well as homogeneous Markov switching models. We also propose a simple procedure to classify load profiles into ones with and without regime-switching behavior.

A Time Variant Log-Linear Learning Approach to the SET K-COVER Problem in Wireless Sensor Networks.

Toward the global optimality of the SET K-COVER problem in wireless sensor networks, we view each sensor node as a rational player and propose a time variant log-linear learning algorithm (TVLLA) that relies on local information only. By defining the local utility as the normalized area covered by one node alone, we formulate the problem as a spatial potential game. The resulting optimal Nash equilibria correspond to the optimal partition. Such equilibria are obtained by designing a time varying parameter that approaches infinity with time. Using inhomogeneous Markov chain theory, we prove that the TVLLA guarantees convergence to the optimal solution with probability 1. Comparison results against traditional methods demonstrate that the algorithm can also provide better near-optimal solutions in a reasonable computation time than the state-of-the-art. Our findings pave a new way to reach the global optimality of the SET K-COVER problem in a distributed manner as well as other potential games from the view of self-organized optimization.

A learned sparseness and IGMRF-based regularization framework for dense disparity estimation using unsupervised feature learning

In this work, we propose a new approach for dense disparity estimation in a global energy minimization framework. We propose to use a feature matching cost which is defined using the learned hierarchical features of given left and right stereo images and we combine it with the pixel-based intensity matching cost in our energy function. Hierarchical features are learned using the deep deconvolutional network which is trained in an unsupervised way using a database consisting of large number of stereo images. In order to perform the regularization, we propose to use the inhomogeneous Gaussian Markov random field (IGMRF) and sparsity priors in our energy function. A sparse autoencoder-based approach is proposed for learning and inferring the sparse representation of disparities. The IGMRF prior captures the smoothness as well as preserves sharp discontinuities while the sparsity prior captures the sparseness in the disparity map. Finally, an iterative two-phase algorithm is proposed to estimate the dense disparity map where in phase one, sparse representation of disparities are inferred from the trained sparse autoencoder, and IGMRF parameters are computed, keeping the disparity map fixed and in phase two, the disparity map is refined by minimizing the energy function using graph cuts, with other parameters fixed. Experimental results on the Middlebury stereo benchmarks demonstrate the effectiveness of the proposed approach.

Fluctuation-dissipation theorems for inhomogeneous Markov jump processes and a biochemical application

In this paper, we establish a rigorous mathematical theory of three types of fluctuation-dissipation theorems (FDTs) for inhomogeneous Markov jump processes. It turns out that the FDTs and the response formula proved in this paper apply to any form of external perturbations and thus are quite general. Further physical and biochemical applications are also discussed. In particular, the FDTs are used to study an important biochemical phenomenon called adaptation.

Parameter Estimation of a Two-Colored Urn Model Class.

Though widely used in applications, reinforced random walk on graphs have never been the subject of a valid statistical inference. We develop in this paper a statistical framework for a general two-colored urn model. The probability to draw a ball at each step depends on the number of balls of each color and on a multidimensional parameter through a function, called choice function. We introduce two estimators of the parameter: the maximum likelihood estimator and a weighted least squares estimator which is less efficient, but is closer to the calibration techniques used in the applied literature. In general, the model is an inhomogeneous Markov chain and this property makes the estimation of the parameter impossible on a single path, even if it were infinite. Therefore we assume that we observe i.i.d. experiments, each of a predetermined finite length. This is coherent with the usual experimental set-ups. We apply the statistical framework to a real life experiment: the selection of a path among pre-existing channels by an ant colony. We performed experiments, which consisted of letting ants pass through the branches of a fork. We consider the particular urn model proposed by J.-L. Deneubourg et al. in 1990 to describe this phenomenon. We simulate this model for several parameter values in order to assess the accuracy of the MLE and the WLSE. Then we estimate the parameter from the experimental data and evaluate confident regions with Bootstrap algorithms. The findings of this paper do not contradict the biological literature, but give statistical significance to the values of the parameter found therein.

Truncation Bounds for Approximations of Inhomogeneous Continuous-Time Markov Chains

Weakly ergodic continuous-time countable Markov chains are studied. We obtain uniform in time bounds for approximations via truncations by analogous smaller chains under some natural assumptions.MSC codesinhomogeneous continuous-time Markov processesapproximationstruncationsweak ergodicity

Population Density, Climate Variables and Poverty Synergistically Structure Spatial Risk in Urban Malaria in India.

BackgroundThe world is rapidly becoming urban with the global population living in cities projected to double by 2050. This increase in urbanization poses new challenges for the spread and control of communicable diseases such as malaria. In particular, urban environments create highly heterogeneous socio-economic and environmental conditions that can affect the transmission of vector-borne diseases dependent on human water storage and waste water management. Interestingly India, as opposed to Africa, harbors a mosquito vector, Anopheles stephensi, which thrives in the man-made environments of cities and acts as the vector for both Plasmodium vivax and Plasmodium falciparum, making the malaria problem a truly urban phenomenon. Here we address the role and determinants of within-city spatial heterogeneity in the incidence patterns of vivax malaria, and then draw comparisons with results for falciparum malaria.Methodology/principal findingsStatistical analyses and a phenomenological transmission model are applied to an extensive spatio-temporal dataset on cases of Plasmodium vivax in the city of Ahmedabad (Gujarat, India) that spans 12 years monthly at the level of wards. A spatial pattern in malaria incidence is described that is largely stationary in time for this parasite. Malaria risk is then shown to be associated with socioeconomic indicators and environmental parameters, temperature and humidity. In a more dynamical perspective, an Inhomogeneous Markov Chain Model is used to predict vivax malaria risk. Models that account for climate factors, socioeconomic level and population size show the highest predictive skill. A comparison to the transmission dynamics of falciparum malaria reinforces the conclusion that the spatio-temporal patterns of risk are strongly driven by extrinsic factors.Conclusion/significanceClimate forcing and socio-economic heterogeneity act synergistically at local scales on the population dynamics of urban malaria in this city. The stationarity of malaria risk patterns provides a basis for more targeted intervention, such as vector control, based on transmission ‘hotspots’. This is especially relevant for P. vivax, a more resilient parasite than P. falciparum, due to its ability to relapse and the operational shortcomings of delivering a “radical cure”.

Global convergence of discrete-time inhomogeneous Markov processes from dynamical systems perspective

Given the continuous real-valued objective function f and the discrete time inhomogeneous Markov process Xt defined by the recursive equation of the form Xt+1=Tt(Xt,Yt), where Yt is an independent sequence, we target the problem of finding conditions under which the Xt converges towards the set of global minimums of f. Our methodology is based on the Lyapunov function technique and extends the previous results to cover the case in which the sequence f(Xt) is not assumed to be a supermartingale. We provide a general convergence theorem. An application example is presented: the general result is applied to the Simulated Annealing algorithm.

An anisotropic and inhomogeneous hidden Markov model for the classification of water quality spatio‐temporal series on a national scale: The case of Scotland

This work presents a case study about the evaluation of the water quality dynamics in each of the 56 major catchments in Scotland, for a period of 10 years. Data are obtained by monthly sampling of water contaminants, in order to monitor discharges from the land to the sea. We are interested in the multivariate time series of ammonia, nitrate, and phosphorus. The time series may present issues that make their analysis complex: non‐linearity, non‐normality, weak dependency, seasonality, and missing values. The goals of this work are the classification of the observations into a small set of homogeneous groups representing ordered categories of pollution, the detection of change‐points, and the modeling of data heterogeneity. These aims are pursued by developing a novel spatio‐temporal hidden Markov model, whose hierarchical structure was motivated by the data set to study: the observations are displayed on a cylindrical lattice and driven by an anisotropic and inhomogeneous hidden Markov random field. As a result, four hidden states were selected, showing that catchments could be grouped spatially, with a strong relationship with the dominating land use. This method represents a useful tool for water managers to have a nationwide picture in combination of temporal dynamics.