Non-homogeneous Markov Research Articles

Leader-Following Consensus for Multi-Agent Systems With Asynchronous Control Modes Under Nonhomogeneous Markovian Jump Network Topology

Under a nonhomogeneous Markovian jump network topology, this paper addresses the problem of leader-following consensus for multi-agent systems with an asynchronous control mode. Specifically, based on an asynchronous mode-dependent Lyapunov function and a suitable relaxation process, sufficient conditions for leader-following consensus are derived in such a way that 1) non-convex terms multiplied by time-varying signed parameters can be addressed, 2) the emergence of an asynchronous control mode can be reflected in the control synthesis of the follower agents, and 3) the constraints of two combined parameters can be explicitly imposed on the process of transforming parameterized linear matrix inequalities (PLMIs) into LMIs. Finally, two illustrative examples are given to show the validity of the proposed method.

Nonparametric tests for transition probabilities in nonhomogeneous Markov processes

This paper proposes nonparametric two-sample tests for the direct comparison of the probabilities of a particular transition between states of a continuous time nonhomogeneous Markov process with a finite state space. The proposed tests are a linear nonparametric test, an -norm-based test and a Kolmogorov–Smirnov-type test. Significance level assessment is based on rigorous procedures, which are justified through the use of modern empirical process theory. Moreover, the -norm and the Kolmogorov–Smirnov-type tests are shown to be consistent for every fixed alternative hypothesis. The proposed tests are also extended to more complex situations such as cases with incompletely observed absorbing states and non-Markov processes. Simulation studies show that the test statistics perform well even with small sample sizes. Finally, the proposed tests are applied to data on the treatment of early breast cancer from the European Organization for Research and Treatment of Cancer (EORTC) trial 10854, under an illness-death model.

A general piecewise multi-state survival model: application to breast cancer

Multi-state models are considered in the field of survival analysis for modelling illnesses that evolve through several stages over time. Multi-state models can be developed by applying several techniques, such as non-parametric, semi-parametric and stochastic processes, particularly Markov processes. When the development of an illness is being analysed, its progression is tracked periodically. Medical reviews take place at discrete times, and a panel data analysis can be formed. In this paper, a discrete-time piecewise non-homogeneous Markov process is constructed for modelling and analysing a multi-state illness with a general number of states. The model is built, and relevant measures, such as survival function, transition probabilities, mean total times spent in a group of states and the conditional probability of state change, are determined. A likelihood function is built to estimate the parameters and the general number of cut-points included in the model. Time-dependent covariates are introduced, the results are obtained in a matrix algebraic form and the algorithms are shown. The model is applied to analyse the behaviour of breast cancer. A study of the relapse and survival times of 300 breast cancer patients who have undergone mastectomy is developed. The results of this paper are implemented computationally with MATLAB and R.

Non-homogeneous hidden Markov model for downscaling of short rains occurrence in Kenya

Non-homogeneous hidden Markov model (NHMM) is applied in modeling of daily rainfall occurrences across 16 synoptic stations in Kenya. The time series of the data sets was during the October–December (OND “short rains”) season from 1979 to 2005. The tool assumes that the diurnal rainfall events at a network of observing stations are influenced by unobserved states, that is, “weather states.” These states’ evolution is modeled based on a first-order Markov criterion with state-to-state transition probabilities conditioned on some atmospheric variable indices. The five states are selected using the Bayes information criterion (BIC). To downscale daily rainfall occurrences across 16 stations, a NHMM employed global circulation model (GCM) projection outputs for daily precipitation and sea surface temperatures during the study period. The interannual variability of the mean GCM simulated precipitation and mean historical stations rainfall depicts a weak correlation though significant at 90% confidence level. Thus, it implies that GCM-NHMM simulations do not simulate the rainfall occurrences well. The consecutive wet spell length between the historical rainfall datasets and GCM-NHMM simulated precipitation for 90-day frequencies shows a strong positive correlation significant at 95% confidence level. The findings from this study reveal that the modeling tool is suitable for statistical downscaling of daily rainfall occurrences at multisite stations network. The statistical inference from the model is applicable for drought/flood preparedness, water resource management, and inputs into crop models.

Abnormality Detection and Monitoring in Multi-Sensor Molecular Communication

In this paper, we investigate the problem of detecting and monitoring changes (abnormality) in molecular communication (MC), using the quickest change detection (QCD) schemes. The objective is to watch an environment using a sensor network and make decisions on the time and location of changes based on the received signals from sensors in the fusion center (FC). Such assumptions call for considering spatial and temporal correlations among sensors’ transmitting signals. We use the framework of Partially Observable Markov Decision Processes (POMDPs) based on non-homogeneous Markov models. The metric in detection (stopping-time) scenario is to minimize the delay of announcing an abnormality occurrence from the real change time, in a two-hypothesis setting, subject to constraints on false alarm and missed identification probabilities. Since the optimum detector is very complicated, by utilizing a myopic policy, sub-optimum detectors with less complexity and acceptable performances are proposed. For monitoring scenario, the goal is to determine how the changes are spread in time, in a multi-hypothesis setting. Such goal asks for deciding on multiple changes in time, which is different from the stopping-time scenario. In the latter scenario, we decide only about whether one change has occurred or not, then we stop decision making. In the monitoring scenario, one optimum and two sub-optimum metrics are defined and their corresponding detectors are derived. In this scenario, the detectors make decisions only in the time slots that a reliable decision can be made. The defined metrics minimize the waiting-time , the time that the detector does not make decision about any of the hypotheses due to lack of assurance, subject to constraints on false alarm and missed identification probabilities. In both scenarios, we mathematically model the system and evaluate the performance of the proposed detectors. For performance evaluation, we consider modeling a previously reported system of tumor growth in a tissue. The results confirm out-performance of the proposed detectors in comparison with existing ones.

A data-assisted reliability model for carrier-assisted cold data storage systems

Cold data storage systems are used to allow long term digital preservation for institutions’ archive. The common functionality among cold and warm/hot data storage is that the data is stored on some physical medium for read-back at a later time. However in cold storage, write and read operations are not necessarily done in the same exact geographical location. Hence, a third party assistance is typically utilized to bring together the medium and the drive. On the other hand, the reliability modeling of such a decomposed system poses few challenges that do not necessarily exist in other warm/hot storage alternatives such as fault detection and absence of the carrier, all totaling up to the data unavailability issues. In this paper, we propose a generalized non-homogenous Markov model that encompasses the aging of the carriers in order to address the requirements of today’s cold data storage systems in which the data is encoded and spread across multiple nodes for the long-term data retention. We have derived useful lower/upper bounds on the overall system availability. Furthermore, the collected field data is used to estimate parameters of a Weibull distribution to accurately predict the lifetime of the carriers in an example scale-out setting.

Mixture of Joint Nonhomogeneous Markov Chains to Cluster and Model Water Consumption Behavior Sequences

The emergence of smart meters has fostered the collection of massive data that support a better understanding of consumer behaviors and better management of water resources and networks. The main focus of this article is to analyze consumption behavior over time; thus, we first identify the main weekly consumption patterns. This approach allows each meter to be represented by a categorical series, where each category corresponds to a weekly consumption behavior. By considering the resulting consumption behavior sequences, we propose a new methodology based on a mixture of nonhomogeneous Markov models to cluster these categorical time series. Using this method, the meters are described by the Markovian dynamics of their cluster. The latent variable that controls cluster membership is estimated alongside the parameters of the Markov model using a novel classification expectation maximization algorithm. A specific entropy measure is formulated to evaluate the quality of the estimated partition by considering the joint Markovian dynamics. The proposed clustering model can also be used to predict future consumption behaviors within each cluster. Numerical experiments using real water consumption data provided by a water utility in France and gathered over 19 months are conducted to evaluate the performance of the proposed approach in terms of both clustering and prediction. The results demonstrate the effectiveness of the proposed method.

Moderate deviations for nonhomogeneous Markov chains

In this work, we establish moderate deviation principles for bounded functionals and empirical measures for nonhomogeneous Markov chains with finite state space under the condition of convergence of transition probability matrices for nonhomogeneous Markov chains in Cesàro sense.

SOME LIMIT THEOREMS OF DELAYED AVERAGES FOR COUNTABLE NONHOMOGENEOUS MARKOV CHAINS

AbstractThe purpose of this paper is to establish some limit theorems of delayed averages for countable nonhomogeneous Markov chains. The definition of the generalized C-strong ergodicity and the generalized uniformly C-strong ergodicity for countable nonhomogeneous Markov chains is introduced first. Then a theorem about the generalized C-strong ergodicity and the generalized uniformly C-strong ergodicity for the nonhomogeneous Markov chains is established, and its applications to the information theory are given. Finally, the strong law of large numbers of delayed averages of bivariate functions for countable nonhomogeneous Markov chains is proved.

Bayesian variable selection in non-homogeneous hidden Markov models through an evolutionary Monte Carlo method

Hidden Markov models (HMMs) are dynamic mixture models applied to time series in order to classify the observations into a small number of homogeneous groups, to understand when change points occur, and to model data heterogeneity through the switching between subseries with different state-dependent parameters. In the most general case, HMMs have an unobserved Markov chain whose transition probabilities are time-varying and dependent on exogenous variables through multinomial logit functions. When many covariates are available it is worthwhile selecting the subsets of variables which might affect most each row of the transition matrices. A Bayesian method for the stochastic selection of subsets of covariates is proposed by developing a novel evolutionary Monte Carlo algorithm. The methodology is illustrated and shown to be effective by performing experiments and comparisons on both synthetic data sets and a real multivariate time series with covariates.

Real time Diagnostics and Prognostics of UAV Lithium-Polymer Batteries

This paper examines diagnostics and prognostics of Lithium-Polymer (Li-Po) batteries for unmanned aerial vehicles (UAVs). Several discharge voltage histories obtained during actual indoor flights constitute the training data for a data-driven approach, utilizing the Non-Homogenous Hidden Semi Markov model (NHHSMM). NHHSMM is a suitable candidate as it has a rich mathematical structure, which is capable of describing the discharge process of Li-Po batteries and providing diagnostic and prognostic measures. Diagnostics and prognostics in unseen data are obtained and compared with the actual remaining flight time in order to validate the effectiveness of the selected model.

Identifying the Effect of Public Holidays on Daily Demand for Gas

Summary To reduce operational costs and to ensure security of supply, gas distribution networks require accurate forecasts of the demand for gas. Among domestic and commercial customers, demand relates primarily to the weather and patterns of life and work. Public holidays have a pronounced effect which often spreads into neighbouring days. We call this spread the ‘proximity effect’. Traditionally, the days over which the proximity effect is felt are prespecified in fixed windows around each holiday, allowing no uncertainty in their identification. We are motivated by an application to modelling daily gas demand in two large British regions. We introduce a novel model which does not fix the days on which the proximity effect is felt. Our approach uses a four-state, non-homogeneous hidden Markov model, with cyclic dynamics, where the classification of days as public holidays is observed, but the assignment of days as ‘pre-holiday’, ‘post-holiday’ or ‘normal’ days is unknown. The number of days to the preceding and succeeding holidays guide transitions between states. We apply Bayesian inference and illustrate the benefit of our modelling approach. A version of the model is now being used by one of the UK's regional distribution networks.

Fault Detection Filtering for a Class of Nonhomogeneous Markov Jump Systems with Random Sensor Saturations

This paper is concerned with the problem of fault detection filtering for a class of nonhomogeneous Markov jump systems with randomly occurring sensor saturations, where it is assumed that sensor saturation is randomly occur satisfying the Bernoulli distribution, meanwhile transition probabilities in nonhomogeneous Markov chain are time-varying. By employing the suitable Lyapunov technique in combination with Linear Matrix Inequalities (LMIs), some sufficient criteria are derived to guarantee that the augmented system is mean square stochastically stable with an H∞ performance level. Finally, a representative example of industrial stirred tank reactor systems is presented to illustrate the theoretical results, and it then turns out that the feasible filtering parameters can be exactly fixed by Matlab Tool Box, and the H∞ performance of the system can be fully guaranteed by the proposed control strategy.

Intelligent data-driven prognostic methodologies for the real-time remaining useful life until the end-of-discharge estimation of the Lithium-Polymer batteries of unmanned aerial vehicles with uncertainty quantification

In this paper, the discharge voltage is utilized as a critical indicator towards the probabilistic estimation of the Remaining Useful Life until the End-of-Discharge of the Lithium-Polymer batteries of unmanned aerial vehicles. Several discharge voltage histories obtained during actual flights constitute the in-house developed training dataset. Three data-driven prognostic methodologies are presented based on state-of-the-art as well as innovative mathematical models i.e. Gradient Boosted Trees, Bayesian Neural Networks and Non-Homogeneous Hidden Semi Markov Models. The training and testing process of all models is described in detail. Remaining Useful Life prognostics in unseen data are obtained from all three methodologies. Beyond the mean estimates, the uncertainty associated with the point predictions is quantified and upper/lower confidence bounds are also provided. The Remaining Useful Life prognostics during six random flights starting from fully charged batteries are presented, discussed and the pros and cons of each methodology are highlighted. Several special metrics are utilized to assess the performance of the prognostic algorithms and conclusions are drawn regarding their prognostic capabilities and potential.

The strong law of large numbers for moving average of continuous state nonhomogeneous Markov chains

In this paper, we will study the limit theorem for moving average of continuous state nonhomogeneous Markov chains. Firstly, we introduce the concept of ergodicity for continuous state homogeneous Markov chains. Meanwhile, we establish some lemmas which are the basis of the main result. Finally, we obtain the strong law of large numbers for moving average of continuous state nonhomogeneous Markov chains.

Importance analysis considering time‐varying parameters and different perturbation occurrence times

AbstractImportance measures are integral parts of risk assessment for risk‐informed decision making. Because the parameters of a risk model, such as the component failure rates, are functions of time and a perturbation (change) in their values can occur during the mission time, time dependence must be considered in the evaluation of the importance measures. In this paper, it is shown that the change in system performance at time t, and consequently the importance of the parameters at time t, depends on the parameters perturbation time and their value functions during the system mission time. We consider a nonhomogeneous continuous time Markov model of a series‐parallel system to propose the mathematical proofs and simulations, while the ideas are also shown to be consistent with general models having nonexponential failure rates. Two new measures of importance and a simulation scheme for their computation are introduced to account for the effect of perturbation time and time‐varying parameters.

A Weather‐Regime‐Based Stochastic Weather Generator for Climate Vulnerability Assessments of Water Systems in the Western United States

AbstractVulnerability‐based frameworks are increasingly used to better understand water system performance under climate change. This work advances the use of stochastic weather generators for climate vulnerability assessments that simulate weather based on patterns of regional atmospheric flow (i.e., weather regimes) conditioned on global‐scale climate features. The model is semiparametric by design and includes (1) a nonhomogeneous Markov chain for weather regime simulation; (2) block bootstrapping and a Gaussian copula for multivariate, multisite weather simulation; and (3) modules to impose thermodynamic and dynamical climate change, including Clausius‐Clapeyron precipitation scaling, elevation‐dependent warming, and shifting dynamics of the El Niño–Southern Oscillation (ENSO). In this way, the model can be used to evaluate climate impacts on water systems based on hypotheses of dynamic and thermodynamic climate change. The model is developed and tested for cold‐season climate in the Tuolumne River Basin in California but is broadly applicable across the western United States. Results show that eight weather regimes exert strong influences over local climate in the Tuolumne Basin. Model simulations adequately preserve many of the historical statistics for precipitation and temperature across sites, including the mean, variance, skew, and extreme values. Annual precipitation and temperature are somewhat underdispersed, and precipitation spell statistics are negatively biased by 1‐2 days. For simulations of future climate, the model can generate a range of Clausius‐Clapeyron scaling relationships and modes of elevation‐dependent warming. Model simulations also suggest a muted response of Tuolumne climate to changes in ENSO variability.

Mean-variance portfolio selection with an uncertain exit-time in a regime-switching market

In this paper, we deal with multi-period mean-variance portfolio selection problems with an exogenous uncertain exit-time in a regime-switching market. The market is modelled by a non-homogeneous Markov chain in which the random returns of assets depend on the states of the market and investment time periods. Applying the Lagrange duality method, we derive explicit closed-form expressions for the optimal investment strategies and the efficient frontier. Also, we show that some known results in the literature can be obtained as special cases of our results. A numerical example is provided to illustrate the results.

Quantized control for nonhomogeneous Markovian jump T-S fuzzy systems with missing measurements

In this paper, in terms of the T-S fuzzy technique, the quantization control designs are resolved for a class of nonhomogeneous Markov jump systems (MJSs) with partially unknown transition probabilities. Different from the previous research, the transition probabilities are time-variant and not known exactly in the MJSs. Particularly in a network environment, it is considered that the effects of data packet dropouts and the occurrence of signal quantization simultaneously emerge in the closed-loop circuit. Furthermore, based on a fuzzy Lyapunov function and a set of linear matrix inequalities, one can achieve the desired H∞ performance and the sufficient conditions such that the corresponding closed-loop system is stochastically stable. By the cone complementarity linearisation (CCL) procedure, a sequential minimization problem is tackled efficiently to gain the solutions of the dynamic output feedback controller (DOFC). Finally, the validity of the suggested technique is showed via a simulation example.

Deviation inequalities for separately Lipschitz functionals of composition of random functions

We consider a class of non-homogeneous Markov chains, that contains many natural examples. Next, using martingale methods, we establish some deviation and moment inequalities for separately Lipschitz functions of such a chain, under moment conditions on some dominating random variables.