Markov Chain Transition Research Articles

A Novel Stochastic Seasonal Fuzzy Time Series Forecasting Model

Fuzzy time series approach has been widely used to analyze real-world time series in recent years since using this approach has some important advantages. Various fuzzy time series models have been proposed in the literature in order to reach better forecasting results. A few of these models have been suggested to forecast seasonal time series and called as seasonal fuzzy time series. In this study, a new seasonal fuzzy time series forecasting model based on Markov chain transition matrix is proposed. In the proposed approach, fuzzy inference process is performed by using transition probabilities. Therefore, fuzzy time series approach proposed in this study is the first stochastic seasonal fuzzy time series method in the literature. To show the forecasting performance of the proposed method, it is applied to two real-world time series: the quarterly U.S. beer production and the number of foreign tourists visiting Turkey. As a result of the implementation, it is observed that the proposed method produces accurate forecasting results for both time series.

Bayesian Annealed Sequential Importance Sampling: An Unbiased Version of Transitional Markov Chain Monte Carlo

Abstract The transitional Markov chain Monte Carlo (TMCMC) is one of the efficient algorithms for performing Markov chain Monte Carlo (MCMC) in the context of Bayesian uncertainty quantification in parallel computing architectures. However, the features that are associated with its efficient sampling are also responsible for its introducing of bias in the sampling. We demonstrate that the Markov chains of each subsample in TMCMC may result in uneven chain lengths that distort the intermediate target distributions and introduce bias accumulation in each stage of the TMCMC algorithm. We remedy this drawback of TMCMC by proposing uniform chain lengths, with or without burn-in, so that the algorithm emphasizes sequential importance sampling (SIS) over MCMC. The proposed Bayesian annealed sequential importance sampling (BASIS) removes the bias of the original TMCMC and at the same time increases its parallel efficiency. We demonstrate the advantages and drawbacks of BASIS in modeling of bridge dynamics using finite elements and a disk-wall collision using discrete element methods.

Closure to “Transitional Markov Chain Monte Carlo: Observations and Improvements” by Wolfgang Betz, Iason Papaioannou, and Daniel Straub

Closure to “Transitional Markov Chain Monte Carlo: Observations and Improvements” by Wolfgang Betz, Iason Papaioannou, and Daniel Straub

Offloading in HetNet: A Coordination of Interference Mitigation, User Association, and Resource Allocation

The use of heterogeneous small cell-based networks to offload the traffic of existing cellular systems has recently attracted significant attention. One main challenge is solving the joint problems of interference mitigation, user association, and resource allocation. These problems are formulated as an optimization which is then analyzed using two different approaches: Markov approximation and log-linear learning. However, finding the optimal solutions of both approaches requires complete information of the whole network which is not scalable with the network size. Thus, an approach based on a Markov approximation with a novel Markov chain design and transition probabilities is proposed. This approach enables the Markov chain to converge to the bounded near optimal distribution without complete information. In the game-theoretic approach, the payoff-based log-linear learning is used, and it converges in probability to a mixed-strategy $\epsilon$ -Nash equilibrium. Based on the principles of these two approaches, a highly randomized self-organizing algorithm is proposed to reduce the gap between optimal and converged distributions. Simulation results show that all of the proposed algorithms effectively offload more than 90 percent of the traffic from the macrocell base station to small cell base stations. Moreover, the results also show that the algorithms converge quickly irrespective of the number of possible configurations.

Boosting, downsizing and optimality of test functions of Markov chains

Test functions play an important role in Markov chain theory. Stability of a Markov chain can be demonstrated by constructing a test function of the chain that satisfies a stochastic drift criterion. The test function defines a class of functions of the process for which limit laws hold, yields bounds on the convergence of the Markov chain transition probabilities to the stationary distribution, and provides information concerning the mixing properties of the chain. Under certain conditions, these results can be improved by using a new test function derived from a known test function of a Markov chain.

MixedTrails: Bayesian hypothesis comparison on heterogeneous sequential data

Sequential traces of user data are frequently observed online and offline, e.g., as sequences of visited websites or as sequences of locations captured by GPS. However, understanding factors explaining the production of sequence data is a challenging task, especially since the data generation is often not homogeneous. For example, navigation behavior might change in different phases of browsing a website, or movement behavior may vary between groups of users. In this work, we tackle this task and propose MixedTrails, a Bayesian approach for comparing the plausibility of hypotheses regarding the generative processes of heterogeneous sequence data. Each hypothesis is derived from existing literature, theory or intuition and represents a belief about transition probabilities between a set of states that can vary between groups of observed transitions. For example, when trying to understand human movement in a city and given some observed data, a hypothesis assuming tourists to be more likely to move towards points of interests than locals, can be shown to be more plausible than a hypothesis assuming the opposite. Our approach incorporates such hypotheses as Bayesian priors in a generative mixed transition Markov chain model, and compares their plausibility utilizing Bayes factors. We discuss analytical and approximate inference methods for calculating the marginal likelihoods for Bayes factors, give guidance on interpreting the results, and illustrate our approach with several experiments on synthetic and empirical data from Wikipedia and Flickr. Thus, this work enables a novel kind of analysis for studying sequential data in many application areas.

Erratum for “Transitional Markov Chain Monte Carlo: Observations and Improvements” by Wolfgang Betz, Iason Papaioannou, and Daniel Straub

Erratum for “Transitional Markov Chain Monte Carlo: Observations and Improvements” by Wolfgang Betz, Iason Papaioannou, and Daniel Straub

Assessing transition probability of bridge deterioration using Dempster–Shafer theory of evidence

Reliable deterioration models for forecasting the condition of bridge elements, systems, and networks are essential components of any bridge management system. Estimating deterioration transition probabilities is the basis of the popular Markov chain deterioration model. Transition probabilities ha ve been produced in the literature by statistical and probabilistic analysis performed on available bridge inspection and condition data. The available data may be limited or inconsistent with the Markov chain transition period. In addition, the conventional probability theory has limitations when applied to problems with a stochastic nature such as bridge deterioration. This paper introduces a novel method based on the theory of evidence for bridge deterioration modeling through expert judgment elicitation. The advantages of the theory of evidence over the traditional probability theory are discussed and the process for the theory implementation is demonstrated with a case study to validate the application of Dempster–Shafer theory of evidence to estimate the transition probabilities. Based on the results, the theory of evidence is proposed as a scientific expert judgement elicitation technique in the area of bridge condition rating and deterioration modeling. Expert judgment elicitation and theory of evidence application hold potential in the field of bridge management and require further investigation and research.

Discussion of “Transitional Markov Chain Monte Carlo: Observations and Improvements” by Wolfgang Betz, Iason Papaioannou, and Daniel Straub

Discussion of “Transitional Markov Chain Monte Carlo: Observations and Improvements” by Wolfgang Betz, Iason Papaioannou, and Daniel Straub

Promptly assessing probability of barge–bridge collision damage of piers through probabilistic-based classification of machine learning

This paper presents a novel probabilistic classification framework for promptly assessing the probability of barge–bridge collision damage of piers based on Bayesian inference with Transitional Markov Chain Monte Carlo simulations. The main idea is to divide the potential damage region into multiple discrete sub-regions and train a probabilistic classifier for each sub-region using data from a set of computationally simulated events or real events or combination of both. The novelty of the presented framework includes: (1) intensive computation of dynamic responses of a structure at different damage scenarios is conducted prior to collision events to generate data for training the probabilistic classifiers; (2) the optimal threshold for reliably triggering damage alarm is determined historically based on all previously known events in the training dataset; (3) uncertainties of the classification model and extracted dynamic features are considered based on the Bayesian model selection and the rate of correction classification and quantified into the probabilities of damage occurrence in different sub-regions. Furthermore, the presented framework can be implemented recursively with different levels of hierarchical divisions of sub-regions to more precisely locate damage in a structure. The applicability of the presented framework is demonstrated through the numerical simulation of identifying barge–bridge collision damage locations in one pier of a prototype bridge. Finally, limitations and future research directions are also discussed.

Fusing heterogeneous data for the calibration of molecular dynamics force fields using hierarchical Bayesian models.

We propose a hierarchical Bayesian framework to systematically integrate heterogeneous data for the calibration of force fields in Molecular Dynamics (MD) simulations. Our approach enables the fusion of diverse experimental data sets of the physico-chemical properties of a system at different thermodynamic conditions. We demonstrate the value of this framework for the robust calibration of MD force-fields for water using experimental data of its diffusivity, radial distribution function, and density. In order to address the high computational cost associated with the hierarchical Bayesian models, we develop a novel surrogate model based on the empirical interpolation method. Further computational savings are achieved by implementing a highly parallel transitional Markov chain Monte Carlo technique. The present method bypasses possible subjective weightings of the experimental data in identifying MD force-field parameters.

Lifelong learning” is a new paradigm of personnel training in enterprises

World practice to overcome the lack of knowledge and meet educational needs under the terms of career growth and personal improvement and development is based on the modern paradigm of continuous postgraduate education ‑ lifelong learning (LLL). In the context of learning management, the analysis of features and synthesis of the structure of interaction of the basic process entities as knowledge holders were performed. Referring basic knowledge holders ‑ team, customer, training and in-depth knowledge systems to certain discrete states, which the system may take, allows representing the system using a Markov chain with four discrete states. A directed graph of the Markov chain reflects the secondary projection of a communication on the knowledge space rather than the physical interaction of the knowledge holders. Markov chain transition probabilities for the project environment define its characteristics that vary in discrete time by the steps in the form of a set of state probabilities. Cognitive properties of the model which is based on the Markov chain makes it possible to set some specific values of the transition probabilities of the system. The results obtained are the justification for continuing training and acquiring additional education of personnel in the deficit of knowledge that is necessary for the implementation of unique projects. Using the developed cognitive model of the interaction of knowledge in project management, the paradigm of “lifelong learning” was justified. The knowledge management structure includes four fundamental entities: the customer, the team, training and in-depth knowledge systems of the knowledge owners. These entities are in a constant process of knowledge sharing. The systems of training and in-depth knowledge play a role of generating new knowledge, which was absent in the project environment. Continuing education ideally is aimed at balancing between the needs of society and the motivational structure of personality that actually should be implemented through the mechanisms of the influence of education on the socialization of the individual.

Guided wave-based identification of multiple cracks in beams using a Bayesian approach

A guided wave damage identification method using a model-based approach is proposed to identify multiple cracks in beam-like structures. The guided wave propagation is simulated using spectral finite element method and a crack element is proposed to take into account the mode conversion effect. The Bayesian model class selection algorithm is employed to determine the crack number and then the Bayesian statistical framework is used to identify the crack parameters and the associated uncertainties. In order to improve the efficiency and ensure the reliability of identification, the Transitional Markov Chain Monte Carlo (TMCMC) method is implemented in the Bayesian approach. A series of numerical case studies are carried out to assess the performance of the proposed method, in which the sensitivity of different guided wave modes and effect of different levels of measurement noise in identifying different numbers of cracks is studied in detail. The proposed method is also experimentally verified using guided wave data obtained from laser vibrometer. The results show that the proposed method is able to accurately identify the number, locations and sizes of the cracks, and also quantify the associated uncertainties. In addition the proposed method is robust under measurement noise and different situations of the cracks.

Calibrating Lattice Boltzmann flow simulations and estimating uncertainty in the permeability of complex porous media

A common way to simulate fluid flow in porous media is to use Lattice Boltzmann (LB) methods. Permeability predictions from such flow simulations are controlled by parameters whose settings must be calibrated in order to produce realistic modelling results. Herein we focus on the simplest and most commonly used implementation of the LB method: the single-relaxation-time BGK model. A key parameter in the BGK model is the relaxation time τ which controls flow velocity and has a substantial influence on the permeability calculation. Currently there is no rigorous scheme to calibrate its value for models of real media. We show that the standard method of calibration, by matching the flow profile of the analytic Hagen-Poiseuille pipe-flow model, results in a BGK-LB model that is unable to accurately predict permeability even in simple realistic porous media (herein, Fontainebleau sandstone). In order to reconcile the differences between predicted permeability and experimental data, we propose a method to calibrate τ using an enhanced Transitional Markov Chain Monte Carlo method, which is suitable for parallel computer architectures. We also propose a porosity-dependent τ calibration that provides an excellent fit to experimental data and which creates an empirical model that can be used to choose τ for new samples of known porosity. Our Bayesian framework thus provides robust predictions of permeability of realistic porous media, herein demonstrated on the BGK-LB model, and should therefore replace the standard pipe-flow based methods of calibration for more complex media. The calibration methodology can also be extended to more advanced LB methods.

A Markovian Model for Coarse-Timescale Channel Variation in Wireless Networks

A wide range of wireless channel models has been developed to model variations in received signal strength. In contrast to prior work, which has focused primarily on channel modeling on a short per-packet timescale (millisecond), we develop and validate a finite-state Markovian model that captures variations due to shadowing, which occur at coarser timescales. The Markov chain is constructed by partitioning the entire range of shadowing into a finite number of intervals. We determine the Markov chain transition matrix in two ways: 1) via an abstract modeling approach, in which shadowing effects are modeled as a lognormally distributed random process affecting the received power, and the transition probabilities are derived as functions of the variance and autocorrelation function of shadowing; and 2) via an empirical approach, in which the transition matrix is calculated by directly measuring the changes in signal strengths. We test the assumptions of our Markovian model using signal strength measurements collected over an 802.16e (WiMAX) network and a wireless multihop network deployed by Rice University, Houston, TX, USA. We compare the steady-state and transient performance of the model with those computed using the empirically derived transition matrix and those observed in the actual traces themselves.

A hierarchical Bayesian framework for force field selection in molecular dynamics simulations.

We present a hierarchical Bayesian framework for the selection of force fields in molecular dynamics (MD) simulations. The framework associates the variability of the optimal parameters of the MD potentials under different environmental conditions with the corresponding variability in experimental data. The high computational cost associated with the hierarchical Bayesian framework is reduced by orders of magnitude through a parallelized Transitional Markov Chain Monte Carlo method combined with the Laplace Asymptotic Approximation. The suitability of the hierarchical approach is demonstrated by performing MD simulations with prescribed parameters to obtain data for transport coefficients under different conditions, which are then used to infer and evaluate the parameters of the MD model. We demonstrate the selection of MD models based on experimental data and verify that the hierarchical model can accurately quantify the uncertainty across experiments; improve the posterior probability density function estimation of the parameters, thus, improve predictions on future experiments; identify the most plausible force field to describe the underlying structure of a given dataset. The framework and associated software are applicable to a wide range of nanoscale simulations associated with experimental data with a hierarchical structure.

Statistical analysis of episensitization using transition probability functions and PFS2 for “ROCKET”, a two stage phase II study of RRx-001, a multi-epigenetic agent, investigating resensitization to irinotecan in colorectal cancer.

TPS775 Background: The development of resistance to chemotherapies in cancer leads to disease progression resulting in impaired survival. RRx-001, a ROS-mediated DNMT and HDAC inhibitor, may resensitize patients to previously effective, now refractory therapies, improving survival. A randomized two-stage Phase 2 trial of RRx-001 vs regorafenib (ROCKET) is underway to assess the impact of resensitization to irinotecan on overall survival in irinotecan refractory tumors. Transition Probability Functions (TPF) using Markov Chains have previously been used in clinical trials and provide a powerful tool for resensitization analysis. Methods: Patients with metastatic colorectal cancer who previously responded, then progressed on irinotecan-based therapies, with or without bevacizumab, cetuximab or panitumumab and an ECOG PS 0-1 are randomized to receive RRx-001 1x/week or regorafenib until progression or unacceptable toxicity in Stage 1. Patients then advance to Stage 2 to receive irinotecan-based therapies. This trial is designed to detect an increase in median OS of 3 mo., corresponding to a 33% reduction in risk of death. The primary endpoint, overall survival (OS), employs a two-sided log-rank test to compare treatment groups. The hazard ratio and its 95% CI are derived from the Cox proportional hazards model. Kaplan-Meier survival curve estimates are graphically depicted by treatment group. Secondary endpoints include PFS, PFS2 and Transition Probability Functions to address the resensitization hypothesis, KRAS or p53 status dependence. The analysis of PFS2, defined as time to progression (or death) from randomization to progression in Stage 2 (post Irinotecan therapy) is a key secondary endpoint and its analysis uses the standard two-sided log-rank test. TPF estimates will be derived from the Markov Chain transition probability matrix. The study should provide compelling evidence in support of the ability to return to previously beneficial chemotherapy treatment. Recruitment is ongoing. Clinical trial information: NCT02096354.

Transitional Markov Chain Monte Carlo: Observations and Improvements

AbstractThe Transitional Markov chain Monte Carlo (TMCMC) method is a widely used method for Bayesian updating and Bayesian model class selection. The method is based on successively sampling from a sequence of distributions that gradually approach the posterior target distribution. The samples of the intermediate distributions are used to obtain an estimate of the evidence, which is needed in the context of Bayesian model class selection. The properties of the TMCMC method are discussed and the following three modifications to the TMCMC method are proposed: (1) The sample weights should be adjusted after each MCMC step; (2) a burn-in period in the MCMC sampling step can improve the posterior approximation; and (3) the scale of the proposal distribution of the MCMC algorithm can be selected adaptively to achieve a near-optimal acceptance rate. The performance of the proposed modifications is compared with the original TMCMC method by means of three example problems. The proposed modifications reduce the b...

TRANSITIONAL ANNEALED ADAPTIVE SLICE SAMPLING FOR GAUSSIAN PROCESS HYPER-PARAMETER ESTIMATION

Surrogate models have become ubiquitous in science and engineering for their capability of emulating expensive computer codes, necessary to model and investigate complex phenomena. Bayesian emulators based on Gaussian processes adequately quantify the uncertainty that results from the cost of the original simulator, and thus the inability to evaluate it on the whole input space. However, it is common in the literature that only a partial Bayesian analysis is carried out, whereby the underlying hyper-parameters are estimated via gradient-free optimisation or genetic algorithms, to name a few methods. On the other hand, maximum a posteriori (MAP) estimation could discard important regions of the hyper-parameter space. In this paper, we carry out a more complete Bayesian inference, that combines Slice Sampling with some recently developed Sequential Monte Carlo samplers. The resulting algorithm improves the mixing in the sampling through delayed-rejection, the inclusion of an annealing scheme akin to Asymptotically Independent Markov Sampling and parallelisation via Transitional Markov Chain Monte Carlo. Examples related to the estimation of Gaussian process hyper-parameters are presented. For the purpose of reproducibility, further development, and use in other applications, the code to generate the examples in this paper is freely available for download at http://github.com/agarbuno/ta2s2_codes

Adjustable hybrid resampling approach to computationally efficient probabilistic inference of structural damage based on vibration measurements

This paper presents a novel Adjustable Hybrid Resampling (AHR) approach to deriving samples representing probabilistic distributions of unknown damage indexes through Bayesian inference based on vibration measurements. The AHR is motivated by the need for reducing computational cost in identifying damage in real-world structures and evaluating its associated uncertainties, and is developed through alternative implementation of two newly developed resampling processes—the reserved sample resampling (RSR) and the direct resampling (DR)—in the transitional Markov chain Monte Carlo (TMCMC) simulation for Bayesian inference. The AHR is able to rapidly identify important regions where samples highly populate, and have the capacity to transform samples to more optimal positions, achieving the desired trade-off between computational expense and accuracy in an adjustable manner controlled by decision-makers. The proposed AHR approach and its efficiency and accuracy are illustrated and examined through identification of damage in truss joints and members by using simulated measurements. The limitation of this study and future research direction in this regard is also discussed.