MCMC Algorithm Research Articles

Maximum Likelihood Algorithm for Spatial Generalized Linear Mixed Models without Numerical Evaluations of Intractable Integrals

Spatial generalized linear mixed effects models are popular in spatial or spatiotemporal data analysis when the responses are counts and the random effects are modeled by multivariate normal distributions. Direct computation of the MLEs of model parameters is impossible because the likelihood functions contain high-dimensional intractable integrals. To overcome the difficulty, a new method called the prediction-maximization algorithm is proposed. The method has a maximization step for the MLEs of spatial linear mixed effects models for normal responses and a prediction step for the prediction of the random effects. None of them involves high-dimensional intractable integrals. Because only algorithms for the normal responses are needed, the derivation of the MLEs of a spatial generalized linear mixed effects model for count responses by the proposed method is not computationally harder than a model for normal responses. The simulation study shows that the performance of the proposed method is comparable to that of the previous maximum likelihood algorithms formulated by high-order Laplace approximations and is better than that of Bayesian methods formulated by MCMC algorithms. Supplementary materials for this article are available online.

Bayesian inference of admixture graphs on Native American and Arctic populations.

Admixture graphs are mathematical structures that describe the ancestry of populations in terms of divergence and merging (admixing) of ancestral populations as a graph. An admixture graph consists of a graph topology, branch lengths, and admixture proportions. The branch lengths and admixture proportions can be estimated using numerous numerical optimization methods, but inferring the topology involves a combinatorial search for which no polynomial algorithm is known. In this paper, we present a reversible jump MCMC algorithm for sampling high-probability admixture graphs and show that this approach works well both as a heuristic search for a single best-fitting graph and for summarizing shared features extracted from posterior samples of graphs. We apply the method to 11 Native American and Siberian populations and exploit the shared structure of high-probability graphs to characterize the relationship between Saqqaq, Inuit, Koryaks, and Athabascans. Our analyses show that the Saqqaq is not a good proxy for the previously identified gene flow from Arctic people into the Na-Dene speaking Athabascans.

Hawkes Processes With Stochastic Exogenous Effects for Continuous-Time Interaction Modelling.

Continuous-time interaction data is usually generated under time-evolving environment. Hawkes processes (HP) are commonly used mechanisms for the analysis of such data. However, typical model implementations (such as e.g., stochastic block models) assume that the exogenous (background) interaction rate is constant, and so they are limited in their ability to adequately describe any complex time-evolution in the background rate of a process. In this paper, we introduce a stochastic exogenous rate Hawkes process (SE-HP) which is able to learn time variations in the exogenous rate. The model affiliates each node with a piecewise-constant membership distribution with an unknown number of changepoint locations, and allows these distributions to be related to the membership distributions of interacting nodes. The time-varying background rate function is derived through combinations of these membership functions. We introduce a stochastic gradient MCMC algorithm for efficient, scalable inference. The performance of the SE-HP is explored on real world, continuous-time interaction datasets, where we demonstrate that the SE-HP strongly outperforms comparable state-of-the-art methods.

Spillover shifts in the FX market: Implication for the behavior of a safe haven currency

This paper explores the behavior of safe haven currencies by analyzing shock transmission among major currencies. To capture state-dependent directional spillovers, we incorporate Markov regime-switching parameters into the spillover model and estimate them using a Bayesian MCMC algorithm. By considering weekly data from September 2000 to March 2020, we find that the Japanese yen and the Swiss franc, both of which yield relatively high excess returns in times of crisis, exhibit larger reductions of shock transmission and reception during periods of high-volatility than during periods of low-volatility. This implies that the safe haven currencies insulate themselves from shocks from other currencies by reducing interdependence across the FX market in crisis.

A Mixed-Membership Model for Social Network Clustering

We propose a simple mixed membership model for social network clustering in this paper. A flexible function is adopted to measure affinities among a set of entities in a social network. The model not only allows each entity in the network to possess more than one membership, but also provides accurate statistical inference about network structure. We estimate the membership parameters using an MCMC algorithm. We evaluate the performance of the proposed algorithm by applying our model to two empirical social network data, the Zachary club data and the bottlenose dolphin network data. We also conduct some numerical studies based on synthetic networks for further assessing the effectiveness of our algorithm. In the end, some concluding remarks and future work are addressed briefly.

Bayesian estimation of nonlinear models parameters in the description of growth coffee fruits

Coffee is one of the main products of Brazilian agriculture and the country is currently the largest producer and exporter in the world. The coffee fruit has a double sigmoidal growth pattern, however, as well as in other fruits that also show such a growth pattern, the authors generally do not estimate parameters of regression models to describe such curve. In the study of fruit growth curves, the sample size is generally small, so the estimation of the parameters should preferably be done by the Bayesian methodology, since a priori information is incorporated, reducing the effects of having few observations. The Markov Chain Monte Carlo algorithms are the most used computational tool in Bayesian statistics. However, these generate dependent samples, can be complicated to implement and, mainly, to teach. There are also other alternatives to the MCMC algorithms to obtain approximations of integrals of interest in Bayesian inference, the main ones are based on the importance resampling techniques. The objective of this work is to use Bayesian inference with the weighted importance resampling technique in the estimation of parameters of double sigmoidal nonlinear regression models to the description of coffee fruit growth. The double nonlinear logistic model was used in the description of the accumulation of fresh weight in coffee fruits. All prioris used have Beta distribution and were obtained by the called prior of specialist technique. Bayesian methodology was efficient, since it provided parameters with practical interpretation to coffee fruit growth, consistent with the reality. Thus, Bayesian inference by weighted importance resampling was a good alternative for the parameters estimation of nonlinear double sigmoid regression models. The logistic model showed that the growth of coffee fruits is more intense in the first sigmoid (until 162 DAF)of the growth curve and stabilizes in its final weight after 262 daf.

On polynomial-time computation of high-dimensional posterior measures by Langevin-type algorithms

The problem of generating random samples of high-dimensional posterior distributions is considered. The main results consist of non-asymptotic computational guarantees for Langevin-type MCMC algorithms which scale polynomially in key quantities such as the dimension of the model, the desired precision level, and the number of available statistical measurements. As a direct consequence, it is shown that posterior mean vectors as well as optimisation based maximum a posteriori (MAP) estimates are computable in polynomial time, with high probability under the distribution of the data. These results are complemented by statistical guarantees for recovery of the ground truth parameter generating the data. Our results are derived in a general high-dimensional non-linear regression setting (with Gaussian process priors) where posterior measures are not necessarily log-concave, employing a set of local ‘geometric’ assumptions on the parameter space, and assuming that a good initialiser of the algorithm is available. The theory is applied to a representative non-linear example from PDEs involving a steady-state Schrödinger equation.

Bayesian Inference for Mixed Gaussian GARCH-Type Model by Hamiltonian Monte Carlo Algorithm.

MCMC algorithm is widely used in parameters' estimation of GARCH-type models. However, the existing algorithms are either not easy to implement or not fast to run. In this paper, Hamiltonian Monte Carlo (HMC) algorithm, which is easy to perform and also efficient to draw samples from posterior distributions, is firstly proposed to estimate for the Gaussian mixed GARCH-type models. And then, based on the estimation of HMC algorithm, the forecasting of volatility prediction is investigated. Through the simulation experiments, the HMC algorithm is more efficient and flexible than the Griddy-Gibbs sampler, and the credibility interval of forecasting for volatility prediction is also more accurate. A real application is given to support the usefulness of the proposed HMC algorithm well.

Detecting Preknowledge Cheating via Innovative Measures: A Mixture Hierarchical Model for Jointly Modeling Item Responses, Response Times, and Visual Fixation Counts.

Preknowledge cheating jeopardizes the validity of inferences based on test results. Many methods have been developed to detect preknowledge cheating by jointly analyzing item responses and response times. Gaze fixations, an essential eye-tracker measure, can be utilized to help detect aberrant testing behavior with improved accuracy beyond using product and process data types in isolation. As such, this study proposes a mixture hierarchical model that integrates item responses, response times, and visual fixation counts collected from an eye-tracker (a) to detect aberrant test takers who have different levels of preknowledge and (b) to account for nuances in behavioral patterns between normally-behaved and aberrant examinees. A Bayesian approach to estimating model parameters is carried out via an MCMC algorithm. Finally, the proposed model is applied to experimental data to illustrate how the model can be used to identify test takers having preknowledge on the test items.

Simulation Calculation of Element Number Density in the Earth’s Atmosphere Based on X-ray Occultation Sounding

The number density profiles of elements N and O in the altitude range of 120–250 km are retrieved by simulation based on X-ray occultation. Based on the parameters of the NICER telescope, the energy spectrum forward model of the Crab Nebula in the energy range of 0.25–8 keV during the occultation is constructed, and the energy spectrum simulation data are obtained by adding noise to the energy spectrum forward model at different tangent point altitudes. The NICER energy band includes the K-shell absorption edges (0.4 keV, 0.53 keV for N, O), and there are significant differences in X-ray cross sections at the K-shell absorption edges, which provides an opportunity to retrieve the atmospheric density of each element. The MCMC algorithm is used to fit the energy spectrum forward model and simulation data, and the density profiles of elements N and O are retrieved. It is found that the retrieved error of O element in the altitude range of 120–140 km is large, which may be related to the low proportion of O in the line of sight and the low signal-to-noise ratio of simulation data. In the altitude range of 140–200 km, the retrieved error of elements N and O is small, but in the altitude range of 200–250 km, the retrieved error of elements N and O becomes larger, and the inconsistency between the retrieved results and NRLMSISE-00 model values increases. This is because the number of absorbed photons is reduced due to the thin atmospheric density at higher altitude, which introduces great uncertainty into the retrieved results. This study lays a foundation for element density retrieval based on X-ray occultation measured data in the future.

Bayesian inference of multivariate-GARCH-BEKK models

The main aim of this paper is to present a Bayesian analysis of Multivariate GARCH(l, m) (M-GARCH) models including estimation of the coefficient parameters as well as the model order, by combining a set of existing MCMC algorithms in the literature. The proposed algorithm focuses on the BEKK formulation of the multivariate GARCH model. The estimation procedure will be designed as a custom MCMC with embedded Reversible Jump MCMC (RJMCMC) and Delayed Rejection Metropolis-Hastings (DRMH) steps implemented using the statistical software R. The RJMCMC steps allow three variants of BEKK models (constant, diagonal and full) to be indexed and this index included as a parameter to be estimated. The proposed MCMC algorithms are validated using extensive simulation experiments followed by a case study using bivariate data derived from the daily share prices for BHP Group Limited, Rio Tinto Group, and Fortescue Metals Group Limited on the ASX over from September 2013 to December 2021.

A Bayesian Change Point Analysis of the USD/CLP Series in Chile from 2018 to 2020: Understanding the Impact of Social Protests and the COVID-19 Pandemic

Exchange rates are determined by factors such as interest rates, political stability, confidence, the current account on balance of payments, government intervention, economic growth and relative inflation rates, among other variables. In October 2019, an increased climate of citizen discontent with current social policies resulted in a series of massive protests that ignited important political changes in Chile. This event along with the global COVID-19 pandemic were two major factors that affected the value of the US dollar and produced sudden changes in the typically stable USD/CLP (Chilean Peso) exchange rate. In this paper, we use a Bayesian approach to detect and locate change points in the currency exchange rate process in order to identify and relate these points with the important dates related to the events described above. The implemented method can successfully detect the onset of the social protests, the beginning of the COVID-19 pandemic in Chile and the economic reactivation in the US and Europe. In addition, we evaluate the performance of the proposed MCMC algorithms using a simulation study implemented in Python and R.

Delivering spatially comparable inference on the risks of multiple severities of respiratory disease from spatially misaligned disease count data.

Population-level disease risk varies between communities, and public health professionals are interested in mapping this spatial variation to monitor the locations of high-risk areas and the magnitudes of health inequalities. Almost all of these risk maps relate to a single severity of disease outcome, such as hospitalization, which thus ignores any cases of disease of a different severity, such as a mild case treated in a primary care setting. These spatially-varying risk maps are estimated from spatially aggregated disease count data, but the set of areal units to which these disease counts relate often varies by severity. Thus, the statistical challenge is to provide spatially comparable inference from multiple sets of spatially misaligned disease count data, and an additional complexity is that the spatial extents of the areal units for some severities are partially unknown. This paper thus proposes a novel spatial realignment approach for multivariate misaligned count data, and applies it to the first study delivering spatially comparable inference for multiple severities of the same disease. Inference is via a novel spatially smoothed data augmented MCMC algorithm, and the methods are motivated by a new study of respiratory disease risk in Scotland in2017.

Approximate bounding of mixing time for multiple-step Gibbs samplers

Abstract Markov chain Monte Carlo (MCMC) methods are important in a variety of statistical applications that require sampling from intractable probability distributions. Among the most common MCMC algorithms is the Gibbs sampler. When an MCMC algorithm is used, it is important to have an idea of how long it takes for the chain to become “close” to its stationary distribution. In many cases, there is high autocorrelation in the output of the chain, so the output needs to be thinned so that an approximate random sample from the desired probability distribution can be obtained by taking a state of the chain every h steps in a process called h-thinning. This manuscript extends the work of [D. A. Spade, Estimating drift and minorization coefficients for Gibbs sampling algorithms, Monte Carlo Methods Appl. 27 2021, 3, 195–209] by presenting a computational approach to obtaining an approximate upper bound on the mixing time of the h-thinned Gibbs sampler.

Bayesian bridge and reciprocal bridge composite quantile regression

Bayesian regularized composite quantile regression approaches with bridge and reciprocal bridge penalties are adopted to conduct variable selection in composite quantile regression. Two MCMC algorithms were developed for posterior inference using the normal-exponential mixture representation of the asymmetric Laplace distribution. Gamma prior is placed on the regularization parameter. The parameters of the Gamma prior are treated as unknowns and estimate them along with the other parameters. The performance of the proposed composite quantile regression methods over the existing methods is then illustrated via simulated studies and a real data set.

A Hierarchical Random Graph Efficient Sampling Algorithm Based on Improved MCMC Algorithm

A hierarchical random graph (HRG) model combined with a maximum likelihood approach and a Markov Chain Monte Carlo algorithm can not only be used to quantitatively describe the hierarchical organization of many real networks, but also can predict missing connections in partly known networks with high accuracy. However, the computational cost is very large when hierarchical random graphs are sampled by the Markov Chain Monte Carlo algorithm (MCMC), so that the hierarchical random graphs, which can describe the characteristics of network structure, cannot be found in a reasonable time range. This seriously limits the practicability of the model. In order to overcome this defect, an improved MCMC algorithm called two-state transitions MCMC (TST-MCMC) for efficiently sampling hierarchical random graphs is proposed in this paper. On the Markov chain composed of all possible hierarchical random graphs, TST-MCMC can generate two candidate state variables during state transition and introduce a competition mechanism to filter out the worse of the two candidate state variables. In addition, the detailed balance of Markov chain can be ensured by using Metropolis–Hastings rule. By using this method, not only can the convergence speed of Markov chain be improved, but the convergence interval of Markov chain can be narrowed as well. Three example networks are employed to verify the performance of the proposed algorithm. Experimental results show that our algorithm is more feasible and more effective than the compared schemes.

An efficient adaptive MCMC algorithm for Pseudo-Bayesian quantum tomography

We revisit the Pseudo-Bayesian approach to the problem of estimating density matrix in quantum state tomography in this paper. Pseudo-Bayesian inference has been shown to offer a powerful paradigm for quantum tomography with attractive theoretical and empirical results. However, the computation of (Pseudo-)Bayesian estimators, due to sampling from complex and high-dimensional distribution, pose significant challenges that hamper their usages in practical settings. To overcome this problem, we present an efficient adaptive MCMC sampling method for the Pseudo-Bayesian estimator by exploring an adaptive proposal scheme together with subsampling method. We show in simulations that our approach is substantially computationally faster than the previous implementation by at least two orders of magnitude which is significant for practical quantum tomography.

Exact Bayesian Inference for Level-Set Cox Processes with Piecewise Constant Intensity Function

This article proposes a new methodology to perform Bayesian inference for a class of multidimensional Cox processes in which the intensity function is piecewise constant. Poisson processes with piecewise constant intensity functions are believed to be suitable to model a variety of point process phenomena and, given its simpler structure, are expected to provide more precise inference when compared to processes with nonparametric and continuously varying intensity functions. The partition of the space domain is flexibly determined by a level-set function of a latent Gaussian process. Despite the intractability of the likelihood function and the infinite dimensionality of the parameter space, inference is performed exactly, in the sense that no space discretization approximation is used and MCMC error is the only source of inaccuracy. That is achieved by using retrospective sampling techniques and devising a pseudo-marginal infinite-dimensional MCMC algorithm that converges to the exact target posterior distribution. Computational efficiency is favored by considering a nearest neighbor Gaussian process, allowing for the analysis of large datasets. An extension to consider spatiotemporal models is also proposed. The efficiency of the proposed methodology is investigated in simulated examples and its applicability is illustrated in the analysis of some real point process datasets.

An adaptively weighted stochastic gradient MCMC algorithm for Monte Carlo simulation and global optimization

An adaptively weighted stochastic gradient MCMC algorithm for Monte Carlo simulation and global optimization

Volatility spillovers across financial markets: the role of oil price uncertainty

ABSTRACT This paper analyzes the state-dependent volatility transmission mechanism between oil, stock, dollar, and bond prices to further examine the role of oil price uncertainty in financial markets. To this end, we extend the Diebold and Yilmaz (2014) spillover framework by incorporating a Markov-switching model and a Bayesian MCMC algorithm. We find that oil prices spills the highest degree of volatility to other markets during crises. The interdependence between the stock and oil markets is solid and stable, regardless of the regime shift. In contrast, the effect of oil price uncertainty on the foreign exchange or bond market during crises is double that during non-crisis periods. This suggests that oil price is closely related to other asset classes and reinforces its role as a risk transmitter during a crisis.