Hastings Algorithm Research Articles

The Bayesian superorganism: externalized memories facilitate distributed sampling.

A key challenge for any animal (or sampling technique) is to avoid wasting time by searching for resources (information) in places already found to be unprofitable. In biology, this challenge is particularly strong when the organism is a central place forager—returning to a nest between foraging bouts—because it is destined repeatedly to cover much the same ground. This problem will be particularly acute if many individuals forage from the same central place, as in social insects such as the ants. Foraging (sampling) performance may be greatly enhanced by coordinating movement trajectories such that each ant (walker) visits separate parts of the surrounding (unknown) space. We find experimental evidence for an externalized spatial memory in Temnothorax albipennis ants: chemical markers (either pheromones or cues such as cuticular hydrocarbon footprints) that are used by nest-mates to mark explored space. We show these markers could be used by the ants to scout the space surrounding their nest more efficiently through indirect coordination. We also develop a simple model of this marking behaviour that can be applied in the context of Markov chain Monte Carlo methods (Baddeley et al. 2019 J. R. Soc. Interface 16, 20190162 (doi:10.1098/rsif.2019.0162)). This substantially enhances the performance of standard methods like the Metropolis–Hastings algorithm in sampling from sparse probability distributions (such as those confronted by the ants) with only a little additional computational cost. Our Bayesian framework for superorganismal behaviour motivates the evolution of exploratory mechanisms such as trail marking in terms of enhanced collective information processing.

Modelling dependence structures of extreme wind speed using bivariate distribution: a Bayesian approach

When investigating extremes of weather variables, it is seldom that a single weather station determines the damage, and extremes may be caused from the combined behaviour of several weather stations. To investigate joint dependence of extreme wind speed, a bivariate generalised extreme value distribution (BGEVD) was considered from frequentist and Bayesian approaches to analyse the extremes of component-wise monthly maximum wind speed at selected weather stations in South Africa. In the frequentist approach, the parameters of extreme value distributions (EVDs) were estimated with maximum likelihood, whereas in the Bayesian approach the Markov Chain Monte Carlo (MCMC) technique was used with the Metropolis–Hastings algorithm. The results showed that when fitted to component-wise maxima of extreme weather variables, the BGEVD provided apparent benefits over the univariate method, which allowed information to be pooled across stations and resulted in improved precision of the estimates for the parameters and return levels of the distributions. The paper also discusses a method to construct informative priors empirically using historical data of the underlying process from weather characteristics of four pairs of surrounding weather stations at various distances. The results from the Bayesian analysis showed that posterior inference might be affected by the choice of priors that were used to formulate the informative priors. From the results, it could be inferred that the Bayesian approach provides a satisfactory estimation strategy in terms of precision, compared with the frequentist approach, because it accounts for uncertainty in parameters and return level estimations.

Optimal scaling of random-walk metropolis algorithms on general target distributions

One main limitation of the existing optimal scaling results for Metropolis–Hastings algorithms is that the assumptions on the target distribution are unrealistic. In this paper, we consider optimal scaling of random-walk Metropolis algorithms on general target distributions in high dimensions arising from practical MCMC models from Bayesian statistics. For optimal scaling by maximizing expected squared jumping distance (ESJD), we show the asymptotically optimal acceptance rate 0.234 can be obtained under general realistic sufficient conditions on the target distribution. The new sufficient conditions are easy to be verified and may hold for some general classes of MCMC models arising from Bayesian statistics applications, which substantially generalize the product i.i.d. condition required in most existing literature of optimal scaling. Furthermore, we show one-dimensional diffusion limits can be obtained under slightly stronger conditions, which still allow dependent coordinates of the target distribution. We also connect the new diffusion limit results to complexity bounds of Metropolis algorithms in high dimensions.

Uncertainty quantification in time-lapse seismic imaging: a full-waveform approach

SUMMARY Time-lapse seismic monitoring using full-wavefield methods aims to accurately and robustly image rock and fluid changes within a reservoir. These changes are typically small and localized. Quantifying the uncertainty related to these changes is crucial for decision making, but traditional methods that use pixel by pixel uncertainty quantification with large models are computationally infeasible. We exploit the structure of the time-lapse seismic problem for fast wavefield computations using a numerically exact local acoustic solver. This allows us to perform a Bayesian inversion using a Metropolis–Hastings algorithm to sample our posterior distribution. We address the well-known dimensionality problem in global optimization using an image compression technique. We run our numerical experiments using a single shot and a single frequency, however we show that various frequencies converge to different local minima. In addition, we test our framework for both uncorrelated and correlated noise, and we retrieve different histograms for each noise type. Through our numerical examples we show the importance of defining quantities of interest in order to setup an appropriate uncertainty quantification framework involving choosing the number of degrees of freedom and model parametrization that best approximate the problem. To our knowledge, there is no work in the literature studying the time-lapse problem using stochastic full-waveform inversion.

Parameter Estimation of Bayesian Multiple Regression Model with Informative Inverse Gamma Prior Distribution: Application to Malaria Symptom Dataset

Objectives: The study aims to develop a Bayesian multiple regression model with informative inverse gamma prior and t the model to malaria symptom dataset.Place and Duration of Study: The study was carried out in Masinde Muliro University of Science and Technology (MMUST). The study used 300 malaria related symptom dataset obtained from Health service records of different patients (students) between the time period of 1st January, 2015 to 20th December, 2015.Methodology: Multiple linear regression model with Bayesian parameter estimation is used. The Normal prior distribution for θ parameter and inverse gamma prior distribution for the σ2 parameter is derived. Gibbs sampler and Metropolis Hasting algorithm is used with Markov Chain Monte Carlo (MCMC) method to produce an iteration of about 102,491 with Burn-in of 2500 and thinning of 10 that resulting to eective sample size of 90000.Results: The results shows that all the estimated posterior predictive p-values are between 0.05 and 0.95 indicating an adequate t for the individual observation of the data in the model. The results also reveals that the data values and the average distance between the data values and the mean tend to be close to each other and the estimated coeffcient of θ′s approximately 95%draws fall within each of the corresponding highest posterior density intervals.
 Conclusion: Though the Least Squares method is sucient for estimating the coeffcients of the regression parameters, the Bayesian estimates recorded comparatively very small standard errors making the Bayesian method more robust in analysing symptom dataset.

MCMC methods applied to the reconstruction of the autumn 2017 Ruthenium-106 atmospheric contamination source

Abstract In autumn 2017, small amounts of Ruthenium-106 of unknown origin were detected in Europe by several independent monitoring networks. To study the dispersion of this radionuclide, inverse modelling methods are applied to retrieve the location, time, duration and magnitude of the source. The inverse problem is solved within the Bayesian framework to yield a full reconstruction of the uncertainties. We first develop a classical Markov chain Monte Carlo (MCMC) method - the Metropolis Hastings (MH) algorithm - to reconstruct the source. However, the algorithm fails to converge in acceptable time because of local minima of the cost function, whose existence is due to a poor distribution of the observations. To overcome this obstacle, the parallel tempering algorithm, an enhanced MCMC method, is assessed and applied to the Ruthenium dispersion event. The convergence of the algorithm is studied and keys to implement and accelerate it are provided. The probability distribution functions of the variables associated with the source and the observation errors are obtained: they point to an area in south Ural and a total release of several hundreds of TBq on the 26 th of September 2017. The results are compared and proven to be consistent with other estimates. Moreover, the method converges fast enough to make it suitable for operational use.

The periodic pulse photothermal radiometry technique within the front face configuration

The front face photothermal radiometry technique has been improved in order to estimate the thermal conductivity of thin films with better accuracy compared to existing techniques. The experimental procedure is based on the front face response to a nanoseconds laser pulse repeated periodically at high frequency, i. e., a Dirac comb waveform. Averaging the thermal response by considering thousands successive pulses allows improving largely the signal noise ratio. The unknown thermal properties and related experimental parameters are identified by minimizing the gap between the measured signal and the theoretical response that accounts with the pulse waveform, the repetition frequency and the detector transfer function. Minimization is first achieved by implementing first a simplex technique that gives an initial set of values to start the Metropolis–Hastings algorithm in a second step. Application of the proposed methodology is done considering amorphous GeTe film deposited on a Si wafer. It is shown that this experimental method as well as the implementation of the Bayes minimization technique allows to identify the thin film intrinsic thermal conductivity with high accuracy considering some uncertainty on the other parameters assumed to be known.

Bayesian analysis of ranking data with the Extended Plackett–Luce model

Multistage ranking models, including the popular Plackett–Luce distribution (PL), rely on the assumption that the ranking process is performed sequentially, by assigning the positions from the top to the bottom one (forward order). A recent contribution to the ranking literature relaxed this assumption with the addition of the discrete-valued reference order parameter, yielding the novel Extended Plackett–Luce model (EPL). Inference on the EPL and its generalization into a finite mixture framework was originally addressed from the frequentist perspective. In this work, we propose the Bayesian estimation of the EPL in order to address more directly and efficiently the inference on the additional discrete-valued parameter and the assessment of its estimation uncertainty, possibly uncovering potential idiosyncratic drivers in the formation of preferences. We overcome initial difficulties in employing a standard Gibbs sampling strategy to approximate the posterior distribution of the EPL by combining the data augmentation procedure and the conjugacy of the Gamma prior distribution with a tuned joint Metropolis–Hastings algorithm within Gibbs. The effectiveness and usefulness of the proposal is illustrated with applications to simulated and real datasets.

A Bayesian inference for the penalized spline joint models of longitudinal and time-to-event data: A prior sensitivity analysis

Abstract Bayesian approaches have been used in the literature to estimate the parameters for joint models of longitudinal and time-to-event data. The main aim of this paper is to analyze the impact of prior distributions on estimating parameters in a proposed fully Bayesian analysis setting for the penalized spline joint models. To achieve this aim, the joint posterior distribution of parameters in survival and longitudinal submodels is presented. The Markov chain Monte Carlo (MCMC) algorithm is then proposed, which consists of the Gibbs sampler (GS) and Metropolis Hastings (MH) algorithms to sample for the target conditional posterior distributions. The prior sensitivity analysis for the baseline hazard rate and association parameters is performed through simulation studies and a case study.

Bayesian and classical inference of reliability in multicomponent stress-strength under the generalized logistic model

In this paper, a multicomponent system which has k independent and identical strength components is considered. Each component is exposed to a common random stress Y when distributions are generalized logistic. This system is operating or failing only if at least s out of k () strength variables exceeds the random stress. The Bayesian and classical inferences of multicomponent stress-strength reliability under the generalized logistic distribution are studied. The small sample comparison of the reliability estimates is made through Monte Carlo simulation and asymptotic confidence interval is obtained based on maximum likelihood estimation. Also the highest posterior density credible interval is calculated based on Bayesian estimation with Gibbs and Metropolis Hastings algorithms. Finally analysis of a real data set has been presented for illustrative purposes too.

How to Explain When the ES Is Lower Than One? A Bayesian Nonlinear Mixed-Effects Approach

Most studies in Vietnam use the Cobb-Douglas production function and its modifications for economic analysis. Extremely rigid presumptions are a main weak point of this functional form, particularly if the elasticity of factor substitution (ES) is equal to one, which hides the role of the ES for economic growth. The CES (constant elasticity of substitution) production function with more flexible presumptions, concretely its ES, is not unitary, and has been used more and more widely in economic investigations. So, this study is conducted to estimate the average ES through the specification of an aggregate CES function for the Vietnamese nonfinancial enterprises. By performing Bayesian nonlinear mixed-effects regression via Random-walk Metropolis Hastings (MH) algorithm, based on the data set of the listed nonfinancial enterprises of Vietnam, the author found that the CES function estimated for the researched enterprises has an ES lower than one, i.e., capital and labor are complimentary. This finding shows that Vietnamese nonfinancial enterprises can confront a downward trend of output growth.

Tracking and climbing behavior recognition of heavy-duty trucks on roadways

The tracking and behavior recognition of heavy-duty trucks on roadways are keys for the development of automated heavy-duty trucks and an advanced driver assistance system. The spatiotemporal information of trucks from trajectory tracking and motions learnt from behavior analysis can be employed to predict possible driving risks and generate safe motion to avoid roadway accidents. This article presents a unified tracking and behavior recognition algorithm that can model the mobility of heavy-duty trucks on long inclined roadways. Random noise within the sampled elevation data is addressed by time-based segmentation to extract time-continuous samples at geographical locations. A Kalman filter is first used to distinguish error offsets from random noise and to estimate the distribution of truck elevations for different time intervals. A Markov chain Monte Carlo model is then applied to classify truck behaviors based on the change in elevation between two geographical locations. A heavy-duty truck mobility (HVMove) model is constructed based on the map information to apply the roadway geometry to the tracking and behavior recognition algorithm. We develop an extended Metropolis–Hastings algorithm to tune the parameters of the HVMove model. The proposed model is verified and evaluated through extensive experiments based on a real-world trajectory dataset covering sections of an expressway and national and provincial highways. From the experimental results, we conclude that the HVMove model provides sufficient accuracy and efficiency for automated heavy-duty trucks and advanced driver assistance system applications. In addition, HVMove can generate maps with the elevation information marked automatically.

Accelerated Life Test Method for the Doubly Truncated Burr Type XII Distribution

The Burr type XII (BurrXII) distribution is very flexible for modeling and has earned much attention in the past few decades. In this study, the maximum likelihood estimation method and two Bayesian estimation procedures are investigated based on constant-stress accelerated life test (ALT) samples, which are obtained from the doubly truncated three-parameter BurrXII distribution. Because computational difficulty occurs for maximum likelihood estimation method, two Bayesian procedures are suggested to estimate model parameters and lifetime quantiles under the normal use condition. A Markov Chain Monte Carlo approach using the Metropolis–Hastings algorithm via Gibbs sampling is built to obtain Bayes estimators of the model parameters and to construct credible intervals. The proposed Bayesian estimation procedures are simple for practical use, and the obtained Bayes estimates are reliable for evaluating the reliability of lifetime products based on ALT samples. Monte Carlo simulations were conducted to evaluate the performance of these two Bayesian estimation procedures. Simulation results show that the second Bayesian estimation procedure outperforms the first Bayesian estimation procedure in terms of bias and mean squared error when users do not have sufficient knowledge to set up hyperparameters in the prior distributions. Finally, a numerical example about oil-well pumps is used for illustration.

A flexible particle Markov chain Monte Carlo method

Particle Markov Chain Monte Carlo methods are used to carry out inference in nonlinear and non-Gaussian state space models, where the posterior density of the states is approximated using particles. Current approaches usually perform Bayesian inference using either a particle marginal Metropolis–Hastings (PMMH) algorithm or a particle Gibbs (PG) sampler. This paper shows how the two ways of generating variables mentioned above can be combined in a flexible manner to give sampling schemes that converge to a desired target distribution. The advantage of our approach is that the sampling scheme can be tailored to obtain good results for different applications. For example, when some parameters and the states are highly correlated, such parameters can be generated using PMMH, while all other parameters are generated using PG because it is easier to obtain good proposals for the parameters within the PG framework. We derive some convergence properties of our sampling scheme and also investigate its performance empirically by applying it to univariate and multivariate stochastic volatility models and comparing it to other PMCMC methods proposed in the literature.

Markov Chain Monte Carlo Based Energy Use Behaviors Prediction of Office Occupants

Prediction of energy use behaviors is a necessary prerequisite for designing personalized and scalable energy efficiency programs. The energy use behaviors of office occupants are different from those of residential occupants and have not yet been studied as intensively as residential occupants. This paper proposes a method based on Markov chain Monte Carlo (MCMC) to predict the energy use behaviors of office occupants. Firstly, an indoor electrical Internet of Things system (IEIoTS) for the office scenario is developed to collect the switching state time series data of selected user electrical equipment (desktop computer, water dispenser, light) and the historical environment parameters. Then, the Metropolis–Hastings (MH) algorithm is used to sample and obtain the optimal solution of the parameters for the office occupants’ behavior function, the model of which includes the energy action model, energy working hours model, and air-conditioner energy use behavior model. Finally, comparative experiments are carried out to evaluate the performance of the proposed method. The experimental results show that while the mean value performs similarly in estimating the energy use model, the proposed method outperforms the Maximum Likelihood Estimation (MLE) method on uncertainty quantification with relatively narrower confidence intervals.

Reliability estimation in a multicomponent stress–strength based on unit-Gompertz distribution

PurposeThe purpose of this paper is to estimate the multicomponent reliability by assuming the unit-Gompertz (UG) distribution. Both stress and strength are assumed to have an UG distribution with common scale parameter.Design/methodology/approachThe reliability of a multicomponent stress–strength system is obtained by the maximum likelihood (MLE) and Bayesian method of estimation. Bayes estimates of system reliability are obtained by using Lindley’s approximation and Metropolis–Hastings (M–H) algorithm methods when all the parameters are unknown. The highest posterior density credible interval is obtained by using M–H algorithm method. Besides, uniformly minimum variance unbiased estimator and exact Bayes estimates of system reliability have been obtained when the common scale parameter is known and the results are compared for both small and large samples.FindingsBased on the simulation results, the authors observe that Bayes method provides better estimation results as compared to MLE. Proposed asymptotic and HPD intervals show satisfactory coverage probabilities. However, average length of HPD intervals tends to remain shorter than the corresponding asymptotic interval. Overall the authors have observed that better estimates of the reliability may be achieved when the common scale parameter is known.Originality/valueMost of the lifetime distributions used in reliability analysis, such as exponential, Lindley, gamma, lognormal, Weibull and Chen, only exhibit constant, monotonically increasing, decreasing and bathtub-shaped hazard rates. However, in many applications in reliability and survival analysis, the most realistic hazard rates are upside-down bathtub and bathtub-shaped, which are found in the unit-Gompertz distribution. Furthermore, when reliability is measured as percentage or ratio, it is important to have models defined on the unit interval in order to have plausible results. Therefore, the authors have studied the multicomponent stress–strength reliability under the unit-Gompertz distribution by comparing the MLEs, Bayes estimators and UMVUEs.

Modelling dependency effect to extreme value distributions with application to extreme wind speed at Port Elizabeth, South Africa: a frequentist and Bayesian approaches

The dependency effect to extreme value distributions (EVDs) using the frequentist and Bayesian approaches have been used to analyse the extremes of annual and daily maximum wind speed at Port Elizabeth, South Africa. In the frequentist approach, the parameters of EVDs were estimated using maximum likelihood, whereas in the Bayesian approach the Markov Chain Monte Carlo technique with the Metropolis–Hastings algorithm was used. The results show that the EVDs fitted considering the dependency and seasonality effects with in the data series provide apparent benefits in terms of improved precision in estimation of the parameters as well as return levels of the distributions. The paper also discusses a method to construct informative priors empirically using historical data of the underlying process from other weather stations. The results from the Bayesian analysis show that posterior inference might be affected by the choice of priors used to formulate the informative priors. The Bayesian approach provides satisfactory estimation strategy in terms of precision compared to the frequentist approach, accounting for uncertainty in parameters and return levels estimation.

Improved localisation algorithm based on Markov chain Monte Carlo-Metropolis Hastings for wireless sensor networks

Accurate and low-cost sensor localisation is the key requirement for deploying wireless sensor networks (WSNs) and the internet of things (IoT) in various applications. Researchers are trying their best to find a way to localise mobile nodes in WSNs. To solve the problem of the moment outside the anchoring range or positioning errors, an improved distance vector hop (DV-Hop) location algorithm based on the Markov chain Monte Carlo Metropolitan Hastings algorithm (MMDV-Hop) is proposed. According to the receiving and transmitting power of received signal strength indication (RSSI), anchor information is taken into account when calculating the distance between unknown nodes and anchor nodes. From the different percentages of anchor nodes and unknown nodes, node density, and node connectivity, MMDVHop shows better position error than traditional algorithms.

Improved localisation algorithm based on Markov chain Monte Carlo-Metropolis Hastings for wireless sensor networks

Accurate and low-cost sensor localisation is the key requirement for deploying wireless sensor networks (WSNs) and the internet of things (IoT) in various applications. Researchers are trying their best to find a way to localise mobile nodes in WSNs. To solve the problem of the moment outside the anchoring range or positioning errors, an improved distance vector hop (DV-Hop) location algorithm based on the Markov chain Monte Carlo Metropolitan Hastings algorithm (MMDV-Hop) is proposed. According to the receiving and transmitting power of received signal strength indication (RSSI), anchor information is taken into account when calculating the distance between unknown nodes and anchor nodes. From the different percentages of anchor nodes and unknown nodes, node density, and node connectivity, MMDVHop shows better position error than traditional algorithms.

Majorize–Minimize Adapted Metropolis–Hastings Algorithm

The dimension and the complexity of inference problems have dramatically increased in statistical signal processing. It thus becomes mandatory to design improved proposal schemes in Metropolis-Hastings algorithms, providing large proposal transitions that are accepted with high probability. The proposal density should ideally provide an accurate approximation to the target density with a low computational cost. In this paper, we derive a novel Metropolis-Hastings proposal, inspired from Langevin dynamics, where the drift term is preconditioned by an adaptive matrix constructed through a Majorization-Minimization strategy. We propose several variants of low-complexity curvature metrics applicable to large scale problems. We demonstrate the geometric ergodicity of the resulting chain for the class of super-exponential distributions. The proposed method is shown to exhibit a good performance in two signal recovery examples.