Adaptive Rejection Metropolis Sampling Research Articles

Generalized gamma distribution based on the Bayesian approach with application to investment modelling

The Generalized Gamma Distribution (GGD) is one of the most popular distributions in analyzing real lifetime datasets. Estimating the parameters of a high dimensional probability distribution is challenging due to the complexities associated with the resulting objectives function. When traditional estimation techniques fail due to complexity in the model objectives function, other powerful computational approaches are employed. In this study, a Bayesian approach to Generalized Gamma Distribution (GGD) based on Markov Chain Monte-Carlo (MCMC) has been employed to estimate model parameters. This study considers the Bayesian approach to GGD parameters using the Adaptive Rejection Metropolis Sampling (ARMS) technique of random variable generation within the Gibbs sampler. The MCMC approach has been used for estimating the multi-dimensional objectives function distribution. The results of the ARMS were compared to the existing Simulated annealing (SA) algorithm and Method of Moment (MM) based on modified internal rate of return data (MIRR). The performances of various derived estimators were recorded using the Markov chain Monte Carlo simulation technique for different sample sizes. The study reveals that ARMS's performance is marginally better than the existing SA and MA approaches. The efficiency of ARMS does not require a larger sample size as the SA does, in the case of simulated data. The performances of ARMS and SA are similar comparing them to the MM as an initial assumption in the case of real MIRR data. However, ARMS gives an acceptable estimated parameter for the different sample sizes due to its ability to evaluate the conditional distributions easily and sample from them exactly.

Bayesian Estimation of The Ex-Gaussian Distribution

Fitting of the exponential modified Gaussian distribution to model reaction times and drawing conclusions from its estimated parameter values is one of the most popular method used in psychology. This paper aims to develop a Bayesian approach to estimate the parameters of the ex-Gaussian distribution. Since the chosen priors yield to posterior densities that are not of known form and that they are not always log-concave, we suggest to use the adaptive rejection Metropolis sampling method. Applications on simulated data and on real data are provided to compare this method to the standard maximum likelihood estimation method as well as the quantile maximum likelihood estimation. Results shows the effectiveness of the proposed Bayesian method by computing the root mean square error of the estimated parameters using the three methods.

Bayesian inference for a single factor copula stochastic volatility model using Hamiltonian Monte Carlo

For modeling multivariate financial time series a single factor copula model with stochastic volatility margins is proposed. It generalizes single factor models based on the multivariate normal distribution by allowing for symmetric and asymmetric tail dependence. A joint Bayesian approach using Hamiltonian Monte Carlo (HMC) within Gibbs sampling is developed. Thus, the information loss caused by the two-step approach for margins and dependence is avoided. Further, the Bayesian approach is tractable in high dimensional parameter spaces in addition to uncertainty quantification through credible intervals. By introducing indicators for different copula families the copula families are selected automatically in the Bayesian framework. In a first simulation study the performance of HMC for the copula part is compared to a procedure based on adaptive rejection Metropolis sampling within Gibbs sampling. It is shown that HMC considerably outperforms this alternative approach in terms of effective sample size per minute. In a second simulation study satisfactory performance is seen for the full HMC within Gibbs procedure. The approach is illustrated for a portfolio of financial assets with respect to one-day ahead value at risk forecasts. Comparison to a two-step estimation procedure and to relevant benchmark models, such as a multivariate factor stochastic volatility model, shows superior performance of the proposed approach.

Bayesian Survival Analysis for Generalized Pareto Distribution Under Progressively Type II Censored Data

In this paper, based on the progressively type II censoring data of generalized Pareto distribution, we consider the maximum likelihood estimation and asymptotic interval estimations of survival function and hazard function by using the Fisher information matrix and delta method. Also, we present a nonparametric Bootstrap-p method to generate the bootstrap samples and derive confidence interval estimation. In addition, we propose the Bayes estimator of Adaptive Rejection Metropolis Sampling algorithm to derive the point estimate and credible intervals. Finally, Monte Carlo simulation study is carried out to compare the performances of the three proposed methods based on different data schemes. An illustrative example is presented.

A nonparametric Bayesian continual reassessment method in single-agent dose-finding studies

BackgroundThe main purpose of dose-finding studies in Phase I trial is to estimate maximum tolerated dose (MTD), which is the maximum test dose that can be assigned with an acceptable level of toxicity. Existing methods developed for single-agent dose-finding assume that the dose-toxicity relationship follows a specific parametric potency curve. This assumption may lead to bias and unsafe dose escalations due to the misspecification of parametric curve.MethodsThis paper relaxes the parametric assumption of dose-toxicity relationship by imposing a Dirichlet process prior on unknown dose-toxicity curve. A hybrid algorithm combining the Gibbs sampler and adaptive rejection Metropolis sampling (ARMS) algorithm is developed to estimate the dose-toxicity curve, and a two-stage Bayesian nonparametric adaptive design is presented to estimate MTD.ResultsFor comparison, we consider two classical continual reassessment methods (CRMs) (i.e., logistic and power models). Numerical results show the flexibility of the proposed method for single-agent dose-finding trials, and the proposed method behaves better than two classical CRMs under our considered scenarios.ConclusionsThe proposed dose-finding procedure is model-free and robust, and behaves satisfactorily even in small sample cases.

Repeated responses in misclassification binary regression: A Bayesian approach

Binary regression models generally assume that the response variable is measured perfectly. However, in some situations, the outcome is subject to misclassification: a success may be erroneously classified as a failure or vice versa. Many methods, described in existing literature, have been developed to deal with misclassification, but we demonstrate that these methods may lead to serious inferential problems when only a single evaluation of the individual is taken. Thus, this study proposes to incorporate repeated and independent responses in misclassification binary regression models, considering the total number of successes obtained or even the simple majority classification. We use subjective prior distributions, as our conditional means prior, to evaluate and compare models. A data augmentation approach, Gibbs sampling, and Adaptive Rejection Metropolis Sampling are used for posterior inferences. Simulation studies suggested that repeated measures significantly improve the posterior estimates, in that these estimates are closer to those obtained in a case with no misclassifications with a lower standard deviation. Finally, we illustrate the usefulness of the new methodology with the analysis about defects in eyeglass lenses.

Adaptive independent sticky MCMC algorithms

Monte Carlo methods have become essential tools to solve complex Bayesian inference problems in different fields, such as computational statistics, machine learning, and statistical signal processing. In this work, we introduce a novel class of adaptive Monte Carlo methods, called adaptive independent sticky Markov Chain Monte Carlo (MCMC) algorithms, to sample efficiently from any bounded target probability density function (pdf). The new class of algorithms employs adaptive non-parametric proposal densities, which become closer and closer to the target as the number of iterations increases. The proposal pdf is built using interpolation procedures based on a set of support points which is constructed iteratively from previously drawn samples. The algorithm’s efficiency is ensured by a test that supervises the evolution of the set of support points. This extra stage controls the computational cost and the convergence of the proposal density to the target. Each part of the novel family of algorithms is discussed and several examples of specific methods are provided. Although the novel algorithms are presented for univariate target densities, we show how they can be easily extended to the multivariate context by embedding them within a Gibbs-type sampler or the hit and run algorithm. The ergodicity is ensured and discussed. An overview of the related works in the literature is also provided, emphasizing that several well-known existing methods (like the adaptive rejection Metropolis sampling (ARMS) scheme) are encompassed by the new class of algorithms proposed here. Eight numerical examples (including the inference of the hyper-parameters of Gaussian processes, widely used in machine learning for signal processing applications) illustrate the efficiency of sticky schemes, both as stand-alone methods to sample from complicated one-dimensional pdfs and within Gibbs samplers in order to draw from multi-dimensional target distributions.

Recent Developments in Copula Models

Copula models have become very popular and well studied among the scientific community.[...]

Bayesian Inference for Latent Factor Copulas and Application to Financial Risk Forecasting

Factor modeling is a popular strategy to induce sparsity in multivariate models as they scale to higher dimensions. We develop Bayesian inference for a recently proposed latent factor copula model, which utilizes a pair copula construction to couple the variables with the latent factor. We use adaptive rejection Metropolis sampling (ARMS) within Gibbs sampling for posterior simulation: Gibbs sampling enables application to Bayesian problems, while ARMS is an adaptive strategy that replaces traditional Metropolis-Hastings updates, which typically require careful tuning. Our simulation study shows favorable performance of our proposed approach both in terms of sampling efficiency and accuracy. We provide an extensive application example using historical data on European financial stocks that forecasts portfolio Value at Risk (VaR) and Expected Shortfall (ES).

A Bayesian hierarchical model with novel prior specifications for estimating HIV testing rates.

Human immunodeficiency virus (HIV) infection is a severe infectious disease actively spreading globally, and acquired immunodeficiency syndrome (AIDS) is an advanced stage of HIV infection. The HIV testing rate, that is, the probability that an AIDS-free HIV infected person seeks a test for HIV during a particular time interval, given no previous positive test has been obtained prior to the start of the time, is an important parameter for public health. In this paper, we propose a Bayesian hierarchical model with two levels of hierarchy to estimate the HIV testing rate using annual AIDS and AIDS-free HIV diagnoses data. At level one, we model the latent number of HIV infections for each year using a Poisson distribution with the intensity parameter representing the HIV incidence rate. At level two, the annual numbers of AIDS and AIDS-free HIV diagnosed cases and all undiagnosed cases stratified by the HIV infections at different years are modeled using a multinomial distribution with parameters including the HIV testing rate. We propose a new class of priors for the HIV incidence rate and HIV testing rate taking into account the temporal dependence of these parameters to improve the estimation accuracy. We develop an efficient posterior computation algorithm based on the adaptive rejection metropolis sampling technique. We demonstrate our model using simulation studies and the analysis of the national HIV surveillance data in the USA.

Fiducial inference for Birnbaum-Saunders distribution

In this paper, we consider fiducial inference for the unknown parameters of the Birnbaum-Saunders distribution. Two generalized fiducial distributions of the parameters are obtained. One is based on the inverse of the structural equation, and the fiducial estimates of the parameters are obtained by a simulation method. The other is based on the method of [Hannig J. Generalized fiducial inference via discretization. Stat. Sinica. 2013;23:489–514], then we use adaptive rejection Metropolis sampling to get the fiducial estimates. We compare the fiducial estimates with the maximum likelihood estimates and Bayesian estimates by simulations. Two real data sets are analysed for illustration.

Hit and run ARMS: adaptive rejection Metropolis sampling with hit and run random direction

An algorithm for sampling from non-log-concave multivariate distributions is proposed, which improves the adaptive rejection Metropolis sampling (ARMS) algorithm by incorporating the hit and run sampling. It is not rare that the ARMS is trapped away from some subspace with significant probability in the support of the multivariate distribution. While the ARMS updates samples only in the directions that are parallel to dimensions, our proposed method, the hit and run ARMS (HARARMS), updates samples in arbitrary directions determined by the hit and run algorithm, which makes it almost not possible to be trapped in any isolated subspaces. The HARARMS performs the same as ARMS in a single dimension while more reliable in multidimensional spaces. Its performance is illustrated by a Bayesian free-knot spline regression example. We showed that it overcomes the well-known ‘lethargy’ property and decisively find the global optimal number and locations of the knots of the spline function.

Probabilistic Principal Component Analysis to Identify Profiles of Physical Activity Behaviours in the Presence of Non-Ignorable Missing Data

SummaryThe paper is motivated by an accelerometer-based study of physical activity (PA) behaviours in a large cohort of UK school-aged children. Advances in research on PA are accompanied by a growing number of results that are contributing to form a complex picture of PA behaviours in children. One source of such complexity is intimately related to the multiplicity of dimensions associated with PA. Currently a comprehensive individual accelerometer summary can include a large number of outcomes and this clearly poses challenges for the analysis. We explore the application of principal component analysis to accelerometer measurements that are aggregated daily over several days of the week and are affected by missingness. The probabilistic approach to principal component analysis with latent scores is extended to include non-ignorable missing data. The extended likelihood is maximized through a Monte Carlo EM algorithm via adaptive rejection Metropolis sampling. Our findings suggest that physical activity and inactivity are two dimensions over which children aggregate into distinct behavioural profiles, characterized by gender and season but not by anthropometric factors.

Adaptive Rejection Metropolis Simulated Annealing for Detecting Global Maximum Regions

A finite mixture model has been used to fit the data from heterogeneous populations to many applications. An Expectation Maximization (EM) algorithm is the most popular method to estimate parameters in a finite mixture model. A Bayesian approach is another method for fitting a mixture model. However, the EM algorithm often converges to the local maximum regions, and it is sensitive to the choice of starting points. In the Bayesian approach, the Markov Chain Monte Carlo (MCMC) sometimes converges to the local mode and is difficult to move to another mode. Hence, in this paper we propose a new method to improve the limitation of EM algorithm so that the EM can estimate the parameters at the global maximum region and to develop a more effective Bayesian approach so that the MCMC chain moves from one mode to another more easily in the mixture model. Our approach is developed by using both simulated annealing (SA) and adaptive rejection metropolis sampling (ARMS). Although SA is a well-known approach for detecting distinct modes, the limitation of SA is the difficulty in choosing sequences of proper proposal distributions for a target distribution. Since ARMS uses a piecewise linear envelope function for a proposal distribution, we incorporate ARMS into an SA approach so that we can start a more proper proposal distribution and detect separate modes. As a result, we can detect the maximum region and estimate parameters for this global region. We refer to this approach as ARMS annealing. By putting together ARMS annealing with the EM algorithm and with the Bayesian approach, respectively, we have proposed two approaches: an EM-ARMS annealing algorithm and a Bayesian-ARMS annealing approach. We compare our two approaches with traditional EM algorithm alone and Bayesian approach alone using simulation, showing that our two approaches are comparable to each other but perform better than EM algorithm alone and Bayesian approach alone. Our two approaches detect the global maximum region well and estimate the parameters in this region. We demonstrate the advantage of our approaches using an example of the mixture of two Poisson regression models. This mixture model is used to analyze a survey data on the number of charitable donations.

Independent Doubly Adaptive Rejection Metropolis Sampling Within Gibbs Sampling

Markov Chain Monte Carlo (MCMC) methods, such as the Metropolis-Hastings (MH) algorithm, are widely used for Bayesian inference. One of the most important issues for any MCMC method is the convergence of the Markov chain, which depends crucially on a suitable choice of the proposal density. Adaptive Rejection Metropolis Sampling (ARMS) is a well-known MH scheme that generates samples from one-dimensional target densities making use of adaptive piecewise proposals constructed using support points taken from rejected samples. In this work we pinpoint a crucial drawback in the adaptive procedure in ARMS: support points might never be added inside regions where the proposal is below the target. When this happens in many regions it leads to a poor performance of ARMS, with the proposal never converging to the target. In order to overcome this limitation we propose two improved adaptive schemes for constructing the proposal. The first one is a direct modification of the ARMS procedure that incorporates support points inside regions where the proposal is below the target, while satisfying the diminishing adaptation property, one of the required conditions to assure the convergence of the Markov chain. The second one is an adaptive independent MH algorithm with the ability to learn from all previous samples except for the current state of the chain, thus also guaranteeing the convergence to the invariant density. These two new schemes improve the adaptive strategy of ARMS, thus simplifying the complexity in the construction of the proposals. Numerical results show that the new techniques provide better performance w.r.t. the standard ARMS.

Adaptive rejection Metropolis sampling using Lagrange interpolation polynomials of degree 2

A crucial problem in Bayesian posterior computation is efficient sampling from a univariate distribution, e.g. a full conditional distribution in applications of the Gibbs sampler. This full conditional distribution is usually non-conjugate, algebraically complex and computationally expensive to evaluate. We propose an alternative algorithm, called ARMS2, to the widely used adaptive rejection sampling technique ARS [Gilks, W.R., Wild, P., 1992. Adaptive rejection sampling for Gibbs sampling. Applied Statistics 41 (2), 337–348; Gilks, W.R., 1992. Derivative-free adaptive rejection sampling for Gibbs sampling. In: Bernardo, J.M., Berger, J.O., Dawid, A.P., Smith, A.F.M. (Eds.), Bayesian Statistics, Vol. 4. Clarendon, Oxford, pp. 641–649] for generating a sample from univariate log-concave densities. Whereas ARS is based on sampling from piecewise exponentials, the new algorithm uses truncated normal distributions and makes use of a clever auxiliary variable technique [Damien, P., Walker, S.G., 2001. Sampling truncated normal, beta, and gamma densities. Journal of Computational and Graphical Statistics 10 (2) 206–215]. Furthermore, we extend this algorithm to deal with non-log-concave densities to provide an enhanced alternative to adaptive rejection Metropolis sampling, ARMS [Gilks, W.R., Best, N.G., Tan, K.K.C., 1995. Adaptive rejection Metropolis sampling within Gibbs sampling. Applied Statistics 44, 455–472]. The performance of ARMS and ARMS2 is compared in simulations of standard univariate distributions as well as in Gibbs sampling of a Bayesian hierarchical state-space model used for fisheries stock assessment.

Bayesian stock assessment using a state–space implementation of the delay difference model

This paper presents a Bayesian approach to fisheries stock assessment using the delay difference model to describe nonlinear population dynamics. Given a time series of annual catch and effort data, models in the Deriso-Schnute family predict exploitable biomass in the following year from biomass in the current and previous year and from past spawning stock. A state-space model is used, as it allows incorporation of random errors in both the biomass dynamics equations and the observations. Because the biomass dynamics are nonlinear, the common Kalman filter is generally not applicable for parameter estimation. However, it is demonstrated that the Bayesian approach can handle any form of nonlinear relationship in the state and observation equations as well as realistic distributional assumptions. Difficulties with posterior calculations are overcome by the Gibbs sampler in conjunction with the adaptive rejection Metropolis sampling algorithm.

Estimation of a Generalized Linear Mixed‐Effects Model with a Finite‐Support Random‐Effects Distribution via Gibbs Sampling

AbstractWe discuss a Bayesian hierarchical generalized linear mixed‐effects model with a finite‐support random‐effects distribution and show how Gibbs sampling can be used for estimating the posterior distribution of the parameters and for clustering on the basis of longitudinal data. When directly sampling from the conditional distributions is laborious, the adaptive rejection sampling (ARS, Gilks and Wild, 1992; Gilks, 1992) algorithm or adaptive rejection Metropolis sampling (ARMS, Gilks et al., 1995) algorithm is used. Log‐concavity, a prerequisite of ARS, of the conditional distributions is examined. We also discuss a Bayesian solution to the uncertainty of the support size of the random‐effects distribution in statistical inference. The methodology is illustrated with an analysis of data from a study of regulation of serum parathyroid hormone secretion.

Adaptive Rejection Metropolis Sampling within Gibbs Sampling

Gibbs sampling is a powerful technique for statistical inference. It involves little more than sampling from full conditional distributions, which can be both complex and computationally expensive to evaluate. Gilks and Wild have shown that in practice full conditionals are often log‐concave, and they proposed a method of adaptive rejection sampling for efficiently sampling from univariate log‐concave distributions. In this paper, to deal with non‐log‐concave full conditional distributions, we generalize adaptive rejection sampling to include a Hastings‐Metropolis algorithm step. One important field of application in which statistical models may lead to non‐log‐concave full conditionals is population pharmacokinetics. Here, the relationship between drug dose and blood or plasma concentration in a group of patients typically is modelled by using nonlinear mixed effects models. Often, the data used for analysis are routinely collected hospital measurements, which tend to be noisy and irregular. Consequently, a robust (t‐distributed) error structure is appropriate to account for outlying observations and/or patients. We propose a robust nonlinear full probability model for population pharmacokinetic data. We demonstrate that our method enables Bayesian inference for this model, through an analysis of antibiotic administration in new‐born babies.

Discussion on the Meeting on the Gibbs Sampler and Other Markov Chain Monte Carlo Methods

Discussion on the Meeting on the Gibbs Sampler and Other Markov Chain Monte Carlo Methods