Adaptive Rejection Sampling Research Articles

Efficient Sampling From the Watson Distribution in Arbitrary Dimensions

In this paper, we present two efficient methods for sampling from the Watson distribution in arbitrary dimensions. The first method adapts the rejection sampling algorithm from Kent et al. (2018), originally designed for Bingham distributions, using angular central Gaussian envelopes. For the Watson distribution, we derive a closed-form expression for the parameters that maximize sampling efficiency, which is further investigated and bounded by asymptotic results. This approach avoids the curse of dimensionality through a smart matrix inversion, enabling fast runtimes even in high dimensions. The second method, based on Saw (1978), employs adaptive rejection sampling from a projected distribution. This algorithm is also effective in all dimensions and offers rapid sampling capabilities. Finally, our simulation study compares the two main methods, revealing that each excels under different conditions: the first method is more efficient for small samples or large dimensions, while the second performs better with larger samples and more concentrated distributions. Both algorithms are available in the R package watson on CRAN. Supplementary materials for this article are available online.

Automated generation of initial points for adaptive rejection sampling of log-concave distributions

Automated generation of initial points for adaptive rejection sampling of log-concave distributions

An adaptive rejection sampler for sampling from the Wiener diffusion model.

The Wiener diffusion model with two absorbing boundaries is one of the most frequently applied models for jointly modeling responses and response latencies in psychological research. We consider four methods for sampling from the model with and without variability in drift rate, starting point, and non-decision time: Inverse transform sampling, rejection sampling, and two new methods based on adaptive rejection sampling (ARS). We implement these four methods in an R package, validate the methods, and compare their sampling speed in different settings. All four implemented methods provide samples that follow the intended distributions. The ARS-based methods, however, outperform the other methods in sampling speed as the requested sample size increases. We provide guidelines for when using ARS is more efficient than using traditional methods and vice versa.

Exact Simulation of Max-Infinitely Divisible Processes

Max-infinitely divisible (max-id) processes play a central role in extreme-value theory and include the subclass of all max-stable processes. They allow for a constructive representation based on the pointwise maximum of random functions drawn from a Poisson point process defined on a suitable function space. Simulating from a max-id process is often difficult due to its complex stochastic structure, while calculating its joint density in high dimensions is often numerically infeasible. Therefore, exact and efficient simulation techniques for max-id processes are useful tools for studying the characteristics of the process and for drawing statistical inferences. Inspired by the simulation algorithms for max-stable processes, theory and algorithms to generalize simulation approaches tailored for certain flexible (existing or new) classes of max-id processes are presented. Efficient simulation for a large class of models can be achieved by implementing an adaptive rejection sampling scheme to sidestep a numerical integration step in the algorithm. The results of a simulation study highlight that our simulation algorithm works as expected and is highly accurate and efficient, such that it clearly outperforms customary approximate sampling schemes. As a by-product, new max-id models, which can be represented as pointwise maxima of general location-scale mixtures and possess flexible tail dependence structures capturing a wide range of asymptotic dependence scenarios, are also developed.

Adaptive rejection sampling with fixed number of nodes

ABSTRACTThe adaptive rejection sampling (ARS) algorithm is a universal random generator for drawing samples efficiently from a univariate log-concave target probability density function (pdf). ARS generates independent samples from the target via rejection sampling with high acceptance rates. Indeed, ARS yields a sequence of proposal functions that converge toward the target pdf, so that the probability of accepting a sample approaches one. However, sampling from the proposal pdf becomes more computational demanding each time it is updated. In this work, we propose a novel ARS scheme, called Cheap Adaptive Rejection Sampling (CARS), where the computational effort for drawing from the proposal remains constant, decided in advance by the user. For generating a large number of desired samples, CARS is faster than ARS.

Parsimonious adaptive rejection sampling

Monte Carlo (MC) methods have become very popular in signal processing during the past decades. The adaptive rejection sampling (ARS) algorithms are well-known MC techniques which draw efficiently independent samples from univariate target densities. The ARS schemes yield a sequence of proposal functions that converge towards the target, so that the probability of accepting a sample approaches one. However, sampling from the proposal pdf becomes more computationally demanding each time it is updated. The parsimonious ARS method, where an efficient trade-off between acceptance rate and proposal complexity is obtained, is proposed. Thus, the resulting algorithm is faster than the standard ARS approach.

Reliability analysis of masked data in adaptive step-stress partially accelerated lifetime tests with progressive removal

ABSTRACTBy combining the progressive hybrid censoring with the step-stress partially accelerated lifetime test, we propose an adaptive step-stress partially accelerated lifetime test, which allows random changing of the number of step-stress levels according to the pre-fixed censoring number and time points. Thus, the time expenditure and economic cost of the test will be reduced greatly. Based on the Lindley-distributed tampered failure rate (TFR) model with masked system lifetime data, the BFGS method is introduced in the expectation maximization (EM) algorithm to obtain the maximum likelihood estimation (MLE), which overcomes the difficulties of the vague maximization procedure in the M-step. Asymptotic confidence intervals of components' distribution parameters are also investigated according to the missing information principle. As comparison, the Bayesian estimation and the highest probability density (HPD) credible intervals are obtained by using adaptive rejection sampling. Furthermore, the reliability of the system and components are estimated at a specified time under usual and severe operating conditions. Finally, a numerical simulation example is presented to illustrate the performance of our proposed method.

A New Adaptive Rejection Sampling Method Using Kernel Density Approximations and Its Application to Subset Simulation

AbstractIn stochastic analysis of engineering systems, the task of generating samples according to a target probability distribution involving some performance function of the system response often arises. This paper introduces an adaptive method for rejection sampling that uses adaptive kernel sampling densities (AKSD) as proposal densities for the rejection sampling algorithm in an iterative approach. The AKSD formulation relies on having available (1) a small number of samples from the target density, as well as (2) evaluations of the system performance function over some other sample set. This information is then used to establish the adaptive features of the stochastic sampling involving (1) an explicit optimization of the kernel characteristics for reduction of the computational burden, and so maximizing sampling efficiency, and (2) selection of the exact model parameters to target so that potential problems when forming proposal densities for high-dimensional vectors are avoided. Beyond this theore...

적응 기각 추출을 기반으로 하는 난수 생성기의 성능 비교

Adaptive Rejection Sampling (ARS) method is a well-known random number generator to acquire a random sample from a probability distribution, and has the advantage of improving the proposal distribution during the sampling procedures, which update it closer to the target distribution. However, the use of ARS is limited since it can be used only for the target distribution in the form of the log-concave function, and thus various methods have been proposed to overcome such a limitation of ARS. In this paper, we attempt to compare five random number generators based on ARS in terms of adequacy and efficiency. Based on empirical analysis using simulations, we discuss their results and make a comparison of five ARS-based methods.

A note on nonparametric inference for species variety with Gibbs-type priors

A Bayesian nonparametric methodology has been recently introduced for estimating, given an initial observed sample, the species variety featured by an additional unobserved sample of size $m$. Although this methodology led to explicit posterior distributions under the general framework of Gibbs-type priors, there are situations of practical interest where $m$ is required to be very large and the computational burden for evaluating these posterior distributions makes impossible their concrete implementation. In this paper we present a solution to this problem for a large class of Gibbs-type priors which encompasses the two parameter Poisson-Dirichlet prior and, among others, the normalized generalized Gamma prior. Our solution relies on the study of the large $m$ asymptotic behaviour of the posterior distribution of the number of new species in the additional sample. In particular we introduce a simple characterization of the limiting posterior distribution in terms of a scale mixture with respect to a suitable latent random variable; this characterization, combined with the adaptive rejection sampling, leads to derive a large $m$ approximation of any feature of interest from the exact posterior distribution. We show how to implement our results through a simulation study and the analysis of a dataset in linguistics.

Mixture of the inverse Rayleigh distribution: Properties and estimation in a Bayesian framework

An engineering process is an output from a set of combined processes which may be homogeneous or heterogeneous. To study the lifetime of such processes, we need a model which can accommodate the nature of such processes. Single probability models are not capable of capturing the heterogeneity of nature. However, mixture models of some suitable lifetime distributions, have the potential to highlight such interesting feature. Due to time and cost constraint, in the most lifetime testing experiments, censoring is an unavoidable feature of most lifetime data sets. This article deals with the modeling of the heterogeneity existing in the lifetime processes using the mixture of the inverse Rayleigh distribution, and the spotlight is the Bayesian inference of the mixture model using non-informative (the Jeffreys and the uniform) and informative (gamma) priors. We are considering this particular distribution due to two reasons; the first one is due to its skewed behavior, i.e. in engineering processes, an engineer suspects that high failure rate in the beginning, but after continuous inspection, the failure goes down and the second reason is due to its vast application in many applied fields. A Gibbs sampling algorithm based on adaptive rejection sampling is designed for the posterior computation. A detailed simulation study is carried out to investigate the performance of the estimators based on different prior distributions. The posterior risks are evaluated under the squared error, the weighted, the quadratic, the entropy, the modified squared error and the precautionary loss functions. Posterior risks of the Bayes estimates are compared to explore the effect of prior information and loss functions.

Inference for accelerated competing failure models from Weibull distribution under Type-I progressive hybrid censoring

This paper considers constant-stress accelerated competing failure models under Type-I progressive hybrid censoring with binomial random removals. A Weibull distributed life of test units is assumed for a specific cause and by the Newton–Raphson iteration and asymptotic likelihood theory, the maximum likelihood estimates (MLEs) and asymptotic confidence intervals of the unknown parameters are obtained. Based on the noninformative prior, a Gibbs sampling algorithm using adaptive rejection sampling is presented to obtain Bayesian estimates and the Monte Carlo (MC) method is employed to construct the HPD credible intervals. The simulation results are provided to show that Bayesian estimates perform better than MLEs and the change of the removal probability has a significant effect on MLEs.

2-모수 파레토분포의 객관적 베이지안 추정

본 연구에서는 2-모수 파레토분포에 대해 무정보사전분포인 준거사전분포의 가정 하에서 객관적 베이지안 모수추정 절차를 제안하였다. 베이지안 추정은 깁스샘플링에 의해서 수행된다. 깁스샘플러에서 모수생성하는 방법은 형태모수는 감마분포로부터 생성하고 척도모수는 적응기각표집 알고리즘에 의해 생성한다. 제안된 베이지안 모수추정 절차는 모의실험과 자료분석에서 기존의 추정방법들인 L-적률추정법, 최우추정법, 공액사전분포 하의 주관적 베이지안 모수추정법과 비교된다. An objective Bayesian estimation procedure of the two-parameter Pareto distribution is presented under the reference prior and the noninformative prior. Bayesian estimators are obtained by Gibbs sampling. The steps to generate parameters in the Gibbs sampler are from the shape parameter of the gamma distribution and then the scale parameter by the adaptive rejection sampling algorism. A numerical study shows that the proposed objective Bayesian estimation outperforms other estimations in simulated bias and mean squared error.

Bayesian analysis for step-stress accelerated life testing using weibull proportional hazard model

In this paper, we present a Bayesian analysis for the Weibull proportional hazard (PH) model used in step-stress accelerated life testings. The key mathematical and graphical difference between the Weibull cumulative exposure (CE) model and the PH model is illustrated. Compared with the CE model, the PH model provides more flexibility in fitting step-stress testing data and has the attractive mathematical properties of being desirable in the Bayesian framework. A Markov chain Monte Carlo algorithm with adaptive rejection sampling technique is used for posterior inference. We demonstrate the performance of this method on both simulated and real datasets.

Concave-Convex Adaptive Rejection Sampling

We describe a method for generating independent samples from univariate density functions using adaptive rejection sampling without the log-concavity requirement. The method makes use of the fact that many functions can be expressed as a sum of concave and convex functions. Using a concave-convex decomposition, we bound the log-density by separately bounding the concave and convex parts using piecewise linear functions. The upper bound can then be used as the proposal distribution in rejection sampling. We demonstrate the applicability of the concave-convex approach on a number of standard distributions and describe an application to the efficient construction of sequential Monte Carlo proposal distributions for inference over genealogical trees. Computer code for the proposed algorithms is available online.

A generalization of the adaptive rejection sampling algorithm

Rejection sampling is a well-known method to generate random samples from arbitrary target probability distributions. It demands the design of a suitable proposal probability density function (pdf) from which candidate samples can be drawn. These samples are either accepted or rejected depending on a test involving the ratio of the target and proposal densities. The adaptive rejection sampling method is an efficient algorithm to sample from a log-concave target density, that attains high acceptance rates by improving the proposal density whenever a sample is rejected. In this paper we introduce a generalized adaptive rejection sampling procedure that can be applied with a broad class of target probability distributions, possibly non-log-concave and exhibiting multiple modes. The proposed technique yields a sequence of proposal densities that converge toward the target pdf, thus achieving very high acceptance rates. We provide a simple numerical example to illustrate the basic use of the proposed technique, together with a more elaborate positioning application using real data.

Generalized rejection sampling schemes and applications in signal processing

Bayesian methods and their implementations by means of sophisticated Monte Carlo techniques, such as Markov chain Monte Carlo (MCMC) and particle filters, have become very popular in signal processing over the last years. However, in many problems of practical interest these techniques demand procedures for sampling from probability distributions with non-standard forms, hence we are often brought back to the consideration of fundamental simulation algorithms, such as rejection sampling (RS). Unfortunately, the use of RS techniques demands the calculation of tight upper bounds for the ratio of the target probability density function (pdf) over the proposal density from which candidate samples are drawn. Except for the class of log-concave target pdf's, for which an efficient algorithm exists, there are no general methods to analytically determine this bound, which has to be derived from scratch for each specific case. In this paper, we introduce new schemes for (a) obtaining upper bounds for likelihood functions and (b) adaptively computing proposal densities that approximate the target pdf closely. The former class of methods provides the tools to easily sample from a posteriori probability distributions (that appear very often in signal processing problems) by drawing candidates from the prior distribution. However, they are even more useful when they are exploited to derive the generalized adaptive RS (GARS) algorithm introduced in the second part of the paper. The proposed GARS method yields a sequence of proposal densities that converge towards the target pdf and enable a very efficient sampling of a broad class of probability distributions, possibly with multiple modes and non-standard forms. We provide some simple numerical examples to illustrate the use of the proposed techniques, including an example of target localization using range measurements, often encountered in sensor network applications.

Adaptive rejection Metropolis sampling using Lagrange interpolation polynomials of degree 2

A crucial problem in Bayesian posterior computation is efficient sampling from a univariate distribution, e.g. a full conditional distribution in applications of the Gibbs sampler. This full conditional distribution is usually non-conjugate, algebraically complex and computationally expensive to evaluate. We propose an alternative algorithm, called ARMS2, to the widely used adaptive rejection sampling technique ARS [Gilks, W.R., Wild, P., 1992. Adaptive rejection sampling for Gibbs sampling. Applied Statistics 41 (2), 337–348; Gilks, W.R., 1992. Derivative-free adaptive rejection sampling for Gibbs sampling. In: Bernardo, J.M., Berger, J.O., Dawid, A.P., Smith, A.F.M. (Eds.), Bayesian Statistics, Vol. 4. Clarendon, Oxford, pp. 641–649] for generating a sample from univariate log-concave densities. Whereas ARS is based on sampling from piecewise exponentials, the new algorithm uses truncated normal distributions and makes use of a clever auxiliary variable technique [Damien, P., Walker, S.G., 2001. Sampling truncated normal, beta, and gamma densities. Journal of Computational and Graphical Statistics 10 (2) 206–215]. Furthermore, we extend this algorithm to deal with non-log-concave densities to provide an enhanced alternative to adaptive rejection Metropolis sampling, ARMS [Gilks, W.R., Best, N.G., Tan, K.K.C., 1995. Adaptive rejection Metropolis sampling within Gibbs sampling. Applied Statistics 44, 455–472]. The performance of ARMS and ARMS2 is compared in simulations of standard univariate distributions as well as in Gibbs sampling of a Bayesian hierarchical state-space model used for fisheries stock assessment.

Bayesian Parameter Estimation of the Four-Parameter Gamma Distribution

A Bayesian estimation of the four-parameter gamma distribution is considered under the noninformative prior. The Bayesian estimators are obtained by the Gibbs sampling. The generation of the shape/power parameter and the power parameter in the Gibbs sampler is implemented using the adaptive rejection sampling algorithm of Gilks and Wild (1992). Also, the location parameter is generated using the adaptive rejection Metropolis sampling algorithm of Gilks, Best and Tan (1995). Finally, the simulation result is presented.

Spatial stochastic volatility for lattice data

Spatial heteroscedasticity may arise jointly with spatial autocorrelation in lattice data collected from agricultural trials and environmental studies. This leads to spatial clustering not only in the level but also in the variation of the data, the latter of which may be very important, for example, in constructing prediction intervals. This article introduces a spatial stochastic volatility (SSV) component into the widely used conditional autoregressive (CAR) model to capture the spatial clustering in heteroscedasticity. The SSV component is a mean zero, conditionally independent Gaussian process given a latent spatial process of the variances. The logarithm of the latent variance process is specified by an intrinsic Gaussian Markov random field. The SSV model relaxes the traditional homoscedasticity assumption for spatial heterogeneity and brings greater flexibility to the popular spatial statistical models. The Bayesian method is used for inference. The full conditional distribution of the heteroscedasticity components can be shown to be log-concave, which facilitates an adaptive rejection sampling algorithm. Application to the well-known wheat yield data illustrates that incorporating spatial stochastic volatility may reveal the spatial heteroscedasticity hidden from existing analyses.