Accept-reject Algorithm Research Articles

Using Rejection Sampling Probability of Acceptance as a Measure of Independence

This article proposes a new association statistic for determining whether random variables are statistically independent. The proposed association statistic can also be used to examine the strength of both linear and nonlinear dependency between variables. This statistic is derived by examining how the conditional probabilities of events differ from their corresponding marginal probabilities. The new statistic is defined in terms of the probability of acceptance commonly associated with the Accept–Reject algorithm and has a very simple formula. The simulated density of the probability of acceptance can be used to measure the degree of uncertainty of the estimated values of the new association statistic. The results from simulations as well as examples that employ real data indicate that this new association statistic is very powerful in detecting linear and nonlinear associations between two random variables. Supplementary materials for this article are available online.

DART: Adaptive Accept Reject Algorithm for Non-Linear Combinatorial Bandits

We consider the bandit problem of selecting K out of N arms at each time step. The joint reward can be a non-linear function of the rewards of the selected individual arms. The direct use of a multi-armed bandit algorithm requires choosing among all possible combinations, making the action space large. To simplify the problem, existing works on combinatorial bandits typically assume feedback as a linear function of individual rewards. In this paper, we prove the lower bound for top-K subset selection with bandit feedback with possibly correlated rewards. We present a novel algorithm for the combinatorial setting without using individual arm feedback or requiring linearity of the reward function. Additionally, our algorithm works on correlated rewards of individual arms. Our algorithm, aDaptive Accept RejecT (DART), sequentially finds good arms and eliminates bad arms based on confidence bounds. DART is computationally efficient and uses storage linear in N. Further, DART achieves a regret bound of Õ(K√KNT) for a time horizon T, which matches the lower bound in bandit feedback up to a factor of √log 2NT. When applied to the problem of cross-selling optimization and maximizing the mean of individual rewards, the performance of the proposed algorithm surpasses that of state-of-the-art algorithms. We also show that DART significantly outperforms existing methods for both linear and non-linear joint reward environments.

Mixture Density Networks-Based Knock Simulator

This paper proposes a statistical simulator for the engine knock based on the Mixture Density Network (MDN) and the accept-reject method. The proposed simulator can generate the random knock intensity signal corresponding to the input signal. The generated knock intensity has a consistent probability distribution with the real engine. Firstly, the statistical analysis is conducted with the experimental data. From the analysis results, some important assumptions on the statistical properties of the knock intensity are made. Regarding the knock intensity as a random variable on the discrete-time index, it is independent and identically distributed if the input of the engine is identical. The probability distribution of the knock intensity under identical input can be approximated by the Gaussian Mixture Model(GMM). The parameter of the GMM is a function of the input. Based on these assumptions, two sub-problems for establishing the statistical simulator are formulated: One is to approximate the function from input to the parameters of the knock intensity distribution with an absolutely continuous function; The other one is to design a random number generator that outputs the random data consistent with the given distribution. The MDN is applied to approximate the probability density of the knock intensity and the accept-reject algorithm is used for the random number generator design. The proposed method is evaluated in experimental data-based validation.

Rejection Sampling Schemes for Simulating from Arbitrary Probability Densities

Simulating random variates from arbitrary non-normalized probability densities, very often they do not have familiar forms, is an increasingly important requirement in many different fields, especially in Bayesian statistics. Accept-reject algorithm is one of the commonly used methods to simulate random variates from such densities but restriction on choosing proposal density under this framework (heavier tails than the target density) limits its applicability to a larger extent. On the other hand, Markov Chain Monte Carlo (MCMC) method can choose proposal density arbitrary which makes this method applicable to a larger class of target densities5. In addition to MCMC method, a more general widely used method known as ratio-of-uniforms (RoU) which requires only two uniform variates to simulate one variates from such densities. However, no empirical comparison among these methods for simulating random variates from such densities was seen in the literature. In this paper, we limit our study only to MCMC and RoU methods to simulate random variates from such densities. Following the generation of random variates from such densities using these two methods, we compare the performance of these two methods based on quality of the generated samples. Finally, we conclude that RoU method performs better than MCMC method as far as quality of the generated sample (randomness) and computational cost are concerned.
 Dhaka Univ. J. Sci. 68(1): 59-64, 2020 (January)

Non-parametric stochastic subset optimization for reliability-based importance ranking of bridges in transportation networks

• Proposes a non-parametric stochastic subset optimization algorithm for reliability-based importance ranking of bridges. • Proposes iterative approach with adaptive stochastic sampling to improve efficiency of algorithm. • Number of iterations only grows logarithmically with respect to the total number of bridges in the transportation network. • Algorithm is highly efficient and especially useful for large-scale networks with many bridges. Establishing reliability-based importance ranking of bridges in a transportation network usually entails significant computational efforts, especially for large-scale networks. This paper proposes a non-parametric stochastic subset optimization (NP-SSO) algorithm to efficiently identify the reliability-based importance ranking of bridges. It first generates failure samples from an augmented failure distribution and then directly establishes the importance ranking by comparing the number of failure samples for each bridge. To improve the efficiency, an iterative NP-SSO algorithm is established by restricting the search space to the subset of bridges identified in the previous iteration. For each iteration, the modified accept-reject algorithm with adaptive kernel sampling density (AKSD) is adopted to efficiently generate failure samples. The proposed NP-SSO algorithm is highly efficient for identification of important bridges, and the number of iterations only grows logarithmically with respect to the number of bridges, and NP-SSO is especially useful for large-scale networks with many bridges. The effectiveness and efficiency of NP-SSO are demonstrated through identifying the importance ranking of bridges in the transportation network of Los Angeles and Orange counties.

The study of properties on generalized Beta distribution

Statistical distributions such as Normal, Log-Normal, Gamma and Beta distributions have been studied to determine the fitting ability of the distributions in different field of studies. The objective of this research is to study the properties of the generalized Beta distribution which is a 6 parameter model from the Beta family as well as to evaluate the prediction level of the model. The advantages of the generalized Beta distribution is they can be very versatile and flexible where it will provide a good description to many different types of data, including unimodal, uniantimodal, increasing, decreasing, or bath-tub shape distribution depending on the parameter values. The distributions were fitted to rainfall data collected from Sg Lui, Selangor using Non-Linear Least-Squares Minimization package of a Python software with the maximum likelihood estimation as the parameter estimation method. Simulation was proceeded using Accept-Reject algorithm where their predictive level were evaluated using model selection criteria such as Mean Absolute Error (MAE) and Root Mean Square Error (RMSE).

An accept-reject algorithm for the positive multivariate normal distribution

The need to simulate from a positive multivariate normal distribution arises in several settings, specifically in Bayesian analysis. A variety of algorithms can be used to sample from this distribution, but most of these algorithms involve Gibbs sampling. Since the sample is generated from a Markov chain, the user has to account for the fact that sequential draws in the sample depend on one another and that the sample generated only follows a positive multivariate normal distribution asymptotically. The user would not have to account for such issues if the sample generated was i.i.d. In this paper, an accept-reject algorithm is introduced in which variates from a positive multivariate normal distribution are proposed from a multivariate skew-normal distribution. This new algorithm generates an i.i.d. sample and is shown, under certain conditions, to be very efficient.

Efficient algorithms for generating truncated multivariate normal distributions

Sampling from a truncated multivariate normal distribution (TMVND) constitutes the core computational module in fitting many statistical and econometric models. We propose two efficient methods, an iterative data augmentation (DA) algorithm and a non-iterative inverse Bayes formulae (IBF) sampler, to simulate TMVND and generalize them to multivariate normal distributions with linear inequality constraints. By creating a Bayesian incomplete-data structure, the posterior step of the DA algorithm directly generates random vector draws as opposed to single element draws, resulting obvious computational advantage and easy coding with common statistical software packages such as S-PLUS, MATLAB and GAUSS. Furthermore, the DA provides a ready structure for implementing a fast EM algorithm to identify the mode of TMVND, which has many potential applications in statistical inference of constrained parameter problems. In addition, utilizing this mode as an intermediate result, the IBF sampling provides a novel alternative to Gibbs sampling and eliminates problems with convergence and possible slow convergence due to the high correlation between components of a TMVND. The DA algorithm is applied to a linear regression model with constrained parameters and is illustrated with a published data set. Numerical comparisons show that the proposed DA algorithm and IBF sampler are more efficient than the Gibbs sampler and the accept-reject algorithm.

A modified adaptive accept–reject algorithm for univariate densities with bounded support

The need to simulate from a univariate density arises in several settings, particularly in Bayesian analysis. An especially efficient algorithm which can be used to sample from a univariate density, f X , is the adaptive accept–reject algorithm. To implement the adaptive accept–reject algorithm, the user has to envelope T ∘ f X , where T is some transformation such that the density g(x) ∝ T −1 (α+β x) is easy to sample from. Successfully enveloping T ∘ f X , however, requires that the user identify the number and location of T ∘ f X ’s inflection points. This is not always a trivial task. In this paper, we propose an adaptive accept–reject algorithm which relieves the user of precisely identifying the location of T ∘ f X ’s inflection points. This new algorithm is shown to be efficient and can be used to sample from any density such that its support is bounded and its log is three-times differentiable.

A fully automatic and highly efficient navigator gating technique for high‐resolution free‐breathing acquisitions: Continuously adaptive windowing strategy

A fully automatic and highly efficient free-breathing navigator gated technique, continuously adaptive windowing strategy (CLAWS), is presented. Using a novel and dynamic acquisition strategy that ensures all potential navigator acceptance windows are possible, CLAWS acquires an image with the highest possible efficiency regardless of variations in the respiratory pattern. Unnecessary prolongation of scan durations due to respiratory drift or navigator acceptance window adjustments are avoided. As CLAWS requires no setting of the acceptance window, nor monitoring of the navigator traces during the scan, operator dependence is minimized and ease of use improved. CLAWS was compared against a standard accept/reject algorithm (ARA) and an end-expiratory following ARA (EE-ARA) in 20 healthy subjects and 10 patients (ARA only). The respiratory efficiency was compared against the retrospectively determined best possible respiratory efficiency for each acquisition. On average, the difference between CLAWS scan times and best possible scan times was 0.6% (± 1.3%). For the ARA and EE-ARA techniques, mean differences were 14.4% (± 20.9%) and 32.6 ± 10.9%, respectively. Had the CLAWS algorithm been used with the ARA and EE-ARA traces, mean differences would have been 0.2% (± 1.1%) and 0.5% (± 1.7%), respectively. Image quality was the same for all techniques: respiratory gating, motion artifacts, navigator, and coronary artery imaging.

Fast simulation of truncated Gaussian distributions

We consider the problem of simulating a Gaussian vector X, conditional on the fact that each component of X belongs to a finite interval [a_i,b_i], or a semi-finite interval [a_i,+infty). In the one-dimensional case, we design a table-based algorithm that is computationally faster than alternative algorithms. In the two-dimensional case, we design an accept-reject algorithm. According to our calculations and our numerical studies, the acceptance rate of this algorithm is bounded from below by 0.5 for semi-finite truncation intervals, and by 0.47 for finite intervals. Extension to 3 or more dimensions is discussed.

Bayesian calibration as a tool for initialising the carbon pools of dynamic soil models

The most widely applied soil carbon models partition the soil organic carbon into two or more kinetically defined conceptual pools. The initial distribution of soil organic matter between these pools influences the simulations. Like many other soil organic carbon models, the DAYCENT model is initialised by assuming equilibrium at the beginning of the simulation. However, as we show here, the initial distribution of soil organic matter between the different pools has an appreciable influence on simulations, and the appropriate distribution is dependent on the climate and management at the site before the onset of a simulated experiment. If the soil is not in equilibrium, the only way to initialise the model is to simulate the pre-experimental period of the site. Most often, the site history, in terms of land use and land management is often poorly defined at site level, and entirely unknown at regional level. Our objective was to identify a method that can be applied to initialise a model when the soil is not in equilibrium and historic data are not available, and which quantifies the uncertainty associated with initial soil carbon distribution. We demonstrate a method that uses Bayesian calibration by means of the Accept–Reject algorithm, and use this method to calibrate the initial distribution of soil organic carbon pools against observed soil respiration measurements. It was shown that, even in short-term simulations, model initialisation can have a major influence on the simulated results. The Bayesian calibration method quantified and reduced the uncertainties in initial carbon distribution.

Beyond accept-reject sampling

We propose a new estimator for estimating a quantity μ via Monte Carlo simulations, where μ = ∫fdP and P is the target probability measure. The new estimator outperforms the usual accept-reject algorithm in terms of reducing the variance. Properties of the estimator are derived. A new Rao-Blackwellised version of the estimator is also produced, but this version requires enormous computing time. Numerical results are provided.

Simulation of right and left truncated gamma distributions by mixtures

We study the properties of truncated gamma distributions and we derive simulation algorithms which dominate the standard algorithms for these distributions. For the right truncated gamma distribution, an optimal accept–reject algorithm is based on the fact that its density can be expressed as an infinite mixture of beta distribution. For integer values of the parameters, the density of the left truncated distributions can be rewritten as a mixture which can be easily generated. We give an optimal accept–reject algorithm for the other values of the parameter. We compare the efficiency of our algorithm with the previous method and show the improvement in terms of minimum acceptance probability. The algorithm proposed here has an acceptance probability which is superior to e/4.

Statistical inference and Monte Carlo algorithms

This review article looks at a small part of the picture of the interrelationship between statistical theory and computational algorithms, especially the Gibbs sampler and the Accept-Reject algorithm. We pay particular attention to how the methodologies affect and complement each other.