Adaptive Multiple Importance Sampling Research Articles

Bayesian Inference for Multiple Datasets

Estimating parameters for multiple datasets can be time consuming, especially when the number of datasets is large. One solution is to sample from multiple datasets simultaneously using Bayesian methods such as adaptive multiple importance sampling (AMIS). Here, we use the AMIS approach to fit a von Mises distribution to multiple datasets for wind trajectories derived from a Lagrangian Particle Dispersion Model driven from 3D meteorological data. A posterior distribution of parameters can help to characterise the uncertainties in wind trajectories in a form that can be used as inputs for predictive models of wind-dispersed insect pests and the pathogens of agricultural crops for use in evaluating risk and in planning mitigation actions. The novelty of our study is in testing the performance of the method on a very large number of datasets (>11,000). Our results show that AMIS can significantly improve the efficiency of parameter inference for multiple datasets.

A distributed particle filter with sampling-based consensus density fusion for speaker tracking in distributed microphone networks

This paper proposes a novel distributed particle filter with sampling-based consensus density fusion for speaker tracking in distributed microphone networks (DMNs). Firstly, to combat the reverberation and ambient noise, multi-hypothesis likelihood model is applied to describe the likelihood distribution in the state-space model of speaker tracking problem. Then, the sequential importance resampling (SIR) is performed on each node to obtain the local posteriors of the state vector of the speaker. Since the non-Gaussian noise and the reverberation may make local posteriors presenting non-Gaussian and multi-modal characters, in this paper, the local posterior densities are approximated by Gaussian mixtures whose parameters are further transmitted to neighboring nodes to complete the distributed fusion based on the consensus method. We investigate a novel adaptive multiple importance sampling (AMIS) algorithm combining parallel tempering Monte Carlo (PTMC) samplers to achieve the sampling-based consensus density fusion of the non-Gaussian local posteriors and estimate the global posterior locally on each node. At last, the optimal global estimation of the speaker state vector could be calculated by using the estimated global posterior on each node in DMNs. Simulation and real-world recording experiment results show that the proposed speaker tracking method exhibits satisfactory tracking performance under reverberant and noisy environments. Moreover, it is easy and feasible to extend the proposed tracking algorithm to solve the multi-speaker tracking problem.

Forecasting Pathogen Dynamics with Bayesian Model-Averaging: Application to Xylella fastidiosa.

Forecasting invasive-pathogen dynamics is paramount to anticipate eradication and containment strategies. Such predictions can be obtained using a model grounded on partial differential equations (PDE; often exploited to model invasions) and fitted to surveillance data. This framework allows the construction of phenomenological but concise models relying on mechanistic hypotheses and real observations. However, it may lead to models with overly rigid behavior and possible data-model mismatches. Hence, to avoid drawing a forecast grounded on a single PDE-based model that would be prone to errors, we propose to apply Bayesian model averaging (BMA), which allows us to account for both parameter and model uncertainties. Thus, we propose a set of different competing PDE-based models for representing the pathogen dynamics, we use an adaptive multiple importance sampling algorithm (AMIS) to estimate parameters of each competing model from surveillance data in a mechanistic-statistical framework, we evaluate the posterior probabilities of models by comparing different approaches proposed in the literature, and we apply BMA to draw posterior distributions of parameters and a posterior forecast of the pathogen dynamics. This approach is applied to predict the extent of Xylella fastidiosa in South Corsica, France, a phytopathogenic bacterium detected in situ in Europe less than 10 years ago (Italy 2013, France 2015). Separating data into training and validation sets, we show that the BMA forecast outperforms competing forecast approaches.

Fitting double hierarchical models with the integrated nested Laplace approximation

Double hierarchical generalized linear models (DHGLM) are a family of models that are flexible enough as to model hierarchically the mean and scale parameters. In a Bayesian framework, fitting highly parameterized hierarchical models is challenging when this problem is addressed using typical Markov chain Monte Carlo (MCMC) methods due to the potential high correlation between different parameters and effects in the model. The integrated nested Laplace approximation (INLA) could be considered instead to avoid dealing with these problems. However, DHGLM do not fit within the latent Gaussian Markov random field (GMRF) models that INLA can fit. In this paper, we show how to fit DHGLM with INLA by combining INLA and importance sampling (IS) algorithms. In particular, we will illustrate how to split DHGLM into submodels that can be fitted with INLA so that the remainder of the parameters are fit using adaptive multiple IS (AMIS) with the aid of the graphical representation of the hierarchical model. This is illustrated using a simulation study on three different types of models and two real data examples.

Importance Sampling with the Integrated Nested Laplace Approximation

The integrated nested Laplace approximation (INLA) is a deterministic approach to Bayesian inference on latent Gaussian models (LGMs) and focuses on fast and accurate approximation of posterior marginals for the parameters in the models. Recently, methods have been developed to extend this class of models to those that can be expressed as conditional LGMs by fixing some of the parameters in the models to descriptive values. These methods differ in the manner descriptive values are chosen. This article proposes to combine importance sampling with INLA (IS-INLA), and extends this approach with the more robust adaptive multiple importance sampling algorithm combined with INLA (AMIS-INLA). This article gives a comparison between these approaches and existing methods on a series of applications with simulated and observed datasets and evaluates their performance based on accuracy, efficiency, and robustness. The approaches are validated by exact posteriors in a simple bivariate linear model; then, they are applied to a Bayesian lasso model, a Poisson mixture, a zero-inflated Poisson model and a spatial autoregressive combined model. The applications show that the AMIS-INLA approach, in general, outperforms the other methods compared, but the IS-INLA algorithm could be considered for faster inference when good proposals are available. Supplementary materials for this article are available online.

Integrating geostatistical maps and infectious disease transmission models using adaptive multiple importance sampling

The Adaptive Multiple Importance Sampling algorithm (AMIS) is an iterative technique which recycles samples from all previous iterations in order to improve the efficiency of the proposal distribution. We have formulated a new statistical framework, based on AMIS, to take the output from a geostatistical model of infectious disease prevalence, incidence or relative risk, and project it forward in time under a mathematical model for transmission dynamics. We adapted the AMIS algorithm so that it can sample from multiple targets simultaneously by changing the focus of the adaptation at each iteration. By comparing our approach against the standard AMIS algorithm, we showed that these novel adaptations greatly improve the efficiency of the sampling. We tested the performance of our algorithm on four case studies: ascariasis in Ethiopia, onchocerciasis in Togo, human immunodeficiency virus (HIV) in Botswana, and malaria in the Democratic Republic of the Congo.

Uncertainty Quantification of Regional Cardiac Tissue Properties in Arrhythmogenic Cardiomyopathy Using Adaptive Multiple Importance Sampling.

Introduction: Computational models of the cardiovascular system are widely used to simulate cardiac (dys)function. Personalization of such models for patient-specific simulation of cardiac function remains challenging. Measurement uncertainty affects accuracy of parameter estimations. In this study, we present a methodology for patient-specific estimation and uncertainty quantification of parameters in the closed-loop CircAdapt model of the human heart and circulation using echocardiographic deformation imaging. Based on patient-specific estimated parameters we aim to reveal the mechanical substrate underlying deformation abnormalities in patients with arrhythmogenic cardiomyopathy (AC).Methods: We used adaptive multiple importance sampling to estimate the posterior distribution of regional myocardial tissue properties. This methodology is implemented in the CircAdapt cardiovascular modeling platform and applied to estimate active and passive tissue properties underlying regional deformation patterns, left ventricular volumes, and right ventricular diameter. First, we tested the accuracy of this method and its inter- and intraobserver variability using nine datasets obtained in AC patients. Second, we tested the trueness of the estimation using nine in silico generated virtual patient datasets representative for various stages of AC. Finally, we applied this method to two longitudinal series of echocardiograms of two pathogenic mutation carriers without established myocardial disease at baseline.Results: Tissue characteristics of virtual patients were accurately estimated with a highest density interval containing the true parameter value of 9% (95% CI [0–79]). Variances of estimated posterior distributions in patient data and virtual data were comparable, supporting the reliability of the patient estimations. Estimations were highly reproducible with an overlap in posterior distributions of 89.9% (95% CI [60.1–95.9]). Clinically measured deformation, ejection fraction, and end-diastolic volume were accurately simulated. In presence of worsening of deformation over time, estimated tissue properties also revealed functional deterioration.Conclusion: This method facilitates patient-specific simulation-based estimation of regional ventricular tissue properties from non-invasive imaging data, taking into account both measurement and model uncertainties. Two proof-of-principle case studies suggested that this cardiac digital twin technology enables quantitative monitoring of AC disease progression in early stages of disease.

Metamodelling for geotechnical reliability analysis with noisy and incomplete models

ABSTRACT A kriging-based metamodelling approach for analysing the structural reliability of a sheetpile wall in a dyke is formulated. This specific problem is characterised by high target reliabilities () in combination with a noisy and incomplete numerical model response. Starting from the original formulation of active learning kriging-based Monte Carlo simulation (AK-MCS), a robust two-stage metamodel framework is formulated in combination with adaptive multiple importance sampling, Gaussian process classification and kernel enhancements. Learning functions and convergence criteria are revised to maintain consistency with the metamodel enhancements. The developed metamodel is applied in the reliability analysis of a soil-structure interaction problem involving a sheetpile wall in a dyke body, which is representative of a class of problems encountered in engineering practice. Low dimensional example studies demonstrate the workings of the model and give insight into the model response. Full probabilistic analyses are then performed to estimate the probabilities of structural failure in a reliability updating context. The results show that after several necessary enhancements of the classical formulations, metamodelling approaches can be used successfully in combination with noisy and incomplete computational models as are often encountered in geotechnical engineering practice.

A bayesian inference procedure based on inverse dispersion modelling for source term estimation in built-up environments

Abstract In atmospheric physics, reconstructing a pollution source is a challenging and important question. It provides better input parameters to dispersion models, and gives useful information to first-responder teams in case of an accidental toxic release. Various methods already exist, but using them requires an important amount of computational resources, especially when the accuracy of the dispersion model increases which is necessary in complex built-up environments. In this paper, a Bayesian probabilistic approach to estimate the location and the temporal emission profile of a pointwise source is proposed. More precisely, an Adaptive Multiple Importance Sampling (AMIS) algorithm is considered and enhanced by an efficient use of a Lagrangian Particle Dispersion Model (LPDM) in backward mode. Twin experiments empirically demonstrate the efficiency of the proposed inference strategy in very complex cases.

A Derivative-Free Trust-Region Algorithm for the Optimization of Functions Smoothed via Gaussian Convolution Using Adaptive Multiple Importance Sampling

In this paper we consider the optimization of a functional $F$ defined as the convolution of a function $f$ with a Gaussian kernel. We propose this type of objective function for the optimization of the output of complex computational simulations, which often present some form of deterministic noise and need to be smoothed for the results to be meaningful. We introduce a derivative-free algorithm that computes trial points from the minimization of a regression model of the noisy function $f$ over a trust region. The regression model is constructed from function values at sample points that are chosen randomly around iterates and trial points of the algorithm. The weights given to the individual sample points in the regression problem are obtained according to an adaptive multiple importance sampling strategy. This has two advantages. First, it makes it possible to reuse all noisy function values collected over the course of the optimization. Second, the resulting regression model converges to the second-o...

Sample Reuse with Adaptive Multiple Importance Sampling for Policy Search

Sample Reuse with Adaptive Multiple Importance Sampling for Policy Search

Adaptive multiple importance sampling for general functions

We propose a mathematical expression for the optimal distribution of the number of samples in multiple importance sampling (MIS) and also give heuristics that work well in practice. The MIS balance...

Adaptive multiple importance sampling for Gaussian processes

ABSTRACTIn applications of Gaussian processes (GPs) where quantification of uncertainty is a strict requirement, it is necessary to accurately characterize the posterior distribution over Gaussian process covariance parameters. This is normally done by means of standard Markov chain Monte Carlo (MCMC) algorithms, which require repeated expensive calculations involving the marginal likelihood. Motivated by the desire to avoid the inefficiencies of MCMC algorithms rejecting a considerable amount of expensive proposals, this paper develops an alternative inference framework based on adaptive multiple importance sampling (AMIS). In particular, this paper studies the application of AMIS for GPs in the case of a Gaussian likelihood, and proposes a novel pseudo-marginal-based AMIS algorithm for non-Gaussian likelihoods, where the marginal likelihood is unbiasedly estimated. The results suggest that the proposed framework outperforms MCMC-based inference of covariance parameters in a wide range of scenarios.

An adaptive Bayesian inference algorithm to estimate the parameters of a hazardous atmospheric release

In the eventuality of an accidental or intentional atmospheric release, the reconstruction of the source term using measurements from a set of sensors is an important and challenging inverse problem. A rapid and accurate estimation of the source allows faster and more efficient action for first-response teams, in addition to providing better damage assessment. This paper presents a Bayesian probabilistic approach to estimate the location and the temporal emission profile of a pointwise source. The release rate is evaluated analytically by using a Gaussian assumption on its prior distribution, and is enhanced with a positivity constraint to improve the estimation. The source location is obtained by the means of an advanced iterative Monte-Carlo technique called Adaptive Multiple Importance Sampling (AMIS), which uses a recycling process at each iteration to accelerate its convergence. The proposed methodology is tested using synthetic and real concentration data in the framework of the Fusion Field Trials 2007 (FFT-07) experiment. The quality of the obtained results is comparable to those coming from the Markov Chain Monte Carlo (MCMC) algorithm, a popular Bayesian method used for source estimation. Moreover, the adaptive processing of the AMIS provides a better sampling efficiency by reusing all the generated samples.

An Adaptive Population Importance Sampler: Learning From Uncertainty

Monte Carlo (MC) methods are well-known computational techniques, widely used in different fields such as signal processing, communications and machine learning. An important class of MC methods is composed of importance sampling (IS) and its adaptive extensions, such as population Monte Carlo (PMC) and adaptive multiple IS (AMIS). In this paper, we introduce a novel adaptive and iterated importance sampler using a population of proposal densities. The proposed algorithm, named adaptive population importance sampling (APIS), provides a global estimation of the variables of interest iteratively, making use of all the samples previously generated. APIS combines a sophisticated scheme to build the IS estimators (based on the deterministic mixture approach) with a simple temporal adaptation (based on epochs). In this way, APIS is able to keep all the advantages of both AMIS and PMC, while minimizing their drawbacks. Furthermore, APIS is easily parallelizable. The cloud of proposals is adapted in such a way that local features of the target density can be better taken into account compared to single global adaptation procedures. The result is a fast, simple, robust, and high-performance algorithm applicable to a wide range of problems. Numerical results show the advantages of the proposed sampling scheme in four synthetic examples and a localization problem in a wireless sensor network.

Visual Tracking of Human Head and Arms Using Adaptive Multiple Importance Sampling on a Single Camera in Cluttered Environments

This paper presents a 2D upper body tracking algorithm using a single monocular camera. The proposed method can be applied on a stationary or moving camera platform, and is able to achieve real-time performance in cluttered environments, making it ideal for human-machine interaction. The algorithm extracts body parts even when the target person approaches other objects. This is a common problem for depth-based camera systems. Real-time visual extraction of a subject's head and arms is performed during preprocessing in order to determine their current action and presents two key innovations. First, multiple visual clues are integrated dynamically by an adaptive multiple importance sampling particle filter to generate hypotheses. These hypotheses can efficiently estimate various gestures of arms on images captured from cluttered environments. Second, multiple visual cues of a human face and arms are devised, which quickly and effectively verifies various hypotheses from the multiple importance sampling schemes. To validate the effectiveness of the proposed tracking approach, several experiments are performed whose results appear to be quite promising.

Adaptive Multiple Importance Sampling

Abstract. The Adaptive Multiple Importance Sampling algorithm is aimed at an optimal recycling of past simulations in an iterated importance sampling (IS) scheme. The difference with earlier adaptive IS implementations like Population Monte Carlo is that the importance weights of all simulated values, past as well as present, are recomputed at each iteration, following the technique of the deterministic multiple mixture estimator of Owen & Zhou (J. Amer. Statist. Assoc., 95, 2000, 135). Although the convergence properties of the algorithm cannot be investigated, we demonstrate through a challenging banana shape target distribution and a population genetics example that the improvement brought by this technique is substantial.

Representativity for Robust and Adaptive Multiple Importance Sampling

We present a general method enhancing the robustness of estimators based on multiple importance sampling (MIS) in a numerical integration context. MIS minimizes variance of estimators for a given sampling configuration, but when this configuration is less adapted to the integrand, the resulting estimator suffers from extra variance. We address this issue by introducing the notion of "representativity" of a sampling strategy, and demonstrate how it can be used to increase robustness of estimators, by adapting them to the integrand. We first show how to compute representativities using common rendering informations such as BSDF, photon maps, or caches in order to choose the best sampling strategy for MIS. We then give hints to generalize our method to any integration problem and demonstrate that it can be used successfully to enhance robustness in different common rendering algorithms.