Importance Sampling Estimator Research Articles

Population pharmacokinetic rationale for intravenous contezolid acefosamil followed by oral contezolid dosage regimens.

Contezolid is a novel oxazolidinone antibiotic with a promising safety profile. Oral contezolid and its intravenous (IV) prodrug contezolid acefosamil (CZA) are in development for treatment of diabetic foot and acute bacterial skin and skin structure infections (ABSSSI). The prodrug CZA is converted to active contezolid via intermediate MRX-1352. This study aimed to provide the pharmacokinetic rationale for safe, effective, and flexible dosage regimens with initial IV CZA followed by oral contezolid. We simultaneously modeled plasma concentrations from 110 healthy volunteers and 74 phase 2 patients with ABSSSI via population pharmacokinetics (using the importance sampling estimation algorithm), and optimized dosage regimens by Monte Carlo simulations. This included data on MRX-1352, contezolid, and its metabolite MRX-1320 from 66 healthy volunteers receiving intravenous CZA (150-2400 mg) for up to 28 days, and 74 patients receiving oral contezolid [800 mg every 12 h (q12h)] for 10 days. The apparent total clearance for 800 mg oral contezolid with food was 16.0 L/h (23.4% coefficient of variation) in healthy volunteers and 17.7 L/h (53.8%) in patients. CZA was rapidly converted to MRX-1352, which subsequently transformed to contezolid. The proposed dosage regimen used an IV CZA 2000 mg loading dose with 1000 mg IV CZA q12h as maintenance dose(s), followed by 800 mg oral contezolid q12h (with food). During each 24-h period, Monte Carlo simulations predicted this regimen to achieve consistent areas under the curve of 91.9 mg·h/L (range: 76.3-106 mg·h/L) under all scenarios. Thus, this regimen was predicted to reliably achieve efficacious contezolid exposures independent of timing of switch from IV CZA to oral contezolid.IMPORTANCEThis study provides the population pharmacokinetic rationale for the dosage regimen of the intravenous (IV) prodrug contezolid acefosamil (CZA) followed by oral contezolid. We developed the first integrated population model for the pharmacokinetics of the MRX-1352 intermediate prodrug, active contezolid, and its main metabolite MRX-1320 based on data from three clinical studies in healthy volunteers and phase 2 patients. The proposed regimen was predicted to reliably achieve efficacious contezolid exposures independent of timing of switch from IV CZA to oral contezolid.

The methods of CADIS-NEE and CADIS-DXTRAN in NECP-MCX and their applications

This paper presents two new methods for variance reduction for shielding calculation in Monte Carlo radiation transport. One method is CADIS-NEE, which combines Consistent Adjoint Driven Importance Sampling (CADIS) and next-event estimator (NEE) methods to increase the calculation efficiency of tallies at points. The other is CADIS-deterministic transport (DXTRAN), which combines CADIS and DXTRAN to obtain higher performance than using CADIS and DXTRAN separately. The combination processes are derived and implemented in the hybrid Monte-Carlo-Deterministic particle-transport code NECP-MCX. Various problems are tested to demonstrate the effectiveness of the two methods. According to the results, the two combination methods have higher efficiency than using CADIS, NEE or DXTRAN separately. In a long-distance photon-transport problem, CADIS-NEE converges faster than NEE and the figure of merit (FOM) of CADIS-NEE is 75.6 times of NEE. In a labyrinthine problem, CADIS-DXTRAN's FOM surpasses that of DXTRAN and CADIS by a factor of 45.3 and 17.7, respectively. Therefore, it is advisable to employ these two novel methods selectively in appropriate scenarios to reduce variance.

Robust importance sampling with adaptive winsorization

Importance sampling is a widely used technique to estimate properties of a distribution. The resulting estimator is unbiased but may have huge, potentially infinite, variance. This paper proposes trading-off some bias for variance by adaptively winsorizing the importance sampling estimator. The novel procedure is based on the Balancing Principle (or Lepskii’s Method). As a consequence, it offers a principled way to perform winsorization with finite-sample guarantees in the form of an oracle inequality. In various examples, the proposed estimator is shown to have smaller mean squared error and mean absolute deviation than leading alternatives such as the traditional importance sampling estimator or winsorizing it via cross-validation.

Minimax Off-Policy Evaluation for Multi-Armed Bandits

We study the problem of off-policy evaluation in the multi-armed bandit model with bounded rewards, and develop minimax rate-optimal procedures under three settings. First, when the behavior policy is known, we show that the Switch estimator, a method that alternates between the plug-in and importance sampling estimators, is minimax rate-optimal for all sample sizes. Second, when the behavior policy is unknown, we analyze performance in terms of the competitive ratio, thereby revealing a fundamental gap between the settings of known and unknown behavior policies. When the behavior policy is unknown, any estimator must have mean-squared error larger—relative to the oracle estimator equipped with the knowledge of the behavior policy— by a multiplicative factor proportional to the support size of the target policy. Moreover, we demonstrate that the plug-in approach achieves this worst-case competitive ratio up to a logarithmic factor. Third, we initiate the study of the partial knowledge setting in which it is assumed that the minimum probability taken by the behavior policy is known. We show that the plug-in estimator is optimal for relatively large values of the minimum probability, but is sub-optimal when the minimum probability is low. In order to remedy this gap, we propose a new estimator based on approximation by Chebyshev polynomials that provably achieves the optimal estimation error. Numerical experiments on both simulated and real data corroborate our theoretical findings.

I-SEA: Importance Sampling and Expected Alignment-Based Deep Distance Metric Learning for Time Series Analysis and Embedding

Learning effective embeddings for potentially irregularly sampled time-series, evolving at different time scales, is fundamental for machine learning tasks such as classification and clustering. Task-dependent embeddings rely on similarities between data samples to learn effective geometries. However, many popular time-series similarity measures are not valid distance metrics, and as a result they do not reliably capture the intricate relationships between the multi-variate time-series data samples for learning effective embeddings. One of the primary ways to formulate an accurate distance metric is by forming distance estimates via Monte-Carlo-based expectation evaluations. However, the high-dimensionality of the underlying distribution, and the inability to sample from it, pose significant challenges. To this end, we develop an Importance Sampling based distance metric -- I-SEA -- which enjoys the properties of a metric while consistently achieving superior performance for machine learning tasks such as classification and representation learning. I-SEA leverages Importance Sampling and Non-parametric Density Estimation to adaptively estimate distances, enabling implicit estimation from the underlying high-dimensional distribution, resulting in improved accuracy and reduced variance. We theoretically establish the properties of I-SEA and demonstrate its capabilities via experimental evaluations on real-world healthcare datasets.

Volumetric Multi‐View Rendering

AbstractRendering photo‐realistic images using Monte Carlo path tracing often requires sampling a large number of paths to reach acceptable levels of noise. This is particularly the case when rendering participating media, that complexify light paths with multiple scattering events. Our goal is to accelerate the rendering of heterogeneous participating media by exploiting redundancy across views, for instance when rendering animated camera paths, motion blur in consecutive frames or multi‐view images such as lenticular or light‐field images. This poses a challenge as existing methods for sharing light paths across views cannot handle heterogeneous participating media and classical estimators are not optimal in this context. We address these issues by proposing three key ideas. First, we propose new volume shift mappings to transform light paths from one view to another within the recently introduced null‐scattering framework, taking into account changes in density along the transformed path. Second, we generate a shared path suffix that best contributes to a subset of views, thus effectively reducing variance. Third, we introduce the multiple weighted importance sampling estimator that benefits from multiple importance sampling for combining sampling strategies, and from weighted importance sampling for reducing the variance due to non contributing strategies. We observed significant reuse when views largely overlap, with no visible bias and reduced variance compared to regular path tracing at equal time. Our method further readily integrates into existing volumetric path tracing pipelines.

A Koopman framework for rare event simulation in stochastic differential equations

We exploit the relationship between the stochastic Koopman operator and the Kolmogorov backward equation to construct importance sampling schemes for stochastic differential equations. Specifically, we propose using eigenfunctions of the stochastic Koopman operator to approximate the Doob transform for an observable of interest (e.g., associated with a rare event) which in turn yields an approximation of the corresponding zero-variance importance sampling estimator. Our approach is broadly applicable and systematic, treating non-normal systems, non-gradient systems, and systems with oscillatory dynamics or rank-deficient noise in a common framework. In nonlinear settings where the stochastic Koopman eigenfunctions cannot be derived analytically, we use dynamic mode decomposition (DMD) methods to approximate them numerically, but the framework is agnostic to the particular numerical method employed. Numerical experiments demonstrate that even coarse approximations of a few eigenfunctions, where the latter are built from non-rare trajectories, can produce effective importance sampling schemes for rare events.

Generalizing the Balance Heuristic Estimator in Multiple Importance Sampling.

In this paper, we propose a novel and generic family of multiple importance sampling estimators. We first revisit the celebrated balance heuristic estimator, a widely used Monte Carlo technique for the approximation of intractable integrals. Then, we establish a generalized framework for the combination of samples simulated from multiple proposals. Our approach is based on considering as free parameters both the sampling rates and the combination coefficients, which are the same in the balance heuristics estimator. Thus our novel framework contains the balance heuristic as a particular case. We study the optimal choice of the free parameters in such a way that the variance of the resulting estimator is minimized. A theoretical variance study shows the optimal solution is always better than the balance heuristic estimator (except in degenerate cases where both are the same). We also give sufficient conditions on the parameter values for the new generalized estimator to be better than the balance heuristic estimator, and one necessary and sufficient condition related to divergence. Using five numerical examples, we first show the gap in the efficiency of both new and classical balance heuristic estimators, for equal sampling and for several state of the art sampling rates. Then, for these five examples, we find the variances for some notable selection of parameters showing that, for the important case of equal count of samples, our new estimator with an optimal selection of parameters outperforms the classical balance heuristic. Finally, new heuristics are introduced that exploit the theoretical findings.

Optimality in noisy importance sampling

Many applications in signal processing and machine learning require the study of probability density functions (pdfs) that can only be accessed through noisy evaluations. In this work, we analyze the noisy importance sampling (IS), i.e., IS working with noisy evaluations of the target density. We present the general framework and derive optimal proposal densities for noisy IS estimators. The optimal proposals incorporate the information of the variance of the noisy realizations, proposing points in regions where the noise power is higher. We also compare the use of the optimal proposals with previous optimality approaches considered in a noisy IS framework.

Asymptotically Tight MLD Bounds and Minimum-Variance Importance Sampling Estimator for Linear Block Codes Over BSCs

Asymptotically Tight MLD Bounds and Minimum-Variance Importance Sampling Estimator for Linear Block Codes Over BSCs

Learning from eXtreme Bandit Feedback

We study the problem of batch learning from bandit feedback in the setting of extremely large action spaces. Learning from extreme bandit feedback is ubiquitous in recommendation systems, in which billions of decisions are made over sets consisting of millions of choices in a single day, yielding massive observational data. In these large-scale real-world applications, supervised learning frameworks such as eXtreme Multi-label Classification (XMC) are widely used despite the fact that they incur significant biases due to the mismatch between bandit feedback and supervised labels. Such biases can be mitigated by importance sampling techniques, but these techniques suffer from impractical variance when dealing with a large number of actions. In this paper, we introduce a selective importance sampling estimator (sIS) that operates in a significantly more favorable bias-variance regime. The sIS estimator is obtained by performing importance sampling on the conditional expectation of the reward with respect to a small subset of actions for each instance (a form of Rao-Blackwellization). We employ this estimator in a novel algorithmic procedure---named Policy Optimization for eXtreme Models (POXM)---for learning from bandit feedback on XMC tasks. In POXM, the selected actions for the sIS estimator are the top-p actions of the logging policy, where p is adjusted from the data and is significantly smaller than the size of the action space. We use a supervised-to-bandit conversion on three XMC datasets to benchmark our POXM method against three competing methods: BanditNet, a previously applied partial matching pruning strategy, and a supervised learning baseline. Whereas BanditNet sometimes improves marginally over the logging policy, our experiments show that POXM systematically and significantly improves over all baselines.

Some new results on relative entropy production, time reversal, and optimal control of time-inhomogeneous diffusion processes

This paper studies time-inhomogeneous diffusion processes, including both Brownian dynamics and Langevin dynamics. We derive upper bounds of the relative entropy production for a time-inhomogeneous process with respect to the transient invariant probability measures. We also study the time reversal of the reverse process in Crooks’s fluctuation theorem. We show that the time reversal of the reverse process coincides with the optimally controlled forward process that leads to a zero variance importance sampling estimator for free energy calculations based on Jarzynski’s equality.

Confidence-Based Design Optimization for a More Conservative Optimum Under Surrogate Model Uncertainty Caused by Gaussian Process

Abstract Even though many efforts have been devoted to effective strategies to build accurate surrogate models, surrogate model uncertainty is inevitable due to a limited number of available simulation samples. Therefore, the surrogate model uncertainty, one of the epistemic uncertainties in reliability-based design optimization (RBDO), has to be considered during the design process to prevent unexpected failure of a system that stems from an inaccurate surrogate model. However, there have been limited attempts to obtain a reliable optimum taking into account the surrogate model uncertainty due to its complexity and computational burden. Thus, this paper proposes a confidence-based design optimization (CBDO) under surrogate model uncertainty to find a conservative optimum despite an insufficient number of simulation samples. To compensate the surrogate model uncertainty in reliability analysis, the confidence of reliability is brought to describe the uncertainty of reliability. The proposed method employs the Gaussian process modeling to explicitly quantify the uncertainty of a surrogate model. Thus, metamodel-based importance sampling and expansion optimal linear estimation are exploited to reduce the computational burden on confidence estimation. In addition, stochastic sensitivity analysis of the confidence is developed for CBDO, which is formulated to find a conservative optimum than an RBDO optimum at a specific confidence level. Numerical examples using mathematical functions and finite element analysis show that the proposed confidence analysis and CBDO can prevent overestimation of reliability caused by an inaccurate surrogate model.

A Robust Solution to Variational Importance Sampling of Minimum Variance.

Importance sampling is a Monte Carlo method where samples are obtained from an alternative proposal distribution. This can be used to focus the sampling process in the relevant parts of space, thus reducing the variance. Selecting the proposal that leads to the minimum variance can be formulated as an optimization problem and solved, for instance, by the use of a variational approach. Variational inference selects, from a given family, the distribution which minimizes the divergence to the distribution of interest. The Rényi projection of order 2 leads to the importance sampling estimator of minimum variance, but its computation is very costly. In this study with discrete distributions that factorize over probabilistic graphical models, we propose and evaluate an approximate projection method onto fully factored distributions. As a result of our evaluation it becomes apparent that a proposal distribution mixing the information projection with the approximate Rényi projection of order 2 could be interesting from a practical perspective.

Fast Search and Estimation of Bayesian Nonparametric Mixture Models Using a Classification Annealing EM Algorithm

Bayesian nonparametric (BNP) infinite-mixture models provide flexible and accurate density estimation, cluster analysis, and regression. However, for the posterior inference of such a model, MCMC algorithms are complex, often need to be tailor-made for different BNP priors, and are intractable for large datasets. We introduce a BNP classification annealing EM algorithm which employs importance sampling estimation. This new fast-search algorithm, for virtually any given BNP mixture model, can quickly and accurately calculate the posterior predictive density estimate (by posterior averaging) and the maximum a-posteriori clustering estimate (by simulated annealing), even for datasets containing millions of observations. The algorithm can handle a wide range of BNP priors because it primarily relies on the ability to generate prior samples. The algorithm can be fast because in each iteration, it performs a sampling step for the (missing) clustering of the data points, instead of a costly E-step; and then performs direct posterior calculations in the M-step, given the sampled (imputed) clustering. The new algorithm is illustrated and evaluated through BNP Gaussian mixture model analyses of benchmark simulated data and real datasets. MATLAB code for the new algorithm is provided in the supplementary materials. Supplementary materials for this article are available online.

Efficient Importance Sampling for the Left Tail of Positive Gaussian Quadratic Forms

Estimating the left tail of quadratic forms in Gaussian random vectors is of major practical importance in many applications. In this letter, we propose an efficient importance sampling estimator that is endowed with the bounded relative error property. This property significantly reduces the number of simulation runs required by the proposed estimator compared to naive Monte Carlo (MC), especially when the probability of interest is very small. Selected simulation results are presented to illustrate the efficiency of our estimator compared to naive MC as well as some of the well-known approximations.

The Counterfactual χ-GAN: Finding comparable cohorts in observational health data.

Causal inference often relies on the counterfactual framework, which requires that treatment assignment is independent of the outcome, known as strong ignorability. Approaches to enforcing strong ignorability in causal analyses of observational data include weighting and matching methods. Effect estimates, such as the average treatment effect (ATE), are then estimated as expectations under the re-weighted or matched distribution, P. The choice of P is important and can impact the interpretation of the effect estimate and the variance of effect estimates. In this work, instead of specifying P, we learn a distribution that simultaneously maximizes coverage and minimizes variance of ATE estimates. In order to learn this distribution, this research proposes a generative adversarial network (GAN)-based model called the Counterfactual χ-GAN (cGAN), which also learns feature-balancing weights and supports unbiased causal estimation in the absence of unobserved confounding. Our model minimizes the Pearson χ2-divergence, which we show simultaneously maximizes coverage and minimizes the variance of importance sampling estimates. To our knowledge, this is the first such application of the Pearson χ2-divergence. We demonstrate the effectiveness of cGAN in achieving feature balance relative to established weighting methods in simulation and with real-world medical data.

Importance sampling correction versus standard averages of reversible MCMCs in terms of the asymptotic variance

We establish an ordering criterion for the asymptotic variances of two consistent Markov chain Monte Carlo (MCMC) estimators: an importance sampling (IS) estimator, based on an approximate reversible chain and subsequent IS weighting, and a standard MCMC estimator, based on an exact reversible chain. Essentially, we relax the criterion of the Peskun type covariance ordering by considering two different invariant probabilities, and obtain, in place of a strict ordering of asymptotic variances, a bound of the asymptotic variance of IS by that of the direct MCMC. Simple examples show that IS can have arbitrarily better or worse asymptotic variance than Metropolis–Hastings and delayed-acceptance (DA) MCMC. Our ordering implies that IS is guaranteed to be competitive up to a factor depending on the supremum of the (marginal) IS weight. We elaborate upon the criterion in case of unbiased estimators as part of an auxiliary variable framework. We show how the criterion implies asymptotic variance guarantees for IS in terms of pseudo-marginal (PM) and DA corrections, essentially if the ratio of exact and approximate likelihoods is bounded. We also show that convergence of the IS chain can be less affected by unbounded high-variance unbiased estimators than PM and DA chains.

Simulating false alarm probability in K-distributed sea clutter

Efficient and precise calculation of the probability of false alarms for the K-distributed sea clutter and noise is crucial in radar detection system. We propose efficient and optimal importance sampling algorithms, in which two importance sampling estimators are constructed for both the product form and the compound form of the underlying K distribution. Moreover, we prove the existence and uniqueness of the optimal tilting parameters. Detection accuracy and efficiency are analyzed in simulation studies.

FAVE: A Fast and Efficient Network Flow AVailability Estimation Method With Bounded Relative Error

Capacity planning and sales projection are essential tasks for network operators. This work aims to help network providers to carry out network capacity planning and sales projection by answering: Given topology and capacity, whether the network can serve current flow demands with high probabilities? We name such probability as the “ flow availability ”, and present the flow availability estimation (FAVE) problem with generalizing the classical network connectivity based and maximum flow based reliability estimations. To quickly estimate flow availabilities, we utilize correlations among link and flow failures to figure out the importance of roles played by different links in flow failures (i.e., flow demands could not be satisfied). And we design three sequential importance sampling (SIS) estimation methods, which are: (1) Accurate and efficient : They achieve a bounded or even vanishing relative error with linear computational complexities. Hence they can provide more accurate estimations in less simulation time. (2) Robust and scalable: They maintain such estimation efficiencies even if only a partial SEED set information is available, or when the FAVE problem is extended to the multiple flows case. When applying to a realistic backbone network, our method can reduce the flow availability estimation cost by 900 and 130 times compared with MC and baseline IS methods; and also facilitate capacity planning and sales projection by providing better flow availability guarantees, compared with traditional methods.