Variance Reduction Scheme Research Articles

Robustness of zero-variance Monte Carlo games

Zero-variance Monte Carlo games are ideal sampling strategies that provide the theoretical foundations upon which very successful variancereduction schemes are built. In this work, we explore the robustness of zerovariance games and assess in particular the impact of two common simplifications used in production Monte Carlo codes. First, using a discretized adjoint function to bias the kernels that define the stochastic process. Second, replacing the exact sampling of the flight kernel by approximate sampling strategies that are more amenable to be implemented in production Monte Carlo codes. The resulting effects on the variance and on the Figure of Merit (FoM) will be probed in the framework of a benchmark configuration based on the singlespeed two-direction transport model.

A variance reduction strategy for numerical random homogenization based on the equivalent inclusion method

Using the equivalent inclusion method (a method strongly related to the Hashin–Shtrikman variational principle) as a surrogate model, we propose a variance reduction strategy for the numerical homogenization of random composites made of “inclusions” (or rather inhomogeneities) embedded in a homogeneous matrix. The efficiency of this strategy is demonstrated within the framework of two-dimensional, linear conductivity. Significant computational gains vs full-field simulations are obtained even for high contrast values. We also show that our strategy allows to investigate the influence of parameters of the microstructure on the macroscopic response. A specific attention is given to the computational cost of the surrogate model. In particular, an inexpensive approximation of the so-called influence tensors (that are used to compute the surrogate model) is proposed.

A two-stage Kriging estimation variance reduction method for efficient time-variant reliability-based design optimization

Adaptive Kriging modelling technique is an effective tool to reduce the computational burden for time-variant reliability-based design optimization (TRBDO) significantly. However, the existing methods may allocate sampling resources to unimportant regions and do not consider the correlations between time trajectories, both of which will waste some computationally expensive samples. In this paper, we propose a two-stage Kriging estimation variance reduction method (KEVAR2) to address these challenges. First, the expression of Kriging estimation variance is derived, which can quantify the contribution of correlations between time trajectories to the Kriging prediction uncertainty. Second, the global and local adaptive Kriging approaches are proposed based on the Kriging estimation variance reduction strategy. Meanwhile, two metrics, i.e., the expectation of wrong classification rate and the error of safe rate are proposed to quantify global and local errors of Kriging models respectively. After that, we propose a two-stage framework that integrates the global and local adaptive Kriging approaches together, where the first stage performs global adaptive sampling to quickly reduce the Kriging prediction error, and then the second stage performs local adaptive sampling in the region near the current optimal solution to achieve high efficiency. Finally, five case studies demonstrate the effectiveness of the proposed KEVAR2.

Real variance estimation in iDTMC-based depletion analysis

The Improved Deterministic Truncation of Monte Carlo (iDTMC) is a powerful acceleration and variance reduction scheme in the Monte Carlo analysis. The concept of the iDTMC method and correlated sampling-based real variance estimation are briefly introduced. Moreover, the application of the iterative scheme to the correlated sampling is discussed. The iDTMC method is utilized in a 3-dimensional small modular reactor (SMR) model problem. The real variances of burnup-dependent criticality and power distribution are evaluated and compared with the ones obtained from 30 independent iDTMC calculations. The impact of the inactive cycles on the correlated sampling is also evaluated to investigate the consistency of the correlated sample scheme. In addition, numerical performances and sensitivity analysis on the real variance estimation are performed in view of the figure of merit of the iDTMC method. The numerical results show that the correlated sampling accurately estimates the real variances with high computational efficiencies.

Stochastic adaptive CL-BFGS algorithms for fully complex-valued dendritic neuron model

This paper proposes two stochastic variance reduced gradient algorithms based on adaptive complex-valued limited-memory BFGS (ACL-BFGS) algorithm, which are established on the calculation of Wirtinger gradient for fully complex-valued dendritic neuron model (FCDNM). Basically, the explicit expression of Wirtinger gradient is derived to enable the quasi-Newton algorithm for the training of FCDNM. A stochastic variance reduction strategy is adopted such that the number of samples per batch is increased during the optimization process to ensure the effectiveness of the stochastic algorithm. The stochastic ACL-BFGS algorithm with variance reduction (SACL-BFGS) can be efficiently applied for the training of FCDNM. Furthermore, by generalizing the A3 accumulative multi-step quasi-Newton method, the A3-SACL-BFGS algorithm is presented to achieve fast convergence. The experimental results on some benchmark prediction and classification problems conclusively imply that the proposed SACL-BFGS and A3-SACL-BFGS algorithms for FCDNM are clearly superior to complex-valued backpropagation (CBP), CL-BFGS and ACL-BFGS algorithms.

Breaking the sample complexity barrier to regret-optimal model-free reinforcement learning

AbstractAchieving sample efficiency in online episodic reinforcement learning (RL) requires optimally balancing exploration and exploitation. When it comes to a finite-horizon episodic Markov decision process with $S$ states, $A$ actions and horizon length $H$, substantial progress has been achieved toward characterizing the minimax-optimal regret, which scales on the order of $\sqrt{H^2SAT}$ (modulo log factors) with $T$ the total number of samples. While several competing solution paradigms have been proposed to minimize regret, they are either memory-inefficient, or fall short of optimality unless the sample size exceeds an enormous threshold (e.g. $S^6A^4 \,\mathrm{poly}(H)$ for existing model-free methods).To overcome such a large sample size barrier to efficient RL, we design a novel model-free algorithm, with space complexity $O(SAH)$, that achieves near-optimal regret as soon as the sample size exceeds the order of $SA\,\mathrm{poly}(H)$. In terms of this sample size requirement (also referred to the initial burn-in cost), our method improves—by at least a factor of $S^5A^3$—upon any prior memory-efficient algorithm that is asymptotically regret-optimal. Leveraging the recently introduced variance reduction strategy (also called reference-advantage decomposition), the proposed algorithm employs an early-settled reference update rule, with the aid of two Q-learning sequences with upper and lower confidence bounds. The design principle of our early-settled variance reduction method might be of independent interest to other RL settings that involve intricate exploration–exploitation trade-offs.

A Monte Carlo Method for Fluid Simulation

We present a novel Monte Carlo-based fluid simulation approach capable of pointwise and stochastic estimation of fluid motion. Drawing on the Feynman-Kac representation of the vorticity transport equation, we propose a recursive Monte Carlo estimator of the Biot-Savart law and extend it with a stream function formulation that allows us to treat free-slip boundary conditions using a Walk-on-Spheres algorithm. Inspired by the Monte Carlo literature in rendering, we design and compare variance reduction schemes suited to a fluid simulation context for the first time, show its applicability to complex boundary settings, and detail a simple and practical implementation with temporal grid caching. We validate the correctness of our approach via quantitative and qualitative evaluations - across a range of settings and domain geometries - and thoroughly explore its parameters' design space. Finally, we provide an in-depth discussion of several axes of future work building on this new numerical simulation modality.

Distributed Variable Sample-Size Stochastic Optimization With Fixed Step-Sizes

In this article, we consider distributed stochastic optimization over randomly switching networks, where agents collaboratively minimize the average of all agents’ local expectation-valued convex cost functions. Due to the stochasticity in gradient observations, distributedness of local functions, and randomness of communication topologies, distributed algorithms with an exact convergence guarantee under fixed step-sizes have not been achieved yet. This work incorporates variance reduction scheme into the distributed stochastic gradient tracking algorithm, where local gradients are estimated by averaging across a variable number of sampled gradients. With an identically and independently distributed random network, we show that all agents’ iterates converge almost surely to the same optimal solution under fixed step-sizes. When the global cost function is strongly convex and the sample size increases at a geometric rate, we prove that the iterates geometrically converge to the unique optimal solution, and establish the iteration, oracle, and communication complexity. The algorithm performance, including rate and complexity analysis, are further investigated with constant step-sizes and a polynomially increasing sample size. Finally, the empirical algorithm performance are illustrated with numerical examples.

Towards efficient Monte Carlo N-Particle simulation of a positron emission tomography (PET) via source volume definition

Monte Carlo N-Particle (MCNP) simulation has been extensively proven in nuclear medicine imaging systems, most notably in designing and optimizing new medical imaging tools. It enables more complicated geometries and the simulation of particles passing through and interacting with materials. However, a relatively long simulation time is a drawback of Monte Carlo simulation, mainly when complex geometry exists. The current study presents an alternative variance reduction technique for a modeled positron emission tomography (PET) camera by reducing the height of the source volume definition while maintaining the geometry of the simulated model. The National Electrical Manufacturers Association (NEMA) of the International Electrotechnical Commission (IEC) PET's phantom was used with a 1 cm diameter and 7 cm height of line source placed in the middle. The first geometry was fully filled the line source with 0.50 mCi radioactivity. In contrast, the second geometry decreased the source definition to 2.4 cm in height, covering 1 cm above and below the sub-block detector level. The source volume definition approach led to a 71% reduction in the total photons to be simulated. Results showed that the proposed variance reduction strategy could produce spatial resolution as precise as fully filled geometry and sped up the simulation time by approximately 65%. Hence, this strategy can be utilized for further PET optimizing simulation studies.

Stochastic relaxed inertial forward-backward-forward splitting for monotone inclusions in Hilbert spaces

We consider monotone inclusions defined on a Hilbert space where the operator is given by the sum of a maximal monotone operator T and a single-valued monotone, Lipschitz continuous, and expectation-valued operator V. We draw motivation from the seminal work by Attouch and Cabot (Attouch in AMO 80:547–598, 2019, Attouch in MP 184: 243–287) on relaxed inertial methods for monotone inclusions and present a stochastic extension of the relaxed inertial forward–backward-forward method. Facilitated by an online variance reduction strategy via a mini-batch approach, we show that our method produces a sequence that weakly converges to the solution set. Moreover, it is possible to estimate the rate at which the discrete velocity of the stochastic process vanishes. Under strong monotonicity, we demonstrate strong convergence, and give a detailed assessment of the iteration and oracle complexity of the scheme. When the mini-batch is raised at a geometric (polynomial) rate, the rate statement can be strengthened to a linear (suitable polynomial) rate while the oracle complexity of computing an epsilon -solution improves to {mathcal {O}}(1/epsilon ). Importantly, the latter claim allows for possibly biased oracles, a key theoretical advancement allowing for far broader applicability. By defining a restricted gap function based on the Fitzpatrick function, we prove that the expected gap of an averaged sequence diminishes at a sublinear rate of {mathcal {O}}(1/k) while the oracle complexity of computing a suitably defined epsilon -solution is {mathcal {O}}(1/epsilon ^{1+a}) where a > 1. Numerical results on two-stage games and an overlapping group Lasso problem illustrate the advantages of our method compared to competitors.

Analysis of the HI-TRAC VW Transfer Cask Dose Rates for Spent Fuel Assemblies Loaded in Nuclear Power Plant Krsko Storage Campaign One

Abstract In this paper, shielding analysis was performed to determine neutron and gamma dose rates around the transfer cask HI-TRAC VW (Holtec International Transfer Cask Variable Weight) loaded with spent fuel assemblies (SFA) from nuclear power plant (NPP) Krsko spent fuel dry storage (SFDS) campaign one. The HI-TRAC VW is a multilayered cylindrical vessel designed to accept a multipurpose canister (MPC) during loading, unloading, and transfer to dry storage building. The MPC can contain up to 37 spent fuel assemblies. The analysis was divided into two steps. The first step was the source term generation using origen-s module of the scale code package. The source was calculated based on the operating history of spent fuel assemblies currently located in the NPP Krsko spent fuel pool. The obtained particle intensities and source spectra of the SFA were used in the second step to calculate the dose rates around the transfer cask. A comprehensive hybrid shielding analysis included the calculation of dose rates resulting from fuel neutrons and gammas, neutron-induced gammas (n-γ reaction), and hardware activation gammas under normal conditions and during accident scenario. To obtain the dose rates within the acceptable uncertainties, forward weighted consistent adjoint driven importance sampling (FW-CADIS) variance reduction scheme, as implemented in advantg code, was adopted for accelerating final mcnp6.2 calculations. The dose rates around HI-TRAC VW cask were calculated using mcnp6.2 code for all 16 cask loadings belonging to campaign one in order to illustrate the impact of fuel assembly selection schemes proposed by company responsible for project realization - Holtec International (HI).

Adaptive Stochastic Variance Reduction for Subsampled Newton Method with Cubic Regularization

The cubic regularized Newton method of Nesterov and Polyak has become increasingly popular for nonconvex optimization because of its capability of finding an approximate local solution with a second order guarantee and its low iteration complexity. Several recent works extend this method to the setting of minimizing the average of N smooth functions by replacing the exact gradients and Hessians with subsampled approximations. It is shown that the total Hessian sample complexity can be reduced to be sublinear in N per iteration by leveraging stochastic variance reduction techniques. We present an adaptive variance reduction scheme for a subsampled Newton method with cubic regularization and show that the expected Hessian sample complexity is [Formula: see text] for finding an [Formula: see text]-approximate local solution (in terms of first and second order guarantees, respectively). Moreover, we show that the same Hessian sample complexity is retained with fixed sample sizes if exact gradients are used. The techniques of our analysis are different from previous works in that we do not rely on high probability bounds based on matrix concentration inequalities. Instead, we derive and utilize new bounds on the third and fourth order moments of the average of random matrices, which are of independent interest on their own.

Fast and Accurate Multidimensional Free Energy Integration.

Enhanced sampling and free energy calculation algorithms of the thermodynamic integration family (such as the adaptive biasing force (ABF) method) are not based on the direct computation of a free energy surface but rather of its gradient. Integrating the free energy surface is nontrivial in dimensions higher than one. Here, the author introduces a flexible, portable implementation of a Poisson equation formalism to integrate free energy surfaces from estimated gradients in dimensions 2 and 3 using any combination of periodic and nonperiodic (Neumann) boundary conditions. The algorithm is implemented in portable C++ and provided as a standalone tool that can be used to integrate multidimensional gradient fields estimated on a grid using any algorithm, such as umbrella integration as a post-treatment of umbrella sampling simulations. It is also included in the implementation of ABF (and its extended-system variant eABF) in the Collective Variables Module, enabling the seamless computation of multidimensional free energy surfaces within ABF and eABF simulations. A Python-based analysis toolchain is provided to easily plot and analyze multidimensional ABF simulation results, including metrics to assess their convergence. The Poisson integration algorithm can also be used to perform Helmholtz decomposition of noisy gradient estimates on the fly, resulting in an efficient implementation of the projected ABF (pABF) method proposed by Leliévre and co-workers. In numerical tests, pABF is found to lead to faster convergence with respect to ABF in simple cases of low intrinsic dimension but seems detrimental to convergence in a more realistic case involving degenerate coordinates and hidden barriers due to slower exploration. This suggests that variance reduction schemes do not always yield convergence improvements when applied to enhanced sampling methods.

Comparison of a semi-analytic variance reduction technique to classical Monte Carlo variance reduction techniques for high aspect ratio pencil beam collimators for emission tomography applications

A semi-analytic variance reduction technique developed for collimated gamma emission tomography problems was compared to classic Monte Carlo variance reduction techniques within the Monte Carlo N Particle Transport (MCNP) code. In the semi-analytic technique, a computationally efficient, non-analog, monodirectional source biased Monte Carlo simulation is first performed. Analytical expressions or empirical values are then used to correct for solid angle and field-of-view effects introduced by the non-analog source definition. This variance reduction technique was compared with deterministic transport sphere (DXTRAN) and geometry splitting variance reduction schemes to determine the accuracy and computational savings of each technique relative to an analog pulse height tally (F8 tally) at 1, 3, 5, and 10 mm collimator aperture radii. For radii 0.5 mm and 0.3 mm a DXTRAN sphere was used in place of an analog F8 tally, due to large particle history demands, to analyze the accuracy of the semi-analytic variance reduction technique. The computational savings and accuracy were evaluated for six to seven photopeaks depending on the method used. The monodirectional source biasing technique overestimated the count rates by approximately 9%–19% when the radius is less than 3 mm, but the technique overestimated by a factor of 2 to 7, when the radius is greater than or equal to 3 mm. The monodirectionally source biased technique offered computational saving factors on the order of 108-1013 over 1-10 mm collimator radii studied. DXTRAN and geometry splitting methods yielded higher accuracy, but computational savings range from approximately 0.13 to 2.2 and 0.07 to 2.9, respectively indicating marginal improvement at best.

A General Framework for Decentralized Optimization With First-Order Methods

Decentralized optimization to minimize a finite sum of functions, distributed over a network of nodes, has been a significant area within control and signal-processing research due to its natural relevance to optimal control and signal estimation problems. More recently, the emergence of sophisticated computing and large-scale data science needs have led to a resurgence of activity in this area. In this article, we discuss decentralized first-order gradient methods, which have found tremendous success in control, signal processing, and machine learning problems, where such methods, due to their simplicity, serve as the first method of choice for many complex inference and training tasks. In particular, we provide a general framework of decentralized first-order methods that is applicable to directed and undirected communication networks alike and show that much of the existing work on optimization and consensus can be related explicitly to this framework. We further extend the discussion to decentralized stochastic first-order methods that rely on stochastic gradients at each node and describe how local variance reduction schemes, previously shown to have promise in the centralized settings, are able to improve the performance of decentralized methods when combined with what is known as gradient tracking. We motivate and demonstrate the effectiveness of the corresponding methods in the context of machine learning and signal-processing problems that arise in decentralized environments.

A generalized approximate control variate framework for multifidelity uncertainty quantification

We describe and analyze a variance reduction approach for Monte Carlo (MC) sampling that accelerates the estimation of statistics of computationally expensive simulation models using an ensemble of models with lower cost. These lower cost models — which are typically lower fidelity with unknown statistics — are used to reduce the variance in statistical estimators relative to a MC estimator with equivalent cost. We derive the conditions under which our proposed approximate control variate framework recovers existing multifidelity variance reduction schemes as special cases. We demonstrate that existing recursive/nested strategies are suboptimal because they use the additional low-fidelity models only to efficiently estimate the unknown mean of the first low-fidelity model. As a result, they cannot achieve variance reduction beyond that of a control variate estimator that uses a single low-fidelity model with known mean. However, there often exists about an order-of-magnitude gap between the maximum achievable variance reduction using all low-fidelity models and that achieved by a single low-fidelity model with known mean. We show that our proposed approach can exploit this gap to achieve greater variance reduction by using non-recursive sampling schemes. The proposed strategy reduces the total cost of accurately estimating statistics, especially in cases where only low-fidelity simulation models are accessible for additional evaluations. Several analytic examples and an example with a hyperbolic PDE describing elastic wave propagation in heterogeneous media are used to illustrate the main features of the methodology.

Strangeness production in the new version of the Liège intranuclear cascade model

The capabilities of the new version of the Li\`ege Intra-Nuclear Cascade model (INCL++6) are presented in detail. This new version INCL is able to handle strange particles, such as kaons and the $\Lambda$ particle, and the associated reactions and also allows extending nucleon-nucleon collisions up to about $15-20$ GeV incident energy. Compared to the previous version, new observables can be studied, e.g., kaon, hyperon, and hypernuclei production cross sections (with the use of a suitable de-excitation code) as well as aspects of kaon-induced spallation reactions. The main purpose of this paper is to present the specific ingredients of the new INCL version and its new features, notably the new variance reduction scheme. We also compare for some illustrative strangeness production cases calculated using this version of INCL with experimental data.

Learning to Importance Sample in Primary Sample Space

AbstractImportance sampling is one of the most widely used variance reduction strategies in Monte Carlo rendering. We propose a novel importance sampling technique that uses a neural network to learn how to sample from a desired density represented by a set of samples. Our approach considers an existing Monte Carlo rendering algorithm as a black box. During a scene‐dependent training phase, we learn to generate samples with a desired density in the primary sample space of the renderer using maximum likelihood estimation. We leverage a recent neural network architecture that was designed to represent real‐valued non‐volume preserving (“Real NVP”) transformations in high dimensional spaces. We use Real NVP to non‐linearly warp primary sample space and obtain desired densities. In addition, Real NVP efficiently computes the determinant of the Jacobian of the warp, which is required to implement the change of integration variables implied by the warp. A main advantage of our approach is that it is agnostic of underlying light transport effects, and can be combined with an existing rendering technique by treating it as a black box. We show that our approach leads to effective variance reduction in several practical scenarios.

SVR-AMA: An Asynchronous Alternating Minimization Algorithm With Variance Reduction for Model Predictive Control Applications

This paper focuses on the design of an asynchronous dual solver suitable for model predictive control (MPC) applications. The proposed solver relies on a state-of-the-art variance reduction (VR) scheme, previously used in the context of proximal stochastic gradient methods (Prox-SVRG) and on the alternating minimization algorithm (AMA). The resultant algorithm, a stochastic AMA with VR (SVR-AMA), shows geometric convergence (in the expectation) to a suboptimal solution of the MPC problem and, compared to other state-of-the-art dual asynchronous algorithms, allows one to tune the probability of the asynchronous updates to improve the quality of the estimates. Two novel accelerated versions of the Prox-SVRG (and, by duality, of SVR-AMA) are also provided. We apply the proposed algorithm to a specific class of splitting methods, that is, the decomposition along the length of the prediction horizon. Numerical results on the longitudinal control problem of an Airbus passenger aircraft show the benefits that we can gain in terms of computation time when using our proposed solver with an adaptive probability distribution.

Response Matrix Method–Based Importance Solver and Variance Reduction Scheme in the Serpent 2 Monte Carlo Code

A deterministic importance solver has been implemented as an internal subroutine in the Serpent 2 Monte Carlo code for the purpose of producing weight-window meshes for variance reduction. The routine solves the adjoint transport problem using the response matrix method with coupling coefficients obtained from a conventional forward Monte Carlo simulation. The methodology can be applied to photon and neutron external source problems, and the solver supports multiple energy groups and several mesh types. Importances can be generated with respect to multiple responses, and an iterative global variance reduction sequence enables distributing the transported particle population evenly throughout the geometry. This paper describes the methodology applied in the response matrix solver and presents a verification for the generated importance functions through simple demonstrations. A practical example involving a photon shielding problem is included for performance evaluation.