On the Improvement of the Barzilai–Borwein Step Size in Variance Reduction Methods

This study compares the accuracy, efficiency, and reliability of variance reduction (VR) methods for Monte Carlo radiation transport simulations involving wide-area ground plane (i.e., “surface”) and buried (i.e., “volumetric”) gamma source emissions from environmental soil. The simulation models are idealized external exposure scenarios intended as a basis for deriving site-specific dose-based or carcinogenic risk–based regulatory limits in the radiological site remediation process. These simulations are computationally resource intensive since particle tracks are transported from an extremely large source region to a relatively small detector region. For each simulation, several VR methods are compared with metrics of accuracy, efficiency, and reliability. The MCNP deterministic transport (DXTRAN) VR method was most effective for problems involving sources emitting low-energy gamma rays, and a coupled multicode method was more effective for problems involving sources emitting higher-energy gamma rays that undergo significant attenuation in the soil.

Reliability assessment of test embankments on soft Bangkok clay by variance reduction and nearest-neighbor methods

Variance reduction (VR) methods for finite-sum minimization typically require the knowledge of problem-dependent constants that are often unknown and difficult to estimate. To address this, we use ideas from adaptive gradient methods to propose AdaSVRG, which is a more-robust variant of SVRG, a common VR method. AdaSVRG uses AdaGrad, a common adaptive gradient method, in the inner loop of SVRG, making it robust to the choice of step-size. When minimizing a sum of n smooth convex functions, we prove that a variant of AdaSVRG requires \(\tilde{O}(n + 1/\epsilon )\) gradient evaluations to achieve an \(O(\epsilon )\)-suboptimality, matching the typical rate, but without needing to know problem-dependent constants. Next, we show that the dynamics of AdaGrad exhibit a two-phase behavior – the step-size remains approximately constant in the first phase, and then decreases at a \(O\left( {1}/{\sqrt{t}}\right)\) rate. This result maybe of independent interest, and allows us to propose a heuristic that adaptively determines the length of each inner-loop in AdaSVRG. Via experiments on synthetic and real-world datasets, we validate the robustness and effectiveness of AdaSVRG, demonstrating its superior performance over standard and other “tune-free” VR methods.

A faster path-based algorithm with Barzilai-Borwein step size for solving stochastic traffic equilibrium models

There are many available variance reduction methods and these are described in the sampling theory literature in such works as Kish (1965) and Raj (1968), and in the literature of Monte Carlo methods (Hammersley & Handscomb, 1964) and in the survey paper by Halton (1970). In sampling problems where a mean is to be estimated, these variance reduction methods may induce extreme nonnormality in the distribution of the resulting estimate which in turn may cause difficulties with assessment of the error of the estimate and with other inferential procedures. We have emphasized the effects on ,81 = 2/(J6 of four major techniques of variance reduction, namely, importance or probability proportional to estimated size sampling, regression, the use of conditional expectation, and stratification. The results are readily extended to include the effects on ,82 = ,04/o-4. To facilitate comparisons with population values, (J2 and ft1 are expressed on a unit observation basis throughout.

Development of new variance reduction methods based on weight window technique in RMC code

Evaluation of the scale of fluctuation based on variance reduction method

A new formulation of the Monte-Carlo method for dynamic models of three-dimensional particle transport -within a homogeneous media is presented. A basic Monte-Carlo procedure is defined ns the so-called ‘ basic model ’. Then each of two variance reduction methods (statistical weighting and exponential transform) is formulated by alterations in the basic modal, so as to produce so-called ‘ biased models’. Each biased model is formally shown to produce the same estimate of the parameter of interest (detector response) as the basic model. This is done using methods of comparison of dynamic models derived from system theory. It is also shown that the two variance reduction methods are significant, in reducing the running time of the problem.

This paper discusses the Monte Carlo and quasi-Monte Carlo methods combined with some variance reduction techniques for exotic option pricing where the log returns of the underlying asset prices follow both the NIG and the normal distributions. An arithmetic Asian option and an Up-and-Out Asian option are considered in this paper. Our test results show that variance reduction methods can usually reduce variances significantly if they are chosen carefully. The results also show that the (randomized) quasi-Monte Carlo method is more efficient than the Monte Carlo method if both are combined with the same variance reduction method.

Variance reduction methods are often needed for the reliability assessment of complex industrial systems, we focus on one variance reduction method in a given context, that is, the interacting particle system (IPS) method used on piecewise deterministic Markov processes (PDMPs) for reliability assessment. The PDMPs are a very large class of processes which benefit from high modeling capacities, they can model almost any Markovian phenomenon that does not include diffusion. In reliability assessment, the PDMPs modeling industrial systems generally involve low jump rates and jump kernels favoring one safe arrival, we call such model a "concentrated PDMP." Used on such concentrated PDMPs, the IPS is inefficient and does not always provide a variance reduction. Indeed, the efficiency of the IPS method relies on simulating many different trajectories during its propagation steps, but unfortunately, concentrated PDMPs are likely to generate the same deterministic trajectories over and over. We propose an adaptation of the IPS method called IPS+M that reduces this phenomenon. The IPS+M consists in modifying the propagation steps of the IPS, by conditioning the propagation to avoid generating the same trajectories multiple times. We prove that, compared to the IPS, the IPS+M method always provides an estimator with a lower variance. We also carry out simulations on two-components systems that confirm these results.

Density-extrapolation Global Variance Reduction (DeGVR) method for large-scale radiation field calculation

Improving uplift model evaluation on randomized controlled trial data

The Monte Carlo simulation has become a standard tool in the practice of planning risk-affected projects. In particular, it is frequently applied to testing the impact of risk on schedule networks with deterministic structures and random activity durations defined by distribution functions of any type. The accuracy of simulation-based estimates can be improved by increasing the number of replications or by applying variance reduction methods. This paper focuses on the latter and analyzes the impact of the variance reduction method on the scale of the standard error of the estimated mean value of project duration. Three methods of variance reduction were examined: the Quasi-Monte Carlo with Weyl sequence sampling, the antithetic variates, and the Latin Hypercube Sampling. The object of the simulation experiment was a sample network model with the activity durations of triangular distributions. This type of distribution was selected as it is often applied in the practice of construction scheduling to capture the variability of operating conditions in the absence of grounds for assuming other types of distribution. The results of the sample simulation provided an indirect proof that applying variance reduction measures may reduce the time of the simulation experiment (reduced number of replications) as well as improve the confidence in the estimates of the model’s characteristics.

Monte Carlo simulations are widely considered to be the gold standard for studying the propagation of light in turbid media. However, due to the probabilistic nature of these simulations, large numbers of photons are often required in order to generate relevant results. Here, we present methods for reduction in the variance of dose distribution in a computational volume. Dose distribution is computed via tracing of a large number of rays, and tracking the absorption and scattering of the rays within discrete voxels that comprise the volume. Variance reduction is shown here using quasi-random sampling, interaction forcing for weakly scattering media, and dose smoothing via bi-lateral filtering. These methods, along with the corresponding performance enhancements are detailed here.

To solve the problem that it is difficult to determine the learning rate when training a neural network model, this paper proposes an improved adaptive algorithm based on the Barzilai-Borwein (BB) step size. In this paper, the new algorithm accelerates the model's training through the second-order momentum and adapts the learning rate according to the BB step size. We also set an adequate range for the learning rate to ensure the stability of adaptive adjustment and reduce the error of step size. Compared with different algorithms in a series of popular models, the new algorithm significantly avoids the tediousness of manually adjusting the learning rate and helps to improve the convergence speed. The results show that the new algorithm is feasible and effective.