Statistical Uncertainty Estimates Research Articles

The Effect of Branchless Collisions and Population Control on Correlations in Monte Carlo Power Iteration

The investigation of correlations in Monte Carlo power iteration has long been dominated by the question of generational correlations and their effects on the estimation of statistical uncertainties. More recently, there has been a growing interest in spatial correlations, prompted by the discovery of neutron clustering. Despite several attempts, a comprehensive framework concerning how Monte Carlo sampling strategies, population control, and variance reduction methods affect the strength of such correlations is still lacking. In this work, we propose a set of global and local (i.e., space-dependent) tallies that can be used to characterize the impact of correlations. These tallies encompass Shannon entropy, pair distance, normalized variance, and Feynman moment. In order to have a clean yet fully meaningful setting, we carry out our analysis in a few homogeneous and heterogeneous benchmark problems of varying dominance ratio. Several classes of collision sampling strategies, population control, and variance reduction techniques are tested, and their relative advantages and drawbacks are assessed with respect to the proposed tallies. The major finding of our study is that branchless collisions, which suppress the emergence of branches in neutron histories, also considerably reduce the effects of correlations in most of the explored configurations.

Integral Turbulent Length and Time Scales of Higher Order Moments

Turbulent length and time scales represent a fundamental quantity for analysing and modelling turbulent flows. Although higher order statistical moments have been conveniently used for decades to describe the mean behaviour of turbulent fluid flow, the definition of the integral turbulent scales seems to be limited to the velocity or its fluctuation itself (i.e. the first moment). Higher order moments are characterized by smaller integral scales and a framework is proposed for estimating autocorrelation functions and integral turbulent length or time scales of higher order moments under the assumption that the probability distribution of the velocity field is Gaussian. The new relations are tested for synthetic turbulence as well as for DNS data of a turbulent plane jet at Reynolds number 10000. The present results in particular suggest that the length or time scales of higher order moments can be markedly smaller than those of the turbulent variable itself, which has implications for statistical uncertainty estimates of higher order moments.

Experimental and Simulated Spectral Gamma-Ray Response of a NaI(Tl) Scintillation Detector used in Airborne Gamma-Ray Spectrometry

Abstract. The objective of this work is to simulate the spectral gamma-ray response of NaI(Tl) scintillation detectors for airborne gamma-ray spectrometry (AGRS) using the state-of-the-art multi-purpose Monte Carlo code FLUKA. The study is based on a commercial airborne gamma-ray spectrometry detector system with four individual NaI(Tl) scintillation crystals and a total volume of 16.8 L. To validate the developed model, radiation measurements were conducted using 57Co, 60Co, 88Y, 109Cd, 133Ba, 137Cs and 152Eu calibration point sources with known activities and source-detector geometries under laboratory conditions. In addition, empirical calibration and resolution functions were derived from these measurements combined with additional radiation measurements adopting natural uranium, thorium and potassium volume sources. The simulation results show superior accuracy and precision compared to previous AGRS simulation models with a median relative spectral error < 10 % for most of the radiation sources. Moreover, the implementation of a lower level discriminator model and detailed modelling of the laboratory result in a significant improvement in model accuracy for spectral energies < 100 keV compared to previous studies. Yet thorough statistical analysis incorporating statistical and systematic uncertainty estimates revealed statistically significant deviations between the simulated and measured spectra in the spectral region around the Compton edge, which could be attributed to the scintillator non-proportionality. These findings imply that the linear energy deposition model applied in this and previously developed AGRS simulation models should be revised and considered to be replaced by more accurate non-proportional models.

Statistical uncertainty estimation of higher-order cumulants with finite efficiency and its application in heavy-ion collisions

We derive the general analytical expressions for the statistical uncertainties of cumulants up to fourth order including an efficiency correction. The analytical expressions have been tested with a toy Monte Carlo model analysis. An application to the study of particle multiplicity fluctuations in heavy-ion collisions is investigated. In this derivation, a mathematical proof is given that the validity of an averaged efficiency correction and the fluctuations induced by the non-uniformity of efficiency can be eliminated. The estimation of statistical uncertainties using the analytical formulas is found to be significantly faster than the commonly used bootstrap method. The simplicity and efficiency of using the analytical formulas may be useful for massive data analysis in many fields.

Four-dimensional mesospheric and lower thermospheric wind fields using Gaussian process regression on multistatic specular meteor radar observations

Abstract. Mesoscale dynamics in the mesosphere and lower thermosphere (MLT) region have been difficult to study from either ground- or satellite-based observations. For understanding of atmospheric coupling processes, important spatial scales at these altitudes range between tens and hundreds of kilometers in the horizontal plane. To date, this scale size is challenging observationally, so structures are usually parameterized in global circulation models. The advent of multistatic specular meteor radar networks allows exploration of MLT mesoscale dynamics on these scales using an increased number of detections and a diversity of viewing angles inherent to multistatic networks. In this work, we introduce a four-dimensional wind field inversion method that makes use of Gaussian process regression (GPR), which is a nonparametric and Bayesian approach. The method takes measured projected wind velocities and prior distributions of the wind velocity as a function of space and time, specified by the user or estimated from the data, and produces posterior distributions for the wind velocity. Computation of the predictive posterior distribution is performed on sampled points of interest and is not necessarily regularly sampled. The main benefits of the GPR method include this non-gridded sampling, the built-in statistical uncertainty estimates, and the ability to horizontally resolve winds on relatively small scales. The performance of the GPR implementation has been evaluated on Monte Carlo simulations with known distributions using the same spatial and temporal sampling as 1 d of real meteor measurements. Based on the simulation results we find that the GPR implementation is robust, providing wind fields that are statistically unbiased with statistical variances that depend on the geometry and are proportional to the prior velocity variances. A conservative and fast approach can be straightforwardly implemented by employing overestimated prior variances and distances, while a more robust but computationally intensive approach can be implemented by employing training and fitting of model hyperparameters. The latter GPR approach has been applied to a 24 h dataset and shown to compare well to previously used homogeneous and gradient methods. Small-scale features have reasonably low statistical uncertainties, implying geophysical wind field horizontal structures as low as 20–50 km. We suggest that this GPR approach forms a suitable method for MLT regional and weather studies.

Validation of HWRF-based Probabilistic TC Wind and Precipitation forecasts

AbstractTropical cyclones are associated with a variety of significant social hazards, including wind, rain, and storm surge. Despite this, most of the model validation effort has been directed toward track and intensity forecasts. In contrast, few studies have investigated the skill of state-of-the-art, high-resolution ensemble prediction systems in predicting associated TC hazards, which is crucial since TC position and intensity do not always correlate with the TC-related hazards, and can result in impacts far from the actual TC center. Furthermore, dynamic models can provide flow-dependent uncertainty estimates, which in turn can provide more specific guidance to forecasters than statistical uncertainty estimates based on past errors. This study validates probabilistic forecasts of wind speed and precipitation hazards derived from the HWRF ensemble prediction system and compares its skill to forecasts by the stochastically-based operational Monte Carlo Model (NHC), the IFS (ECMWF), and the GEFS (NOAA) in use 2017-2019. Wind and Precipitation forecasts are validated against NHC best track wind radii information and the National Stage IV QPE Product. The HWRF 34 kn wind forecasts have comparable skill to the global models up to 60 h lead time before HWRF skill decreases, possibly due to detrimental impacts of large track errors. In contrast, HWRF has comparable quality to its competitors for higher thresholds of 50 kn and 64 kn throughout 120 h lead time. In terms of precipitation hazards, HWRF performs similar or better than global models, but depicts higher, although not perfect, reliability, especially for events over 5 in120h−1. Post-processing, like Quantile Mapping, improves forecast skill for all models significantly and can alleviate reliability issues of the global models.

The development of rainfall retrievals from radar at Darwin

Abstract. The U.S. Department of Energy Atmospheric Radiation Measurement program Tropical Western Pacific site hosted a C-band polarization (CPOL) radar in Darwin, Australia. It provides 2 decades of tropical rainfall characteristics useful for validating global circulation models. Rainfall retrievals from radar assume characteristics about the droplet size distribution (DSD) that vary significantly. To minimize the uncertainty associated with DSD variability, new radar rainfall techniques use dual polarization and specific attenuation estimates. This study challenges the applicability of several specific attenuation and dual-polarization-based rainfall estimators in tropical settings using a 4-year archive of Darwin disdrometer datasets in conjunction with CPOL observations. This assessment is based on three metrics: statistical uncertainty estimates, principal component analysis (PCA), and comparisons of various retrievals from CPOL data. The PCA shows that the variability in R can be consistently attributed to reflectivity, but dependence on dual-polarization quantities was wavelength dependent for 1<R<10mmh-1. These rates primarily originate from stratiform clouds and weak convection (median drop diameters less than 1.5 mm). The dual-polarization specific differential phase and differential reflectivity increase in usefulness for rainfall estimators in times with R>10mmh-1. Rainfall estimates during these conditions primarily originate from deep convective clouds with median drop diameters greater than 1.5 mm. An uncertainty analysis and intercomparison with CPOL show that a Colorado State University blended technique for tropical oceans, with modified estimators developed from video disdrometer observations, is most appropriate for use in all cases, such as when 1<R<10mmh-1 (stratiform rain) and when R>10mmh-1 (deeper convective rain).

The Design of Clustered Observational Studies in Education

Clustered observational studies (COSs) are a critical analytic tool for educational effectiveness research. We present a design framework for the development and critique of COSs. The framework is built on the counterfactual model for causal inference and promotes the concept of designing COSs that emulate the targeted randomized trial that would have been conducted were it feasible. We emphasize the key role of understanding the assignment mechanism to study design. We review methods for statistical adjustment and highlight a recently developed form of matching designed specifically for COSs. We review how regression models can be profitably combined with matching and note best practices for estimates of statistical uncertainty. Finally, we review how sensitivity analyses can determine whether conclusions are sensitive to bias from potential unobserved confounders. We demonstrate concepts with an evaluation of a summer school reading intervention in a large U.S. school district.

Improved slant column density retrieval of nitrogen dioxide and formaldehyde for OMI and GOME-2A from QA4ECV: intercomparison, uncertainty characterisation, and trends

Abstract. Nitrogen dioxide (NO2) and formaldehyde (HCHO) column data from satellite instruments are used for air quality and climate studies. Both NO2 and HCHO have been identified as precursors to the ozone (O3) and aerosol essential climate variables, and it is essential to quantify and characterise their uncertainties. Here we present an intercomparison of NO2 and HCHO slant column density (SCD) retrievals from four different research groups (BIRA-IASB, IUP Bremen, and KNMI as part of the Quality Assurance for Essential Climate Variables (QA4ECV) project consortium, and NASA) and from the OMI and GOME-2A instruments. Our evaluation is motivated by recent improvements in differential optical absorption spectroscopy (DOAS) fitting techniques and by the desire to provide a fully traceable uncertainty budget for the climate data record generated within QA4ECV. The improved NO2 and HCHO SCD values are in close agreement but with substantial differences in the reported uncertainties between groups and instruments. To check the DOAS uncertainties, we use an independent estimate based on the spatial variability of the SCDs within a remote region. For NO2, we find the smallest uncertainties from the new QA4ECV retrieval (0.8 × 1015 molec. cm−2 for both instruments over their mission lifetimes). Relative to earlier approaches, the QA4ECV NO2 retrieval shows better agreement between DOAS and statistical uncertainty estimates, suggesting that the improved QA4ECV NO2 retrieval has reduced but not altogether eliminated systematic errors in the fitting approach. For HCHO, we reach similar conclusions (QA4ECV uncertainties of 8–12 × 1015 molec. cm−2), but the closeness between the DOAS and statistical uncertainty estimates suggests that HCHO uncertainties are indeed dominated by random noise from the satellite's level 1 data. We find that SCD uncertainties are smallest for high top-of-atmosphere reflectance levels with high measurement signal-to-noise ratios. From 2005 to 2015, OMI NO2 SCD uncertainties increase by 1–2 % year−1, which is related to detector degradation and stripes, but OMI HCHO SCD uncertainties are remarkably stable (increase < 1 % year−1) and this is related to the use of Earth radiance reference spectra which reduces stripes. For GOME-2A, NO2 and HCHO SCD uncertainties increased by 7–9 and 11–15 % year−1 respectively up until September 2009, when heating of the instrument markedly reduced further throughput loss, stabilising the degradation of SCD uncertainty to < 3 % year−1 for 2009–2015. Our work suggests that the NO2 SCD uncertainty largely consists of a random component ( ∼ 65 % of the total uncertainty) as a result of the propagation of measurement noise but also of a substantial systematic component ( ∼ 35 % of the total uncertainty) mainly from stripe effects. Averaging over multiple pixels in space and/or time can significantly reduce the SCD uncertainties. This suggests that trend detection in OMI, GOME-2 NO2, and HCHO time series is not limited by the spectral fitting but rather by the adequacy of assumptions on the atmospheric state in the later air mass factor (AMF) calculation step.

Using the MCNP Taylor series perturbation feature (efficiently) for shielding problems

The Taylor series or differential operator perturbation method, implemented in MCNP and invoked using the PERT card, can be used for efficient parameter studies in shielding problems. This paper shows how only two PERT cards are needed to generate an entire parameter study, including statistical uncertainty estimates (an additional three PERT cards can be used to give exact statistical uncertainties). One realistic example problem involves a detailed helium-3 neutron detector model and its efficiency as a function of the density of its high-density polyethylene moderator. The MCNP differential operator perturbation capability is extremely accurate for this problem. A second problem involves the density of the polyethylene reflector of the BeRP ball and is an example of first-order sensitivity analysis using the PERT capability. A third problem is an analytic verification of the PERT capability.

Differential geodetic stereo SAR with TerraSAR-X by exploiting small multi-directional radar reflectors

In this paper, we report on the direct positioning of small multi-directional radar reflectors, so-called octahedrons, with the synthetic aperture radar (SAR) satellite TerraSAR-X. Its highest resolution imaging mode termed staring spotlight enables the use of such octahedron reflectors with a dimension of only half a meter, but still providing backscatter equivalent to 1–2 cm observation error. Four octahedrons were deployed at Wettzell geodetic observatory, and observed by TerraSAR-X with 12 acquisitions in three different geometries. By applying our least squares stereo SAR algorithm already tested with common trihedral corner reflectors (CRs), and introducing a novel differential extension using one octahedron as reference point, the coordinates of the remaining octahedrons were directly retrieved in the International Terrestrial Reference Frame (ITRF). Contrary to our standard processing, the differential approach does not require external corrections for the atmospheric path delays and the geodynamic displacements, rendering it particularly useful for joint geodetic networks employing SAR and GNSS. In this paper, we present and discuss both methods based on results when applying them to the aforementioned Wettzell data set of the octahedrons. The comparison with the independently determined reference coordinates confirms the positioning accuracy with 2–5 cm for the standard approach, and 2–3 cm for the differential processing. Moreover, we present statistical uncertainty estimates of the observations and the positioning solutions, which are additionally provided by our parameter estimation algorithms. The results also include our 1.5 m CR available at Wettzell, and the outcomes clearly demonstrate the advantage of the multi-directional octahedrons over conventional CRs for global positioning applications with SAR.

Toward a Standard Protocol for Micelle Simulation.

In this paper, we present protocols for simulating micelles using dissipative particle dynamics (and in principle molecular dynamics) that we expect to be appropriate for computing micelle properties for a wide range of surfactant molecules. The protocols address challenges in equilibrating and sampling, specifically when kinetics can be very different with changes in surfactant concentration, and with minor changes in molecular size and structure, even using the same force field parameters. We demonstrate that detection of equilibrium can be automated and is robust, for the molecules in this study and others we have considered. In order to quantify the degree of sampling obtained during simulations, metrics to assess the degree of molecular exchange among micellar material are presented, and the use of correlation times are prescribed to assess sampling and for statistical uncertainty estimates on the relevant simulation observables. We show that the computational challenges facing the measurement of the critical micelle concentration (CMC) are somewhat different for high and low CMC materials. While a specific choice is not recommended here, we demonstrate that various methods give values that are consistent in terms of trends, even if not numerically equivalent.

Estimating error in diffusion coefficients derived from molecular dynamics simulations.

The computationally expensive nature of molecular dynamics simulation limits the access to length (nanometer) and time scales (nanosecond) that are orders of magnitude smaller than the experiment it models. This limitation warrants a careful estimation of statistical uncertainty associated with the properties calculated from these simulations. The assumption that a simulation is long enough so that the ergodic hypothesis applies is often invoked in the literature for the computation of properties of interest from a single molecular dynamics simulation. Here, we demonstrate that making this assumption without validation results in poor estimates of the self-diffusion coefficient from a single molecular dynamics simulation of Lennard-Jones fluid. This problem is shown to be even more severe when the diffusion coefficient of macromolecules is calculated from a single molecular dynamics simulation. We have shown that conducting multiple independent simulations is necessary to obtain reliable estimates of diffusion coefficients and their associated statistical uncertainties. We show that even a “routine” calculation of the self-diffusion coefficient for a Lennard-Jones fluid, as determined from a linear fit of the mean squared displacement of particles as a function of time, violates the key assumptions of linear regression. A rigorous approach for addressing these issues is presented.

Assessment of bootstrap resampling performance for PET data

Bootstrap resampling has been successfully used for estimation of statistical uncertainty of parameters such as tissue metabolism, blood flow or displacement fields for image registration. The performance of bootstrap resampling as applied to PET list-mode data of the human brain and dedicated phantoms is assessed in a novel and systematic way such that: (1) the assessment is carried out in two resampling stages: the ‘real world’ stage where multiple reference datasets of varying statistical level are generated and the ‘bootstrap world’ stage where corresponding bootstrap replicates are generated from the reference datasets. (2) All resampled datasets were reconstructed yielding images from which multiple voxel and regions of interest (ROI) values were extracted to form corresponding distributions between the two stages. (3) The difference between the distributions from both stages was quantified using the Jensen–Shannon divergence and the first four moments.It was found that the bootstrap distributions are consistently different to the real world distributions across the statistical levels. The difference was explained by a shift in the mean (up to 33% for voxels and 14% for ROIs) being proportional to the inverse square root of the statistical level (number of counts). Other moments were well replicated by the bootstrap although for very low statistical levels the estimation of the variance was poor. Therefore, the bootstrap method should be used with care when estimating systematic errors (bias) and variance when very low statistical levels are present such as in early time frames of dynamic acquisitions, when the underlying population may not be sufficiently represented.

On the estimation of statistical uncertainties on powder diffraction and small-angle scattering data from two-dimensional X-ray detectors

Optimal methods are explored for obtaining one-dimensional powder pattern intensities from two-dimensional planar detectors with good estimates of their standard deviations. Methods are described to estimate uncertainties when the same image is measured in multiple frames as well as from a single frame. The importance of considering the correlation of diffraction points during the integration and the resampling process of data analysis is shown. It is found that correlations between adjacent pixels in the image can lead to seriously overestimated uncertainties if such correlations are neglected in the integration process. Off-diagonal entries in the variance–covariance (VC) matrix are problematic as virtually all data processing and modeling programs cannot handle the full VC matrix. It is shown that the off-diagonal terms come mainly from the pixel-splitting algorithm used as the default integration algorithm in many popular two-dimensional integration programs, as well as from rebinning and resampling steps later in the processing. When the full VC matrix can be propagated during the data reduction, it is possible to get accurate refined parameters and their uncertainties at the cost of increasing computational complexity. However, as this is not normally possible, the best approximate methods for data processing in order to estimate uncertainties on refined parameters with the greatest accuracy from just the diagonal variance terms in the VC matrix is explored.

Statistical error analysis for phenomenological nucleon-nucleon potentials

Nucleon-Nucleon potentials are commonplace in nuclear physics and are determined from a finite number of experimental data with limited precision sampling the scattering process. We study the statistical assumptions implicit in the standard least squares fitting procedure and apply, along with more conventional tests, a tail sensitive quantile-quantile test as a simple and confident tool to verify the normality of residuals. We show that the fulfilment of normality tests is linked to a judicious and consistent selection of a nucleon-nucleon database. These considerations prove crucial to a proper statistical error analysis and uncertainty propagation. We illustrate these issues by analyzing about 8000 proton-proton and neutron-proton scattering published data. This enables the construction of potentials meeting all statistical requirements necessary for statistical uncertainty estimates in nuclear structure calculations.

Satellite Remote Sensing as a Reconnaissance Tool for Assessing Nautical Chart Adequacy and Completeness

National hydrographic offices need a better means of assessing the adequacy of existing nautical charts in order to plan and prioritize future hydrographic surveys. The ability to derive bathymetry from multispectral satellite imagery is a topic that has received considerable attention in scientific literature. However, published studies have not addressed the ability of satellite-derived bathymetry to meet specific hydrographic survey requirements. Specifically, the bathymetry needs to be referenced to a chart datum and statistical uncertainty estimates of the bathymetry should be provided. Ideally, the procedure should be based on readily-available, low-cost software, tools, and data. This paper describes the development and testing of a procedure using publicly-available, multispectral satellite imagery to map and portray shallow-water bathymetry in a GIS environment for three study sites: Northeast United States, Nigeria, and Belize. Landsat imagery and published algorithms were used to derive estimates of the bathymetry in shallow waters, and uncertainty of the satellite-derived bathymetry was then assessed using a Monte Carlo method. Results indicate that the practical procedures developed in this study are suitable for use by national hydrographic offices.

Evolution of spit morphology: a case study using a remote sensing and statistical based approach

The dynamics and factors responsible for morphological changes of spits viz., Uliyargoli-Padukere, Oddu Bengre and Kodi Bengre, southern Karnataka, India, are investigated using multi-dated satellite images and topographic maps during the last 95-years (1910–2005). Variations and overall rate of changes in length, area and coefficient of determination (R2) of each spit are calculated separately for two periods (1910 and 1967 as base years) to find out whether there is any significant trend in the case of change in length and area in all the three spits that are under study. Linear trend lines are fitted using a least squares method and the statistical significance is considered at 80 % level of confidence. The results recorded significant changes in spit morphology, especially in length and area if 1910 and 1967 are considered separately as base years, are may be due to non-availability of data set between 1910 and 1967 period. The study reveals that coastal processes, such as SW-monsoon influenced strong currents and longshore drifts are the main process for formation and growth of spits, whereas rivers influence/drift also plays significant role. The statistical uncertainty estimation in spits morphology is prevalent wherever the coast is affected by human interventions. The study demonstrates that combined use of satellite imagery and statistical techniques can be effectively used to understand the evolution of spits morphology.

Efficient Moment-Based Inference of Admixture Parameters and Sources of Gene Flow

The recent explosion in available genetic data has led to significant advances in understanding the demographic histories of and relationships among human populations. It is still a challenge, however, to infer reliable parameter values for complicated models involving many populations. Here, we present MixMapper, an efficient, interactive method for constructing phylogenetic trees including admixture events using single nucleotide polymorphism (SNP) genotype data. MixMapper implements a novel two-phase approach to admixture inference using moment statistics, first building an unadmixed scaffold tree and then adding admixed populations by solving systems of equations that express allele frequency divergences in terms of mixture parameters. Importantly, all features of the model, including topology, sources of gene flow, branch lengths, and mixture proportions, are optimized automatically from the data and include estimates of statistical uncertainty. MixMapper also uses a new method to express branch lengths in easily interpretable drift units. We apply MixMapper to recently published data for Human Genome Diversity Cell Line Panel individuals genotyped on a SNP array designed especially for use in population genetics studies, obtaining confident results for 30 populations, 20 of them admixed. Notably, we confirm a signal of ancient admixture in European populations—including previously undetected admixture in Sardinians and Basques—involving a proportion of 20–40% ancient northern Eurasian ancestry.

New Cohort Fertility Forecasts for the Developed World: Rises, Falls, and Reversals

With period fertility having risen in many low‐fertility countries, an important emerging question is whether cohort fertility trends are also reversing. We produce new estimates of cohort fertility for 37 developed countries using a new, simple method that avoids the underestimation typical of previous approaches. Consistent with the idea that timing changes were largely responsible for the last decades' low period fertility, we find that family size has remained considerably higher than the period rates of 1.5 in many “low‐fertility” countries, averaging about 1.8 children. Our forecasts suggest that the long‐term decline in cohort fertility is flattening or reversing in many world regions previously characterized by low fertility. We document the marked increase of cohort fertility in the English‐speaking world and in Scandinavia; signs of an upward reversal in many low‐fertility countries, including Japan and Germany; and continued declines in countries such as Taiwan and Portugal. We include in our forecasts estimates of statistical uncertainty and the possible effects of the recent economic recession.