Rigorous Uncertainty Assessment Research Articles

Thermal influences as an uncertainty contributor of the coordinate measuring machine (CMM)

Estimates of the measurement uncertainty are crucial, without which precision measurements are useless from a practical point of view. Dimensional compliance with geometric product specifications (GPS) plays a key role in decision-making in manufacturing engineering. Without rigorous uncertainty assessment, manufacturers risk making incorrect decisions. Coordinate measuring machine (CMM) measurements are suitable when complex measurement tasks of a workpiece must be made, such as measurements of positional tolerances or repetitive measurements. Here, we attempted to solve problems related to the thermal influences in the assessment of uncertainty in Coordinate Metrology. Task-specific experiments based on the identification of residual errors using a hole plate standard calibrated by Physikalisch-Technische Bundesanstalt (PTB) were performed to compare the measurement uncertainty results in the different temperature ranges. The thermal issues of CMMs include measurements of the temperature of the environment and CMM errors, estimation of the uncertainties of the thermal errors of the CMM, and reduction of the uncertainties related to thermal errors of CMMs. Computation of the thermal errors must account for environmental influences and internal heat sources of the CMM. Assessment of the uncertainty of thermal errors covers almost all factors related to the uncertainty of the CMM. The state-of-the-art technique for determining the uncertainty of the measurements is discussed.

Pathline Density Distributions in a Null-Space Monte Carlo Approach to Assess Groundwater Pathways.

A null-space Monte-Carlo (NSMC) approach was applied to account for uncertainty in the calibration of the hydraulic conductivity (K) field for a three-dimensional groundwater flow model of a major water supply system in Switzerland. The approach generates different parameter realizations of the K field using the pilot point methodology. Subsequently, particle tracking (PT) was applied to each calibrated model, and the resulting particles are interpreted as the spatial pathline density distribution of multiple sources. The adopted approach offers advantages over classical PT which does not provide a means for treating uncertainty originating from the incomplete description of subsurface heterogeneity. Uncertainty in the K field is shown to strongly influence the spatial pathline distribution. Pathline spreading is particularly evident in locations where the information content of the head observations does not sufficiently constrain the estimated parameters. Despite the predictive uncertainty, the pumped drinking water at the study site is most likely dominated by artificially-infiltrated groundwater originating from the local infiltration canals. The model suggests that within the well field, the central pumping wells could be extracting regional groundwater, although the probability is relatively low. Nevertheless, a rigorous uncertainty assessment is still required since only a few realizations resulted in flow paths that support the field observations. Model results should therefore not be based on only one model realization; rather, an uncertainty analysis should be carried out to provide a sufficiently large suite of equally probable simulations that include all potential sources and pathways.

Construction and preliminary testing of a guarded hot plate apparatus for thermal conductivity measurements at high temperatures

We constructed a guarded hot plate (GHP) apparatus that can be used for thermal conductivity measurements from 100 °C to 1000 °C. The apparatus measures the effective thermal conductivity of square-shaped thermal insulation materials of 300 mm per side in a controlled gas environment. As a preliminary test for the high temperature GHP apparatus, we first measured the amount of electrical power required for heaters at various locations to maintain the temperature at up to 900 °C. After this, two specimens of thermal insulation material whose thermal conductivities are ≈0.044 W·m–1 K–1 and ≈0.025 W m–1 K–1 at room temperature were tested at a thermally steady state with unilateral heat flow at temperatures between 100 °C and 800 °C. We applied an offset to the guard plate to actively compensate for the lateral heat flow across the air gap. The measurements showed that the linearity between the temperature difference on the two sides of the specimen and the power dissipated across the specimen was extremely good (with a standard deviation of 0.1%). After some modifications suggested from the preliminary test, we expect that the high temperature GHP will work within the technical specification CEN/TS 15548-1:2014 in the range up to 800 °C, but rigorous uncertainty assessment is yet to be performed after such modifications.

Reanalysis of a 10-year record (2004–2013) of seasonal mass balances at Langenferner/Vedretta Lunga, Ortler Alps, Italy

Abstract. Records of glacier mass balance represent important data in climate science and their uncertainties affect calculations of sea level rise and other societally relevant environmental projections. In order to reduce and quantify uncertainties in mass balance series obtained by direct glaciological measurements, we present a detailed reanalysis workflow which was applied to the 10-year record (2004 to 2013) of seasonal mass balance of Langenferner, a small glacier in the European Eastern Alps. The approach involves a methodological homogenization of available point values and the creation of pseudo-observations of point mass balance for years and locations without measurements by the application of a process-based model constrained by snow line observations. We examine the uncertainties related to the extrapolation of point data using a variety of methods and consequently present a more rigorous uncertainty assessment than is usually reported in the literature. Results reveal that the reanalyzed balance record considerably differs from the original one mainly for the first half of the observation period. For annual balances these misfits reach the order of > 300 kg m−2 and could primarily be attributed to a lack of measurements in the upper glacier part and to the use of outdated glacier outlines. For winter balances respective differences are smaller (up to 233 kg m−2) and they originate primarily from methodological inhomogeneities in the original series. Remaining random uncertainties in the reanalyzed series are mainly determined by the extrapolation of point data to the glacier scale and are on the order of ±79 kg m−2 for annual and ±52 kg m−2 for winter balances with values for single years/seasons reaching ±136 kg m−2. A comparison of the glaciological results to those obtained by the geodetic method for the period 2005 to 2013 based on airborne laser-scanning data reveals that no significant bias of the reanalyzed record is detectable.

Statistical rigor in LiDAR-assisted estimation of aboveground forest biomass

Abstract For many decades remotely sensed data have been used as a source of auxiliary information when conducting regional or national surveys of forest resources. In the past decade, airborne scanning LiDAR (Light Detection and Ranging) has emerged as a promising tool for sample surveys aimed at improving estimation of above-ground forest biomass. This technology is now employed routinely in forest management inventories of some Nordic countries, and there is eager anticipation for its application to assess changes in standing biomass in vast tropical regions of the globe in concert with the UN REDD program to limit C emissions. In the rapidly expanding literature on LiDAR-assisted biomass estimation the assessment of the uncertainty of estimation varies widely, ranging from statistically rigorous to ad hoc. In many instances, too, there appears to be no recognition of different bases of statistical inference which bear importantly on uncertainty estimation. Statistically rigorous assessment of uncertainty for four large LiDAR-assisted surveys is expounded.

Phase behavior of (CO2 + H2) and (CO2 + N2) at temperatures between (218.15 and 303.15) K at pressures up to 15 MPa

Vapor–liquid equilibrium data are reported for the binary systems (CO2+H2) and (CO2+N2) at temperatures between (218.15 and 303.15)K at pressures ranging from the vapor pressure of CO2 to approximately 15MPa. These data were measured in a new analytical apparatus which is described in detail. The results are supported by a rigorous assessment of uncertainties and careful validation measurements. The new data help to resolve discrepancies between previous studies, especially for the (CO2+H2) system. Experimental measurements of the three-phase solid–liquid–vapor locus are also reported for both binary systems.The vapor–liquid equilibrium data are modeled with the Peng–Robinson (PR) equation of state with two binary interaction parameters: one, a linear function of inverse temperature, applied to the unlike term in the PR attractive-energy parameter; and the other, taken to be constant, applied to the unlike term in the PR co-volume parameter. This model is able to fit the experimental data in a satisfactory way except in the critical region. We also report alternative binary parameter sets optimized for improved performance at either temperatures below 243K or temperatures above 273K. A simple predictive model for the three-phase locus is also presented and compared with the experimental data.

Uncertainty of measurements of podded propulsor performance characteristics

One of the fundamental aspects of any physical experiment is the uncertainty or error limits of the measurement. Unfortunately, majority of published model experimental results do not come with the much needed error analysis. It is imperative that a fundamental and rigorous uncertainty assessment is carried out for all measurements, especially for measurements using a newly designed apparatus. This paper presents the uncertainty analysis methodology and results for a newly designed fully functional podded propulsor performance measurement apparatus. The measurements include propeller thrust, torque, rotation rate and advance speed as well as global forces and moments of a pod unit. The facility and measurement systems are briefly described, and detailed uncertainty assessment methodologies with examples for each measurement are provided with descriptions of bias and precision limits and total uncertainties. The generalized methodology can also be used for other relevant measurements.

Estimating uncertainties in statistics computed from direct numerical simulation

Rigorous assessment of uncertainty is crucial to the utility of direct numerical simulation (DNS) results. Uncertainties in the computed statistics arise from two sources: finite statistical sampling and the discretization of the Navier–Stokes equations. Due to the presence of non-trivial sampling error, standard techniques for estimating discretization error (such as Richardson extrapolation) fail or are unreliable. This work provides a systematic and unified approach for estimating these errors. First, a sampling error estimator that accounts for correlation in the input data is developed. Then, this sampling error estimate is used as part of a Bayesian extension of Richardson extrapolation in order to characterize the discretization error. These methods are tested using the Lorenz equations and are shown to perform well. These techniques are then used to investigate the sampling and discretization errors in the DNS of a wall-bounded turbulent flow at Reτ ≈ 180. Both small (Lx/δ × Lz/δ = 4π × 2π) and large (Lx/δ × Lz/δ = 12π × 4π) domain sizes are investigated. For each case, a sequence of meshes was generated by first designing a “nominal” mesh using standard heuristics for wall-bounded simulations. These nominal meshes were then coarsened to generate a sequence of grid resolutions appropriate for the Bayesian Richardson extrapolation method. In addition, the small box case is computationally inexpensive enough to allow simulation on a finer mesh, enabling the results of the extrapolation to be validated in a weak sense. For both cases, it is found that while the sampling uncertainty is large enough to make the order of accuracy difficult to determine, the estimated discretization errors are quite small. This indicates that the commonly used heuristics provide adequate resolution for this class of problems. However, it is also found that, for some quantities, the discretization error is not small relative to sampling error, indicating that the conventional wisdom that sampling error dominates discretization error for this class of simulations needs to be reevaluated.

The Maximum Earthquake Magnitude in a Time Horizon: Theory and Case Studies

We show how the maximum magnitude within a predefined future time horizon may be estimated from an earthquake catalog within the context of Gutenberg–Richter statistics. The aim is to carry out a rigorous uncertainty assessment, and calculate precise confidence intervals based on an imposed level of confidence α . In detail, we present a model for the estimation of the maximum magnitude to occur in a time interval T f in the future, given a complete earthquake catalog for a time period T in the past and, if available, paleoseismic events. For this goal, we solely assume that earthquakes follow a stationary Poisson process in time with unknown productivity Λ and obey the Gutenberg–Richter law in magnitude domain with unknown b ‐value. The random variables Λ and b are estimated by means of Bayes theorem with noninformative prior distributions. Results based on synthetic catalogs and on retrospective calculations of historic catalogs from the highly active area of Japan and the low‐seismicity, but high‐risk region lower Rhine embayment (LRE) in Germany indicate that the estimated magnitudes are close to the true values. Finally, we discuss whether the techniques can be extended to meet the safety requirements for critical facilities such as nuclear power plants. For this aim, the maximum magnitude for all times has to be considered. In agreement with earlier work, we find that this parameter is not a useful quantity from the viewpoint of statistical inference.

QnAs with Adrian E. Raftery

Adrian Raftery has a penchant for probability. Over a three-decade career spanning several disciplines, Raftery, a professor of statistics and sociology at the University of Washington, Seattle, and a recently elected member of the National Academy of Sciences, has used statistical tools to unravel uncertainty in all manner of projections. Named the world’s most-cited researcher in mathematics between 1995 and 2005 by Thomson ISI, Raftery has solved a raft of real-world problems related to weather forecasting, population estimation, and personalized medicine. Modeling studies help researchers make reasonable predictions of future scenarios, but Raftery says knowledge of the level of uncertainty surrounding the underlying numbers lends such models the mantle of meaning. Raftery discusses his diverse projects with PNAS. Adrian E. Raftery. > PNAS:In your 2012 PNAS Inaugural Article you reported a method for projecting countries’ future population sizes. Can you explain how this method differs from existing methods? > Raftery:This is a collaborative effort with the United Nations (UN) Population Division, which routinely makes population projections—numbers of people, fertility, and mortality rates—for all countries. The projections are widely used for research, policymaking, and monitoring international development goals, among others. For such projections, the UN had long been using demographic models, which take a deterministic approach, rely on assumptions, and result in absolute values without any rigorous assessment of uncertainty. We …

Global surface wave inversion with model constraints‡

ABSTRACT Practical applications of surface wave inversion demand reliable inverted shear‐wave profiles and a rigorous assessment of the uncertainty associated to the inverted parameters. As a matter of fact, the surface wave inverse problem is severely affected by solution non‐uniqueness: the degree of non‐uniqueness is closely related to the complexity of the observed dispersion pattern and to the experimental inaccuracies in dispersion measurements. Moreover, inversion pitfalls may be connected to specific problems such as inadequate model parametrization and incorrect identification of the surface wave modes. Consequently, it is essential to tune the inversion problem to the specific dataset under examination to avoid unnecessary computations and possible misinterpretations. In the heuristic inversion algorithm presented in this paper, different types of model constraints can be easily introduced to bias constructively the solution towards realistic estimates of the 1D shear‐wave profile. This approach merges the advantages of global inversion, like the extended exploration of the parameter space and a theoretically rigorous assessment of the uncertainties on the inverted parameters, with the practical approach of Lagrange multipliers, which is often used in deterministic inversion, which helps inversion to converge towards models with desired properties (e.g., ‘smooth’ or ‘minimum norm' models). In addition, two different forward kernels can be alternatively selected for direct‐problem computations: either the conventional modal inversion or, instead, the direct minimization of the secular function, which allows the interpreter to avoid mode identification. A rigorous uncertainty assessment of the model parameters is performed by posterior covariance analysis on the accepted solutions and the modal superposition associated to the inverted models is investigated by full‐waveform modelling. This way, the interpreter has several tools to address the more probable sources of inversion pitfalls within the framework of a rigorous and well‐tested global inversion algorithm. The effectiveness and the versatility of this approach, as well as the impact of the interpreter's choices on the final solution and on its posterior uncertainty, are illustrated using both synthetic and real data. In the latter case, the inverted shear velocity profiles are blind compared with borehole data.

Pitch and Heave Tests and Uncertainty Assessment for a Surface Combatant in Regular Head Waves

Towing-tank experiments of coupled pitch and heave motions are presented for a surface combatant advancing in regular head waves. The data include ballasting parameters, time histories, fast Fourier transform (FFT), Fourier series amplitudes, and pitch and heave transfer functions and phases for a range of speeds, wave steepnesses, and wave frequencies. The geometry is David Taylor Model Basin (DTMB) model 5512, which is a 1/46.6 scale geosim of DTMB model 5415 (DDG-51) with Lpp = 3.048 m. The experiments are performed in a 3.048 × 3.048 × 100 m towing tank equipped with a plunger-type wave maker. The test program is undertaken to provide a validation data set for unsteady Reynolds-averaged Navier-Stokes and other computational fluid dynamics (CFD) codes, including rigorous uncertainty assessment of the experimental results following standard procedures. Results indicate that the regular head waves are linear with second- and third-order magnitudes consistent with third-order Stokes waves. Pitch and heave responses and phases show expected trends for long and short wavelengths and are linear or Ak independent for all test conditions. Maximum response occurs for frequency of encounter equal to pitch and heave natural frequencies and Lpp / λ = 0.75. Under these conditions, an equation is derived that predicts the Froude number for maximum response as a function of ship geometrical coefficients.

Predicting the Frequency of Water Quality Standard Violations Using Bayesian Calibration of Eutrophication Models

The water quality standard setting process usually relies on mathematical models with strong mechanistic basis, as this provides assurance that the model will more realistically project the effects of alternative management schemes. From an operational standpoint, the interpretation of model results should be coupled with rigorous error analysis and explicit consideration of the predictive uncertainty and natural variability. In this study, our main objective is to attain effective model calibration and rigorous uncertainty assessment by integrating environmental mathematical modeling with Bayesian analysis. We use a complex aquatic biogeochemical model that simulates multiple elemental cycles (org. C, N, P, Si, O), multiple functional phytoplankton (diatoms, green algae and cyanobacteria) and zooplankton (copepods and cladocerans) groups. The Bayesian calibration framework is illustrated using three synthetic datasets that represent oligo-, meso- and eutrophic lake conditions. Scientific knowledge, expert judgment, and observational data were used to formulate prior probability distributions and characterize the uncertainty pertaining to a subset of the model parameters, i.e., a vector comprising the 35 most influential parameters based on an earlier sensitivity analysis of the model. Our study also underscores the lack of perfect simulators of natural system dynamics using a statistical formulation that explicitly accounts for the discrepancy between mathematical models and environmental systems. The model reproduces the key epilimnetic temporal patterns and provides realistic estimates of predictive uncertainty for water quality variables of environmental management interest. Our analysis also demonstrates how the Bayesian parameter estimation can be used for assessing the exceedance frequency and confidence of compliance of different water quality criteria. The proposed methodological framework can be very useful in the policy-making process and can facilitate environmental management decisions in the Laurentian Great Lakes region.

Uncertainties and shortcomings of ground surface temperature histories derived from inversion of temperature logs

Analysing borehole temperature data in terms of ground surface history can add useful information to reconstructions of past climates. Therefore, a rigorous assessment of uncertainties and error sources is a necessary prerequisite for the meaningful interpretation of such ground surface temperature histories. This study analyses the most prominent sources of uncertainty. The diffusive nature of the process makes the inversion relatively robust against incomplete knowledge of the thermal diffusivity. Similarly the influence of heat production is small. It turns out that for investigations of the last 1000 to 100000 years the maximum depth of the temperature log is crucial. More than 3000 m are required for an optimal inversion. Reconstructions of the last one or two millennia require only modestly deep logs (>300 m) but suffer severely from noisy data.

SUB-SURFACE UNCERTAINTY IN OIL FIELDS: LEARNINGS FROM EARLY PRODUCTION OF LEGENDRE OIL FIELDS

The Legendre North and South Oil Fields (together referred to as the field) have been producing since May 2001 from high rate horizontal wells and had produced 18 MMBBL by end 2002. This represents about 45% of the proven and probable reserves for the field.Many pre-drill uncertainties remain. The exploration and development wells are located primarily along the crest of the structure, leaving significant gross rock volume uncertainty on the flanks of the field. Qualitative use of amplitudes provides some insight into the Legendre North Field but not the Legendre South Field where the imaging is poor. The development wells were drilled horizontally and did not intersect any fluid contacts.Early field life has brought some surprises, despite a rigorous assessment of uncertainty during the field development planning process. Higher than expected gas-oil ratios suggested a saturated oil with small primary gas caps, rather than the predicted under-saturated oil. Due to the larger than expected gas volumes, the gas reinjection system proved to have inadequate redundancy resulting in constrained production from the field. The pre-drill geological model has required significant changes to reflect the drilling and production results to date. The intra-field shales needed to be areally much smaller than predicted to explain well intersections and production performance. This is consistent with outcrop analogues.Surprises are common when an oil field is first developed and often continue to arise during secondary development phases. Learnings, in the context of subsurface uncertainty, from other oil fields in the greater North West Shelf are compared briefly to highlight the importance of managing uncertainty during field development planning. It is important to have design flexibility to enable facility adjustments to be made easily, early in field life.

Rigorous uncertainty assessment in contaminant transport inverse modelling: a case study of fluoride diffusion through clay liners

Inverse methods used in assessing landfill liner design have not yet taken advantage of current developments in inverse procedures. Here, a method for inverting contaminant transport models is presented including a general error model and procedures for differentially weighted multiple response regression. General error models are employed in cases where the residuals are heteroscedastic and correlated, and lead to valid inference on model parameter and predictive uncertainty. The Shuffled Complex Evolution algorithm is used to optimise model parameters. Model parameter uncertainty is assessed by exploring the posterior probability distribution with the Metropolis algorithm, a Markov chain Monte Carlo sampling method. The inverse method is applied to simultaneously determine the sorption and diffusion parameters from laboratory diffusion cell experiments. In these experiments, fluoride migration through kaolin clays was measured by sampling the source and collector cells over time. To uniquely determine the transport model parameters, it was necessary to simultaneously fit the observed data from two independent diffusion cell experiments with different initial concentrations. The jointly fitted transport model parameters compared well with those fitted to independent batch experiments.