Quantification Of Errors Research Articles

Comparison of traditional method and triple collocation analysis for evaluation of multiple gridded precipitation products across Germany

AbstractEvaluating the accuracy of precipitation products is essential for many applications. The traditional method for evaluation is to calculate error metrics of products with gauge measurements that are considered as ground-truth. The multiplicative triple collocation (MTC) method has been demonstrated powerful in error quantification of precipitation products when ground-truth is not known. This study applied MTC to evaluate five precipitation products in Germany: two raw satellite-based (CMORPH and PERSIANN), one reanalysis (ERA-Interim), one soil moisture-based (SM2RAIN-ASCAT), and one gauge-based (REGNIE) products. Evaluation was performed at the 0.5° -daily spatial-temporal scales. MTC involves a log transformation of data, necessitating dealing with zero values in daily precipitation. Effects of 12 different strategies for dealing with zero value on MTC results were investigated. Seven different triplet combinations were tested to evaluate the stability of MTC. Results showed that different strategies for replacing zero values had considerable effects on MTC-derived error metrics particularly for root mean squared error (RMSE). MTC with different triplet combinations generated different error metrics for individual products. MTC-derived correlation coefficient (CC) was more reliable than RMSE. It is more appropriate to use MTC to compare the relative accuracy of different precipitation products. Based on CC with unknown truth, MTC with different triplet combinations produced the same ranking of products as the traditional method. A comparison of results from MTC and the classic TC with additive error model showed the potential limitation of MTC in arid area or dry time periods with large ratio of zero daily precipitation.

Error analysis of higher order Trace Finite Element Methods for the surface Stokes equation

Abstract The paper studies a higher order unfitted finite element method for the Stokes system posed on a surface in ℝ3. The method employs parametric P k -P k−1 finite element pairs on tetrahedral bulk mesh to discretize the Stokes system on embedded surface. Stability and optimal order convergence results are proved. The proofs include a complete quantification of geometric errors stemming from approximate parametric representation of the surface. Numerical experiments include formal convergence studies and an example of the Kelvin–Helmholtz instability problem on the unit sphere.

LiDAR and SCADA data processing for interacting wind turbine wakes with comparison to analytical wake models

This study is a follow up on a previous one carried out within the frame of the French project SMARTEOLE, during which, a ground-based scanning LiDAR measurement campaign was conducted in the onshore wind farm of Sole du Moulin Vieux. That previous study focused on the wakes of two wind turbines that experienced different degrees of interaction depending on the incoming wind direction, through the processing of LiDAR measurements. The measurement duration (7 months) ensured the statistical convergence of the ensemble-averaged flow fields obtained after holding a categorisation process based on the wind speed at hub height, wind direction, and atmospheric stability, where only near-neutral stability conditions were considered. The present study focuses on integrating the operational data of the wind turbines through SCADA processing to complement the LiDAR wake field observations and to be used as an input for analytical wake models. First, the correlation between the atmospheric stability, deduced from MERRA-2 dataset, and the free-stream turbulence intensity, measured by the wind turbines’ anemometers, is studied for different wind speed ranges. It is observed that the turbulence intensity tends towards a consistent value as the atmospheric stability approaches near-neutral stability conditions, giving confidence into the applied strategy of data categorisation based on MERRA-2 outputs. The influence of the degree of wake interaction on the wake added turbulence, the velocity and power deficits between both turbines is assessed. Clear trends between the wake added turbulence and both the velocity and power deficits are detected. Consequently, two fitting laws are proposed. Then, different analytical wake models and wake superposition methods are fed with the operational data deduced from the processed SCADA data, and are used for predicting the evolution of the velocity deficit within the wake. Some statistical metrics are used for error quantification of the different engineering wake models compared to the scanning LiDAR measurements, used as reference, and Blondel and Cathelain produces the closest results to the field measurements.

Architecture-Driven Digital Image Correlation Technique (ADDICT) for the measurement of sub-cellular kinematic fields in speckle-free cellular materials

Measuring displacement and strain fields at low observable scales of complex microstructures still remains a challenge in experimental mechanics often because of the combination of low definition images with poor texture at this scale. This is the case for cellular materials, for which complex local phenomena can occur. The aim of this paper is to design and validate numerically and experimentally a Digital Image Correlation (DIC) technique for the measurement of local displacement fields of samples with complex cellular geometries (i.e. samples presenting multiple random holes). It consists of a DIC method assisted with a physically sound weak regularization using an elastic B-spline image-based model. This technique introduces a separation of scales above which DIC is dominant and below which it is assisted with image-based modeling. Several in silico experimentations are performed in order to finely analyze the influence of the introduced regularization lengths for different input mechanical behaviors (elastic, elasto-plastic and geometrically non-linear) and in comparison with true error quantification. We show that the method can estimate complex local displacement and strain fields with speckle-free low definition images, even in non-linear regimes such as local buckling or plasticity. Finally, an experimental validation is performed in 2D-DIC to allow for the comparison of the proposed method on low resolution speckle-free images with a classic DIC on speckled high resolution images.

Recent Progresses in Verification, Validation and Uncertainly Analysis in Computational Fluid Dynamic

Verification and validation (V&V) are indispensable tools to evaluate the accuracy and reliability in computational simulations. This paper highlights guidelines, related works, as well as recent progresses for evaluating the credibility of model and simulation in computational fluid dynamics (CFD). Herein, detailed definitions of V&V that aims to help researchers to distinguish between the term verification and the term validation, define the conceptual sources of error and uncertainty as well as understanding the difference between them. Then, we discuss related works to V&V for CFD simulations, the most useful methodologies and algorithms have been introduced, problems have been faced as well as solutions has been proposed. Finally, an outlook and perspectives for future challenges in V&V for CFD Simulation have been proposed. For verification assessment, in order to assure the accuracy, the computational solution is compared with an analytical solution or a highly accurate solution. The main strategy of validation is to reflect the accuracy between the computational results and the experimental data, as well as the quantification and estimation of both error and uncertainty. It is hoped from this review to help in research, development as well as the use of CFD simulations by establishing robust methodologies and terminologies for V&V.

Virtual error quantification of cross-correlation algorithm for solids velocity measurement in different gas fluidization regimes

Cross-correlation algorithm has been widely used in experimental measurement of solids velocity. However, its reliability and validation range remain unquantified. To this end, a CFD-based “virtual error quantification” method was proposed to quantify the reliability of cross-correlation algorithm in the measurement of solids velocity in two different gas fluidization regimes, i.e., bubbling and fast fluidization regimes (BFB and FFB). It was found that the cross-correlation algorithm works quite well in the core region of FFB, but its applications in the annulus region of FFB and in BFB should be considered with great caution. Finally, the findings were delineated by solids convection-mixing mechanism based on the relative standard deviation (RSD) of solids velocity signals and solids Péclet number. The present study highlights the deficiency of the cross-correlation based solids velocity measurement in fluidization systems and addresses the need to develop better measurement methods for solids mixing dominated systems.

Representation of Dropsonde‐Observed Atmospheric River Conditions in Reanalyses

AbstractAtmospheric rivers (ARs) are the primary mechanism for mid‐latitude water vapor transport, and are identified by a key variable, integrated water vapor transport (IVT). The ability of atmospheric reanalyses in providing a ground‐truth dataset for the IVT field is assessed by comparing ERA5, MERRA‐2, and JRA‐55 data against a large sample (>1,700) of dropsonde profiles deployed in and around ARs. Bias and error increase with IVT magnitude, although asymmetrically around the AR core. A partitioning of the source of error reveals that humidity contributes more to the difference in IVT above 800 hPa, while wind is the dominant source in the lowest levels (to 950 hPa). This quantification of reanalysis error and bias identifies ERA5 as the dataset with the lowest IVT errors and demonstrates remaining challenges in representing the observed state in ARs.

Quantification of the aerosol-induced errors in solar irradiance modeling

Quantification of the aerosol-induced errors in solar irradiance modeling

Evaluating Precipitation Errors Using the Environmentally Conditioned Intensity‐Frequency Decomposition Method

AbstractA fundamental issue when evaluating the simulation of precipitation is the difficulty of quantifying specific sources of errors and recognizing compensation of errors. We assess how well a large ensemble of high‐resolution simulations represents the precipitation associated with strong cyclones. We propose a framework to breakdown precipitation errors according to different dynamical (vertical velocity) and thermodynamical (vertically integrated water vapor) regimes and the frequency and intensity of precipitation. This approach approximates the error in the total precipitation of each regime as the sum of three terms describing errors in the large‐scale environmental conditions, the frequency of precipitation and its intensity. We show that simulations produce precipitation too often, that its intensity is too weak, that errors are larger for weak than for strong dynamical forcing and that biases in the vertically integrated water vapor can be large. Using the error breakdown presented above, we define four new error metrics differing on the degree to which they include the compensation of errors. We show that convection‐permitting simulations consistently improve the simulation of precipitation compared to coarser‐resolution simulations using parameterized convection, and that these improvements are revealed by our new approach but not by traditional metrics which can be affected by compensating errors. These results suggest that convection‐permitting models are more likely to produce better results for the right reasons. We conclude that the novel decomposition and error metrics presented in this study give a useful framework that provides physical insights about the sources of errors and a reliable quantification of errors.

ENVIRONMENTAL INFLUENCES ON THE STABILITY OF A PERMANENTLY INSTALLED LASER SCANNER

Abstract. The advancement of permanently measuring laser scanners has opened up a wide range of new applications, but also led to the need for more advanced approaches on error quantification and correction. Time-dependent and systematic error influences may only become visible in data of quasi-permanent measurements. During a scan experiment in February/March 2020 point clouds were acquired every thirty minutes with a Riegl VZ-2000 laser scanner, and various other sensors (inclination sensors, weather station and GNSS sensors) were used to survey the environment of the laser scanner and the study site. Using this measurement configuration, our aim is to identify apparent displacements in multi-temporal scans due to systematic error influences and to investigate data quality for assessment of geomorphic changes in coastal regions. We analyse scan data collected around two storm events around 09/02/2020 (Ciara) and around 22/02/2020 (Yulia) and derive the impact of heavy storms on the point cloud data through comparison with the collected auxiliary data. To investigate the systematic residuals on data acquired by permanent laser scanning, we extracted several stable flat surfaces from the point cloud data. From a plane fitted through the respective surfaces of each scan, we estimated the mean displacement of each plane with the respective root mean square errors. Inclination sensors, internal and external, recorded pitch and roll values during each scan. We derived a mean inclination per scan (in pitch and roll) and the standard deviation from the mean as a measure of the stability of the laser scanner during each scan. Evaluation of the data recorded by a weather station together with knowledge of the movement behaviour, allows to derive possible causes of displacements and/or noise and correction models. The results are compared to independent measurements from GNSS sensors for validation. For wind speeds of 10 m/s and higher, movements of the scanner considerably increase the noise level in the point cloud data.

Quantification and mitigation of PIV bias errors caused by intermittent particle seeding and particle lag by means of large eddy simulations

In the present work, a standard large eddy simulation is combined with tracer particle seeding simulations to investigate the different PIV bias errors introduced by intermittent particle seeding and particle lag. The intermittency effect is caused by evaluating the velocity from tracer particles with inertia in a region where streams mix with different seeding densities. This effect, which is different from the vastly-discussed particle lag, is frequently observed in the literature but scarcely addressed. Here, bias errors in the velocity are analysed in the framework of a turbulent annular gaseous jet weakly confined by low-momentum co-flowing streams. The errors are computed between the gaseous flow velocity, obtained directly from the simulation, and the velocities estimated from synthetic PIV evaluations. Tracer particles with diameters of 0.037, 0.37 and 3.7 µm are introduced into the simulated flow through the jet only, intermediate co-flowing stream only and through both regions. Results quantify the influence of intermittency in the time-averaged velocities and Reynolds stresses when only one of the streams is seeded, even when tracers fulfil the Stokes-number criterion. Additionally, the present work proposes assessing unbiased velocity statistics from large eddy simulations, after validation of biased seeded simulations with biased PIV measurements. The approach can potentially be applied to a variety of flows and geometries, mitigating the bias errors.

Quantification method for extrapolation errors of constitutive models and a demonstration on C/SiC composite

Phenomenological constitutive models are always used in structural analysis due to its simplicity in modeling and implementation. However, these models are always fitted from limited experimental curves, leading to errors and insufficient confidence in analysis results. Previous constitutive model error quantification methods rely on a-posteriori experimental observations. In this paper, a priori error quantification method for extrapolation errors of constitutive models is established. Gaussian Process models are fitted with stress states as inputs and strains as outputs. Prediction variations of the GP model are used as the error indicator. The method naturally leads to zero error at known load cases and larger error at load cases away from existing experimental data. It can be integrated with FEM analysis easily and provide error visualization capabilities along with stress fields. Stress states at large error areas provide directions for further experiments and constitutive refinement. The method is demonstrated on the constitutive model and FEM analysis of a C/SiC frame and showed good validity.

Inter-methodological quantification of the target change for performance test outcomes relevant to elite female soccer players

ABSTRACT Introduction Valid and informed interpretations of changes in physical performance test data are important within athletic development programmes. At present, there is a lack of consensus regarding a suitable method for deeming whether a change in physical performance is practically relevant or not. Methods We compared true population variance in mean test scores between those derived from evidence synthesis of observational studies to those derived from practioner opinion (n = 30), and to those derived from a measurement error (minimal detectable change) quantification (n = 140). All these methods can help to obtain ‘target’ change score values for performance variables. Results We found that the conventional ‘blanket’ target change of 0.2 (between-subjects SD) systematically underestimated practically relevant and more informed changes derived for 5-m sprinting, 30-m sprinting, CMJ, and Yo-Yo Intermittent Recovery Level 1 (IR1) tests in elite female soccer players. Conclusions For the first time in the field of sport and exercise sciences, we have illustrated the use of a principled approach for comparing different methods for the definition of changes in physical performance test variables that are practically relevant. Our between-method comparison approach provides preliminary guidance for arriving at target change values that may be useful for research purposes and tracking of individual female soccer player’s physical performance.

Integrating multi-fidelity blood flow data with reduced-order data assimilation

High-fidelity patient-specific modeling of cardiovascular flows and hemodynamics is challenging. Direct blood flow measurement inside the body with in-vivo measurement modalities such as 4D flow magnetic resonance imaging (4D flow MRI) suffer from low resolution and acquisition noise. In-vitro experimental modeling and patient-specific computational fluid dynamics (CFD) models are subject to uncertainty in patient-specific boundary conditions and model parameters. Furthermore, collecting blood flow data in the near-wall region (e.g., wall shear stress) with experimental measurement modalities poses additional challenges. In this study, a computationally efficient data assimilation method called reduced-order modeling Kalman filter (ROM-KF) was proposed, which combined a sequential Kalman filter with reduced-order modeling using a linear model provided by dynamic mode decomposition (DMD). The goal of ROM-KF was to overcome low resolution and noise in experimental and uncertainty in CFD modeling of cardiovascular flows. The accuracy of the method was assessed with 1D Womersley flow, 2D idealized aneurysm, and 3D patient-specific cerebral aneurysm models. Synthetic experimental data were used to enable direct quantification of errors using benchmark datasets. The accuracy of ROM-KF in reconstructing near-wall hemodynamics was assessed by applying the method to problems where near-wall blood flow data were missing in the experimental dataset. The ROM-KF method provided blood flow data that were more accurate than the computational and synthetic experimental datasets and improved near-wall hemodynamics quantification.

A probabilistic finite element method based on random meshes: A posteriori error estimators and Bayesian inverse problems

We present a novel probabilistic finite element method (FEM) for the solution and uncertainty quantification of elliptic partial differential equations based on random meshes, which we call random mesh FEM (RM-FEM). Our methodology allows to introduce a probability measure on classical FEMs to quantify the uncertainty due to numerical errors either in the context of a-posteriori error quantification or for FE based Bayesian inverse problems. The new approach involves only a perturbation of the mesh and an interpolation that are very simple to implement We present a posteriori error estimators and a rigorous a posteriori error analysis based uniquely on probabilistic information for standard piecewise linear FEM. A series of numerical experiments illustrates the potential of the RM-FEM for error estimation and validates our analysis. We furthermore demonstrate how employing the RM-FEM enhances the quality of the solution of Bayesian inverse problems, thus allowing a better quantification of numerical errors in pipelines of computations.

Improved high‐resolution global and regionalized isoscapes of δ18O, δ2H and d‐excess in precipitation

AbstractPatterns of δ18O and δ2H in Earth's precipitation provide essential scientific data for use in hydrological, climatological, ecological and forensic research. Insufficient global spatial data coverage promulgated the use of gridded datasets employing geostatistical techniques (isoscapes) for spatiotemporally coherent isotope predictions. Cluster‐based isoscape regionalization combines the advantages of local or regional prediction calibrations into a global framework. Here we present a revision of a Regionalized Cluster‐Based Water Isotope Prediction model (RCWIP2) incorporating new isotope data having extensive spatial coverage and a wider array of predictor variables combined with high‐resolution gridded climatic data. We introduced coupling of δ18O and δ2H (e.g., d‐excess constrained) in the model predictions to prevent runaway isoscapes when each isotope is modelled separately and cross‐checked observed versus modelled d‐excess values. We improved model error quantification by adopting full uncertainty propagation in all calculations. RCWIP2 improved the RMSE over previous isoscape models by ca. 0.3 ‰ for δ18O and 2.5 ‰ for δ2H with an uncertainty <1.0 ‰ for δ18O and < 8 ‰ for δ2H for most regions of the world. The determination of the relative importance of each predictor variable in each ecoclimatic zone is a new approach to identify previously unrecognized climatic drivers on mean annual precipitation δ18O and δ2H. The improved RCWIP2 isoscape grids and maps (season, monthly, annual, regional) are available for download at https://isotopehydrologynetwork.iaea.org.

Recording fine-scale movement of ground beetles by two methods: Potentials and methodological pitfalls.

Movement trajectories are usually recorded as a sequence of discrete movement events described by two parameters: step length (distance) and turning angle (bearing). One of the most widespread methods to record the geocoordinates of each step is by a GPS device. Such devices have limited suitability for recording fine movements of species with low dispersal ability including flightless carabid beetles at small spatio‐temporal scales. As an alternative, the distance‐bearing approach can avoid the measurement error of GPS units since it uses directly measured distances and compass azimuths. As no quantification of measurement error between distance‐bearing and GPS approaches exists so far, we generated artificial fine‐scale trajectories and in addition radio‐tracked living carabids in a temperate forest and recorded each movement step by both methods. Trajectories obtained from distance‐bearing were compared to those obtained by a GPS device in terms of movement parameters. Consequently, both types of trajectories were segmented by state‐switching modeling into two distinct movement stages typical for carabids: random walk and directed movement. We found that the measurement error of GPS compared to distance‐bearing was 1.878 m (SEM = 0.181 m) for distances and 31.330° (SEM = 2.066°) for bearings. Moreover, these errors increased under dense forest canopy and rainy weather. Distance error did not change with increasing distance recorded by distance‐bearing but bearings were significantly more sensitive to error at short distances. State‐switching models showed only slight, not significant, differences in movement states between the two methods in favor of the random walk in the distance‐bearing approach. However, the shape of the GPS‐measured trajectories considerably differed from those recorded by distance‐bearing caused especially by bearing error at short distances. Our study showed that distance‐bearing could be more appropriate for recording movement steps not only of ground‐dwelling beetles but also other small animals at fine spatio‐temporal scales.

Verification of an agent-based disease model of human Mycobacterium tuberculosis infection.

Agent‐based models (ABMs) are a powerful class of computational models widely used to simulate complex phenomena in many different application areas. However, one of the most critical aspects, poorly investigated in the literature, regards an important step of the model credibility assessment: solution verification. This study overcomes this limitation by proposing a general verification framework for ABMs that aims at evaluating the numerical errors associated with the model. A step‐by‐step procedure, which consists of two main verification studies (deterministic and stochastic model verification), is described in detail and applied to a specific mission critical scenario: the quantification of the numerical approximation error for UISS‐TB, an ABM of the human immune system developed to predict the progression of pulmonary tuberculosis. Results provide indications on the possibility to use the proposed model verification workflow to systematically identify and quantify numerical approximation errors associated with UISS‐TB and, in general, with any other ABMs.

Which particles to select, and if yes, how many?

Micro- and nanoplastic contamination is becoming a growing concern for environmental protection and food safety. Therefore, analytical techniques need to produce reliable quantification to ensure proper risk assessment. Raman microspectroscopy (RM) offers identification of single particles, but to ensure that the results are reliable, a certain number of particles has to be analyzed. For larger MP, all particles on the Raman filter can be detected, errors can be quantified, and the minimal sample size can be calculated easily by random sampling. In contrast, very small particles might not all be detected, demanding a window-based analysis of the filter. A bootstrap method is presented to provide an error quantification with confidence intervals from the available window data. In this context, different window selection schemes are evaluated and there is a clear recommendation to employ random (rather than systematically placed) window locations with many small rather than few larger windows. Ultimately, these results are united in a proposed RM measurement algorithm that computes confidence intervals on-the-fly during the analysis and, by checking whether given precision requirements are already met, automatically stops if an appropriate number of particles are identified, thus improving efficiency.To provide quality control in the MP quantification by Raman microspectroscopy, a window subsampling and bootstrap protocol is presented, which can provide confidence intervals that enable the assessment of the reliability of the data. This is brought together with a proposed on-the-fly algorithm that assesses the precision during the measurement and stops at the optimal point

A Code Verification for the Cavity SGEMP Simulation Code LASER-SGEMP

The stochastic Richardson extrapolation-based numerical error quantification (StREEQ) technique has been applied to accomplish a code verification for the cavity systemgenerated electromagnetic pulse (SGEMP) simulation code LASER-SGEMP. Here, the grid convergence index method estimates the discretization error caused by the finite difference, while the bootstrap sampling demonstrates the sampling error arising from the inherent stochastic noise. The electric field waveforms obtained by the LASER-SGEMP code agree well with the high precision solution from a reference. According to the StREEQ method, the convergence value of the electric field at 1.8 ns is calculated to be 0.955 V/m and the numerical error characterized by the 95% confidence interval is [0.948, 0.964] V/m, while the high precision solution is 0.961 V/m. Therefore, the code verification has supported the accuracy of the LASER-SGEMP code, and the StREEQ technique is proved to be capable of estimating the numerical error of the cavity SGEMP simulation.