Uncertainty Evaluation Research Articles

Traceability and measurement uncertainty of non-removable field flowmeters using clamp-on ultrasonic flowmeters as reference

This study examines the traceability and measurement uncertainty of in situ hydraulic calibration of flowmeters using clamp-on ultrasonic flowmeters as reference. The procedure compares the equipment readings with the reference ones. Measurement uncertainty evaluation uses Guide to the expression of Uncertainty in Measurement (GUM) formulation, considering the linearity conditions of the mathematical models applied. Experimental values are used to test the procedure and its suitability for actual cases where the expected accuracy needs to be achieved.

Determination of Uncertainty in calibration of External Micrometer

In recent years there has been a growing awareness of measurement uncertainty, mainly for two reasons. First, the ever- increasing number of laboratory accreditations requires the evaluation of measurement uncertainty, especially in the area of calibration. Second, quality system protocols get mature with process and time. This is because manufacturers ensure the reliability of their measurements through correct calibration of their test and measuring equipment. International and national regulations require calibration and testing laboratories to provide result of measurement uncertainty. Calibration of the measuring instrument is the process in which the readings obtained from the instrument are compared with the standard values in the laboratory at several points along the scale of the instruments i.e. micrometer, gauges, calipers etc. In general, a calibration without information about the uncertainty of these estimates is incomplete. Calculating measurement uncertainty is an attempt to reasonably limit the results of measurements as referring to standard operating procedures. Since each measurement only provides a result of the outcome, the main requirement of the uncertainty estimation is to make reader confident that reading is within stated range. Proper calibration involves more than just estimating measurement uncertainty. It also requires that the estimated value is small compared to the stated accuracy of the meter

Evaluation of Uncertainty in Linear Cylindrical Diameter Measurement based on Coordinate Measuring Machines

According to the measurement method of linear cylindrical diameter based on coordinate measuring machine (CMM), establish a black box evaluation model for the uncertainty of CMM linear cylindrical diameter measurement, and analyze the sources of each uncertainty. Combining measurement examples, this paper elaborates on the uncertainty evaluation process of linear cylindrical diameter measurement based on CMM, calculates the extended uncertainty of linear cylindrical diameter measurement based on CMM, and provides a reference for the uncertainty evaluation of linear cylindrical diameter measurement based on CMM.

Uncertainty evaluation in density and viscosity of nanofluids at different temperatures using Gaussian process regression-based Monte-Carlo simulations

Uncertainty in experimental measurements is likely to be incorporated due to various limitations in measurement instruments, experimental procedures, quantity of chemicals, etc. Obtaining the true properties of nanofluids is a difficult task. Hence, the evaluation of the dynamic viscosity and density is strenuous. The present work aims to study the influence of uncertainties in temperature and concentration on density and dynamic viscosity of Graphene nanoparticle/distilled water, multiwall carbon nanotube/distilled water, and alumina/distilled water nanofluids using Gaussian Process Regression (GPR) learning algorithm guided Monte-Carlo simulations. The GPR algorithm is employed to predict the properties of the nanofluids which are then fed to Monte-Carlo simulations to predict the uncertainty. The noise and uncertainties are introduced in the mole fraction and temperature artificially and are validated using the Kolmogorov-Smirnov test and Quantile-Quantile plot. The influence of different noise levels in the data is also carried out.

A probabilistic full waveform inversion of surface waves

AbstractOver the past decades, surface wave methods have been routinely employed to retrieve the physical characteristics of the first tens of meters of the subsurface, particularly the shear wave velocity profiles. Traditional methods rely on the application of the multichannel analysis of surface waves to invert the fundamental and higher modes of Rayleigh waves. However, the limitations affecting this approach, such as the 1D model assumption and the high degree of subjectivity when extracting the dispersion curve, motivate us to apply the elastic full‐waveform inversion, which, despite its higher computational cost, enables leveraging the complete information embedded in the recorded seismograms. Standard approaches solve the full‐waveform inversion using gradient‐based algorithms minimizing an error function, commonly measuring the misfit between observed and predicted waveforms. However, these deterministic approaches lack proper uncertainty quantification and are susceptible to get trapped in some local minima of the error function. An alternative lies in a probabilistic framework, but, in this case, we need to deal with the huge computational effort characterizing the Bayesian approach when applied to non‐linear problems associated with expensive forward modelling and large model spaces. In this work, we present a gradient‐based Markov chain Monte Carlo full‐waveform inversion where we accelerate the sampling of the posterior distribution by compressing data and model spaces through the discrete cosine transform. Additionally, a proposal is defined as a local, Gaussian approximation of the target density, constructed using the local Hessian and gradient information of the log posterior. We first validate our method through a synthetic test where the velocity model features lateral and vertical velocity variations. Then we invert a real dataset from the InterPACIFIC project. The obtained results prove the efficiency of our proposed algorithm, which demonstrates to be robust against cycle‐skipping issues and able to provide reasonable uncertainty evaluations with an affordable computational cost.

Evaluation of Uncertainty in Cylindricity Measurement based on Coordinate Measuring Machine

Evaluation of Uncertainty in Cylindricity Measurement based on Coordinate Measuring Machine

Evaluation of Uncertainty in Linear Plane Angle Measurement based on Coordinate Measuring Machines

Evaluation of Uncertainty in Linear Plane Angle Measurement based on Coordinate Measuring Machines

PyCHAMP: A crop-hydrological-agent modeling platform for groundwater management

The Crop-Hydrological-Agent Modeling Platform (PyCHAMP) is a Python-based open-source package designed for modeling agro-hydrological systems. The modular design, incorporating aquifer, crop field, groundwater well, finance, and behavior components, enables users to simulate and analyze the interactions between human and natural systems, considering both environmental and socio-economic factors. This study demonstrates PyCHAMP's capabilities by simulating the dynamics in the Sheridan 6 Local Enhanced Management Area, a groundwater conservation program in the High Plains Aquifer in Kansas. We highlight how a model, empowered by PyCHAMP, accurately captures human-water dynamics, including groundwater level, water withdrawal, and the fraction of cropland dedicated to each crop. We also show how farmer behavior, and its representation, drives system outcomes more strongly than environmental conditions. The results indicate PyCHAMP's potential as a useful tool for human-water research and sustainable groundwater management, offering prospects for future integration with detailed sub-models and systematic evaluation of model structural uncertainty.

Uncertainty evaluation of dam inflow under the influence of climate change (study area: Latiyan Dam)

Uncertainty evaluation of dam inflow under the influence of climate change (study area: Latiyan Dam)

Wide cone beam CT dosimetry: Analysis of results and uncertainty evaluation

ObjectiveTo investigate the consistency of the recommended methodology, we analyzed the dosimetric results obtained for each setup beam shaping filter and tube voltage and evaluated the uncertainty associated with the full dose measurement process. MethodsA 300-mm PTIC was used to validate the results obtained with the 100-mm PTIC. Considering the other dosimetric parts, a cylindrical 160-mm diameter PMMA phantom and a cylindrical 320-mm diameter PMMA phantom were also used in the experimental protocol. ResultsFor the lowest and highest tube voltages available, the CTDIair,160 values obtained by 1-step dosimetry with the 300-mm PTIC were greater than the respective values obtained by 2- and 3-step dosimetry with the 100-mm PTIC. ConclusionThis study established that careful positioning of the 100-mm PTIC in 2 or 3 steps, as well as proper execution of the other dosimetric parts recommended by the IAEA, represents a validated approach within up to 20% uncertainty for wide cone beam CT dosimetry.

Physics-Informed Neural Networks With Weighted Losses by Uncertainty Evaluation for Accurate and Stable Prediction of Manufacturing Systems.

The state prediction of key components in manufacturing systems tends to be risk-sensitive tasks, where prediction accuracy and stability are the two key indicators. The physics-informed neural networks (PINNs), which integrate the advantages of both data-driven models and physics models, are deemed as an effective approach and research trends for stable prediction; however, the potential advantages of PINN are limited for the situations with inaccurate physics models or noisy data, where the balancing of the weights of the data-driven model and physics model is very important for improving the performance of PINN, and it is also a challenge urgently to be addressed. This article proposed a kind of PINN with weighted losses (PNNN-WLs) by uncertainty evaluation for accurate and stable prediction of manufacturing systems, where a novel weight allocation strategy based on uncertainty evaluation by quantifying the variance of prediction errors is proposed, and an improved PINN framework is established for accurate and stable prediction. The proposed approach is verified with open datasets on tool wear prediction, and experimental results show that the prediction accuracy and stability could be obviously improved over existing methods.

Quantitative evaluation of uncertainty and interpretability in machine learning-based landslide susceptibility mapping through feature selection and explainable AI

Landslide susceptibility mapping (LSM) is essential for determining risk regions and guiding mitigation strategies. Machine learning (ML) techniques have been broadly utilized, but the uncertainty and interpretability of these models have not been well-studied. This study conducted a comparative analysis and uncertainty assessment of five ML algorithms—Random Forest (RF), Light Gradient-Boosting Machine (LGB), Extreme Gradient Boosting (XGB), K-Nearest Neighbor (KNN), and Support Vector Machine (SVM)—for LSM in Inje area, South Korea. We optimized these models using Bayesian optimization, a method that refines model performance through probabilistic model-based tuning of hyperparameters. The performance of these algorithms was evaluated using accuracy, Kappa score, and F1 score, with accuracy in detecting landslide-prone locations ranging from 0.916 to 0.947. Among them, the tree-based models (RF, LGB, XGB) showed competitive performance and outperformed the other models. Prediction uncertainty was quantified using bootstrapping and Monte Carlo simulation methods, with the latter providing a more consistent estimate across models. Further, the interpretability of ML predictions was analyzed through sensitivity analysis and SHAP values. We also expanded our investigation to include both the inclusion and exclusion of predictors, providing insights into each significant variable through a comprehensive sensitivity analysis. This paper provides insights into the predictive uncertainty and interpretability of ML algorithms for LSM, contributing to future research in South Korea and beyond.

Deriving a Continuous Point of Departure for Skin Sensitization Risk Assessment Using a Bayesian Network Model.

Regulations of cosmetic ingredients and products have been the most advanced in embracing new approach methodologies (NAMs). Consequently, the cosmetic industry has assumed a forerunner role in the development and implementation of animal-free next-generation risk assessment (NGRA) that incorporates defined approaches (DAs) to assess the skin sensitization potency of ingredients. A Bayesian network DA predicting four potency categories (SkinSens-BN) was constructed against reference Local Lymph Node Assay data for a total of 297 substances, achieving a predictive performance similar to that of other DAs. With the aim of optimally informing risk assessment with a continuous point of departure (PoD), a weighted sum of the SkinSens-BN probabilities for four potency classes (non-, weak, moderate, and strong/extreme sensitizer) was calculated, using fixed weights based on associated LLNA EC3-values. The approach was promising, e.g., the derived PoDs for substances classified as non-sensitizers did not overlap with any others and 77% of PoDs were similar or more conservative than LLNA EC3. In addition, the predictions were assigned a level of confidence based on the probabilities to inform the evaluation of uncertainty in an NGRA context. In conclusion, the PoD derivation approach can substantially contribute to reliable skin sensitization NGRAs.

Identification and optimization of parameters for accurate quantification of RNA by RT-dPCR.

Reverse transcription-digital PCR (RT-dPCR) is attracting attention as a method that enables SI-traceable RNA quantification without calibration, but its accuracy and bias have not been thoroughly studied. In this study, the accurate quantification of RNA by the RT-dPCR method was investigated using NMIJ CRM 6204-b, an RNA certified reference material whose certified value was assigned by orthogonal chemical measurement methods. Moreover, a two-step RT-dPCR method was adopted to examine in detail the conditions for the RT reaction process, which was expected to be the major uncertainty component in the RT-dPCR measurement. Optimization experiments revealed that the type of reverse transcriptase, the concentration of template RNA, and the type and concentration of primers in the RT reaction affected the value quantifiedby RT-dPCR. Under the optimal conditions, the value quantifiedby RT-dPCR, 76.4ng/μL ± 6.7ng/μL (the quantified value ± expanded uncertainty (k = 2)), was consistent with the certified value, 68.2ng/μL ± 5.8ng/μL, of NMIJ CRM 6204-b RNA 1000-A within the expanded uncertainty. From the results of the uncertainty evaluation, the relative combined uncertainty of the RT-dPCR method was 4.42%, and the major uncertainty components in the RT-dPCR method were the preparation of RT solution (3.68%), the inter-day difference (1.80%), and the RT reaction (1.30%). Together, the results suggested that the contribution of the RT reaction process to the total uncertainty was greater than that of the dPCR process.

Validation and Conformity Testing of Sentinel-3 Green Instantaneous FAPAR and Canopy Chlorophyll Content Products

This article presents validation and conformity testing of the Sentinel-3 Ocean Land Colour Instrument (OLCI) green instantaneous fraction of absorbed photosynthetically active radiation (FAPAR) and OLCI terrestrial chlorophyll index (OTCI) canopy chlorophyll content (CCC) products with fiducial reference measurements (FRM) collected in 2018 and 2021 over two sites (Las Tiesas—Barrax, Spain, and Wytham Woods, UK) in the context of the European Space Agency (ESA) Fiducial Reference Measurement for Vegetation (FRM4Veg) initiative. Following metrological principles, an end-to-end uncertainty evaluation framework developed in the project is used to account for the uncertainty of reference data based on a two-stage validation approach. The process involves quantifying uncertainties at the elementary sampling unit (ESU) level and incorporating these uncertainties in the upscaling procedures using orthogonal distance regression (ODR) between FRM and vegetation indices derived from Sentinel-2 data. Uncertainties in the Sentinel-2 data are also accounted for. FRM-based high spatial resolution reference maps and their uncertainties were aggregated to OLCI’s native spatial resolution using its apparent point spread function (PSF). The Sentinel-3 mission requirements, which give an uncertainty of 5% (goal) and 10% (threshold), were considered for conformity testing. GIFAPAR validation results revealed correlations > 0.95, RMSD ~0.1, and a slight negative bias (~−0.06) for both sites. This bias could be partly explained by the differences in the FAPAR definitions between the satellite product and the FRM-based reference. For the OTCI-based CCC, leave-one-out cross-validation demonstrated correlations > 0.8 and RMSDcv ~0.28 g·m−2. Despite the encouraging validation results, conclusive conformity with the strict mission requirements was low, with most cases providing inconclusive results (driven by large uncertainties in the satellite products as well as by the uncertainties in the upscaling approach). It is recommended that mission requirements for bio-geophysical products are reviewed, at least at the threshold level. It is also suggested that the large uncertainties associated with the two-stage validation approach may be avoided by directly comparing with spatially representative FRM.

Interference with Signaling Track Circuits Caused by Rolling Stock: Uncertainty and Variability on a Test Case

The demonstration of compliance of rolling stock against disturbance limits for railway signaling, and in particular track circuits, is subject to a large deal of variability, caused by the diverse values of the electrical parameters of the railway line and resulting transfer functions, as well as the operating conditions of the rolling stock during tests. Instrumental uncertainty is evaluated with a type B approach and shown to be much less than the experimental variability. Repeated test runs in acceleration, coasting, cruising, and braking conditions are considered, deriving both max-hold (spread) and sample (or experimental) standard deviation curves compared to the respective mean values (type A approach to the evaluation of uncertainty, as defined in of the Guide to the Uncertainty in Measurement. The major source of variability affecting a significant portion of the spectrum is caused by the superposed oscillations of the onboard LC filter, for which different choices of the transformation window duration are discussed. The test runs and the acquired data covered, overall, 1 day of tests along about 300 km of the Italian 3 kV DC railway network.

Assessing the credibility of a drug's effects: identification and judgment of uncertainty by the Dutch Medicines Evaluation Board.

Medicine regulators need to judge whether a drug's favorable effects outweigh its unfavorable effects based on a dossier submitted by an applicant, such as a pharmaceutical company. Because scientific knowledge is inherently uncertain, regulators also need to judge the credibility of these effects by identifying and evaluating uncertainties. We performed an ethnographic study of assessment procedures at the Dutch Medicines Evaluation Board (MEB) and describe how regulators evaluate the credibility of an applicant's claims about the benefits and risks of a drug in practice. Our analysis shows that regulators use an investigative approach, which illustrates the effort required to identify uncertainties. Moreover, we show that regulators' expectations about the presentation, the design, and the results of studies can shape how they perceive a medicine's dossier. We highlight the importance of regulatory experience and expertise in the identification and evaluation of uncertainties. In light of our observations, we provide two recommendations to reduce avoidable uncertainty: less reliance on evidence generated by the applicant; and better communication about, and enforcement of, regulatory frameworks toward drug developers.

Experimental uncertainty evaluation by measuring a micro gear standard using focus variation

Abstract Micro gears play an increasingly important role in various industrial applications, and the minimization of their deviations is challenging for metrology and manufacturing. A promising method is the focus variation technology, which enables areal measurements of micro gears. Practice-related standards are used to determine measurement uncertainties by comparison with calibration values. In this work, the external micro gear standard of the Physikalisch-Technische Bundesanstalt (PTB) is used to evaluate experimental measurement uncertainties of a focus variation coordinate measurement system for the first time. The traceable standard with modules between 0.1 and 1 mm is calibrated using micro tactile coordinate measurements. Optical and tactile measurements are then compared. As a result, small expanded measurement uncertainties of less than 4 µm are achieved.

Improving Accuracy and Reliability of Hearing Tests: Measurement Standards for Audiometric Devices.

Pure-tone audiometry, using an audiometer, is the fundamental hearing test for diagnosing hearing loss. The requirements of the devices and the detailed process for calibrating the related equipment are described in international standards. However, traceable calibration and uncertainty evaluation processes are not widely accepted or applied to the qualification and maintenance of audiometric equipment. Here, we briefly review standard measurement systems for audiometric devices and introduce their calibration procedures. The uncertainty of each calibration process was investigated, and its impact on hearing test results was considered. Our findings show that the traceability of each procedure can be secured, satisfying the uncertainty requirement and being sufficiently smaller than the permissible deviation from the audiometer requirement. To guarantee the objectivity and reliability of hearing tests and maintain low uncertainty, close cooperation and mutual understanding between the metrology field and the medical community are necessary.

Test of conformance or non-conformance with geometrical specifications

Abstract Systematic and random measurement errors are the cause of uncertain conformance and non-conformance statements. A wrong conformance statement occurs if the workpiece is accepted and is a reject part (type II error or false-negative) and a wrong non-conformance statement if the workpiece is rejected and is an in-spec part (type I error or false-positive). In order to avoid type I and type II errors, measurement uncertainty must be taken into account in the conformance and non-conformance testing. In practice, some procedures are used to consider measurement errors or the uncertainty that deviate from the state-of-the-art in research and technology. As these methods have become established over many years, they are still widely used despite better theoretical knowledge. The standard ISO 14253-1:2017 specifies a procedure based on probability and measurement uncertainty that is in accordance to the internationally accepted “Guide to the expression of uncertainty in measurement” and its supplements but is often not used due to the complexity of the evaluation of measurement uncertainty. In this contribution we give an overview and comparison of the different existing methods and provide an suggestion for supplementing the standard ISO 14253-1:2017, as Monte Carlo simulations enable a direct probability-based conformance and non-conformance testing even for complex measurement processes.