Detection Of Systematic Errors Research Articles

Selective editing for asymmetry analysis in intra-EU trade Micro-Data Exchange (MDE)

Since January 2022, the Regulation on European Business Statistics (EU 2019/2152) requires EU Member States to compulsorily share microdata on intra-EU exports. Establishing intra-EU export Micro-Data Exchange (MDE) provides National Statistical Institutes with a new data source to compile intra-EU import statistics. The availability of MDE tackles two key challenges: diminishing the overall response burden on data providers and meeting user expectations regarding the quality of the produced statistics. However, transitioning to a data production system based on MDE data requires the assessment of the coherence and comparability between MDE and National import data. To identify asymmetries between the two data sources, Istat developed an innovative application designed to foster cooperation among Member States. The tool was developed using the Shiny package in R. The implemented solution allows users to perform exploratory analysis, systematic error detection, and selective editing. The most relevant asymmetries are identified through relative contribution and the asymmetry suspicion indices assessed by user-defined thresholds. Sharing the open tool within the European Statistical System enhances interoperability, promotes method harmonization, and encourages the adoption of official statistical standards.

Machine Learning for Patient-Based Real-Time Quality Control (PBRTQC), Analytical and Preanalytical Error Detection in Clinical Laboratory.

The rapidly evolving field of machine learning (ML), along with artificial intelligence in a broad sense, is revolutionising many areas of healthcare, including laboratory medicine. The amalgamation of the fields of ML and patient-based real-time quality control (PBRTQC) processes could improve the traditional PBRTQC and error detection algorithms in the laboratory. This narrative review discusses published studies on using ML for the detection of systematic errors, non-systematic errors, and combinations of different types of errors in clinical laboratories. The studies discussed used ML for detecting bias, the requirement for re-calibration, samples contaminated with intravenous fluid or EDTA, delayed sample analysis, wrong-blood-in-tube errors, interference or a combination of different types of errors, by comparing the performance of ML models with human validators or traditional PBRTQC algorithms. Advantages, limitations, the creation of standardised ML models, ethical and regulatory aspects and potential future developments have also been discussed in brief.

Diffusion tensor imaging metrics as natural markers of multiple sclerosis-induced brain disorders with a low Expanded Disability Status Scale score

Non-invasive and effective differentiation along with determining the degree of deviations compared to the healthy cohort is important in the case of various brain disorders, including multiple sclerosis (MS). Evaluation of the effectiveness of diffusion tensor metrics (DTM) in 3T DTI for recording MS-related deviations was performed using a time-acceptable MRI protocol with unique comprehensive detection of systematic errors related to spatial heterogeneity of magnetic field gradients. In a clinical study, DTMs were acquired in segmented regions of interest (ROIs) for 50 randomly selected healthy controls (HC) and 50 multiple sclerosis patients. Identical phantom imaging was performed for each clinical measurement to estimate and remove the influence of systematic errors using the b-matrix spatial distribution in the DTI (BSD-DTI) technique. In the absence of statistically significant differences due to age in healthy volunteers and patients with multiple sclerosis, the existence of significant differences between groups was proven using DTM. Moreover, a statistically significant impact of spatial systematic errors occurs for all ROIs and DTMs in the phantom and for approximately 90 % in the HC and MS groups. In the case of a single patient measurement, this appears for all the examined ROIs and DTMs. The obtained DTMs effectively discriminate healthy volunteers from multiple sclerosis patients with a low mean score on the Expanded Disability Status Scale. The magnitude of the group differences is typically significant, with an effect size of approximately 0.5, and similar in both the standard approach and after elimination of systematic errors. Differences were also observed between metrics obtained using these two approaches. Despite a small alterations in mean DTMs values for groups and ROIs (1–3 %), these differences were characterized by a huge effect (effect size ∼0.8 or more). These findings indicate the importance of determining the spatial distribution of systematic errors specific to each MR scanner and DTI acquisition protocol in order to assess their impact on DTM in the ROIs examined. This is crucial to establish accurate DTM values for both individual patients and mean values for a healthy population as a reference. This approach allows for an initial reliable diagnosis based on DTI metrics.

Exploring the performance and explainability of fine-tuned BERT models for neuroradiology protocol assignment

BackgroundDeep learning has demonstrated significant advancements across various domains. However, its implementation in specialized areas, such as medical settings, remains approached with caution. In these high-stake environments, understanding the model's decision-making process is critical. This study assesses the performance of different pretrained Bidirectional Encoder Representations from Transformers (BERT) models and delves into understanding its decision-making within the context of medical image protocol assignment.MethodsFour different pre-trained BERT models (BERT, BioBERT, ClinicalBERT, RoBERTa) were fine-tuned for the medical image protocol classification task. Word importance was measured by attributing the classification output to every word using a gradient-based method. Subsequently, a trained radiologist reviewed the resulting word importance scores to assess the model’s decision-making process relative to human reasoning.ResultsThe BERT model came close to human performance on our test set. The BERT model successfully identified relevant words indicative of the target protocol. Analysis of important words in misclassifications revealed potential systematic errors in the model.ConclusionsThe BERT model shows promise in medical image protocol assignment by reaching near human level performance and identifying key words effectively. The detection of systematic errors paves the way for further refinements to enhance its safety and utility in clinical settings.

Assay error detection when using common quality control targets across multiple instruments: An analysis using simulated and real-world data.

Clinical laboratories frequently implement the same tests and internal quality control (QC) rules on identical instruments. It is unclear whether individual QC targets for each analyser or ones that are common to all instruments are preferable. This study modelled how common QC targets influence assay error detection before examining their effect on real-world data. The effect of variable bias and imprecision on error detection and false rejection rates when using common or individual QC targets on two instruments was simulated. QC data from tests run on two identical Beckman instruments (6-month period, same QC lot, n > 100 points for each instrument) determined likely real-world consequences. Compared to individual QC targets, common targets had an asymmetrical effect on systematic error detection, with one instrument assay losing detection power more than the other gained. If individual in-control assay standard deviations (SDs) differed, then common targets led to one assay failing QC more frequently. Applied to two analysers (95 QC levels and 45 tests), common targets reduced one instrument's error detection by ≥ 0.4 sigma on 15/45 (33%) of tests. Such targets also meant 14/45 (31%) of assays on one in-control instrument would fail over twice as frequently as the other (median ratio 1.62, IQR 1.20-2.39) using a 2SD rule. Compared to instrument-specific QC targets, common targets can reduce the probability of detecting changes in individual assay performance and cause one in-control assay to fail QC more frequently than another. Any impact on clinical care requires further investigation.

Application of systematic error detection method to integral parameter measurements in sophisticated production processes and operations

This paper uses the Lagrange multiplier method to substantiate the general solution for validity control problem in terms of independently measured parameters of material flows. The paper shows that the efficiency of the above method can be raised if certain process parameters are measured in a number of independent ways by a dispatcher control system. A technique is described to adjust the initial measurement signals in case some channels have redundant information. Such technique is applicable if the values to be adjusted are identified over a period of time that greatly exceeds the dynamic memory of the object or if the dynamic links between certain channels are completely compensated and the execution of static equations of the studied processes is a prerequisite for the values to be adjusted. A method for systematic error detection in application to aggregate integral uncorrelated measurements was used as part of the NavoiAzot dispatcher control system for adjustment of per-shift and per-day performance indicators as calculated by the system. The paper considers the problem of selecting and retrieving valid data on the basis of redundant measurements of material flows and flow constraint equations. The paper describes a solution for the problem of selecting and retrieving valid data on the basis of redundant measurements of material flows and flow constraint equations in application to a complex fertilizer production monitoring system.

Optimization and Validation of Limit Check Error-Detection Performance Using a Laboratory-Specific Data-Simulation Approach: A Prerequisite for an Evidence-Based Practice.

Autoverification procedures based on limit checks (LCs) provide important support to preanalytical, analytical, and postanalytical quality assurance in medical laboratories. A recently described method, based on laboratory-specific error-detection performances, was used to determine LCs for all chemistry analytes performed on random-access chemistry analyzers prior to application. Using data sets of historical test results, error-detection simulations of limit checks were performed using the online MA Generator system (www.huvaros.com). Errors were introduced at various positions in the data set, and the number of tests required for an LC alarm to occur was plotted in bias detection curves. Random error detection was defined as an LC alarm occurring in 1 test result, whereas systematic error detection was defined as an LC alarm occurring within an analytical run, both with ≥97.5% probability. To enable the lower limit check (LLC) and the upper limit check (ULC) to be optimized, the simulation results and the LC alarm rates for specific LLCs and ULCs were presented in LC performance tables. Optimal LLCs and ULCs were obtained for 31 analytes based on their random and systematic error-detection performances and the alarm rate. Reliable detection of random errors greater than 60% was only possible for analytes known to have a rather small variation of results. Furthermore, differences for negative and positive errors were observed. The used method brings objectivity to the error-detection performance of LCs, thereby enabling laboratory-specific LCs to be optimized and validated prior to application.

Internal quality control: Moving average algorithms outperform Westgard rules

IntroductionInternal quality control (IQC) is traditionally interpreted against predefined control limits using multi-rules or ‘Westgard rules’. These include the commonly used 1:3s and 2:2s rules. Either individually or in combination, these rules have limited sensitivity for detection of systematic errors. In this proof-of-concept study, we directly compare the performance of three moving average algorithms with Westgard rules for detection of systematic error. MethodsIn this simulation study, ‘error-free’ IQC data (control case) was generated. Westgard rules (1:3s and 2:2s) and three moving average algorithms (simple moving average (SMA), weighted moving average (WMA), exponentially weighted moving average (EWMA); all using ±3SD as control limits) were applied to examine the false positive rates. Following this, systematic errors were introduced to the baseline IQC data to evaluate the probability of error detection and average number of episodes for error detection (ANEed). ResultsFrom the power function graphs, in comparison to Westgard rules, all three moving average algorithms showed better probability of error detection. Additionally, they also had lower ANEed compared to Westgard rules. False positive rates were comparable between the moving average algorithms and Westgard rules (all <0.5%). The performance of the SMA algorithm was comparable to the weighted algorithms forms (i.e. WMA and EWMA). ConclusionApplication of an SMA algorithm on IQC data improves systematic error detection compared to Westgard rules. Application of SMA algorithms can simplify laboratories IQC strategy.

Abstract 5463: Accuracy improvements in somatic whole-genome small-variant calling with the DRAGEN platform

Abstract Introduction: Next-generation whole-genome sequencing promises to enable dramatic expansion of precision oncology and personalized cancer care. To support this effort, it is critical to develop computational tools that can analyze sequence data accurately and with rapid turn-around-time. One particularly challenging problem is the calling of somatic variants in matched tumor and normal samples. We present the DRAGENTM 3.5 somatic pipeline that performs 110x/40x whole-genome end-to-end analysis in under two hours with accuracy superior to that of all other tools we have benchmarked, including Mutect2/GATK4 and Strelka2. In addition, DRAGEN 3.5 is robust against variations in coverage, sequencing platform, sample preparation chemistry, and tumor purity. It also tolerates tumor-in-normal contamination, thereby making the pipeline applicable to late-stage solid tumors or hematological cancers. Methods: DRAGEN 3.5 replaces the legacy genotyping model (originally developed as part of MuTect2) with that of Strelka2. Whereas the MuTect2 model performs separate analyses on the tumor and normal samples, the Strelka2 model performs a joint analysis, allowing (1) detection of systematic errors that affect both samples simultaneously and (2) modeling of tumor-in-normal contamination, where the amount of contamination in the normal sample depends on the allele frequency observed in the tumor sample at the locus in question. In addition, we improved the probabilistic model of systematic error by incorporating models of strand bias and mismapping. Results: We benchmarked DRAGEN against Mutect2 (included in GATK 4.1.2) and Strelka2 (version 2.9.9) on five public synthetic and real datasets with known truth sets. DRAGEN greatly outperformed the other methods on all five datasets, producing 14-67% and 22-91% fewer false SNV calls, and 35-86% and 48-89% fewer false indel calls than Strelka2 and Mutect2 respectively. DRAGEN also exhibits higher tolerance to tumor-in-normal (TiN) contamination than Strelka2 which is already equipped with a model tolerating TiN contamination. The average end-to-end workflow runtime of the DRAGEN somatic pipeline was 77 minutes, 75% and 830% faster than Strelka2 and Mutect2 taking DRAGEN alignments as input. Conclusion: The DRAGEN pipeline enables reliable whole-genome analysis that can be scaled to large numbers of samples, leading to better tumor characterization and improved interpretation. We anticipate that it will ultimately fuel progress in oncology, cancer research and precision medicine. The DRAGEN 3.5 somatic pipeline can be run either locally on a DRAGEN server or remotely in the cloud via https://basespace.illumina.com. Citation Format: Konrad Scheffler, Sangtae Kim, Varun Jain, Jeffrey Yuan, Westley Sherman, Taylor O'Connell, Eric Ojard, Lisa Murray, Rami Mehio, Severine Catreux. Accuracy improvements in somatic whole-genome small-variant calling with the DRAGEN platform [abstract]. In: Proceedings of the Annual Meeting of the American Association for Cancer Research 2020; 2020 Apr 27-28 and Jun 22-24. Philadelphia (PA): AACR; Cancer Res 2020;80(16 Suppl):Abstract nr 5463.

ANALYSIS OF SYSTEMATIC ERRORS OF MOBILE LiDAR SYSTEMS: A SIMULATION APPROACH

Abstract. The systematic error analysis of the mobile LiDAR system (MLS) is always a challenging task in real-world situations. This challenge is mainly due to the mixture of systematic errors with non-systematic errors. To tackle this issue, in this paper, we introduce a conceptual model of an MLS simulator. The main advantage of the simulation-based approach is the full control over the erroneous systematic and non-systematic parameters that affect an MLS’s output. In the proposed simulation approach, we only take into account systematic errors that affect the simulated georeferenced point cloud. These systematic errors are as follows, POS-LiDAR boresight angles, POS-LiDAR leverarms, range offset, and scan angle offset. To simplify our analysis, we concentrate only on modeling the effects of systematic errors on planar targets and we focus solely on the terrestrial platform. Based on an independent analysis performed on each of the eight systematic errors of an MLS, to obtain strong visibility over systematic errors of an MLS, we suggest two planar targets of 1m x 1m dimensions with vertical and inclined orientations and a five-line pattern for MLS, two parallel and three side-looking passages. The proposed configuration generates an ideal input point cloud for the detection of systematic errors (except for the Z-Leverarm error) and ultimately it will lead to the proper input data for calibration of a terrestrial MLS. To validate our methodology, with an in-house assembled terrestrial MLS, we scanned a set of planar targets with three different orientations (vertical, inclined, and horizontal). This real-data validation test illustrated that with only two out of three planar targets (vertical and inclined) and with five out of six passages (two parallel to the planar targets and three side-looking passages), we will obtain expected visibility over the systematic errors of a terrestrial MLS, which approves the results with the simulation data.

Applications of statistical process control in the management of unaccounted for gas

The phenomenon of Unaccounted-for-Gas (UAG) in natural gas transmission networks can be summarised as the failure to account for a percentage of network throughput on either side of the balancing equation - typically around 0.3% of throughput per annum, a cost directly absorbed by the transporter. This article presents a case study of historical UAG data from the UK's National Gas Transmission System, compromising two instances of significant under-reads amounting to 1617 GWh and 1141 GWh respectively. Statistical process control techniques are applied and the usefulness of such analysis in relation to drawing inferences about the system state is assessed. Practical difficulties relevant to the gas industry in particular are discussed. Whilst we focus on transmission in the UK, the techniques and analysis presented herein should be pertinent to most grids around the world. It is demonstrated that significant practical advantages can be gained by implementing statistical monitoring in the balancing process. The importance of using multiple such measures is highlighted, in addition to a thorough analysis of the UAG time series focusing on the presence of seasonality. Furthermore, we discuss issues regarding the quantification of UAG sources, and present practical numerical limits to online systematic error detection with gas time series data via a simulation study.

Use of Miniature Step Gauges to Assess the Performance of 3D Optical Scanners and to Evaluate the Accuracy of a Novel Additive Manufacture Process.

In this work, we show how miniature step gauges featuring unidirectional and bidirectional lengths can be used to assess the performance of 3D optical scanners as well as the accuracy of novel Additive Manufacturing (AM) processes. A miniature step gauge made of black polyphenylene sulfide (PPS) was used for the performance verification of three different optical scanners: a structured light scanner (SLS), a laser line scanner (LLS), and a photogrammetry-based scanner (PSSRT), having comparable resolutions and working volumes. Results have shown a good agreement between the involved scanners, with errors below 5 μm and expanded uncertainties below 10 μm. The step gauge geometry due to the bidirectional lengths, highlights that there is a different interaction between the optical properties of the step gauge under measurement and each optical instrument involved and this aspect has to be considered in the uncertainty budget. The same geometry, due to its great significance in the detection of systematic errors, was used, as a novelty, to evaluate the accuracy of Lithography-based Ceramics Manufacturing (LCM), a proprietary additive manufacturing technology used for the fabrication of medical implants. In particular, two miniature step gauges made of Tricalcium Phosphate (TCP) were produced. Measurements conducted with the SLS scanner were characterized by a negligible error and by an uncertainty of about 5 μm. Deviations of the manufactured step gauges with respect to the Computer Aided Designed (CAD) model were comprised between ±50 μm, with positive deviations in the order of 100 μm on vertical sides. Differences in the order of 50 μm between the two step gauges were registered.

DuinoBlocks4Kids: Using Free Technology and Low-Cost Materials for the Exercise of Computational Thinking in Primary School via Programming and Educational Robotics

This paper presents a case study about the development of Computational Thinking in primary school children (3st to 4th grade) via the teaching of programming abilities with the use of educational robotics, free technology and recyclable, low cost materials. We aimed at raising some hypotheses on whether there is a straight relationship between some cognitive aspects of children aged 8-10 (such as the ability to put events and ideas in sequence, the ability to execute mental operations on the basis of concrete experience, among others) and the ability to execute activities that may be linked to the learning of computer programming. The observed results indicated (from the use of a didactic kit developed for the accomplishment of this study) the possibility to develop the following computational thinking skills: abstract thinking ability, understanding of flows of control, Debugging and systematic error detection, iterative thinking, use of conditional logic and problem decomposition. Regarding the investigations related to cognitive maturity, we found evidence of a correlation between the cognitive characteristics analyzed and the performance of certain tasks related to computer programming, such as the development of purely sequential programs and understanding of processing idea.

Dynamic System State Estimation and Outlier Detection Using Robust Data Reconciliation

State estimation and detection of measurement systematic errors are critical components of plant monitoring and control procedures. Reliable estimations of the process variables are attained by Classic Dynamic Data Reconciliation procedures when measurements follow exactly a known distribution. However, if this assumption happens approximately due to the presence of systematic errors, as outliers, classic dynamic data reconciliation provides biased results. In this work, a two-step methodology of Robust Dynamic Data Reconciliation and Systematic Error Detection is proposed. It takes advantages of a moving measurement window of fixed dimension and the features of the M-estimators. Furthermore, the presence of outliers is detected using a Robust Measurement Test. Two case studies are proposed, which work with the Huber and Biweigth M-estimators. A nonlinear benchmark extracted from the literature is considered, and performance measures are reported. The results obtained demonstrate the effectiveness of the proposed methodology.

Plate Design for and Cherry Picking of Bacterial RNAi Clones for Systematic Error Detection in High-Throughput Caenorhabditis elegans RNAi Screens.

Automated or semi-automated high-throughput RNAi screens are highly prone to systematic errors because of multistep repetitive protocols and extensive use of automated instruments. A well-designed RNAi library can help detect and minimize systematic human and robotic errors. In this unit, we describe how to design an RNAi bacterial library for use in conjunction with the well-studied nematode Caenorhabditis elegans for automated phenotypic screens. We provide strategies to design and assemble RNAi libraries to reduce or eliminate systematic errors. These strategies serve as a good quality-control check and facilitate obtaining high-quality data from a genome-wide and sub-library RNAi screen. © 2018 by John Wiley & Sons, Inc.

Aircraft aerodynamic coefficients identification using flight tests data

This paper discusses the problems of aerodynamic coefficients identification from flight test data. The principal stages of identification procedure are considered, the more sophisticated stages such as data compatibility check and parameter estimation from flight data are discussed in details. The paper also presents the examples of flight test data processing, which confirm the opportunities for on-board measurement systematic error detection and wind tunnel aerodynamic parameter values correction.

A Generic Transferable EEG Decoder for Online Detection of Error Potential in Target Selection.

Reliable detection of error from electroencephalography (EEG) signals as feedback while performing a discrete target selection task across sessions and subjects has a huge scope in real-time rehabilitative application of Brain-computer Interfacing (BCI). Error Related Potentials (ErrP) are EEG signals which occur when the participant observes an erroneous feedback from the system. ErrP holds significance in such closed-loop system, as BCI is prone to error and we need an effective method of systematic error detection as feedback for correction. In this paper, we have proposed a novel scheme for online detection of error feedback directly from the EEG signal in a transferable environment (i.e., across sessions and across subjects). For this purpose, we have used a P300-speller dataset available on a BCI competition website. The task involves the subject to select a letter of a word which is followed by a feedback period. The feedback period displays the letter selected and, if the selection is wrong, the subject perceives it by the generation of ErrP signal. Our proposed system is designed to detect ErrP present in the EEG from new independent datasets, not involved in its training. Thus, the decoder is trained using EEG features of 16 subjects for single-trial classification and tested on 10 independent subjects. The decoder designed for this task is an ensemble of linear discriminant analysis, quadratic discriminant analysis, and logistic regression classifier. The performance of the decoder is evaluated using accuracy, F1-score, and Area Under the Curve metric and the results obtained is 73.97, 83.53, and 73.18%, respectively.

One-dimensional cavity mode-dispersion spectroscopy for validation of CRDS technique

Detection of systematic errors of the measurement method is a demanding task, since it requires an independent, high-accuracy reference method for comparison. Here, we validate the cavity ring-down spectroscopy (CRDS) in relation to the one-dimensional cavity mode-dispersion spectroscopy (1D-CMDS). The complex line-shape function is used for the first time in the simultaneous analysis of absorptive and dispersive spectra and its potential in accurate line-shape studies is indicated. A new approach providing insensitivity of the 1D-CMDS to the drift of the laser-to-cavity locking point is demonstrated.

A New Approach to Detection of Systematic Errors in Secondary Substation Monitoring Equipment Based on Short Term Load Forecasting.

In recent years, Secondary Substations (SSs) are being provided with equipment that allows their full management. This is particularly useful not only for monitoring and planning purposes but also for detecting erroneous measurements, which could negatively affect the performance of the SS. On the other hand, load forecasting is extremely important since they help electricity companies to make crucial decisions regarding purchasing and generating electric power, load switching, and infrastructure development. In this regard, Short Term Load Forecasting (STLF) allows the electric power load to be predicted over an interval ranging from one hour to one week. However, important issues concerning error detection by employing STLF has not been specifically addressed until now. This paper proposes a novel STLF-based approach to the detection of gain and offset errors introduced by the measurement equipment. The implemented system has been tested against real power load data provided by electricity suppliers. Different gain and offset error levels are successfully detected.

Statistical tests against systematic errors in data sets based on the equality of residual means and variances from control samples: theory and applications.

Statistical tests are applied for the detection of systematic errors in data sets from least-squares refinements or other residual-based reconstruction processes. Samples of the residuals of the data are tested against the hypothesis that they belong to the same distribution. For this it is necessary that they show the same mean values and variances within the limits given by statistical fluctuations. When the samples differ significantly from each other, they are not from the same distribution within the limits set by the significance level. Therefore they cannot originate from a single Gaussian function in this case. It is shown that a significance cutoff results in exactly this case. Significance cutoffs are still frequently used in charge-density studies. The tests are applied to artificial data with and without systematic errors and to experimental data from the literature.