Automated Quality Control Research Articles

Automated Data Quality Control in FDOPA brain PET Imaging using Deep Learning

Introduction. With biomedical imaging research increasingly using large datasets, it becomes critical to find operator-free methods to quality control the data collected and the associated analysis. Attempts to use artificial intelligence (AI) to perform automated quality control (QC) for both single-site and multi-site datasets have been explored in some neuroimaging techniques (e.g. EEG or MRI), although these methods struggle to find replication in other domains. The aim of this study is to test the feasibility of an automated QC pipeline for brain [18F]-FDOPA PET imaging as a biomarker for the dopamine system.Methods. Two different Convolutional Neural Networks (CNNs) were used and combined to assess spatial misalignment to a standard template and the signal-to-noise ratio (SNR) relative to 200 static [18F]-FDOPA PET images that had been manually quality controlled from three different PET/CT scanners. The scans were combined with an additional 400 scans, in which misalignment (200 scans) and low SNR (200 scans) were simulated. A cross-validation was performed, where 80% of the data were used for training and 20% for validation. Two additional datasets of [18F]-FDOPA PET images (50 and 100 scans respectively with at least 80% of good quality images) were used for out-of-sample validation.Results. The CNN performance was excellent in the training dataset (accuracy for motion: 0.86 ± 0.01, accuracy for SNR: 0.69 ± 0.01), leading to 100% accurate QC classification when applied to the two out-of-sample datasets. Data dimensionality reduction affected the generalizability of the CNNs, especially when the classifiers were applied to the out-of-sample data from 3D to 1D datasets.Conclusions. This feasibility study shows that it is possible to perform automatic QC of [18F]-FDOPA PET imaging with CNNs. The approach has the potential to be extended to other PET tracers in both brain and non-brain applications, but it is dependent on the availability of large datasets necessary for the algorithm training.

A machine learning approach to quality control oceanographic data

Sensor errors are inevitable when measuring the ocean; thus, a reliable dataset of observations requires a quality control (QC) procedure capable of detecting spurious measurements. While manual QC by human experts minimizes errors, it is inefficient to handle large datasets and vulnerable to inconsistencies between different experts. Although automatic QC addresses some of these issues, the traditional methods result in high rates of false positives. Here, I propose a machine learning approach to automatically QC oceanographic data based on the Anomaly Detection technique. Multiple tests are combined into a single multidimensional criterion that learns the behavior of the valid measurements and identifies bad samples as outliers. When applied to 13 years of hydrographic profiles, the Anomaly Detection resulted in the best classification performance, reducing the error by at least 50%. The Anomaly Detection approach introduced here was implemented in the Python package CoTeDe, an open-source framework to quality control oceanographic data.

MARRYING DEEP LEARNING AND DATA FUSION FOR ACCURATE SEMANTIC LABELING OF SENTINEL-2 IMAGES

Abstract. The understanding of the Earth through global land monitoring from satellite images paves the way towards many applications including flight simulations, urban management and telecommunications. The twin satellites from the Sentinel-2 mission developed by the European Space Agency (ESA) provide 13 spectral bands with a high observation frequency worldwide. In this paper, we present a novel multi-temporal approach for land-cover classification of Sentinel-2 images whereby a time-series of images is classified using fully convolutional network U-Net models and then coupled by a developed probabilistic algorithm. The proposed pipeline further includes an automatic quality control and correction step whereby an external source can be introduced in order to validate and correct the deep learning classification. The final step consists of adjusting the combined predictions to the cloud-free mosaic built from Sentinel-2 L2A images in order for the classification to more closely match the reference mosaic image.

Self-supervised pairing image clustering for automated quality control

Self-supervised pairing image clustering for automated quality control

Quality control strategies for brain MRI segmentation and parcellation: Practical approaches and recommendations - insights from the Maastricht study

Quality control of brain segmentation is a fundamental step to ensure data quality. Manual quality control strategies are the current gold standard, although these may be unfeasible for large neuroimaging samples. Several options for automated quality control have been proposed, providing potential time efficient and reproducible alternatives. However, those have never been compared side to side, which prevents consensus in the appropriate quality control strategy to use. This study aimed to elucidate the changes manual editing of brain segmentations produce in morphological estimates, and to analyze and compare the effects of different quality control strategies on the reduction of the measurement error.Structural brain MRI from 259 participants of The Maastricht Study were used. Morphological estimates were automatically extracted using FreeSurfer 6.0. Segmentations with inaccuracies were manually edited, and morphological estimates were compared before and after editing. In parallel, 12 quality control strategies were applied to the full sample. Those included: two manual strategies, in which images were visually inspected and either excluded or manually edited; five automated strategies, where outliers were excluded based on the tools “MRIQC” and “Qoala-T”, and the metrics “morphological global measures”, “Euler numbers” and “Contrast-to-Noise ratio”; and five semi-automated strategies, where the outliers detected through the mentioned tools and metrics were not excluded, but visually inspected and manually edited. In order to quantify the effects of each quality control strategy, the proportion of unexplained variance relative to the total variance was extracted after the application of each strategy, and the resulting differences compared.Manually editing brain surfaces produced particularly large changes in subcortical brain volumes and moderate changes in cortical surface area, thickness and hippocampal volumes. The performance of the quality control strategies depended on the morphological measure of interest. Overall, manual quality control strategies yielded the largest reduction in relative unexplained variance. The best performing automated alternatives were those based on Euler numbers and MRIQC scores. The exclusion of outliers based on global morphological measures produced an increase of relative unexplained variance.Manual quality control strategies are the most reliable solution for quality control of brain segmentation and parcellation. However, measures must be taken to prevent the subjectivity associated with these strategies. The detection of inaccurate segmentations based on Euler numbers or MRIQC provides a time efficient and reproducible alternative. The exclusion of outliers based on global morphological estimates must be avoided.

Hopes and Hypes for Artificial Intelligence in Colorectal Cancer Screening

Hopes and Hypes for Artificial Intelligence in Colorectal Cancer Screening

Motion-based visual inspection of optically indiscernible defects on the example of hazelnuts

Automatic quality control has long been an integral part of the processing of food and agricultural products. Visual inspection offers solutions for many issues in this context and can be employed in the form of sensor-based sorting to automatically remove foreign and low quality entities from a product stream. However, these methods are limited to defects that can be made visible by the employed sensor, which usually restricts the system to defects appearing on the surface. An alternative non-visual solution lies in impact-acoustic methods, which do not suffer from this constraint. However, these are strongly limited in terms of material throughput and consequently not suitable for large scale industrial application. In this paper, we present a novel approach that performs inspection based on optically acquired motion data. A high-speed camera captures image sequences of test objects during a transportation process on a chute with a specific structured surface. The trajectory data is then used to classify test objects based on their motion behavior. The approach is evaluated experimentally on the example of distinguishing defect-free hazelnuts from ones that suffer from insect damage. Results show that by merely utilizing the motion data, a recognition rate of up to 80% for undamaged hazelnuts can be achieved. A major advantage of our approach is that it can be integrated in sensor-based sorting systems and is suitable for high throughput applications.

SalaciaML: A Deep Learning Approach for Supporting Ocean Data Quality Control

We present a skillful deep learning algorithm for supporting quality control of ocean temperature measurements, which we name SalaciaML according to Salacia the roman goddess of sea waters. Classical attempts to algorithmically support and partly automate the quality control of ocean data profiles are especially helpful for the gross errors in the data. Range filters, spike detection, and data distribution checks remove reliably the outliers and errors in the data, still wrong classifications occur. Various automated quality control procedures have been successfully implemented within the main international and EU marine data infrastructures (WOD, CMEMS, IQuOD, SDN) but their resulting data products are still containing data anomalies, bad data flagged as good and vice-versa. They also include visual inspection of suspicious measurements, which is a time consuming activity, especially if the number of suspicious data detected is large. A deep learning approach could highly improve our capabilities to quality assess big data collections and contemporary reducing the human effort. Our algorithm SalaciaML is meant to complement classical automated quality control procedures in supporting the time consuming visually inspection of data anomalies by quality control experts. As a first approach we applied the algorithm to a large dataset from the Mediterranean Sea. SalaciaML has been able to detect correctly more than 90% of all good and/or bad data in 11 out of 16 Mediterranean regions.

Automated quality control and cell identification of droplet-based single-cell data using dropkick.

A major challenge for droplet-based single-cell sequencing technologies is distinguishing true cells from uninformative barcodes in data sets with disparate library sizes confounded by high technical noise (i.e., batch-specific ambient RNA). We present dropkick, a fully automated software tool for quality control and filtering of single-cell RNA sequencing (scRNA-seq) data with a focus on excluding ambient barcodes and recovering real cells bordering the quality threshold. By automatically determining data set–specific training labels based on predictive global heuristics, dropkick learns a gene-based representation of real cells and ambient noise, calculating a cell probability score for each barcode. Using simulated and real-world scRNA-seq data, we benchmarked dropkick against conventional thresholding approaches and EmptyDrops, a popular computational method, showing greater recovery of rare cell types and exclusion of empty droplets and noisy, uninformative barcodes. We show for both low- and high-background data sets that dropkick's weakly supervised model reliably learns which genes are enriched in ambient barcodes and draws a multidimensional boundary that is more robust to data set–specific variation than existing filtering approaches. dropkick provides a fast, automated tool for reproducible cell identification from scRNA-seq data that is critical to downstream analysis and compatible with popular single-cell Python packages.

Deep-learning-based precipitation observation quality control

AbstractWe present a novel approach for the automated quality control (QC) of precipitation for a sparse station observation network within the complex terrain of British Columbia, Canada. Our QC approach uses Convolutional Neural Networks (CNNs) to classify bad observation values, incorporating a multi-classifier ensemble to achieve better QC performance. We train CNNs using human QC’d labels from 2016 to 2017 with gridded precipitation and elevation analyses as inputs. Based on the classification evaluation metrics, our QC approach shows reliable and robust performance across different geographical environments (e.g., coastal and inland mountains), with 0.927 Area Under Curve (AUC) and type I/type II error lower than 15%. Based on the saliency-map-based interpretation studies, we explain the success of CNN-based QC by showing that it can capture the precipitation patterns around, and upstream of the station locations. This automated QC approach is an option for eliminating bad observations for various applications, including the pre-processing of training datasets for machine learning. It can be used in conjunction with human QC to improve upon what could be accomplished with either method alone.

Retrospective non-target analysis to support regulatory water monitoring: from masses of interest to recommendations via in silico workflows

BackgroundApplying non-target analysis (NTA) in regulatory environmental monitoring remains challenging—instead of having exploratory questions, regulators usually already have specific questions related to environmental protection aims. Additionally, data analysis can seem overwhelming because of the large data volumes and many steps required. This work aimed to establish an open in silico workflow to identify environmental chemical unknowns via retrospective NTA within the scope of a pre-existing Swiss environmental monitoring campaign focusing on industrial chemicals. The research question addressed immediate regulatory priorities: identify pollutants with industrial point sources occurring at the highest intensities over two time points. Samples from 22 wastewater treatment plants obtained in 2018 and measured using liquid chromatography–high resolution mass spectrometry were retrospectively analysed by (i) performing peak-picking to identify masses of interest; (ii) prescreening and quality-controlling spectra, and (iii) tentatively identifying priority “known unknown” pollutants by leveraging environmentally relevant chemical information provided by Swiss, Swedish, EU-wide, and American regulators. This regulator-supplied information was incorporated into MetFrag, an in silico identification tool replete with “post-relaunch” features used here. This study’s unique regulatory context posed challenges in data quality and volume that were directly addressed with the prescreening, quality control, and identification workflow developed.ResultsOne confirmed and 21 tentative identifications were achieved, suggesting the presence of compounds as diverse as manufacturing reagents, adhesives, pesticides, and pharmaceuticals in the samples. More importantly, an in-depth interpretation of the results in the context of environmental regulation and actionable next steps are discussed. The prescreening and quality control workflow is openly accessible within the R package Shinyscreen, and adaptable to any (retrospective) analysis requiring automated quality control of mass spectra and non-target identification, with potential applications in environmental and metabolomics analyses.ConclusionsNTA in regulatory monitoring is critical for environmental protection, but bottlenecks in data analysis and results interpretation remain. The prescreening and quality control workflow, and interpretation work performed here are crucial steps towards scaling up NTA for environmental monitoring.

Особливості створення (оновлення) цифрових топографічних карт для формування основноїдержавної топографічної карти

Purpose. The purpose of the article is to research the peculiarities of creation (updating) of digital topographic maps at the scale of 1:50 000/1:10 000 which would satisfy the requirements for the development of the seamless Topographic Database of the Main State Topographic Map at the scale of 1:50 000. Methodology. The basis for the research is the analysis of the possibilities of applying the theory of databases and knowledge bases, International Standards and specifications and vectorization method. Results. The article examines the peculiarities of creation (updating) digital topographic maps of the scale 1:50000 for the formation of the Main State Topographic Map of Ukraine for the purpose of the creation and maintain the seamless topographic database for national needs, which is located on the Geoportal to ensure the relevance of a single digital topographic basis by topographical monitoring of the territories and for the development of the National Spatial Data Infrastructure in Ukraine. The rules of topological relations between features of the digital topographic maps of the scale 1:50 000 are also defined and given. The peculiarities of providing automated quality control of updated digital topographic maps are investigated. The creation of the seamless Topographic Database of the Main State Topographic Map in the conditions of transfer the cartographic paradigm to geoinformation creates new requirements for the creation (updating) of digital topographic maps of the scale 1:50 000/10 000: creation of spatial schemes, description of the internal design of models and rules of digital description of geospatial features, unification of the features catalog and their attributes, as well as rules of topology between topographic features to ensure topological consistency of geometry in accordance with standards and specifications; creation of the “Validate” software package for checking of created (updated) digital topographic maps at a scale of 1:50 000/10 000 to ensure automated quality control of updated digital topographic maps; creation of new virtual and associated features in the TDB of the Main State Topographic Map. This will increase the intellectual level of geospatial data creation. Scientific novelty and practical significance. The creation of the Main Topographic Map Topographic Database takes into account the use of new virtual and associated features, the use of rules of topological relations between digital topographic map features, providing automated quality control of updated digital topographic maps.

ASHLEYS: automated quality control for single-cell Strand-seq data.

SummarySingle-cell DNA template strand sequencing (Strand-seq) enables chromosome length haplotype phasing, construction of phased assemblies, mapping sister-chromatid exchange events and structural variant discovery. The initial quality control of potentially thousands of single-cell libraries is still done manually by domain experts. ASHLEYS automates this tedious task, delivers near-expert performance and labels even large datasets in seconds.Availability and implementationgithub.com/friendsofstrandseq/ashleys-qc, MIT license.Supplementary information Supplementary data are available at Bioinformatics online.

Tire defect classification using a deep convolutional sparse-coding network

Automatic quality control has garnered great interest from academia and industry in recent decades. Tire defect detection and classification are critical research topics for driving safety and improving yields. In this work, a novel deep convolutional sparse-coding network (DCScNet) is built for tire defect classification. In DCScNet, sparse coding is utilized to replace the convolution kernel of a convolutional neural network (CNN) for the extraction of features. Compared with CNN, it requires unsupervised training, reduces the subjectivity of artificially defined labels, and achieves higher classification accuracy. The experimental results validate its superior classification performance, compared with state-of-the-art classification algorithms. The classification accuracy reaches 96.8%, which meets industrial requirements.

Using crowd-sourced data for real-time monitoring of food prices during the COVID-19 pandemic: Insights from a pilot project in northern Nigeria

The COVID-19 pandemic and related lockdown measures have disrupted food supply chains globally and caused threats to food security, especially in Sub-Saharan Africa. Yet detailed, localized, and timely data on food security threats are rarely available to guide targeted policy interventions. Based on real-time evidence from a pilot project in northern Nigeria, where food insecurity is severe, we illustrate how a digital crowdsourcing platform can provide validated real-time, high frequency, and spatially rich information on the evolution of commodity prices. Daily georeferenced price data of major food commodities were submitted by active volunteer citizens through a mobile phone data collection app and filtered through a stepwise quality control algorithm. We analyzed a total of 23,961 spatially distributed datapoints, contributed by 236 active volunteers, on the price of four commodities (local rice, Thailand rice, white maize and yellow maize) to assess the magnitude of price change over eleven weeks (week 20 to week 30) during and after the first COVID-related lockdown (year 2020), relative to the preceding year (2019). Results show that the retail price of maize (yellow and white) and rice (local and Thailand rice) increased on average by respectively 26% and 44% during this COVID-related period, compared to prices reported in the same period in 2019. GPS-tracked data showed that mobility and market access of active volunteers were reduced, travel-distance to market being 54% less in 2020 compared to 2019, and illustrates potential limitations on consumers who often seek lower pricing by accessing broader markets. Combining the price data with a spatial richness index grid derived from UN-FAO, this study shows the viability of a contactless data crowdsourcing system, backed by an automated quality control process, as a decision-support tool for rapid assessment of price-induced food insecurity risks, and to target interventions (e.g. COVID relief support) at the right time and location(s).

A Century of French Railways: The Value of Remote Sensing and VGI in the Fusion of Historical Data

Providing long-term data about the evolution of railway networks in Europe may help us understand how European Union (EU) member states behave in the long-term, and how they can comply with present EU recommendations. This paper proposes a methodology for collecting data about railway stations, at the maximal extent of the French railway network, a century ago.The expected outcome is a geocoded dataset of French railway stations (gares), which: (a) links gares to each other, (b) links gares with French communes, the basic administrative level for statistical information. Present stations are well documented in public data, but thousands of past stations are sparsely recorded, not geocoded, and often ignored, except in volunteer geographic information (VGI), either collaboratively through Wikipedia or individually. VGI is very valuable in keeping track of that heritage, and remote sensing, including aerial photography is often the last chance to obtain precise locations. The approach is a series of steps: (1) meta-analysis of the public datasets, (2) three-steps fusion: measure-decision-combination, between public datasets, (3) computer-assisted geocoding for ‘gares’ where fusion fails, (4) integration of additional gares gathered from VGI, (5) automated quality control, indicating where quality is questionable. These five families of methods, form a comprehensive computer-assisted reconstruction process (CARP), which constitutes the core of this paper. The outcome is a reliable dataset—in geojson format under open license—encompassing (by January 2021) more than 10,700 items linked to about 7500 of the 35,500 communes of France: that is 60% more than recorded before. This work demonstrates: (a) it is possible to reconstruct transport data from the past, at a national scale; (b) the value of remote sensing and of VGI is considerable in completing public sources from an historical perspective; (c) data quality can be monitored all along the process and (d) the geocoded outcome is ready for a large variety of further studies with statistical data (demography, density, space coverage, CO2 simulation, environmental policies, etc.).

Texture defect classification with multiple pooling and filter ensemble based on deep neural network

Fabric quality control is one of the most important phases of production in order to ensure high-quality standards in the fabric production sector. For this reason, the development of successful automatic quality control systems has been a very important research subject. In this study, we propose a Multiple Pooling and Filter approach based on a Deep Neural Network (MPF-DNN) for the classification of texture defects. This model consists of three basic stages: preprocessing, feature extraction, and classification. In the preprocessing stage, the texture images were first divided into n × n equal parts. Then, median filtering and pooling processes were applied to each piece prior to performing image merging. In the proposed pre-treatment stage, it is aimed to clarify fabric errors and increase performance. For the feature extraction stage, deep features were extracted from the texture images using the pretrained ResNet101 model based on the transfer learning approach. Finally, classification and testing procedures were conducted on the obtained deep-effective properties using the SVM method. The multiclass TILDA dataset was used in order to test the proposed model. In experimental work, the MPF-DNN model for all four classes achieved a significant overall accuracy score of 95.82%. In the results obtained from extensive experimental studies, it was observed that the proposed MPF-DNN model was more successful than previous studies that used pretrained deep architectures.

SeqQscorer: automated quality control of next-generation sequencing data using machine learning

Controlling quality of next-generation sequencing (NGS) data files is a necessary but complex task. To address this problem, we statistically characterize common NGS quality features and develop a novel quality control procedure involving tree-based and deep learning classification algorithms. Predictive models, validated on internal and external functional genomics datasets, are to some extent generalizable to data from unseen species. The derived statistical guidelines and predictive models represent a valuable resource for users of NGS data to better understand quality issues and perform automatic quality control. Our guidelines and software are available at https://github.com/salbrec/seqQscorer.

Videomics: bringing deep learning to diagnostic endoscopy.

Machine learning (ML) algorithms have augmented human judgment in various fields of clinical medicine. However, little progress has been made in applying these tools to video-endoscopy. We reviewed the field of video-analysis (herein termed 'Videomics' for the first time) as applied to diagnostic endoscopy, assessing its preliminary findings, potential, as well as limitations, and consider future developments. ML has been applied to diagnostic endoscopy with different aims: blind-spot detection, automatic quality control, lesion detection, classification, and characterization. The early experience in gastrointestinal endoscopy has recently been expanded to the upper aerodigestive tract, demonstrating promising results in both clinical fields. From top to bottom, multispectral imaging (such as Narrow Band Imaging) appeared to provide significant information drawn from endoscopic images. Videomics is an emerging discipline that has the potential to significantly improve human detection and characterization of clinically significant lesions during endoscopy across medical and surgical disciplines. Research teams should focus on the standardization of data collection, identification of common targets, and optimal reporting. With such a collaborative stepwise approach, Videomics is likely to soon augment clinical endoscopy, significantly impacting cancer patient outcomes.

Non-invasive identification of commercial green tea blends using NIR spectroscopy and support vector machine

In this study, we aimed to discriminate four commercial blends of green tea in bagged (inside its sachet) and non-bagged conditions using near-infrared (NIR) spectroscopy and support vector machines (SVM) for data modelling. To choose optimal parameters for the models, we applied Bayesian optimization, which provided accurate models. Two spectrometers were evaluated: a benchtop and a handheld, both presenting reliable results for non-bagged tea (accuracies of 90% and 93%, respectively). However, for bagged tea models, the classification performance of benchtop was superior to handheld equipment, yielding accuracies of 93% and 82%, respectively. Classification accuracies using SVM outperformed partial least squares discriminant analysis (PLS–DA) for handheld and tea inside teabag models. The results indicated that the proposed methodology has the potential to be applied in automatic quality control coupling NIR sensors and machine learning for data processing.