Automated Quality Control Research Articles

Benchmarking framework for anomaly localization: Towards real-world deployment of automated visual inspection

Localizing defects in products is a critical component of industrial pipelines in manufacturing, retail, and many other industries to ensure consistent delivery of high quality products. Automated anomaly localization systems leveraging computer vision have the potential to replace laborious and subjective manual inspection of products. Recently, there have been tremendous efforts in this research domain investigating deep learning-based anomaly localization methods. However, such proposed methods, mainly considering product-specific evaluation, cannot be directly implemented for real-world use. Therefore, despite the advancements, there is still a gap between research and deployment of those methods to real-world production environments. Implementing any automated solution for manufacturing can involve a steep upfront investment. It is important to develop an industry-friendly benchmarking framework to ensure the feasibility and robustness of an automated quality control system. In this paper, we present a new anomaly localization benchmarking framework considering different aspects - (1) understand the performance of models in a generalizable product-agnostic manner, (2) explore pros and cons to find the most optimal modeling approach, (3) develop an efficient training and inference scheme with defect-free training samples and very few defective samples for evaluation, and (4) perform an ablation study of threshold estimation techniques to determine optimal threshold level for segmentation. We release a newly-labeled dataset for the research community with product-agnostic categorization of defective product images. To the best of our knowledge, this is the first anomaly localization work on developing a benchmarking framework focusing on real-world use. We believe domain experts from different industries will find this useful and can gain valuable insights to deploy automated visual inspection in production pipelines.

An AI-Based Image Quality Control Framework for Knee Radiographs

Image quality control (QC) is crucial for the accurate diagnosis of knee diseases using radiographs. However, the manual QC process is subjective, labor intensive, and time-consuming. In this study, we aimed to develop an artificial intelligence (AI) model to automate the QC procedure typically performed by clinicians. We proposed an AI-based fully automatic QC model for knee radiographs using high-resolution net (HR-Net) to identify predefined key points in images. We then performed geometric calculations to transform the identified key points into three QC criteria, namely, anteroposterior (AP)/lateral (LAT) overlap ratios and LAT flexion angle. The proposed model was trained and validated using 2212 knee plain radiographs from 1208 patients and an additional 1572 knee radiographs from 753 patients collected from six external centers for further external validation. For the internal validation cohort, the proposed AI model and clinicians showed high intraclass consistency coefficients (ICCs) for AP/LAT fibular head overlap and LAT knee flexion angle of 0.952, 0.895, and 0.993, respectively. For the external validation cohort, the ICCs were also high, with values of 0.934, 0.856, and 0.991, respectively. There were no significant differences between the AI model and clinicians in any of the three QC criteria, and the AI model required significantly less measurement time than clinicians. The experimental results demonstrated that the AI model performed comparably to clinicians and required less time. Therefore, the proposed AI-based model has great potential as a convenient tool for clinical practice by automating the QC procedure for knee radiographs.

A machine learning‐based quality control tool for functional MRI images

AbstractBackgroundChanges in functional MRI (fMRI) have been associated with Alzheimer’s disease and other neurological disorders. Quality control (QC) of images prior to their computational analysis is a key step in a clinical trial analysis workflow, and it is traditionally implemented visually by image analysts. However, the number of images acquired from large clinical studies makes neuroimaging QC a difficult task that diverts skilled resources and introduces inter‐rater variability. This problem is accentuated in fMRI, where image analysts need to visually inspect 4D sequences, and although image processing reports can be used as aid, the problem of having to inspect each report remains.MethodWe implemented an automated fMRI QC process using a stacking classifier (STC), random forest + support vector machine, with imaging quality metrics (IQM) as inputs. The IQMs were estimated with MRIQC. We also added a designed IQM that measures how centred the participant’s brain is within the image. As train/test dataset we used the ABIDE multisite database. For our experiments, only adult brains from the ABIDE database were used, leaving 14 sites available for a leave‐one‐site‐out (LOSO) cross‐validation experiment. All images included in the experiment were visually QC’d using validated protocols at IXICO.ResultThe LOSO validation procedure resulted in a mean pass‐vs‐fail accuracy of 92% (with 5.7% standard deviation, SD). Pass/fail sensitivity and specificity resulted in 97% (6.7% SD) and 69% (29% SD) respectively. We tested a fully trained STC model on an independent dataset of healthy ageing participants from OpenNeuro (ds002872), which resulted in 87% accuracy, 94% sensitivity and 50% specificity, which validates our modelling procedure.ConclusionIt is possible to design and deploy an automatic QC model for fMRI. Although sensitivity was high in all models from the LOSO experiments, specificity was on average low, highlighting the difficulty of detecting fails in a QC pipeline. However, the STC model can be deployed to detect clear passes that will not need visual QC, reliably reducing the number of cases that require visual inspection. Our future work will focus on testing our developed model on additional datasets (e.g. ADNI) to validate its performance before roll‐out to clinical trial applications.aging after age 90.

Addressing labour exploitation in the data science pipeline: views of precarious US-based crowdworkers on adversarial and co-operative interventions

PurposeUnderlying much recent development in data science and artificial intelligence (AI) is a dependence on the labour of precarious crowdworkers via platforms such as Amazon Mechanical Turk. These platforms have been widely critiqued for their exploitative labour relations, and over recent years, there have been various efforts by academic researchers to develop interventions aimed at improving labour conditions. The aim of this paper is to explore US-based crowdworkers’ views on two proposed interventions: a browser plugin that detects automated quality control “Gold Question” (GQ) checks and a proposal for a crowdworker co-operative.Design/methodology/approachThe authors interviewed 20 US-based crowdworkers and undertook a thematic analysis of collected data.FindingsThe findings indicate that US-based crowdworkers tend to have negative and mixed feelings about the GQ detector, but were more enthusiastic about the crowdworker co-operative.Originality/valueDrawing on theories of precarious labour, this study suggests an explanation for the findings based on US-based workers’ objective and subjective experiences of precarity. The authors argue that for US-based crowdworkers “constructive” interventions such as a crowdworker co-operative have more potential to improve labour conditions.

Analysis of the applicability and state monitoring of material extrusion–printed acrylonitrile butadiene styrene injection mould inserts with different infill levels

The widespread appearance of additive manufacturing (AM) proved to be a game changer for many traditional plastic processing technologies in the last few years. The application of AM for injection mould making has become a new and promising direction of rapid tooling (RT), especially with the emerging demand for more customizable products. A possible solution to satisfy the desire for mass customization is the application of additively manufactured low-volume moulds and inserts. These prototype moulds also give an excellent opportunity to check and validate designs in a real-life operational environment, before the expensive large-volume steel mould is made. Material extrusion printing was used to produce injection mould inserts from acrylonitrile butadiene styrene (ABS). These inserts proved to be suitable for low-volume injection moulding. The inserts were printed with different infill levels: 80 %, 50 % and 25 %. In previous articles, an earlier unknown complex state monitoring method was created to analyse and quantify in-mould operational behaviour of additively manufactured injection mould inserts. In the present article, this measurement system was applied to material extrusion–printed prototype moulds. This system measures strains (at two locations) and the temperature distribution of the inserts (both volumetric temperature and cavity surface temperature). 3D scanning and conventional thickness and mass measurements were also applied to characterize the dimensional accuracy of the injection moulded products and the inserts as well. The correlation between the definite integral of measured operational strains (occurring during the injection moulding cycles, from the start of holding to the end of the holding phase) and the resulting product properties (mass and thicknesses) was also analysed. An excellent linear correlation was found between them (mostly R2 > 0.9). This finding can pave the way for a more automated quality control system because of its practical applicability and relative ease of use.

Automatic uncertainty-based quality controlled T1 mapping and ECV analysis from native and post-contrast cardiac T1 mapping images using Bayesian vision transformer.

Deep learning-based methods for cardiac MR segmentation have achieved state-of-the-art results. However, these methods can generate incorrect segmentation results which can lead to wrong clinical decisions in the downstream tasks. Automatic and accurate analysis of downstream tasks, such as myocardial tissue characterization, is highly dependent on the quality of the segmentation results. Therefore, it is of paramount importance to use quality control methods to detect the failed segmentations before further analysis. In this work, we propose a fully automatic uncertainty-based quality control framework for T1 mapping and extracellular volume (ECV) analysis. The framework consists of three parts. The first one focuses on segmentation of cardiac structures from a native and post-contrast T1 mapping dataset (n=295) using a Bayesian Swin transformer-based U-Net. In the second part, we propose a novel uncertainty-based quality control (QC) to detect inaccurate segmentation results. The QC method utilizes image-level uncertainty features as input to a random forest-based classifier/regressor to determine the quality of the segmentation outputs. The experimental results from four different types of segmentation results show that the proposed QC method achieves a mean area under the ROC curve (AUC) of 0.927 on binary classification and a mean absolute error (MAE) of 0.021 on Dice score regression, significantly outperforming other state-of-the-art uncertainty based QC methods. The performance gap is notably higher in predicting the segmentation quality from poor-performing models which shows the robustness of our method in detecting failed segmentations. After the inaccurate segmentation results are detected and rejected by the QC method, in the third part, T1 mapping and ECV values are computed automatically to characterize the myocardial tissues of healthy and cardiac pathological cases. The native myocardial T1 and ECV values computed from automatic and manual segmentations show an excellent agreement yielding Pearson coefficients of 0.990 and 0.975 (on the combined validation and test sets), respectively. From the results, we observe that the automatically computed myocardial T1 and ECV values have the ability to characterize myocardial tissues of healthy and cardiac diseases like myocardial infarction, amyloidosis, Tako-Tsubo syndrome, dilated cardiomyopathy, and hypertrophic cardiomyopathy.

Automatic detection surface defects based on convolutional neural networks and deflectometry

AbstractSurface defects in industrial refrigerator manufacturing processes can cause significant production losses and compromise product quality. This area is underexplored and currently, visual quality inspection is a subjective process that requires expert intervention, which limits process efficiency and can lead to errors in defect detection. This paper presents a novel approach for automatic surface defect detection using a combination of convolutional neural networks (CNN) and deflectometry. The proposed method takes advantage of the high accuracy and robustness of CNNs in image classification tasks and the sensitivity of deflectometry to detect subtle surface variations. First, a prototype was built to get the images from the refrigerator. Second, using video recordings, we captured surface topographic data using deflectometry, which we then use to generate surface images. Next, we train a CNN to classify the surface images as defective or normal. The proposed method offers a promising solution for automatic detection and quality control of surface defects in refrigerator manufacturing processes. However, this method could also improve the production of vehicles, household appliances in general, and any product that can suffer scratches and dents.

Automatic quality control of aluminium parts welds based on 3D data and artificial intelligence

Automatic quality control of aluminium parts welds based on 3D data and artificial intelligence

Digital by design approach to develop a universal deep learning AI architecture for automatic chromatographic peak integration.

Chromatographic data processing has garnered attention due to multiple Food and Drug Administration 483 citations and warning letters, highlighting the need for a robust technological solution. The healthcare industry has the potential to greatly benefit from the adoption of digital technologies, but the process of implementing these technologies can be slow and complex. This article presents a "Digital by Design" managerial approach, adapted from pharmaceutical quality by design principles, for designing and implementing an artificial intelligence (AI)-based solution for chromatography peak integration process in the healthcare industry. We report the use of a convolutional neural network model to predict analytical variability for integrating chromatography peaks and propose a potential GxP framework for using AI in the healthcare industry that includes elements on data management, model management, and human-in-the-loop processes. The component on analytical variability prediction has a great potential to enable Industry 4.0 objectives on real-time release testing, automated quality control, and continuous manufacturing.

Automatic quality control of the numerical accuracy of EEG lead fields

Precise individualized EEG source localization is predicated on having accurate subject-specific Lead Fields (LFs) obtained from their Magnetic Resonance Images (MRI). LF calculation is a complex process involving several error-prone steps that start with obtaining a realistic head model from the MRI and finalizing with computationally expensive solvers such as the Boundary Element Method (BEM) or Finite Element Method (FEM). Current Big-Data applications require the calculation of batches of hundreds or thousands of LFs. LF. Quality Control is conventionally checked subjectively by experts, a procedure not feasible in practice for larger batches. To facilitate this step, we introduce the Lead Field Automatic-Quality Control Index (LF-AQI) that flags LF with potential errors. We base our LF-AQI on the assumption that LFs obtained from simpler head models, i.e., the homogeneous head model LF (HHM-LF) or spherical head model LF (SHM-LF), deviate only moderately from a "good" realistic test LF. Since these simpler LFs are easier to compute and check for errors, they may serve as "reference LF" to detect anomalous realistic test LF. We investigated this assumption by comparing correlation-based channel ρmin(ref,test)and source τmin(ref,test) similarity indices (SI) between "gold standards," i.e., very accurate FEM and BEM LFs, and the proposed references (HHM-LF and SHM-LF). Surprisingly we found that the most uncomplicated possible reference, HHM-LF had high SI values with the gold standards—leading us to explore further use of the channel ρmin(HHM−LF,test)and source τmin(HHM−LF,test)SI as a basis for our LF-AQI. Indeed, these SI successfully detected five simulated scenarios of LFs artifacts. This result encouraged us to evaluate the SI on a large dataset and thus define our LF-AQI. We thus computed the SI of 1251 LFs obtained from the Child Mind Institute (CMI) MRI dataset. When ρmin(HHM−LF,test)and source τmin(HHM−LF,test) were plotted for all test subjects on a 2D space, most were tightly clustered around the median of a high similarity centroid (HSC), except for a smaller proportion of outliers. We define the LF-AQI for a given LF as the log Euclidean distance between its SI and the HSC median. To automatically detect outliers, the threshold is at the 90th percentile of the CMI LF-AQIs (-0.9755). LF-AQI greater than this threshold flag individual LF to be checked. The robustness of this LF-AQI screening was checked by repeated out-of-sample validation. Strikingly, minor corrections in re-processing the flagged cases eliminated their status as outliers. Furthermore, the "doubtful" labels assigned by LF-AQI were validated by neuroscience students using a Likert scale questionnaire designed to manually check the LF's quality. Item Response Theory (IRT) analysis was applied to the questionnaire results to compute an optimized model and a latent variable θ for that model. A linear mixed model (LMM) between the θ and LF-AQI resulted in an effect with a Cohen's d value of 1.3 and a p-value <0.001, thus validating the correspondence of LF-AQI with the visual quality control. We provide an open-source pipeline to implement both LF calculation and its quality control to allow further evaluation of our index.

Siam-VAE: A hybrid deep learning based anomaly detection framework for automated quality control of head CT scans.

An automated quality control (QC) system is essential to ensure streamlined head computed tomography (CT) scan interpretations that do not affect subsequent image analysis. Such a system is advantageous compared to current human QC protocols, which are subjective and time-consuming. In this work, we aim to develop a deep learning-based framework to classify a scan to be of usable or unusable quality. Supervised deep learning models have been highly effective in classification tasks, but they are highly complex and require large, annotated data for effective training. Additional challenges with QC datasets include - 1) class-imbalance - usable cases far exceed the unusable ones and 2) weak-labels - scan level labels may not match slice level labels. The proposed framework utilizes these weak labels to augment a standard anomaly detection technique. Specifically, we proposed a hybrid model that consists of a variational autoencoder (VAE) and a Siamese Neural Network (SNN). While the VAE is trained to learn how usable scans appear and reconstruct an input scan, the SNN compares how similar this input scan is to its reconstruction and flags the ones that are less similar than a threshold. The proposed method is more suited to capture the differences in non-linear feature structure between the two classes of data than typical anomaly detection methods that depend on intensity-based metrics like root mean square error (RMSE). Comparison with state-of-the-art anomaly detection methods using multiple classification metrics establishes superiority of the proposed framework in flagging inferior quality scans for review by radiologists, thus reducing their workload and establishing a reliable and consistent dataflow.

Abstract 5442: SlideQC: An AI-based tool for automated quality control of whole-slide digital pathology images

Abstract Introduction: Artifacts are often introduced during tissue collection and processing, slide preparation, and/or when generating whole slide images (WSI). The presence of artifacts has a negative impact on the digital pathology workflow as artifacts may hinder diagnostic reporting and can lead to false positive and false negative results when using image analysis algorithms or computer-aided diagnosis systems. Manual quality control of WSI is a time-consuming procedure and therefore automated quality control tools, which report and exclude artifacts, are highly desirable to streamline digital pathology workflows. To automate the quality control step, we developed SlideQC, an AI-based quality control tool that automatically detects, reports, and outlines artifacts such as air bubbles, dust/debris, folds, out-of-focus,and pen marks, in both research and clinical workflows. Methods: SlideQC was trained with a DenseNet-based network using 1984 annotations for artifacts including air bubbles, dust/debris, folds, out-of-focus, and pen markers, across 254 Haematoxylin and Eosin (H&amp;E) stained WSI from more than 9 tissue types. A set of 2048 annotations from synthetically generated out-of-focus images was added to supplement the training data. The performance of the SlideQC was evaluated on an external test cohort of 49 WSI H&amp;E images sourced from the open-source database ‘HistoQCRepo’, across 375 annotations (tissue and artifact), and compared with the performance of HistoQC, an open-source quality control tool for digital pathology slides. Results: On the external test cohort, SlideQC showed high sensitivity, specificity, and F1-score with average values of 0.93, 0.99, and 0.93, across the five artifact types. In the same cohort, HistoQC attained an average sensitivity, specificity, and F1-score of 0.65, 0.79, and 0.54, respectively. Conclusions: SlideQC achieved high sensitivity, specificity, and F1-score on an external test cohort. SlideQC can add efficiency gains to a workflow by performing quality control on 100% of slides rather than the currently manually performed on only a subset of the slides in clinical pathology departments. SlideQC can allowthe triaging and alerting of slides containing a high level of artifact within a digital pathology workflow. The tool can also be used to exclude the artifact region from downstream analysis by subsequent image analysis algorithms. Citation Format: Daniela Rodrigues, Stefan Reinhard, Therese Waldburger, Daniel Martin, Suzana Couto, Inti Zlobec, Peter Caie, Erik Burlingame. SlideQC: An AI-based tool for automated quality control of whole-slide digital pathology images. [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2023; Part 1 (Regular and Invited Abstracts); 2023 Apr 14-19; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2023;83(7_Suppl):Abstract nr 5442.

Automated quality control tool for high-content imaging data by building 2D prediction intervals on reference biosignatures.

Recent advances in automated microscopy and image analysis enables quantitative profiling of cellular phenotypes (Cell Painting). It paves the way for studying the broad effects of chemical perturbations on biological systems at large scale during lead optimization. Comparison of perturbation biosignatures with biosignatures of annotated compounds can inform on both on- and off-target effects. When building databases with phenotypic profiles of thousands of compounds, it is vital to control the quality of Cell Painting assays over time. A tool for this to our knowledge does not yet exist within the imaging community. In this paper, we introduce an automated tool to assess the quality of Cell Painting assays by quantifying the reproducibility of biosignatures of annotated reference compounds. The tool learns the biosignature of those treatments from a historical dataset, and subsequently, it builds a two-dimensional probabilistic quality control (QC) limit. The limit will then be used to detect aberrations in new Cell Painting experiments. The tool is illustrated using simulated data and further demonstrated on Cell Painting data of the A549 cell line. In general, the tool provides a sensitive, detailed and easy-to-interpret mechanism to validate the quality of Cell Painting assays.

Do the Math: Making Mathematics in Wikipedia Computable.

Wikipedia combines the power of AI solutions and human reviewers to safeguard article quality. Quality control objectives include detecting malicious edits, fixing typos, and spotting inconsistent formatting. However, no automated quality control mechanisms currently exist for mathematical formulae. Spell checkers are widely used to highlight textual errors, yet no equivalent tool exists to detect algebraically incorrect formulae. Our paper addresses this shortcoming by making mathematical formulae computable. We present a method that (1) gathers the semantic information surrounding the context of each mathematical formulae, (2) provides access to the information in a graph-structured dependency hierarchy, and (3) performs automatic plausibility checks on equations. We evaluate the performance of our approach on 6,337 mathematical expressions contained in 104 Wikipedia articles on the topic of orthogonal polynomials and special functions. Our system, [Formula: see text], verified 358 out of 1,516 equations as error-free. [Formula: see text] successfully translated 27% of the mathematical expressions and outperformed existing translation approaches by 16%. Additionally, [Formula: see text] achieved an F1 score of .495 for annotating mathematical expressions with relevant textual descriptions, which is a significant step towards advancing searchability, readability, and accessibility of mathematical formulae in Wikipedia. A prototype of [Formula: see text] and the semantically enhanced Wikipedia articles are available at: https://tpami.wmflabs.org.

A new automatic quality control system for ocean profile observations and impact on ocean warming estimate

A new automatic quality control system for ocean profile observations and impact on ocean warming estimate

Possibility of infrared (IR) thermography camera to investigate properties of injected plastic product and optimisation injection process parameter using a design of experiment

ABSTRACT The injection processing parameters have an important role in product quality. The purpose of this research is to estimate the usage of infrared (IR) thermal camera in automatic quality control of polymer parts in the production lines compared to simulation. The evaluation of injected part properties monitored from an IR camera is studied in terms of the defect and mechanical properties. The optimisation of the processing parameters to increase mechanical properties and decrease injection defects is examined. There are four processing factors with four levels namely, melt temperature, mould temperature, injection pressure, and flow rate designed by using Taguchi’s experimental design. The effect of average temperature of injection moulded parts monitored by IR camera on defect and mechanical properties is determined. The variance (ANOVA) method is used to evaluate the influence of each factors. The parameter factors of all experiments are definitely different because of P-values less than 0.05. The optimum processing parameters of PP were a melting temperature of 193°C, mould temperature of 30°C, injection pressure of 50 bar, and flow rate of 40 cm3/s. The suitable processing conditions for ABS are melting temperature, mould temperature, injection pressure, and flow rate of 216°C, 50°C, 90 bar, and 20 cm3/s respectively. IR thermograph accurately monitors short shot, sink mark, and large flash in moulding part; moreover, IR camera can estimate the mechanical properties of ABS by average temperature but cannot accurately measure the mechanical properties of high glossy PP. It is meant that IR thermal imaging technique can automatically control the product quality in the injection moulding process.

Design of Automatic Pond Water Quality Control in Koi Fish Farm

Koi fish is a type of freshwater ornamental fish that has high economic value. Koi fish have attractive body colors and ideal body shapes so they have good sales prospects. However, koi fish farming has several obstacles, one of which is the low recovery rate and the relatively slow growth of fish. Water quality is the main parameter in the success of a fishing effort. Therefore, this study designed a tool that can help koi fish farmers in unifying the quality of pond water so that koi fish can grow and develop properly. This monitoring system is designed by implementing an automatic control monitoring system to produce good water quality for koi fish farming. Data obtained from water quality monitoring are parameters of pH, ammonia, and water temperature. The results of this cultivator's research can monitor the quality of fish pond water and immediately get water quality control automatically.

Optical character recognition on engineering drawings to achieve automation in production quality control

Introduction: Digitization is a crucial step towards achieving automation in production quality control for mechanical products. Engineering drawings are essential carriers of information for production, but their complexity poses a challenge for computer vision. To enable automated quality control, seamless data transfer between analog drawings and CAD/CAM software is necessary.Methods: This paper focuses on autonomous text detection and recognition in engineering drawings. The methodology is divided into five stages. First, image processing techniques are used to classify and identify key elements in the drawing. The output is divided into three elements: information blocks and tables, feature control frames, and the rest of the image. For each element, an OCR pipeline is proposed. The last stage is output generation of the information in table format.Results: The proposed tool, called eDOCr, achieved a precision and recall of 90% in detection, an F1-score of 94% in recognition, and a character error rate of 8%. The tool enables seamless integration between engineering drawings and quality control.Discussion: Most OCR algorithms have limitations when applied to mechanical drawings due to their inherent complexity, including measurements, orientation, tolerances, and special symbols such as geometric dimensioning and tolerancing (GD&amp;amp;T). The eDOCr tool overcomes these limitations and provides a solution for automated quality control.Conclusion: The eDOCr tool provides an effective solution for automated text detection and recognition in engineering drawings. The tool's success demonstrates that automated quality control for mechanical products can be achieved through digitization. The tool is shared with the research community through Github.

Demonstrating quality control procedures for fMRI in DPABI.

Quality control (QC) is an important stage for functional magnetic resonance imaging (fMRI) studies. The methods for fMRI QC vary in different fMRI preprocessing pipelines. The inflating sample size and number of scanning sites for fMRI studies further add to the difficulty and working load of the QC procedure. Therefore, as a constituent part of the Demonstrating Quality Control Procedures in fMRI research topic in Frontiers, we preprocessed a well-organized open-available dataset using DPABI pipelines to illustrate the QC procedure in DPABI. Six categories of DPABI-derived reports were used to eliminate images without adequate quality. After the QC procedure, twelve participants (8.6%) were categorized as excluded and eight participants (5.8%) were categorized as uncertain. More automatic QC tools were needed in the big-data era while visually inspecting images was still indispensable now.

Applying machine learning classifiers to automate quality assessment of paediatric dynamic susceptibility contrast (DSC-) MRI data.

Investigate the performance of qualitative review (QR) for assessing dynamic susceptibility contrast (DSC-) MRI data quality in paediatric normal brain and develop an automated alternative to QR. 1027 signal-time courses were assessed by Reviewer 1 using QR. 243 were additionally assessed by Reviewer 2 and % disagreements and Cohen's κ (κ) were calculated. The signal drop-to-noise ratio (SDNR), root mean square error (RMSE), full width half maximum (FWHM) and percentage signal recovery (PSR) were calculated for the 1027 signal-time courses. Data quality thresholds for each measure were determined using QR results. The measures and QR results trained machine learning classifiers. Sensitivity, specificity, precision, classification error and area under the curve from a receiver operating characteristic curve were calculated for each threshold and classifier. Comparing reviewers gave 7% disagreements and κ = 0.83. Data quality thresholds of: 7.6 for SDNR; 0.019 for RMSE; 3 s and 19 s for FWHM; and 42.9 and 130.4% for PSR were produced. SDNR gave the best sensitivity, specificity, precision, classification error and area under the curve values of 0.86, 0.86, 0.93, 14.2% and 0.83. Random forest was the best machine learning classifier, giving sensitivity, specificity, precision, classification error and area under the curve of 0.94, 0.83, 0.93, 9.3% and 0.89. The reviewers showed good agreement. Machine learning classifiers trained on signal-time course measures and QR can assess quality. Combining multiple measures reduces misclassification. A new automated quality control method was developed, which trained machine learning classifiers using QR results.