Automated Machine Learning Pipeline Research Articles

Evaluation of Deep Learning Estimation of Whole Heart Anatomy from Automated Cardiovascular Magnetic Resonance Short- and Long-Axis Analyses in UK Biobank.

Standard methods of heart chamber volume estimation in cardiovascular magnetic resonance (CMR) typically utilize simple geometric formulae based on a limited number of slices. We aimed to evaluate whether an automated deep learning neural network prediction of 3D anatomy of all four chambers would show stronger associations with cardiovascular risk factors and disease than standard volume estimation methods in the UK Biobank. A deep learning network was adapted to predict 3D segmentations of left and right ventricles (LV, RV) and atria (LA, RA) at ∼1mm isotropic resolution from CMR short and long axis 2D segmentations obtained from a fully automated machine learning pipeline in 4723 individuals with cardiovascular disease (CVD) and 5733 without in the UK Biobank. Relationships between volumes at end-diastole (ED) and end-systole (ES) and risk/disease factors were quantified using univariate, multivariate and logistic regression analyses. Strength of association between deep learning volumes and standard volumes was compared using the area under the receiving operator characteristic curve (AUC). Univariate and multivariate associations between deep learning volumes and most risk and disease factors were stronger than for standard volumes (higher R2 and more significant P values), particularly for sex, age, and body mass index. AUC for all logistic regressions were higher for deep learning volumes than standard volumes (p<0.001 for all four chambers at ED and ES). Neural network reconstructions of whole heart volumes had significantly stronger associations with cardiovascular disease and risk factors than standard volume estimation methods in an automatic processing pipeline.

Computational kinematics of dance: distinguishing hip hop genres.

Dance plays a vital role in human societies across time and culture, with different communities having invented different systems for artistic expression through movement (genres). Differences between genres can be described by experts in words and movements, but these descriptions can only be appreciated by people with certain background abilities. Existing dance notation schemes could be applied to describe genre-differences, however they fall substantially short of being able to capture the important details of movement across a wide spectrum of genres. Our knowledge and practice around dance would benefit from a general, quantitative and human-understandable method of characterizing meaningful differences between aspects of any dance style; a computational kinematics of dance. Here we introduce and apply a novel system for encoding bodily movement as 17 macroscopic, interpretable features, such as expandedness of the body or the frequency of sharp movements. We use this encoding to analyze Hip Hop Dance genres, in part by building a low-cost machine-learning classifier that distinguishes genre with high accuracy. Our study relies on an open dataset (AIST++) of pose-sequences from dancers instructed to perform one of ten Hip Hop genres, such as Breakdance, Popping, or Krump. For comparison we evaluate moderately experienced human observers at discerning these sequence's genres from movements alone (38% where chance = 10%). The performance of a baseline, Ridge classifier model was fair (48%) and that of the model resulting from our automated machine learning pipeline was strong (76%). This indicates that the selected features represent important dimensions of movement for the expression of the attitudes, stories, and aesthetic values manifested in these dance forms. Our study offers a new window into significant relations of similarity and difference between the genres studied. Given the rich, complex, and culturally shaped nature of these genres, the interpretability of our features, and the lightweight techniques used, our approach has significant potential for generalization to other movement domains and movement-related applications.

Using Machine Learning to Predict Outcomes of Patients with Blunt Traumatic Aortic Injuries.

The optimal management of blunt thoracic aortic injury (BTAI) remains controversial, with experienced centers offering therapy ranging from medical management to TEVAR. We investigated the utility of a machine learning (ML) algorithm to develop a prognostic model of risk factors on mortality in patients with BTAI. The Aortic Trauma Foundation registry was utilized to examine demographics, injury characteristics, management and outcomes of patients with BTAI. A STREAMLINE (A Simple, Transparent, End-To-End Automated Machine Learning Pipeline Facilitating Data Analysis and Algorithm Comparison) model as well as logistic regression (LR) analysis with imputation using chained equations was developed and compared. From a total of 1018 patients in the registry, 702 patients were included in the final analysis. Of the 258 (37%) patients who were medically managed, 44 (17%) died during admission, 14 (5.4%) of which were aortic related deaths. 444 (63%) patients underwent TEVAR and 343 of which underwent TEVAR within 24 hours of admission. Amongst TEVAR patients, 39 (8.8%) patients died and 7 (1.6%) had aortic related deaths. (Table 1) Comparison of the STREAMLINE and LR model showed no significant difference in ROC curves and high AUCs of 0.869 (95% CI, 0.813 - 0.925) and 0.840 (95% CI, 0.779 - 0.900) respectively in predicting in-hospital mortality. Unexpectedly, however, the variables prioritized in each model differed between models (Figure 1A-B). The top three variables identified from the LR model were similar to that from existing literature. The STREAMLINE model, however, prioritized location of the injury along the lesser curve, age and aortic injury grade (Figure 1A). Machine learning provides insight on prioritization of variables not typically identified in standard multivariable logistic regression. Further investigation and validation in other aortic injury cohorts are needed to delineate the utility of ML models. Level IIIStudy TypeOriginal research (prognostic/epidemiological).

Expert-augmented machine learning to accelerate the discovery of copolymers for anion exchange membrane

We constructed an automated machine learning pipeline to screen the broad chemical space of candidate copolymers for anion exchange membranes (AEMs). The pipeline utilizes a genetic algorithm integrated twelve machine learning algorithms, to screen copolymers that are potentially capable to improve AEM performance through the evolution of polymer hydrophobic and hydrophilic backbones and cation groups. An exhaustive data mining accumulated 749 experimentally reported AEMs, then the pipeline automatically generated over 172 million hypothetical copolymers, and 2519 potential candidates were screened out according to their predicted performance metrics including the OH− conductivity, the conductivity – dimensional stability trade-off coefficient, and the LUMO of cations in Pareto frontier set. Expert experience was adopted in the construction of robust regression and classification predictive models, and the importance of physical, chemical and topological features were illustrated from Shapley additive explanations (SHAP) analysis. We have released a standalone software that integrates these predictive models and the screened copolymer candidates with predicted AEM performance metrics at https://github.com/polySML/polySML-AEM. This study can facilitate the development of advanced AEMs through rational design of copolymers.

Machine Learning Pipeline for Predicting Bone Marrow Edema Along the Sacroiliac Joints on Magnetic Resonance Imaging.

We aimed to develop and validate a fully automated machine learning (ML) algorithm that predicts bone marrow edema (BME) on a quadrant level in sacroiliac (SI) joint magnetic resonance imaging (MRI). A computer vision workflow automatically locates the SI joints, segments regions of interest (ilium and sacrum), performs objective quadrant extraction, and predicts presence of BME, suggestive of inflammatory lesions, on a quadrant level in semicoronal slices of T1/T2-weighted MRI scans. Ground truth was determined by consensus among human readers. The inflammation classifier was trained using a ResNet18 backbone and five-fold cross-validated on scans of patients with spondyloarthritis (SpA) (n=279), postpartum individuals (n=71), and healthy subjects (n=114). Independent SpA patient MRI scans (n=243) served as test data set. Patient-level predictions were derived from aggregating quadrant-level predictions, ie, at least one positive quadrant. The algorithm automatically detects the SI joints with a precision of 98.4% and segments ilium/sacrum with an intersection over union of 85.6% and 67.9%, respectively. The inflammation classifier performed well in cross-validation: area under the curve (AUC) 94.5%, balanced accuracy (B-ACC) 80.5%, and F1 score 64.1%. In the test data set, AUC was 88.2%, B-ACC 72.1%, and F1 score 50.8%. On a patient level, the model achieved a B-ACC of 81.6% and 81.4% in the cross-validation and test data set, respectively. We propose a fully automated ML pipeline that enables objective and standardized evaluation of BME along the SI joints on MRI. This method has the potential to screen large numbers of patients with (suspected) SpA and is a step closer towards artificial intelligence-assisted diagnosis and follow-up.

The development of an automated machine learning pipeline for the detection of Alzheimer’s Disease

Although Alzheimer’s disease is the most prevalent form of dementia, there are no treatments capable of slowing disease progression. A lack of reliable disease endpoints and/or biomarkers contributes in part to the absence of effective therapies. Using machine learning to analyze EEG offers a possible solution to overcome many of the limitations of current diagnostic modalities. Here we develop a logistic regression model with an accuracy of 81% that addresses many of the shortcomings of previous works. To our knowledge, no other study has been able to solve the following problems simultaneously: (1) a lack of automation and unbiased removal of artifacts, (2) a dependence on a high level of expertise in data pre-processing and ML for non-automated processes, (3) the need for very large sample sizes and accurate EEG source localization using high density systems, (4) and a reliance on black box ML approaches such as deep neural nets with unexplainable feature selection. This study presents a proof-of-concept for an automated and scalable technology that could potentially be used to diagnose AD in clinical settings as an adjunct to conventional neuropsychological testing, thus enhancing efficiency, reproducibility, and practicality of AD diagnosis.

Automated machine learning pipeline for geochemical analysis

Biplot diagrams are traditionally used for rock discrimination using geochemical data from samples. However, this approach has limitations when facing a high number of variables. Machine learning has been proposed as an alternative to analyze multivariate data for more than 70 years. However, the application of machine learning by geoscientists is still complicated since there are no tools that propose a pipeline that can be followed from preparing the data to evaluating the models. Automated machine learning aims to face this issue by automating the creation and evaluation of machine learning models. The contribution of this work is twofold. First, we propose a methodology that follows a pipeline for the application of supervised and unsupervised learning to geochemical data. Both methods were applied to a dataset of granitic rock samples from 6 blocks in the Peninsular Ranges and the Transverse Ranges Provinces in Southern California. For supervised learning, the Decision Trees model offered the best values to classify the samples from this region: accuracy: 87%; precision: 89%; recall: 89%; and F-score: 81%. For unsupervised learning, 2 components were related to pressure effects, and another 2 could be related to water effects. As a second contribution, we propose a web application that follows the proposed methodology to analyze geochemical data using automated machine learning. It allows data preparation using techniques such as imputation and upsampling, the application of supervised and unsupervised learning, and the evaluation of the models. All this without the need to program.

Automated Machine Learning Pipeline Framework for Classification of Pediatric Functional Nausea Using High-Resolution Electrogastrogram.

Pediatric functional nausea is challenging for patients to manage and for clinicians to treat since it lacks objective diagnosis and assessment. A data-driven non-invasive diagnostic screening tool that distinguishes the electro-pathophysiology of pediatric functional nausea from healthy controls would be an invaluable aid to support clinical decision-making in diagnosis and management of patient treatment methodology. The purpose of this paper is to present an innovative approach for objectively classifying pediatric functional nausea using cutaneous high-resolution electrogastrogram data. We present an Automated Electrogastrogram Data Analytics Pipeline framework and demonstrate its use in a 3x8 factorial design to identify an optimal classification model according to a defined objective function. Low-fidelity synthetic high-resolution electrogastrogram data were generated to validate outputs and determine SOBI-ICA noise reduction effectiveness. A 10 parameter support vector machine binary classifier with a radial basis function kernel was selected as the overall top-performing model from a pool of over 1000 alternatives via maximization of an objective function. This resulted in a 91.6% test ROC AUC score. Using an automated machine learning pipeline approach to process high-resolution electrogastrogram data allows for clinically significant objective classification of pediatric functional nausea. To our knowledge, this is the first study to demonstrate clinically significant performance in the objective classification of pediatric nausea patients from healthy control subjects using experimental high-resolution electrogastrogram data. These results indicate a promising potential for high-resolution electrogastrography to serve as a data-driven screening tool for the objective diagnosis of pediatric functional nausea.

빔서치 기반 전력산업용 머신러닝 자동화 파이프라인 시스템

As Artificial Intelligence (AI) shows excellent performances, electric power industry also applies AI to various fields. Even though a lot of data and infrastructures for data analysis are prepared, AI experts in the industry are insufficient. Automated machine learning can be a solution in the industry to apply the excellence of AI to many fields despite the shortage of professionals. Recently, several automated machine learning services have shown good performances on general purpose. However, due to the specificity of data, they does not perform well in the electric power industry. Therefore, we develop an automated machine learning pipeline system based on beam search especially for electric power industry. The proposed system is applied to three real-word problems, one for each of regression, classification, and text classification. The significance of our work is first to apply automated machine learning to electric power industry with high performance. The performances of the models to predict hourly peak power demand and to detect illegal use of electricity for Bitcoin mining are improved by 3.42% and 3.23% respectively compared to the existing models. Moreover, our work shows 95.23% accuracy in classification of the questions’ indents of chatbot, which gives the possibility to replace the existing model.

Combining an Integrated Sensor Array with Machine Learning for the Simultaneous Quantification of Multiple Cations in Aqueous Mixtures.

The direct quantification of multiple ions in aqueous mixtures is achieved by combining an automated machine learning pipeline with transient potentiometric data obtained from a single miniaturized array of polymeric sensors electrodeposited on a conventional printed circuit board (PCB) substrate. A proof-of-concept system was demonstrated by employing 16 polymeric sensors in combination with features extracted from the transient differential voltages produced by these sensors when transitioning from a reference solution to a test solution, thereby obviating the need for a conventional reference electrode. A tree-based regression model enabled concentrations of various metal cations in pure solutions to be determined in less than 2 min. In a model mixture comprising Al3+, Cu2+, Na+, and Fe3+, the mean relative error was found to depend on the type of ion and varied between 1% for Fe3+ and 44% for Na+ in the concentration range 1-10 mg/L. Overall, a mean relative error of 16% was obtained for quantification of these four ions across a total of 124 tests in different solutions spanning concentrations between 2 and 360 mg/L. These results demonstrate how the analytical capability of a multiselective sensor array can leverage data-driven approaches through training by examples for accelerated testing and can be proposed to complement traditional analytical tools to meet industrial demands, including traceability of chemicals.

Chemometric approaches to resolving base oil mixtures.

In the lubrication industry, commercial base oils are commonly made up of blends of base oil stocks from different sources in different ratios to reduce production costs and modulate rheological properties. This practice introduces complexity in lubricant design because as the chemistry of the base oil becomes more complicated, it can become harder to formulate the base oil - particularly when the ratio of the original base oil stocks is unknown. In this study, field ionisation mass spectrometry is used to collect chemical information on a range of base oil mixtures. The resultant data are processed within the Python workspace where molecular formulae are assigned to the components and statistical analyses are performed. A variety of regression techniques including regularised linear models and automated machine learning are evaluated on the data. The use of an automated machine learning pipeline yields insight into effective modelling strategies that could be applied to the data obtained. The best results were obtained using polynomial feature generation combined with ridge cross-validation regression. Overall, with this methodology it is possible to resolve the ratio of group 2 and group 3 base oil within a blended mixture to an accuracy of ±5%. The strategies outlined in this study show how modern data science and chemometrics can be applied successfully to resolve the ratio of a complex mixture.

Prediction of Complete Remission and Survival in Acute Myeloid Leukemia Using Supervised Machine Learning

Prediction of Complete Remission and Survival in Acute Myeloid Leukemia Using Supervised Machine Learning

Automated Machine Learning Pipeline for Traffic Count Prediction

Research indicates that the projection of traffic volumes is a valuable tool for traffic management. However, few studies have examined the application of a universal automated framework for car traffic volume prediction. Within this limited literature, studies using broad data sets and inclusive predictors have been inadequate; such works have not incorporated a comprehensive set of linear and nonlinear algorithms utilizing a robust cross-validation approach. The proposed model pipeline introduced in this study automatically identifies the most appropriate feature-selection method and modeling approach to reduce the mean absolute percentage error. We utilized hyperparameter optimization to generate a universal automated framework, distinct from model optimization techniques that rely on a single case study. The resulting model can be independently customized to any respective project. Automating much of this process minimizes the work and expertise required for traffic count forecasting. To test the applicability of our models, we used Florida historical traffic data from between 2001 and 2017. The results confirmed that nonlinear models outperformed linear models in predicting passenger vehicles’ monthly traffic volumes in this specific case study. By employing the framework developed in this study, transportation planners could identify the critical links on US roads that incur overcapacity issues.

AutoWeka4MCPS-AVATAR: Accelerating automated machine learning pipeline composition and optimisation

Automated machine learning pipeline (ML) composition and optimisation aim at automating the process of finding the most promising ML pipelines within allocated resources (i.e., time, CPU and memory). Existing methods, such as Bayesian-based and genetic-based optimisation, which are implemented in Auto-Weka, Auto-sklearn and TPOT, evaluate pipelines by executing them. Therefore, the pipeline composition and optimisation of these methods frequently require a tremendous amount of time that prevents them from exploring complex pipelines to find better predictive models. To further explore this research challenge, we have conducted experiments showing that many of the generated pipelines are invalid in the first place, and attempting to execute them is a waste of time and resources. To address this issue, we propose a novel method to evaluate the validity of ML pipelines, without their execution, using a surrogate model (AVATAR). The AVATAR generates a knowledge base by automatically learning the capabilities and effects of ML algorithms on datasets’ characteristics. This knowledge base is used for a simplified mapping from an original ML pipeline to a surrogate model which is a Petri net based pipeline. Instead of executing the original ML pipeline to evaluate its validity, the AVATAR evaluates its surrogate model constructed by capabilities and effects of the ML pipeline components and input/output simplified mappings. Evaluating this surrogate model is less resource-intensive than the execution of the original pipeline. As a result, the AVATAR enables the pipeline composition and optimisation methods to evaluate more pipelines by quickly rejecting invalid pipelines. We integrate the AVATAR into the sequential model-based algorithm configuration (SMAC). Our experiments show that when SMAC employs AVATAR, it finds better solutions than on its own. This is down to the fact that the AVATAR can evaluate more pipelines within the same time budget and allocated resources.

Developing an ML pipeline for asthma and COPD: The case of a Dutch primary care service

A complex combination of clinical, demographic and lifestyle parameters determines the correct diagnosis and the most effective treatment for asthma and Chronic Obstructive Pulmonary Disease patients. Artificial Intelligence techniques help clinicians in devising the correct diagnosis and designing the most suitable clinical pathway accordingly, tailored to the specific patient conditions. In the case of machine learning (ML) approaches, availability of real-world patient clinical data to train and evaluate the ML pipeline deputed to assist clinicians in their daily practice is crucial. However, it is common practice to exploit either synthetic data sets or heavily preprocessed collections cleaning and merging different data sources. In this paper, we describe an automated ML pipeline designed for a real-world data set including patients from a Dutch primary care service, and provide a performance comparison of different prediction models for (i) assessing various clinical parameters, (ii) designing interventions, and (iii) defining the diagnosis.

Using hospital Admission, Discharge & Transfer (ADT) data for predicting readmissions

Readmission of patients within a specific period after their discharge from a hospital is a cause of concern for the healthcare industry due to the cost involved. Most of the work done for predicting such readmissions using machine learning (ML) have been based on EHR, claims or authorization data from specific sources, which are mostly snapshot data at one static point in time and hence delayed. ADT being dynamic as the data is available instantaneous on occurrence of a medical event/visit adds value. Our goal is to utilize machine learning on unlabeled ADT data to identify patients who are at a high risk of being readmitted. We approached the problem in three parts. First, we labeled patient events using logical rules and finalized one of many readmission definitions that was more encapsulating of varied scenarios. Second, feature engineering was done which encapsulates the longitudinal timeline of each patient in a representative way considering all the contextual information. Third, we developed an automated machine learning pipeline which takes modeling inputs from the user, runs various models to generate readmission prediction, does a cross validation and returns the best model. We tried multiple combinations of models and cross-validation strategies and decided on a random forest model with specific hyper-parameter values and to be the most effective method to classify high risk patients. It had a test AUC-ROC of 72% which is better than quite a few industry standards. The model currently implemented in the client environment identifies the high-risk patients in real-time to care nurses who in turn take proper interventions to reduce their chances of readmission.

Quantification of Multi-Ion Mixtures Using a Machine Learning Assisted Integrated Electronic Tongue Leveraging Mobile and Cloud Platforms

Quantification of Multi-Ion Mixtures Using a Machine Learning Assisted Integrated Electronic Tongue Leveraging Mobile and Cloud Platforms

A large-scale comparative study on peptide encodings for biomedical classification.

Owing to the great variety of distinct peptide encodings, working on a biomedical classification task at hand is challenging. Researchers have to determine encodings capable to represent underlying patterns as numerical input for the subsequent machine learning. A general guideline is lacking in the literature, thus, we present here the first large-scale comprehensive study to investigate the performance of a wide range of encodings on multiple datasets from different biomedical domains. For the sake of completeness, we added additional sequence- and structure-based encodings. In particular, we collected 50 biomedical datasets and defined a fixed parameter space for 48 encoding groups, leading to a total of 397 700 encoded datasets. Our results demonstrate that none of the encodings are superior for all biomedical domains. Nevertheless, some encodings often outperform others, thus reducing the initial encoding selection substantially. Our work offers researchers to objectively compare novel encodings to the state of the art. Our findings pave the way for a more sophisticated encoding optimization, for example, as part of automated machine learning pipelines. The work presented here is implemented as a large-scale, end-to-end workflow designed for easy reproducibility and extensibility. All standardized datasets and results are available for download to comply with FAIR standards.

Machine learning for automated EEG-based biomarkers of cognitive impairment during Deep Brain Stimulation screening in patients with Parkinson’s Disease

ObjectiveA downside of Deep Brain Stimulation (DBS) for Parkinson’s Disease (PD) is that cognitive function may deteriorate postoperatively. Electroencephalography (EEG) was explored as biomarker of cognition using a Machine Learning (ML) pipeline. MethodsA fully automated ML pipeline was applied to 112 PD patients, taking EEG time-series as input and predicted class-labels as output. The most extreme cognitive scores were selected for class differentiation, i.e. best vs. worst cognitive performance (n = 20 per group). 16,674 features were extracted per patient; feature-selection was performed using a Boruta algorithm. A random forest classifier was modelled; 10-fold cross-validation with Bayesian optimization was performed to ensure generalizability. The predicted class-probabilities of the entire cohort were compared to actual cognitive performance. ResultsBoth groups were differentiated with a mean accuracy of 0.92; using only occipital peak frequency yielded an accuracy of 0.67. Class-probabilities and actual cognitive performance were negatively linearly correlated (β = −0.23 (95% confidence interval (−0.29, −0.18))). ConclusionsParticularly high accuracies were achieved using a compound of automatically extracted EEG biomarkers to classify PD patients according to cognition, rather than a single spectral EEG feature. SignificanceAutomated EEG assessment may have utility for cognitive profiling of PD patients during the DBS screening.

Differentiation of low and high grade renal cell carcinoma on routine MRI with an externally validated automatic machine learning algorithm

Pre-treatment determination of renal cell carcinoma aggressiveness may help guide clinical decision-making. We aimed to differentiate low-grade (Fuhrman I–II) from high-grade (Fuhrman III–IV) renal cell carcinoma using radiomics features extracted from routine MRI. 482 pathologically confirmed renal cell carcinoma lesions from 2008 to 2019 in a multicenter cohort were retrospectively identified. 439 lesions with information on Fuhrman grade from 4 institutions were divided into training and test sets with an 8:2 split for model development and internal validation. Another 43 lesions from a separate institution were set aside for independent external validation. The performance of TPOT (Tree-Based Pipeline Optimization Tool), an automatic machine learning pipeline optimizer, was compared to hand-optimized machine learning pipeline. The best-performing hand-optimized pipeline was a Bayesian classifier with Fischer Score feature selection, achieving an external validation ROC AUC of 0.59 (95% CI 0.49–0.68), accuracy of 0.77 (95% CI 0.68–0.84), sensitivity of 0.38 (95% CI 0.29–0.48), and specificity of 0.86 (95% CI 0.78–0.92). The best-performing TPOT pipeline achieved an external validation ROC AUC of 0.60 (95% CI 0.50–0.69), accuracy of 0.81 (95% CI 0.72–0.88), sensitivity of 0.12 (95% CI 0.14–0.30), and specificity of 0.97 (95% CI 0.87–0.97). Automated machine learning pipelines can perform equivalent to or better than hand-optimized pipeline on an external validation test non-invasively predicting Fuhrman grade of renal cell carcinoma using conventional MRI.