Tree-based Pipeline Optimization Tool Research Articles

New Hybrid Approach for Developing Automated Machine Learning Workflows: A Real Case Application in Evaluation of Marcellus Shale Gas Production

The success of machine learning (ML) techniques implemented in different industries heavily rely on operator expertise and domain knowledge, which is used in manually choosing an algorithm and setting up the specific algorithm parameters for a problem. Due to the manual nature of model selection and parameter tuning, it is impossible to quantify or evaluate the quality of this manual process, which in turn limits the ability to perform comparison studies between different algorithms. In this study, we propose a new hybrid approach for developing machine learning workflows to help automated algorithm selection and hyperparameter optimization. The proposed approach provides a robust, reproducible, and unbiased workflow that can be quantified and validated using different scoring metrics. We have used the most common workflows implemented in the application of artificial intelligence (AI) and ML in engineering problems including grid/random search, Bayesian search and optimization, genetic programming, and compared that with our new hybrid approach that includes the integration of Tree-based Pipeline Optimization Tool (TPOT) and Bayesian optimization. The performance of each workflow is quantified using different scoring metrics such as Pearson correlation (i.e., R2 correlation) and Mean Square Error (i.e., MSE). For this purpose, actual field data obtained from 1567 gas wells in Marcellus Shale, with 121 features from reservoir, drilling, completion, stimulation, and operation is tested using different proposed workflows. A proposed new hybrid workflow is then used to evaluate the type well used for evaluation of Marcellus shale gas production. In conclusion, our automated hybrid approach showed significant improvement in comparison to other proposed workflows using both scoring matrices. The new hybrid approach provides a practical tool that supports the automated model and hyperparameter selection, which is tested using real field data that can be implemented in solving different engineering problems using artificial intelligence and machine learning. The new hybrid model is tested in a real field and compared with conventional type wells developed by field engineers. It is found that the type well of the field is very close to P50 predictions of the field, which shows great success in the completion design of the field performed by field engineers. It also shows that the field average production could have been improved by 8% if shorter cluster spacing and higher proppant loading per cluster were used during the frac jobs.

Application of Automated Machine Learning Based on Radiomics Features of T2WI and RS-EPI DWI to Predict Preoperative T Staging of Rectal Cancer

To explore the radiomics features of T2 weighted image (T2WI) and readout-segmented echo-planar imaging (RS-EPI) plus difusion-weighted imaging (DWI), to develop an automated mahchine-learning model based on the said radiomics features, and to test the value of this model in predicting preoperative T staging of rectal cancer. The study retrospectively reviewed 131 patients who were diagnosed with rectal cancer confirmed by the pathology results of their surgical specimens at West China Hospital of Sichuan University between October, 2017 and December, 2018. In addition, these patients had preoperative rectal MRI. Tumor regions from preoperative MRI were manually segmented by radiologists with the ITK-SNAP software from T2WI and RS-EPI DWI images. PyRadiomics was used to extract 200 features-100 from T2WI and 100 from the apparent diffusion coefficient (ADC) calculated from the RS-EPI DWI. MWMOTE and NEATER were used to resample and balance the dataset, and 13 cases of T 1-2 stage simulation cases were added. The overall dataset was divided into a training set (111 cases) and a test set (37 cases) by a ratio of 3∶1. Tree-based Pipeline Optimization Tool (TPOT) was applied on the training set to optimize model parameters and to select the most important radiomics features for modeling. Five independent T stage models were developed accordingly. Accuracy and the area under the curve ( AUC) of receiver operating characteristic (ROC) were used to pick out the optimal model, which was then applied on the training set and the original dataset to predict the T stage of rectal cancer. The performance of the the five T staging models recommended by automated machine learning were as follows: The accuracy for the training set ranged from 0.802 to 0.838, sensitivity, from 0.762 to 0.825, specificity, from 0.833 to 0.896, AUC, from 0.841 to 0.893, and average precision (AP) from 0.870 to 0.901. After comparison, an optimal model was picked out, with sensitivity, specificity and AUC for the training set reaching 0.810, 0.875, and 0.893, respectively. The sensitivity, specificity and AUC for the test set were 0.810, 0.813, and 0.810, respectively. The sensitivity, specificity and AUC for the original dataset were 0.810, 0.830, and 0.860, respectively. Based on the radiomics data of T2WI and RS-EPI DWI, the model established by automated machine learning showed a fairly high accuracy in predicting rectal cancer T stage.

Accelerating the Selection of Covalent Organic Frameworks with Automated Machine Learning.

Covalent organic frameworks (COFs) have the advantages of high thermal stability and large specific surface and have great application prospects in the fields of gas storage and catalysis. This article mainly focuses on COFs’ working capacity of methane (CH4). Due to the vast number of possible COF structures, it is time-consuming to use traditional calculation methods to find suitable materials, so it is important to apply appropriate machine learning (ML) algorithms to build accurate prediction models. A major obstacle for the use of ML algorithms is that the performance of an algorithm may be affected by many design decisions. Finding appropriate algorithm and model parameters is quite a challenge for nonprofessionals. In this work, we use automated machine learning (AutoML) to analyze the working capacity of CH4 based on 403,959 COFs. We explore the relationship between 23 features such as the structure, chemical characteristics, atom types of COFs, and the working capacity. Then, the tree-based pipeline optimization tool (TPOT) in AutoML and the traditional ML methods including multiple linear regression, support vector machine, decision tree, and random forest that manually set model parameters are compared. It is found that the TPOT can not only save complex data preprocessing and model parameter tuning but also show higher performance than traditional ML models. Compared with traditional grand canonical Monte Carlo simulations, it can save a lot of time. AutoML has broken through the limitations of professionals so that researchers in nonprofessional fields can realize automatic parameter configuration for experiments to obtain highly accurate and easy-to-understand results, which is of great significance for material screening.

Nuclear Reactor Transient Diagnostics Using Classification and AutoML

Artificial intelligence is becoming a larger part of operations for many industries. One industry where this is occurring rapidly is the nuclear industry. Researchers from around the world are looking to implement this technology in various areas of the nuclear industry. This paper explores the use of machine learning to diagnose problems. This project makes use of synthetic data collected from a Generic Pressurized Water Reactor (GPWR) simulator on whether a reactor is operating normally or experiencing one of four different transient events. A dataset was created consisting of over 30 000 reactor operational states. The data were explored and wrangled using Python and the Pandas package, using a variety of methods. Once ready, the data were randomly shuffled, with half the data being used for training and the other half being used for testing. Six different machine learning models were created using scikit-learn and the AutoML package Tree-based Pipeline Optimization Tool (TPOT). These models were created using six data scaling methods along with six feature reduction/selection methods. These models were validated using accuracy, precision, recall, and F1 score. The accuracy of the individual transients was also calculated. All six of the models had validation scores above 95%, with the decision tree and logistic regression models performing the best. These results are promising for the possible future use of machine learning in reactor diagnostics.

Indirect identification of wheel rail contact forces of an instrumented heavy haul railway vehicle using machine learning

Rail car safety and stability are important factors for good performance. The most used safety index to quantify railway safety is the ratio between the lateral and vertical wheel rail contact forces (L/Q). This study aims to use machine learning (ML) models to evaluate the viability of estimating the wheel rail contact forces from sensors in a regular instrumented railway vehicle (IRV). A virtual model of a BRA1 railway vehicle was created to generate an artificial dataset with variables measured with the actual BRA1 vehicle plus different variables measured by other IRVs found in the literature. The desired output variable is the L/Q ratio for each wheel of the leading bogie. Exploratory data analysis was done to clarify the correlation between the variables while model explainability techniques were applied to evaluate the contribution of these input variables to the output (L/Q). A total of 24 embedded machine learning models were trained and optimized using the tree-based pipeline optimization tool (TPOT) to generate a ML pipeline capable of producing an accurate L/Q ratio as the output for different cases of sampling rate, input variables and track irregularities. The results show that machine learning models can predict the wheel rail contact forces indirectly – without using instrumented wheelsets – with the highest mean squared error being equal to 0.01113.

A Continuous Cuffless Blood Pressure Estimation Using Tree-Based Pipeline Optimization Tool

High blood pressure (BP) may lead to further health complications if not monitored and controlled, especially for critically ill patients. Particularly, there are two types of blood pressure monitoring, invasive measurement, whereby a central line is inserted into the patient’s body, which is associated with infection risks. The second measurement is cuff-based that monitors BP by detecting the blood volume change at the skin surface using a pulse oximeter or wearable devices such as a smartwatch. This paper aims to estimate the blood pressure using machine learning from photoplethysmogram (PPG) signals, which is obtained from cuff-based monitoring. To avoid the issues associated with machine learning such as improperly choosing the classifiers and/or not selecting the best features, this paper utilized the tree-based pipeline optimization tool (TPOT) to automate the machine learning pipeline to select the best regression models for estimating both systolic BP (SBP) and diastolic BP (DBP) separately. As a pre-processing stage, notch filter, band-pass filter, and zero phase filtering were applied by TPOT to eliminate any potential noise inherent in the signal. Then, the automated feature selection was performed to select the best features to estimate the BP, including SBP and DBP features, which are extracted using random forest (RF) and k-nearest neighbors (KNN), respectively. To train and test the model, the PhysioNet global dataset was used, which contains 32.061 million samples for 1000 subjects. Finally, the proposed approach was evaluated and validated using the mean absolute error (MAE). The results obtained were 6.52 mmHg for SBS and 4.19 mmHg for DBP, which show the superiority of the proposed model over the related works.

Metabolomics Insights in Early Childhood Caries

Dental caries is characterized by a dysbiotic shift at the biofilm–tooth surface interface, yet comprehensive biochemical characterizations of the biofilm are scant. We used metabolomics to identify biochemical features of the supragingival biofilm associated with early childhood caries (ECC) prevalence and severity. The study’s analytical sample comprised 289 children ages 3 to 5 (51% with ECC) who attended public preschools in North Carolina and were enrolled in a community-based cross-sectional study of early childhood oral health. Clinical examinations were conducted by calibrated examiners in community locations using International Caries Detection and Classification System (ICDAS) criteria. Supragingival plaque collected from the facial/buccal surfaces of all primary teeth in the upper-left quadrant was analyzed using ultra-performance liquid chromatography–tandem mass spectrometry. Associations between individual metabolites and 18 clinical traits (based on different ECC definitions and sets of tooth surfaces) were quantified using Brownian distance correlations (dCor) and linear regression modeling of log2-transformed values, applying a false discovery rate multiple testing correction. A tree-based pipeline optimization tool (TPOT)–machine learning process was used to identify the best-fitting ECC classification metabolite model. There were 503 named metabolites identified, including microbial, host, and exogenous biochemicals. Most significant ECC-metabolite associations were positive (i.e., upregulations/enrichments). The localized ECC case definition (ICDAS ≥1 caries experience within the surfaces from which plaque was collected) had the strongest correlation with the metabolome (dCor P = 8 × 10−3). Sixteen metabolites were significantly associated with ECC after multiple testing correction, including fucose (P = 3.0 × 10−6) and N-acetylneuraminate (p = 6.8 × 10−6) with higher ECC prevalence, as well as catechin (P = 4.7 × 10−6) and epicatechin (P = 2.9 × 10−6) with lower. Catechin, epicatechin, imidazole propionate, fucose, 9,10-DiHOME, and N-acetylneuraminate were among the top 15 metabolites in terms of ECC classification importance in the automated TPOT model. These supragingival biofilm metabolite findings provide novel insights in ECC biology and can serve as the basis for the development of measures of disease activity or risk assessment.

Automated machine learning to predict the co-occurrence of isocitrate dehydrogenase mutations and O6 -methylguanine-DNA methyltransferase promoter methylation in patients with gliomas.

Combining isocitrate dehydrogenase mutation (IDHmut) with O6 -methylguanine-DNA methyltransferase promoter methylation (MGMTmet) has been identified as a critical prognostic molecular marker for gliomas. The aim of this study was to determine the ability of glioma radiomics features from magnetic resonance imaging (MRI) to predict the co-occurrence of IDHmut and MGMTmet by applying the tree-based pipeline optimization tool (TPOT), an automated machine learning (autoML) approach. This was a retrospective study, in which 162 patients with gliomas were evaluated, including 58 patients with co-occurrence of IDHmut and MGMTmet and 104 patients with other status comprising: IDH wildtype and MGMT unmethylated (n=67), IDH wildtype and MGMTmet (n=36), and IDHmut and MGMT unmethylated (n=1). Three-dimensional (3D) T1-weighted images, gadolinium-enhanced 3D T1-weighted images (Gd-3DT1WI), T2-weighted images, and fluid-attenuated inversion recovery (FLAIR) images acquired at 3.0 T were used. Radiomics features were extracted from FLAIR and Gd-3DT1WI images. The TPOT was employed to generate the best machine learning pipeline, which contains both feature selector and classifier, based on input feature sets. A 4-fold cross-validation was used to evaluate the performance of automatically generated models. For each iteration, the training set included 121 subjects, while the test set included 41 subjects. Student's t-test or a chi-square test was applied on different clinical characteristics between two groups. Sensitivity, specificity, accuracy, kappa score, and AUC were used to evaluate the performance of TPOT-generated models. Finally, we compared the above metrics of TPOT-generated models to identify the best-performing model. Patients' ages and grades between two groups were significantly different (p=0.002 and p=0.000, respectively). The 4-fold cross-validation showed that gradient boosting classifier trained on shape and textual features from the Laplacian-of-Gaussian-filtered Gd-3DT1 achieved the best performance (average sensitivity=81.1%, average specificity=94%, average accuracy=89.4%, average kappa score=0.76, average AUC=0.951). Using autoML based on radiomics features from MRI, a high discriminatory accuracy was achieved for predicting co-occurrence of IDHmut and MGMTmet in gliomas. LEVEL OF EVIDENCE: 3 TECHNICAL EFFICACY STAGE: 3.

Preoperative prediction of the stage, size, grade, and necrosis score in clear cell renal cell carcinoma using MRI-based radiomics.

Clear cell renal cell carcinoma (ccRCC) is the most common subtype of renal cell carcinoma. Currently, there is a lack of noninvasive methods to stratify ccRCC prognosis prior to any invasive therapies. The purpose of this study was to preoperatively predict the tumor stage, size, grade, and necrosis (SSIGN) score of ccRCC using MRI-based radiomics. A multicenter cohort of 364 histopathologically confirmed ccRCC patients (272 low [< 4] and 92 high [≥ 4] SSIGN score) with preoperative T2-weighted and T1-contrast-enhanced MRI were retrospectively identified and divided into training (254 patients) and testing sets (110 patients). The performance of a manually optimized radiomics model was assessed by measuring accuracy, sensitivity, specificity, area under receiver operating characteristic curve (AUROC), and area under precision-recall curve (AUPRC) on an independent test set, which was not included in model training. Lastly, its performance was compared to that of a machine learning pipeline, Tree-Based Pipeline Optimization Tool (TPOT). The manually optimized radiomics model using Random Forest classification and Analysis of Variance feature selection methods achieved an AUROC of 0.89, AUPRC of 0.81, accuracy of 0.89 (95% CI 0.816-0.937), specificity of 0.95 (95% CI 0.875-0.984), and sensitivity of 0.72 (95% CI 0.537-0.852) on the test set. The TPOT using Extra Trees Classifier achieved an AUROC of 0.94, AUPRC of 0.83, accuracy of 0.89 (95% CI 0.816-0.937), specificity of 0.95 (95% CI 0.875-0.984), and sensitivity of 0.72 (95% CI 0.537-0.852) on the test set. Preoperative MR radiomics can accurately predict SSIGN score of ccRCC, suggesting its promise as a prognostic tool that can be used in conjunction with diagnostic markers.

Differentiation of low and high grade renal cell carcinoma on routine MRI with an externally validated automatic machine learning algorithm

Pre-treatment determination of renal cell carcinoma aggressiveness may help guide clinical decision-making. We aimed to differentiate low-grade (Fuhrman I–II) from high-grade (Fuhrman III–IV) renal cell carcinoma using radiomics features extracted from routine MRI. 482 pathologically confirmed renal cell carcinoma lesions from 2008 to 2019 in a multicenter cohort were retrospectively identified. 439 lesions with information on Fuhrman grade from 4 institutions were divided into training and test sets with an 8:2 split for model development and internal validation. Another 43 lesions from a separate institution were set aside for independent external validation. The performance of TPOT (Tree-Based Pipeline Optimization Tool), an automatic machine learning pipeline optimizer, was compared to hand-optimized machine learning pipeline. The best-performing hand-optimized pipeline was a Bayesian classifier with Fischer Score feature selection, achieving an external validation ROC AUC of 0.59 (95% CI 0.49–0.68), accuracy of 0.77 (95% CI 0.68–0.84), sensitivity of 0.38 (95% CI 0.29–0.48), and specificity of 0.86 (95% CI 0.78–0.92). The best-performing TPOT pipeline achieved an external validation ROC AUC of 0.60 (95% CI 0.50–0.69), accuracy of 0.81 (95% CI 0.72–0.88), sensitivity of 0.12 (95% CI 0.14–0.30), and specificity of 0.97 (95% CI 0.87–0.97). Automated machine learning pipelines can perform equivalent to or better than hand-optimized pipeline on an external validation test non-invasively predicting Fuhrman grade of renal cell carcinoma using conventional MRI.

Embedding covariate adjustments in tree-based automated machine learning for biomedical big data analyses

BackgroundA typical task in bioinformatics consists of identifying which features are associated with a target outcome of interest and building a predictive model. Automated machine learning (AutoML) systems such as the Tree-based Pipeline Optimization Tool (TPOT) constitute an appealing approach to this end. However, in biomedical data, there are often baseline characteristics of the subjects in a study or batch effects that need to be adjusted for in order to better isolate the effects of the features of interest on the target. Thus, the ability to perform covariate adjustments becomes particularly important for applications of AutoML to biomedical big data analysis.ResultsWe developed an approach to adjust for covariates affecting features and/or target in TPOT. Our approach is based on regressing out the covariates in a manner that avoids ‘leakage’ during the cross-validation training procedure. We describe applications of this approach to toxicogenomics and schizophrenia gene expression data sets. The TPOT extensions discussed in this work are available at https://github.com/EpistasisLab/tpot/tree/v0.11.1-resAdj.ConclusionsIn this work, we address an important need in the context of AutoML, which is particularly crucial for applications to bioinformatics and medical informatics, namely covariate adjustments. To this end we present a substantial extension of TPOT, a genetic programming based AutoML approach. We show the utility of this extension by applications to large toxicogenomics and differential gene expression data. The method is generally applicable in many other scenarios from the biomedical field.

Performance of Automatic Machine Learning versus Radiologists in the Evaluation of Endometrium on Computed Tomography

Objectives: In this study, we developed radiomic models that utilize a combination of imaging features and clinical variables to distinguish endometrial cancer (EC) from non-EC diagnoses on computed tomography (CT). Methods: A total of 926 patients consisting of 416 EC and 510 non-EC diagnoses were included. Uterus and the endometrium were manually segmented on CT. Fourteen feature selection and ten classification methods were manually examined to select the most optimized machine learning pipeline. Automatic machine learning using Tree-Based Pipeline Optimization Tool (TPOT) was performed. 847 patients were portioned into training, validation, testing sets, and another 79 patients were as our external testing set. The performance of the machine learning pipelines on the testing sets was compared to radiologists. Results: There was significant difference in age between the EC and non-EC groups (64.0 vs. 53.7, p <0.001). The manual expert optimized pipeline using the “Relief” feature selection method and “Bagging” classifier on the external testing set achieved accuracy of 0.73 (95% CI: 0.62-0.82), sensitivity of 0.64 (95% CI: 0.45-0.79), and specificity of 0.78 (95% CI: 0.65-0.87), while TPOT achieved accuracy of 0.80 (95% CI: 0.70-0.87), sensitivity of 0.61 (95% CI: 0.43-0.77), and specificity of 0.90 (95% CI: 0.78-0.96). When compared to all radiologists averaged, the TPOT achieved higher test accuracy (0.80 vs. 0.49, p<0.001) and specificity (0.90 vs. 0.51, p<0.001), with comparable sensitivity (0.61 vs. 0.46, p=0.130). Conclusions: Our results demonstrate that automatic machine learning can distinguish EC from non-EC diagnoses on CT imaging with higher accuracy and specificity than radiologists. Funding Statement: This work was supported by the National Cancer Institute (NCI) of the National Institutes of Health under Award Number R03CA249554 to H. Bai, by a training grant from the National Institute of Biomedical Imaging and Bioengineering (NIBIB) of the National Institutes of Health (5T32EB1680), by the National Cancer Institute (NCI) of the National Institutes of Health (F30CA239407) to K. Chang and by the 111 project of ministry of education discipline innovation and wisdom base under grant Number B18059 to Beiji Zou. Declaration of Interests: The authors declare no potential conflicts of interest. Ethics Approval Statement: The study was approved by the Institutional Review Board (IRB) at the first and third institutions. With the agreement to use the second institution data, the IRB approval of our study was waived for the second institution.

Automated classification of a tropical landscape infested by Parthenium weed (Parthenium hyterophorus)

ABSTRACT The invasive Parthenium weed (Parthenium hyterophorus) adversely affects animal and human health, agricultural productivity, rural livelihoods, local and national economies, and the environment. Its fast spreading capability requires consistent monitoring for adoption of relevant mitigation approaches, potentially through remote sensing. To date, studies that have endeavoured to map the Parthenium weed have commonly used popular classification algorithms that include support vector machines and random forest classifiers, which do not capture the complex structural characteristics of the weed. Furthermore, determination of site or data specific algorithms, often achieved through intensive comparison of algorithms, is often laborious and time consuming. In addition, selected algorithms may not be optimal on datasets collected in other sites. Hence, this study adopted the Tree-based Pipeline Optimization Tool (TPOT), an automated machine learning approach that can be used to overcome high data variability during the classification process. Using Sentinel-2 and Land Satellite (Landsat) 8 imagery to map Parthenium weed, we compared the outcome of the TPOT to the best performing and optimized algorithm selected from sixteen classifiers on different training datasets. Results showed that the TPOT model yielded a higher overall classification accuracy (88.15%) using Sentinel-2 and 74% using Landsat 8, accuracies that were higher than the commonly used robust classifiers. This study is the first to demonstrate the value of TPOT in mapping Parthenium weed infestations using satellite imagery. Its adoption would therefore be useful in limiting human intervention while optimizing classification accuracies for mapping invasive plants. Based on these findings, we propose TPOT as an efficient method for selecting and tuning algorithms for Parthenium discrimination and monitoring, and indeed general vegetation mapping.

Automatic Machine Learning to Differentiate Pediatric Posterior Fossa Tumors on Routine MR Imaging.

Differentiating the types of pediatric posterior fossa tumors on routine imaging may help in preoperative evaluation and guide surgical resection planning. However, qualitative radiologic MR imaging review has limited performance. This study aimed to compare different machine learning approaches to classify pediatric posterior fossa tumors on routine MR imaging. This retrospective study included preoperative MR imaging of 288 patients with pediatric posterior fossa tumors, including medulloblastoma (n = 111), ependymoma (n = 70), and pilocytic astrocytoma (n = 107). Radiomics features were extracted from T2-weighted images, contrast-enhanced T1-weighted images, and ADC maps. Models generated by standard manual optimization by a machine learning expert were compared with automatic machine learning via the Tree-Based Pipeline Optimization Tool for performance evaluation. For 3-way classification, the radiomics model by automatic machine learning with the Tree-Based Pipeline Optimization Tool achieved a test micro-averaged area under the curve of 0.91 with an accuracy of 0.83, while the most optimized model based on the feature-selection method χ2 score and the Generalized Linear Model classifier achieved a test micro-averaged area under the curve of 0.92 with an accuracy of 0.74. Tree-Based Pipeline Optimization Tool models achieved significantly higher accuracy than average qualitative expert MR imaging review (0.83 versus 0.54, P < .001). For binary classification, Tree-Based Pipeline Optimization Tool models achieved an area under the curve of 0.94 with an accuracy of 0.85 for medulloblastoma versus nonmedulloblastoma, an area under the curve of 0.84 with an accuracy of 0.80 for ependymoma versus nonependymoma, and an area under the curve of 0.94 with an accuracy of 0.88 for pilocytic astrocytoma versus non-pilocytic astrocytoma. Automatic machine learning based on routine MR imaging classified pediatric posterior fossa tumors with high accuracy compared with manual expert pipeline optimization and qualitative expert MR imaging review.

Rapid identification of human ovarian cancer in second harmonic generation images using radiomics feature analyses and tree-based pipeline optimization tool.

Ovarian cancer is currently one of the most common cancers of the female reproductive organs, and its mortality rate is the highest among all types of gynecologic cancers. Rapid and accurate classification of ovarian cancer plays an important role in the determination of treatment plans and prognoses. Nevertheless, the most commonly used classification method is based on histopathological specimen examination, which is time-consuming and labor-intensive. Thus, in this study, we utilize radiomics feature extraction methods and the automated machine learning tree-based pipeline optimization tool (TOPT) for analysis of 3D, second harmonic generation images of benign, malignant and normal human ovarian tissues, to develop a high-efficiency computer-aided diagnostic model. Area under the receiver operating characteristic curve values of 0.98, 0.96 and 0.94 were obtained, respectively, for the classification of the three tissue types. Furthermore, this approach can be readily applied to other related tissues and diseases, and has great potential for improving the efficiency of medical diagnostic processes.

Self-Service Data Science in Healthcare with Automated Machine Learning

(1) Background: This work investigates whether and how researcher-physicians can be supported in their knowledge discovery process by employing Automated Machine Learning (AutoML). (2) Methods: We take a design science research approach and select the Tree-based Pipeline Optimization Tool (TPOT) as the AutoML method based on a benchmark test and requirements from researcher-physicians. We then integrate TPOT into two artefacts: a web application and a notebook. We evaluate these artefacts with researcher-physicians to examine which approach suits researcher-physicians best. Both artefacts have a similar workflow, but different user interfaces because of a conflict in requirements. (3) Results: Artefact A, a web application, was perceived as better for uploading a dataset and comparing results. Artefact B, a Jupyter notebook, was perceived as better regarding the workflow and being in control of model construction. (4) Conclusions: Thus, a hybrid artefact would be best for researcher-physicians. However, both artefacts missed model explainability and an explanation of variable importance for their created models. Hence, deployment of AutoML technologies in healthcare remains currently limited to the exploratory data analysis phase.

Machine-Learning-Accelerated Perovskite Crystallization

Summary Perovskites have seen significant research interest in the last decade. As ternary and quaternary compounds, their chemical space is exceptionally large, yet perovskite development has been limited to a restricted set of chemical constituents often discovered through trial and error. Here, we report a high-throughput experimental framework for the discovery of new perovskite single crystals. We use machine learning (ML) to guide the sequence of ever-improved robotic synthetic trials. We perform high-throughput syntheses of perovskite single crystals with a protein crystallization robot and characterize the outcomes with the aid of convolutional neural network-based image recognition. We then use an ML model to predict the optimal conditions for the synthesis of a new perovskite single crystal, enabling us to report the first synthesis of (3-PLA)2PbCl4.This material exhibits strong blue emission, illustrating the applicability of the method in identifying new optoelectronic materials.

3:09 PM Abstract No. 299 Automatic machine learning based on routine pre procedure magnetic resonance imaging to predict response of hepatocellular carcinoma following transcatheter arterial chemoembolization

The purpose of this study was to apply automatic machine learning on routine pre procedure MR to predict treatment response of patients with hepatocellular carcinoma (HCC) following transcatheter arterial chemoembolization (TACE). 212 patients with histologically confirmed singe-lesion HCC who only underwent one TACE procedure with available pre and post procedure MR scans were identified from one large academic institution from 2008 to 2018. The preoperative MR examination including T1-contrast-enhanced sequences (T1C) and T2-weighted sequence (T2WI) were manually segmented using 3D Slicer software. For each image space, 56 non-texture (morphology and intensity-based) and 94 texture features were extracted according to the guidelines defined by the Image Biomarker Standardization Initiative. Each of the 94 texture features were computed 32 times using all possible combinations of the following extraction parameters, a process known as “texture optimization”: (i) isotropic voxels of size 1 mm, 2 mm, 3 mm, and 4 mm, (ii) fixed bin number (FBN) discretization algorithm, with and without equalization, (iii) the number of gray levels of 8, 16, 32, and 64 for FBN for a total of (56 + 32*94), or 3064 radiomics features. All the features were normalized and features from T1C and T2WI were combined with 41 clinical variables to be inputted into TPOT, a Tree-Based Pipeline Optimization Tool that chooses the most optimal machine learning pipeline for an inputted dataset through genetic programming, to predict complete response (CR) vs. partial, stable or progressive disease (Non-CR) according to modified RECIST criteria. Among the 212 patients, 126 patients achieved CR and 86 non-CR. The dataset was randomly divided into 70% training and 30% testing set. After running TPOT (generation=50, population_size=100, cv=5, scoring=“balanced_accuracy”), the final cross-validation score was 0.68. The best model achieved validation accuracy of 75.0% (AUC=0.72) with 87.2% sensitivity and 0.56 specificity. Our preliminary results demonstrate automatic machine learning based on routine pre procedure MR can predict response after TACE with good accuracy. Further validation is needed in a larger multi-institutional cohort.

Model selection for metabolomics: predicting diagnosis of coronary artery disease using automated machine learning.

MotivationSelecting the optimal machine learning (ML) model for a given dataset is often challenging. Automated ML (AutoML) has emerged as a powerful tool for enabling the automatic selection of ML methods and parameter settings for the prediction of biomedical endpoints. Here, we apply the tree-based pipeline optimization tool (TPOT) to predict angiographic diagnoses of coronary artery disease (CAD). With TPOT, ML models are represented as expression trees and optimal pipelines discovered using a stochastic search method called genetic programing. We provide some guidelines for TPOT-based ML pipeline selection and optimization-based on various clinical phenotypes and high-throughput metabolic profiles in the Angiography and Genes Study (ANGES).ResultsWe analyzed nuclear magnetic resonance-derived lipoprotein and metabolite profiles in the ANGES cohort with a goal to identify the role of non-obstructive CAD patients in CAD diagnostics. We performed a comparative analysis of TPOT-generated ML pipelines with selected ML classifiers, optimized with a grid search approach, applied to two phenotypic CAD profiles. As a result, TPOT-generated ML pipelines that outperformed grid search optimized models across multiple performance metrics including balanced accuracy and area under the precision-recall curve. With the selected models, we demonstrated that the phenotypic profile that distinguishes non-obstructive CAD patients from no CAD patients is associated with higher precision, suggesting a discrepancy in the underlying processes between these phenotypes.Availability and implementationTPOT is freely available via http://epistasislab.github.io/tpot/.Supplementary information Supplementary data are available at Bioinformatics online.

Automated machine learning based on radiomics features predicts H3 K27M mutation in midline gliomas of the brain.

Conventional MRI cannot be used to identify H3 K27M mutation status. This study aimed to investigate the feasibility of predicting H3 K27M mutation status by applying an automated machine learning (autoML) approach to the MR radiomics features of patients with midline gliomas. This single-institution retrospective study included 100 patients with midline gliomas, including 40 patients with H3 K27M mutations and 60 wild-type patients. Radiomics features were extracted from fluid-attenuated inversion recovery images. Prior to autoML analysis, the dataset was randomly stratified into separate 75% training and 25% testing cohorts. The Tree-based Pipeline Optimization Tool (TPOT) was applied to optimize the machine learning pipeline and select important radiomics features. We compared the performance of 10 independent TPOT-generated models based on training and testing cohorts using the area under the curve (AUC) and average precision to obtain the final model. An independent cohort of 22 patients was used to validate the best model. Ten prediction models were generated by TPOT, and the accuracy obtained with the best pipeline ranged from 0.788 to 0.867 for the training cohort and from 0.60 to 0.84 for the testing cohort. After comparison, the AUC value and average precision of the final model were 0.903 and 0.911 in the testing cohort, respectively. In the validation set, the AUC was 0.85, and the average precision was 0.855 for the best model. The autoML classifier using radiomics features of conventional MR images provides high discriminatory accuracy in predicting the H3 K27M mutation status of midline glioma.