AutoML Methods Research Articles

A quantitative causal analysis and optimization framework for inclusions of steel products

Inclusions generated during steelmaking and continuous casting processes could seriously reduce the mechanical properties of steel products and greatly increase the risk of corrosion failure. Much attention has been paid to perceiving the origin and classification of inclusions based on the process knowledge and mechanism model, while the value of industrial big data is underestimated. Machine learning (ML) methods have demonstrated potential in the processing of nonlinear and strong coupling relationships in industrial data. However, existing ML models are frequently unable to analyze causal relationships between manufacturing process, which is essential for the quality optimization. To address the limitation, this paper proposes a quantitative causal analysis and optimization framework. This framework starts with a feature selection module using propensity score matching (PSM) to measure the quantitative and reliable causal relationship between the process parameters and inclusions. Subsequently, a high-performance AutoML method, AutoGluon-Tabular (AGT), is applied to quantify the optimization range and verify the optimization effect. Specifically, AGT is used to estimate the inclusion level in each window of the process parameter to be optimized, choose the best window of which to determine adjustment strategies, and eventually verify the change of inclusion levels after optimization. The comparative experiments based on the real data collected from a steel company demonstrate that the proposed framework reduces the inclusion rate by 8.85%, which is effective, efficient and has guiding significance for the inclusions control of steel products.

Combinatorial progressive architecture search for crowd counting

Recently, the automated machine learning system, including neural architecture search (NAS), has been introduced to the task of crowd counting. However, there are several concerns about applying existing AutoML methods to crowd counting: (i) The previous AutoML system automates the CNNs of crowd counting without acceleration design and ignores the reproducibility on edge computation circumstances; (ii) a simple concatenation of the fused multi-scale information cannot reflect the pixel-wise information; thus it demands a high-level combinatorial system to learn these singular pieces of information jointly. In this paper, we propose an efficient automation system to tackle the lightweight neural architecture search and combinatorial problem. At first, we design a combinatorial search space for the automatic combination of the singular information (tensors). We then introduce a progressive neural architecture search mechanism that progressively combines the information to be more simple and precise. For efficiency, the combination controller controls both the depth and width of the search space, acquiring simple shallow combinations first and complex and heavy ones later. Our mobile-based searched feature fusion architecture shows strong transferability with the use of the recurrent neural network as a controller and combinatorial dynamic system. The proposed combinatorial progressive architecture search (CPAS) algorithm achieves competitive results on benchmark datasets and sets the new state-of-the-art results on the UCF-CC 50 dataset. The proposed CPAS system acquires to-edge ability.

Combining Multi-View UAV Photogrammetry, Thermal Imaging, and Computer Vision Can Derive Cost-Effective Ecological Indicators for Habitat Assessment

Recent developments in Unmanned Aircraft Vehicles (UAVs), thermal imaging, and Auto-machine learning (AutoML) have shown high potential for precise wildlife surveys but have rarely been studied for habitat assessment. Here, we propose a framework that leverages these advanced techniques to achieve cost-effective habitat quality assessment from the perspective of actual wildlife community usage. The framework exploits vision intelligence hidden in the UAV thermal images and AutoML methods to achieve cost-effective wildlife distribution mapping, and then derives wildlife use indicators to imply habitat quality variance. We conducted UAV-based thermal wildlife surveys at three wetlands in the Rainwater Basin, Nebraska. Experiments were set to examine the optimal protocols, including various flight designs (61 and 122 m), feature types, and AutoML. The results showed that UAV images collected at 61 m with a spatial resolution of 7.5 cm, combined with Faster R-CNN, returned the optimal wildlife mapping (more than 90% accuracy). Results also indicated that the vision intelligence exploited can effectively transfer the redundant AutoML adaptation cycles into a fully automatic process (with around 33 times efficiency improvement for data labeling), facilitating cost-effective AutoML adaptation. Eventually, the derived ecological indicators can explain the wildlife use status well, reflecting potential within- and between-habitat quality variance.

Untrained and Unmatched: Fast and Accurate Zero-Training Classification for Tabular Engineering Data

Abstract In engineering design, navigating complex decision-making landscapes demands a thorough exploration of the design, performance, and constraint spaces, often impeded by resource-intensive simulations. Data-driven methods can mitigate this challenge by harnessing historical data to delineate feasible domains, accelerate optimization, or evaluate designs. However, the implementation of these methods usually demands machine learning expertise and multiple trials to choose the right method and hyperparameters. This makes them less accessible for numerous engineering situations. Additionally, there is an inherent trade-off between training speed and accuracy, with faster methods sometimes compromising precision. In our paper, we demonstrate that a recently released general-purpose transformer-based classification model, TabPFN, is both fast and accurate. Notably, it requires no dataset-specific training to assess new tabular data. TabPFN is a prior-data fitted network, which undergoes a one-time offline training across a broad spectrum of synthetic datasets and performs in-context learning. We evaluated TabPFN’s efficacy across eight engineering design classification problems, contrasting it with seven other algorithms, including a state-of-the-art automated machine learning (AutoML) method. For these classification challenges, TabPFN consistently outperforms in speed and accuracy. It is also the most data-efficient and provides the added advantage of being differentiable and giving uncertainty estimates. Our findings advocate for the potential of pre-trained models that learn from synthetic data and require no domain-specific tuning to make data-driven engineering design accessible to a broader community and open ways to efficient general-purpose models valid across applications. Furthermore, we share a benchmark problem set for evaluating new classification algorithms in engineering design.

A Survey on AutoML Methods and Systems for Clustering

Automated Machine Learning (AutoML) aims to identify the best-performing machine learning algorithm along with its input parameters for a given dataset and a specific machine learning task. This is a challenging problem, as the process of finding the best model and tuning it for a particular problem at hand is both time-consuming for a data scientist and computationally expensive. In this survey, we focus on unsupervised learning, and we turn our attention on AutoML methods for clustering. We present a systematic review that includes many recent research works for automated clustering. Furthermore, we provide a taxonomy for the classification of existing works, and we perform a qualitative comparison. As a result, this survey provides a comprehensive overview of the field of AutoML for clustering. Moreover, we identify open challenges for future research in this field.

Susceptibility of AutoML mortality prediction algorithms to model drift caused by the COVID pandemic

BackgroundConcept drift and covariate shift lead to a degradation of machine learning (ML) models. The objective of our study was to characterize sudden data drift as caused by the COVID pandemic. Furthermore, we investigated the suitability of certain methods in model training to prevent model degradation caused by data drift.MethodsWe trained different ML models with the H2O AutoML method on a dataset comprising 102,666 cases of surgical patients collected in the years 2014–2019 to predict postoperative mortality using preoperatively available data. Models applied were Generalized Linear Model with regularization, Default Random Forest, Gradient Boosting Machine, eXtreme Gradient Boosting, Deep Learning and Stacked Ensembles comprising all base models. Further, we modified the original models by applying three different methods when training on the original pre-pandemic dataset: (1) we weighted older data weaker, (2) used only the most recent data for model training and (3) performed a z-transformation of the numerical input parameters. Afterwards, we tested model performance on a pre-pandemic and an in-pandemic data set not used in the training process, and analysed common features.ResultsThe models produced showed excellent areas under receiver-operating characteristic and acceptable precision-recall curves when tested on a dataset from January-March 2020, but significant degradation when tested on a dataset collected in the first wave of the COVID pandemic from April-May 2020. When comparing the probability distributions of the input parameters, significant differences between pre-pandemic and in-pandemic data were found. The endpoint of our models, in-hospital mortality after surgery, did not differ significantly between pre- and in-pandemic data and was about 1% in each case. However, the models varied considerably in the composition of their input parameters. None of our applied modifications prevented a loss of performance, although very different models emerged from it, using a large variety of parameters.ConclusionsOur results show that none of our tested easy-to-implement measures in model training can prevent deterioration in the case of sudden external events. Therefore, we conclude that, in the presence of concept drift and covariate shift, close monitoring and critical review of model predictions are necessary.

AutoSR4EO: An AutoML Approach to Super-Resolution for Earth Observation Images

Super-resolution (SR), a technique to increase the resolution of images, is a pre-processing step in the pipelines of applications of Earth observation (EO) data. The manual design and optimisation of SR models that are specific to every possible EO use case is a laborious process that creates a bottleneck for EO analysis. In this work, we develop an automated machine learning (AutoML) method to automate the creation of dataset-specific SR models. AutoML is the study of the automatic design of high-performance machine learning models. We present the following contributions. (i) We propose AutoSR4EO, an AutoML method for automatically constructing neural networks for SR. We design a search space based on state-of-the-art residual neural networks for SR and incorporate transfer learning. Our search space is extendable, making it possible to adapt AutoSR4EO to future developments in the field. (ii) We introduce a new real-world single-image SR (SISR) dataset, called SENT-NICFI. (iii) We evaluate the performance of AutoSR4EO on four different datasets against the performance of four state-of-the-art baselines and a vanilla AutoML SR method, with AutoSR4EO achieving the highest average ranking. Our results show that AutoSR4EO performs consistently well over all datasets, demonstrating that AutoML is a promising method for improving SR techniques for EO images.

Using Auto-ML on Synthetic Point Cloud Generation

Automated Machine Learning (Auto-ML) has primarily been used to optimize network hyperparameters or post-processing parameters, while the most critical component for training a high-quality model, the dataset, is usually left untouched. In this paper, we introduce a novel approach that applies Auto-ML methods to the process of generating synthetic datasets for training machine learning models. Our approach addresses the problem that generating synthetic datasets requires a complex data generator, and that developing and tuning a data generator for a specific scenario is a time-consuming and expensive task. Being able to reuse this data generator for multiple purposes would greatly reduce the effort and cost, once the process of tuning it to the specific domains of each task is automated. To demonstrate the potential of this idea, we have implemented a point cloud generator for simple scenes. The scenes from this generator can be used to train a neural network to semantically segment cars from the background. The simple composition of the scene allows us to reuse the generator for several different semantic segmentation tasks. The models trained on the datasets with the optimized domain parameters easily outperform a model without such optimizations, while the optimization effort is minimal due to our Auto-ML approach. Although the development of such complex data generators requires considerable effort, we believe that using Auto-ML for dataset creation has the potential to speed up the development of machine learning applications in domains where high-quality labeled data is difficult to obtain.

Groundwater contaminant source identification considering unknown boundary condition based on an automated machine learning surrogate

Groundwater contamination source identification (GCSI) is a prerequisite for contamination risk evaluation and efficient groundwater contamination remediation programs. The boundary condition generally is set as known variables in previous GCSI studies. However, in many practical cases, the boundary condition is complicated and cannot be estimated accurately in advance. Setting the boundary condition as known variables may seriously deviate from the actual situation and lead to distorted identification results. And the results of GCSI are affected by multiple factors, including contaminant source information, model parameters, boundary condition, etc. Therefore, if the boundary condition is not estimated accurately, other factors will also be estimated inaccurately. This study focuses on the unknown boundary condition and proposed to identify three types of unknown variables (contaminant source information, model parameters and boundary condition) innovatively. When simulation–optimization (S-O) method is applied to GCSI, the huge computational load is usually reduced by building surrogate models. However, when building surrogate models, the researchers need to select the models and optimize the hyperparameters to make the model powerful, which can be a lengthy process. The automated machine learning (AutoML) method was used to build surrogate model, which automates the model selection and hyperparameter optimization in machine learning engineering, largely reducing human operations and saving time. The accuracy of AutoML surrogate model is compared with the surrogate model used in eXtreme Gradient Boosting method (XGBoost), random forest method (RF), extra trees regressor method (ETR) and elasticnet method (EN) respectively, which are automatically selected in AutoML engineering. The results show that the surrogate model constructed by AutoML method has the best accuracy compared with the other four methods. This study provides reliable and strong support for GCSI.

Explainable Remaining Tool Life Prediction for Individualized Production Using Automated Machine Learning.

The increasing demand for customized products is a core driver of novel automation concepts in Industry 4.0. For the case of machining complex free-form workpieces, e.g., in die making and mold making, individualized manufacturing is already the industrial practice. The varying process conditions and demanding machining processes lead to a high relevance of machining domain experts and a low degree of manufacturing flow automation. In order to increase the degree of automation, online process monitoring and the prediction of the quality-related remaining cutting tool life is indispensable. However, the varying process conditions complicate this as the correlation between the sensor signals and tool condition is not directly apparent. Furthermore, machine learning (ML) knowledge is limited on the shop floor, preventing a manual adaption of the models to changing conditions. Therefore, this paper introduces a new method for remaining tool life prediction in individualized production using automated machine learning (AutoML). The method enables the incorporation of machining expert knowledge via the model inputs and outputs. It automatically creates end-to-end ML pipelines based on optimized ensembles of regression and forecasting models. An explainability algorithm visualizes the relevance of the model inputs for the decision making. The method is analyzed and compared to a manual state-of-the-art approach for series production in a comprehensive evaluation using a new milling dataset. The dataset represents gradual tool wear under changing workpieces and process parameters. Our AutoML method outperforms the state-of-the-art approach and the evaluation indicates that a transfer of methods designed for series production to variable process conditions is not easily possible. Overall, the new method optimizes individualized production economically and in terms of resources. Machining experts with limited ML knowledge can leverage their domain knowledge to develop, validate and adapt tool life models.

Classification model for student dropouts using machine learning: A case study

Information and communication technologies have been fulfilling a highly relevant role in the different fields of knowledge, addressing problems in various disciplines; there is an increased capacity to identify patterns and anomalies in an organization's data using data mining; In this context, the study aimed to develop a classification model for student dropout, applying machine learning with the autoML method of the H2O.ai framework; the dimensionality of the socioeconomic and academic characteristics has been taken into account, with the purpose that the directors make reasonable decisions to counteract the abandonment of the students in the study programs. The methodology used was of a technological type, purposeful level, incremental innovation, temporal scope, and synchronous; data collection was prospective. For this, a 20-item questionnaire was applied to 237 students enrolled in the master's degree programs in the education of the Graduate School. The research resulted in a supervised machine learning model, Gradient Reinforcement Machine (GBM), to classify student dropout, thus identifying the main associated factors that influence dropout, obtaining a Gini coefficient of 92.20%, AUC of 96.10% and a LogLoss of 24.24% representing a model with efficient performance.

BioAutoMATED: An end-to-end automated machine learning tool for explanation and design of biological sequences.

The design choices underlying machine-learning (ML) models present important barriers to entry for many biologists who aim to incorporate ML in their research. Automated machine-learning (AutoML) algorithms can address many challenges that come with applying ML to the life sciences. However, these algorithms are rarely used in systems and synthetic biology studies because they typically do not explicitly handle biological sequences (e.g., nucleotide, amino acid, or glycan sequences) and cannot be easily compared with other AutoML algorithms. Here, we present BioAutoMATED, an AutoML platform for biological sequence analysis that integrates multiple AutoML methods into a unified framework. Users are automatically provided with relevant techniques for analyzing, interpreting, and designing biological sequences. BioAutoMATED predicts gene regulation, peptide-drug interactions, and glycan annotation, and designsoptimized synthetic biology components, revealing salient sequence characteristics. By automating sequence modeling, BioAutoMATED allows life scientists to incorporate ML more readily into their work.

AutoML for estimating grass height from ETM+/OLI data from field measurements at a nature reserve

ABSTRACT Remote sensing (RS) is now a standard tool used for grassland monitoring thanks to the availability of data at an unprecedented spatial and temporal resolution. The approaches to monitor grasslands often rely on the use of vegetation indices (e.g. NDVI) and empirical models trained on field data collected in tandem with the RS data. The best combination of models and features is often found by ad-hoc experimentation by the expert. This “classic” approach does not necessarily result in the best possible model. Automatic machine learning (AutoML) allows to automate this procedure by identifying the best possible pipeline in a data-driven manner. This study assesses the applicability of two distinct AutoML algorithms – AutoSklearn and AutoGluon – to monitor grass height from RS data and to systematically compare them to “classic” RS approaches. Grass height was estimated from Landsat ETM+ and OLI for a well-known conservation area as a case study. The “classic” RS approach emulated all possible ad hoc decisions by comparing all combinations of bands and vegetation indices against a naive use of the AutoML systems. While model selection and optimization are automated within AutoML models, for the “classic” RS approach, we used Bayesian optimization for hyperparameter tuning. We found that AutoML methods outperformed “classic” methods with the test error varying between ~1.73 cm ± 0.02 and ~1.78 cm ± 0.03 while for the “classic” methods it varied between ~1.84 cm ± 0.03 and ~2.81 cm ± 0.02. In the case of the “classic” methods, our exhaustive exploration of the possible feature combinations showed that while vegetation indices were always selected for the best models, which index got selected depended on the algorithm. The performance of AutoML compared to “classic” RS approaches clearly demonstrates the ability of these methods to quickly and effectively identify high-performing models. However, as this work focused on a single case-study, the results cannot be directly generalized to other study areas. Nevertheless, it provided a number of insights into future research opportunities to improve the use of AutoML in RS.

AutoML Loss Landscapes

As interest in machine learning and its applications becomes more widespread, how to choose the best models and hyper-parameter settings becomes more important. This problem is known to be challenging for human experts, and consequently, a growing number of methods have been proposed for solving it, giving rise to the area of automated machine learning (AutoML). Many of the most popular AutoML methods are based on Bayesian optimization, which makes only weak assumptions about how modifying hyper-parameters effects the loss of a model. This is a safe assumption that yields robust methods, as the AutoML loss landscapes that relate hyper-parameter settings to loss are poorly understood. We build on recent work on the study of one-dimensional slices of algorithm configuration landscapes by introducing new methods that test n -dimensional landscapes for statistical deviations from uni-modality and convexity, and we use them to show that a diverse set of AutoML loss landscapes are highly structured. We introduce a method for assessing the significance of hyper-parameter partial derivatives, which reveals that most (but not all) AutoML loss landscapes only have a small number of hyper-parameters that interact strongly. To further assess hyper-parameter interactions, we introduce a simplistic optimization procedure that assumes each hyper-parameter can be optimized independently, a single time in sequence, and we show that it obtains configurations that are statistically tied with optimal in all of the n -dimensional AutoML loss landscapes that we studied. Our results suggest many possible new directions for substantially improving the state of the art in AutoML.

Application of Automated Machine Learning (AutoML) Method in Wind Turbine Fault Detection

Fault detection has drew much attention nowadays, as it can save time and operational maintenance costs, especially in the wind turbine (WT) that is becoming familiar with renewable energy. Machine learning became widespread use in fault detection methods. However, most available machine learning needs more data and much time to train. Therefore, there is a need to detect faults using a few data during the training process. This paper aims to apply Automated Machine Learning (AutoML) method for fault detection in WT systems. The fault detection in the WT system focuses on the internal stator fault in the generator as it is the main part of the WT. The AutoML model was developed using a neural network (NN) algorithm in python based on the Auto-Keras model. The model was developed using four inputs, i.e. stator and rotor currents in the d-q axis (Iqs, Ids, Iqr and Idr ) while the outputs are impedance values, i.e. stator resistance, Rs , and stator inductance, Ls . The WT system used in this research is the doubly-fed induction generator (DFIG) in MATLAB/Simulink. In the Auto-Keras model, the impedance values (Rs and L s) indicated the condition of the DFIG, either normal or fault conditions. Two fault types were applied to the WT system, i.e. inter-turn short circuit and open circuit fault. The Auto-Keras model was trained and tested with the various values of data. The accuracy and the root means square error (RMSE) value of the model were calculated. The result shows that the accuracy is high as it is more than 93% in most conditions, and the RMSE value is low, close to the zero value. Applying the AutoML method in fault detection of the WT system shows its capability to identify faults accurately.

Dialectal Arabic sentiment analysis based on tree-based pipeline optimization tool

The heavy involvement of the Arabic internet users resulted in spreading data written in the Arabic language and creating a vast research area regarding natural language processing (NLP). Sentiment analysis is a growing field of research that is of great importance to everyone considering the high added potential for decision-making and predicting upcoming actions using the texts produced in social networks. Arabic used in microblogging websites, especially Twitter, is highly informal. It is not compliant with neither standards nor spelling regulations making it quite challenging for automatic machine-learning techniques. In this paper’s scope, we propose a new approach based on AutoML methods to improve the efficiency of the sentiment classification process for dialectal Arabic. This approach was validated through benchmarks testing on three different datasets that represent three vernacular forms of Arabic. The obtained results show that the presented framework has significantly increased accuracy than similar works in the literature.

Groundwater Potential Mapping in Hubei Region of China Using Machine Learning, Ensemble Learning, Deep Learning and AutoML Methods

Groundwater Potential Mapping in Hubei Region of China Using Machine Learning, Ensemble Learning, Deep Learning and AutoML Methods

Exploring and comparing various machine and deep learning technique algorithms to detect domain generation algorithms of malicious variants

Domain generation algorithm (DGA) is used as the main source of script in different groups of malwares, which generates the domain names of points and will further be used for command-and-control servers. The security measures usually identify the malware but the domain name algorithms will be updating themselves in order to avoid the less efficient older security detection methods. The reason being the older detection methods does not use either the machine learning or deep learning algorithms to detect the DGAs. Thus, the impact of incorporating the machine learning and deep learning techniques to detect the DGA is well discussed. As a result, they can create a huge number of domains to avoid debar and henceforth, block the hackers and zombie systems with the older methods itself. The main purpose of this research work is to compare and analyse by implementing various machine learning algorithms that suits the respective dataset yielding better results. In this research paper, the obtained dataset is pre-processed and the respective data is processed by different machine learning algorithms such as random forest (RF), support vector machine (SVM), Naive Bayes classifier, H20 AutoML, convolutional neural network (CNN), long short-term memory neural network (LSTM) for the classification. It is observed and understood that the LSTM provides a better classification efficiency of 98% and the H20 AutoML method giving the least efficiency of 75%.

Exploring and comparing various machine and deep learning technique algorithms to detect domain generation algorithms of malicious variants

Domain generation algorithm (DGA) is used as the main source of script in different groups of malwares, which generates the domain names of points and will further be used for command-and-control servers. The security measures usually identify the malware but the domain name algorithms will be updating themselves in order to avoid the less efficient older security detection methods. The reason being the older detection methods does not use either the machine learning or deep learning algorithms to detect the DGAs. Thus, the impact of incorporating the machine learning and deep learning techniques to detect the DGA is well discussed. As a result, they can create a huge number of domains to avoid debar and henceforth, block the hackers and zombie systems with the older methods itself. The main purpose of this research work is to compare and analyse by implementing various machine learning algorithms that suits the respective dataset yielding better results. In this research paper, the obtained dataset is pre-processed and the respective data is processed by different machine learning algorithms such as random forest (RF), support vector machine (SVM), Naive Bayes classifier, H20 AutoML, convolutional neural network (CNN), long short-term memory neural network (LSTM) for the classification. It is observed and understood that the LSTM provides a better classification efficiency of 98% and the H20 AutoML method giving the least efficiency of 75%.

Natural Language Processing for Imaging Protocol Assignment: Machine Learning for Multiclass Classification of Abdominal CT Protocols Using Indication Text Data

A correct protocol assignment is critical to high-quality imaging examinations, and its automation can be amenable to natural language processing (NLP). Assigning protocols for abdominal imaging CT scans is particularly challenging given the multiple organ specific indications and parameters. We compared conventional machine learning, deep learning, and automated machine learning builder workflows for this multiclass text classification task. A total of 94,501 CT studies performed over 4 years and their assigned protocols were obtained. Text data associated with each study including the ordering provider generated free text study indication and ICD codes were used for NLP analysis and protocol class prediction. The data was classified into one of 11 abdominal CT protocol classes before and after augmentations used to account for imbalances in the class sample sizes. Four machine learning (ML) algorithms, one deep learning algorithm, and an automated machine learning (AutoML) builder were used for the multilabel classification task: Random Forest (RF), Tree Ensemble (TE), Gradient Boosted Tree (GBT), multi-layer perceptron (MLP), Universal Language Model Fine-tuning (ULMFiT), and Google’s AutoML builder (Alphabet, Inc., Mountain View, CA), respectively. On the unbalanced dataset, the manually coded algorithms all performed similarly with F1 scores of 0.811 for RF, 0.813 for TE, 0.813 for GBT, 0.828 for MLP, and 0.847 for ULMFiT. The AutoML builder performed better with a F1 score of 0.854. On the balanced dataset, the tree ensemble machine learning algorithm performed the best with an F1 score of 0.803 and a Cohen’s kappa of 0.612. AutoML methods took a longer time for completion of NLP model training and evaluation, 4 h and 45 min compared to an average of 51 min for manual methods. Machine learning and natural language processing can be used for the complex multiclass classification task of abdominal imaging CT scan protocol assignment.