Automated Machine Learning Systems Research Articles

Combinatorial progressive architecture search for crowd counting

Recently, the automated machine learning system, including neural architecture search (NAS), has been introduced to the task of crowd counting. However, there are several concerns about applying existing AutoML methods to crowd counting: (i) The previous AutoML system automates the CNNs of crowd counting without acceleration design and ignores the reproducibility on edge computation circumstances; (ii) a simple concatenation of the fused multi-scale information cannot reflect the pixel-wise information; thus it demands a high-level combinatorial system to learn these singular pieces of information jointly. In this paper, we propose an efficient automation system to tackle the lightweight neural architecture search and combinatorial problem. At first, we design a combinatorial search space for the automatic combination of the singular information (tensors). We then introduce a progressive neural architecture search mechanism that progressively combines the information to be more simple and precise. For efficiency, the combination controller controls both the depth and width of the search space, acquiring simple shallow combinations first and complex and heavy ones later. Our mobile-based searched feature fusion architecture shows strong transferability with the use of the recurrent neural network as a controller and combinatorial dynamic system. The proposed combinatorial progressive architecture search (CPAS) algorithm achieves competitive results on benchmark datasets and sets the new state-of-the-art results on the UCF-CC 50 dataset. The proposed CPAS system acquires to-edge ability.

Evolutionary learning of selection hyper-heuristics for text classification

This paper introduces an evolutionary model in the scope of automated machine learning. This model is in charge of learning hyper-heuristics that represent selection rules of the form if-then, such that given a dataset for a text classification problem, the hyper-heuristics select the best classification method to use with it, based on the data distribution of the dataset. The evolutionary model starts by building a set of hyper-heuristics using a series of meta-features extracted from a training group of datasets that represent their data distribution. Hyper-heuristics are then evolved using adapted crossover and mutation operators. During the evolution, each hyper-heuristic is evaluated on its performance to classify each dataset in the training group. When the evolutionary process is done, the best hyper-heuristic is selected and evaluated for its generality with an independent test group of datasets. The results show that the best learned hyper-heuristic obtains an average classification performance close to the general optimum, and has a similar performance to the two most popular state-of-the-art automated machine learning systems, but with less computational cost. The approach used by the present model is relevant for automated machine learning in three aspects, the generality of the hyper-heuristics so they could be applied to groups of datasets; the interpretability of the representations that facilitate the understanding of the method selection by non-expert users; and the reduction of computational time and resources to reach a decision. Furthermore, the model extends the applicability of evolutionary computation methods, with their problem-independent properties and their ability to explore search spaces, to tackle new complex problems, such as the decision of the best classifier for a text classification dataset.

Screening native pepper, Tasmannia lanceolata (Poir.) A.C. Smith, for resistance against Phytophthora cinnamomi dieback

Tasmannia lanceolata (native pepper) has been reported as susceptible to Phytophthora cinnamomi and the objective of this study was to identify variability in native pepper resistance to P. cinnamomi. Plant material was collected from native pepper populations across Tasmania (four regions) and selected commercially grown cultivars, and 47 clones were successfully propagated. Two disease screening experiments were conducted in “soil-free plant growth system” (SPS) units. Native pepper roots were inoculated with P. cinnamomi zoospores and maintained in controlled conditions. After one week, the pathogen was re-isolated to confirm infection success, and after two weeks root discolouration was visually assessed with images, which were then analysed with an automated machine learning system, using colour thresholds. The SPS was a successful approach to screen the early response of native pepper to P. cinnamomi. Based on pathogen re-isolation success and total root discolouration percentage, clones were categorised using a susceptibility rating method across multiple categories from highly resistant to highly susceptible. In the first experiment, all 47 propagated clones were challenged with one isolate of P. cinnamomi (Pc1), and pathogen re-isolation percentage and total root discolouration (brown and black) percentage differed significantly with clone (P < 0.001). In the second experiment, three representative clones of experiment 1 were challenged with two isolates of P. cinnamomi (Pc1 and Pc2) and clone response was similar. This study has highlighted that there is a range of responses to P. cinnamomi, from highly susceptible to highly resistant, in native pepper clones from different regions of Tasmania.

TPOT-MTR: A Multiple Target Regression Based on Genetic Algorithm of Automated Machine Learning Systems

The concept that a cross correlation might improve prediction error underpins machine learning algorithms for multi-target regression (MTR). Numerous MTR approaches have been created in recent years, however there are still uncertainties concerning how their performances are impacted by dataset properties such as linearity, number of targets, and cross correlational complexity. In order to contribute to a better understanding of the relationship between dataset properties and MTR methods, authors proposed a new model of TPOT-MTR, which its result will be compared to previously generated 33 synthetic datasets with controlled characteristics and tested their performance against other two MTR methods, Random Forest and SVM. The results demonstrated that TPOT-MTR approaches could enhance performance even in datasets with non-linearly correlated targets, although the prediction improvement varies depending on the method and regressor combinations used.

Deep Active Learning for Computer Vision: Past and Future

As an important data selection schema, active learning emerges as the essential component when iterating an Artificial Intelligence (AI) model. It becomes even more critical given the dominance of deep neural network based models, which are composed of a large number of parameters and data hungry, in application. Despite its indispensable role for developing AI models, research on active learning is not as intensive as other research directions. In this paper, we present a review of active learning through deep active learning approaches from the following perspectives: 1) technical advancements in active learning, 2) applications of active learning in computer vision, 3) industrial systems leveraging or with potential to leverage active learning for data iteration, 4) current limitations and future research directions. We expect this paper to clarify the significance of active learning in a modern AI model manufacturing process and to bring additional research attention to active learning. By addressing data automation challenges and coping with automated machine learning systems, active learning will facilitate democratization of AI technologies by boosting model production at scale.

The Roles and Modes of Human Interactions with Automated Machine Learning Systems: A Critical Review and Perspectives

The Roles and Modes of Human Interactions with Automated Machine Learning Systems: A Critical Review and Perspectives

Automated Machine Learning System for Defect Detection on Cylindrical Metal Surfaces.

Metal workpieces are indispensable in the manufacturing industry. Surface defects affect the appearance and efficiency of a workpiece and reduce the safety of manufactured products. Therefore, products must be inspected for surface defects, such as scratches, dirt, and chips. The traditional manual inspection method is time-consuming and labor-intensive, and human error is unavoidable when thousands of products require inspection. Therefore, an automated optical inspection method is often adopted. Traditional automated optical inspection algorithms are insufficient in the detection of defects on metal surfaces, but a convolutional neural network (CNN) may aid in the inspection. However, considerable time is required to select the optimal hyperparameters for a CNN through training and testing. First, we compared the ability of three CNNs, namely VGG-16, ResNet-50, and MobileNet v1, to detect defects on metal surfaces. These models were hypothetically implemented for transfer learning (TL). However, in deploying TL, the phenomenon of apparent convergence in prediction accuracy, followed by divergence in validation accuracy, may create a problem when the image pattern is not known in advance. Second, our developed automated machine-learning (AutoML) model was trained through a random search with the core layers of the network architecture of the three TL models. We developed a retraining criterion for scenarios in which the model exhibited poor training results such that a new neural network architecture and new hyperparameters could be selected for retraining when the defect accuracy criterion in the first TL was not met. Third, we used AutoKeras to execute AutoML and identify a model suitable for a metal-surface-defect dataset. The performance of TL, AutoKeras, and our designed AutoML model was compared. The results of this study were obtained using a small number of metal defect samples. Based on TL, the detection accuracy of VGG-16, ResNet-50, and MobileNet v1 was 91%, 59.00%, and 50%, respectively. Moreover, the AutoKeras model exhibited the highest accuracy of 99.83%. The accuracy of the self-designed AutoML model reached 95.50% when using a core layer module, obtained by combining the modules of VGG-16, ResNet-50, and MobileNet v1. The designed AutoML model effectively and accurately recognized defective and low-quality samples despite low training costs. The defect accuracy of the developed model was close to that of the existing AutoKeras model and thus can contribute to the development of new diagnostic technologies for smart manufacturing.

Customized AutoML: An Automated Machine Learning System for Predicting Severity of Construction Accidents

Construction companies are under pressure to enhance their site safety condition, being constantly challenged by rapid technological advancements, growing public concern, and fierce competition. To enhance construction site safety, literature investigated Machine Learning (ML) approaches as risk assessment (RA) tools. However, their deployment requires knowledge for selecting, training, testing, and employing the most appropriate ML predictor. While different ML approaches are recommended by literature, their practicality at construction sites is constrained by the availability, knowledge, and experience of data scientists familiar with the construction sector. This study develops an automated ML system that automatically trains and evaluates different ML to select the most accurate ML-based construction accident severity predictors for the use of construction professionals with limited data science knowledge. A real-life accident dataset is evaluated through automated ML approaches: Auto-Sklearn, AutoKeras, and customized AutoML. The investigated AutoML approaches offer higher scalability, accuracy, and result-oriented severity insight due to their simple input requirements and automated procedures.

A decision support system for selecting the most suitable machine learning in healthcare using user parameters and requirements

The application of machine learning in the medical field is still limited. The main reason behind the lack of use is the unavailability of an easy-to-use machine learning system that targets non-technical users. The objective of this paper is to propose an automated machine learning system to aid non-technical users. The proposed system provides the user with simple choices to provide suggestions to the system. The system uses the combination of the user’s choices and performance evaluation to select the most suited model from available options.In this study, we employed the system on a Parkinson’s disease dataset. The templates for support vector machine and random forest algorithms are provided to the system. Support vector machines and random forests were able to produce 80% and 75% accuracy, respectively. The system used performance parameters of the system and user choices to select the most suited models for each test case. The support vector machine was selected as the most suited model in three test cases, while random forest was selected as the most suited for one test case. The test cases also showed that the weighted time parameter impacted the results heavily.

Deep learning for evaluating left atrium stress echocardiography: a proof of principle study

Abstract Introduction Left atrial (LA) size is a dynamic variable that changes during stress echocardiography (SE) and provides valuable information within and beyond coronary artery disease. However, its measurement remains subjective, time-consuming and based on manual tracings. Recent advances in deep learning might save analysis time and deflate variability by removing the subjectivity of LA assessment. Purpose In this proof of principle study we aim to validate the potential of an automated machine learning system for the evaluation of LA in SE. Methods From the image data bank of Stress Echo 2030 study, we selected 20 consecutive patients who underwent SE (using a variety of stress methods) in 2 recruiting centers. Imaging data was acquired in DICOM format and anonymized. The studies were reviewed by an expert cardiologist trained in SE evaluation who selected apical four chamber (A4Ch) view images during stress and rest phases and marked end-systolic (ES) and end-diastolic frames to be used in further evaluation. Endocardial borders for LA were traced in ES. The tracings were repeated by three different evaluators (expert cardiologist [C1], cardiologist from external center [C2] and a machine learning [ML] model – a convolutional neural network trained on an unrelated set of images). LA area (LAA), volume (LAV) using area-length formula and proportional LAV changes between stress and rest (ΔLAV = [rest − stress] / rest) were calculated. Each evaluator was blinded from each other's measurements. Results In total, 40 A4Ch images were acquired (20 at rest and 20 at stress) of which all were of sufficient quality for performing LA measurements using an automated system. Pearson correlation coefficients (R) for LAA were 0.95 (C1-ML), 0.96 (C2-ML) at rest and 0.88 (C1-ML), 0.79 (C2-ML) at stress. The C1-C2 pair had R of 0.98 and 0.86 for LAA at rest and stress. LAV also showed good correlation between different raters with R values of 0.90 (C1-ML), 0.94 (C2-ML), 0.94 (C1-C2) at rest and 0.86 (C1-ML), 0.87 (C2-ML), 0.84 (C1-C2) at stress. Root mean squared errors (RMSEs) for LAV were 13.48 ml (C1-ML), 7.44 ml (C2-ML), 10.26 ml (C1-C2) at rest and 14.03 ml (C1-ML), 8.53 ml (C2-ML), 13.69 ml (C1-C2) at stress. There were high level correlations between all raters for ΔLAV with R values of 0.93, 0.94 and 0.95 for C1-ML, C2-ML and C1-C2 pairs respectively. Mean (95% CI) ΔLAV values were −20.1 (−51.59 to +11.39), −14.95 (−43.77 to +13.87) and −14.37 (−34.46 to +5.72) for ML, C1 and C2. Comparison with one-way ANOVA did not show significant differences in mean ΔLAV values between operators (p=0.94). Conclusions Automated ML based system produces LA measurements that are comparable to human operators and can reduce the need for manual tracing. There was a tendency for all operators to have lower levels of agreement in stress images compared to rest, further showing the need for additional standardization of SE evaluation for machine and human operators. Funding Acknowledgement Type of funding sources: None.

Automated machine learning: the new data science challenge

The world is changing quite rapidly while increasingly tuning into digitalization. However, it is important to note that data science is what most technology is evolving around and data is definitely the future of everything. For industries, adopting a “data science approach” is no longer an option, it becomes an obligation in order to enhance their business rather than survive. This paper offers a roadmap for anyone interested in this research field or getting started with “machine learning” learning while enabling the reader to easily comprehend the key concepts behind. Indeed, it examines the benefits of automated machine learning systems, starting with defining machine learning vocabulary and basic concepts. Then, explaining how to, concretely, build up a machine learning model by highlighting the challenges related to data and algorithms. Finally, exposing a summary of two studies applying machine learning in two different fields, namely transportation for road traffic forecasting and supply chain management for demand prediction where the predictive performance of various models iscompared based on different metrics.

Machine-Identified Patterns of Visual Field Loss and an Association with Rapid Progression in the Ocular Hypertension Treatment Study

To identify patterns of visual field (VF) loss based on unsupervised machine learning and to identify patterns that are associated with rapid progression. Cross-sectional and longitudinal study. A total of 2231 abnormal VFs from 205 eyes of 176 Ocular Hypertension Treatment Study (OHTS) participants followed over approximately 16 years. Visual fields were assessed by an unsupervised deep archetypal analysis algorithm and an OHTS-certified VF reader to identify prevalent patterns of VF loss. Machine-identified patterns of glaucoma damage were compared against those patterns previously identified (expert-identified) in the OHTS in 2003. Based on the longitudinal VFs of each eye, VF loss patterns that were strongly associated with rapid glaucoma progression were identified. Machine-expert correspondence and type of patterns of VF loss associated with rapid progression. The average VF mean deviation (MD) at conversion to glaucoma was-2.7 decibels (dB) (standard deviation [SD]= 2.4 dB), whereas the average MD of the eyes at the last visit was-5.2 dB (SD= 5.5 dB). Fifty out of 205 eyes had MD rate of-1 dB/year or worse and were considered rapid progressors. Eighteen machine-identified patterns of VF loss were compared with expert-identified patterns, in which 13 patterns of VF loss were similar. The most prevalent expert-identified patterns included partial arcuate, paracentral, and nasal step defects, and the most prevalent machine-identified patterns included temporal wedge, partial arcuate, nasal step, and paracentral VF defects. One of the machine-identified patterns of VF loss predicted future rapid VF progression after adjustment for age, sex, and initial MD. An automated machine learning system can identify patterns of VF loss and could provide objective and reproducible nomenclature for characterizing early signs of visual defects and rapid progression in patients with glaucoma.

Credit Card Fraud Detection with Automated Machine Learning Systems

ABSTRACT The steady increase at the turnover of online trading during the last decade and the increasing use of credit cards has subsequently made credit card frauds more prevalent. Machine Learning (ML) models are among the most prominent techniques in detecting illicit transactions. In this paper, we apply the Just-Add-Data (JAD), a system that automates the selection of Machine Learning algorithms, the tuning of their hyper-parameter values, and the estimation of performance in detecting fraudulent transactions using a highly unbalanced dataset, swiftly providing prediction model for credit card fraud detection. The training of the model does not require the user setting up any of the methods’ (hyper)parameters. In addition, it is trivial to retrain the model with the arrival of new data, to visualize, interpret, and share the results at all management levels within a credit card organization, as well as to apply the model. The model selected by JAD identifies 32 out of a total of 39 fraudulent transactions of the test sample, with all missed fraudulent transactions being small transactions below 50€. The comparison with other methods on the same dataset reveals that all the above come with a high forecasting performance that matches the existing literature.

Dsa-PAML: a parallel automated machine learning system via dual-stacked autoencoder

Dsa-PAML: a parallel automated machine learning system via dual-stacked autoencoder

Use of automated artificial intelligence to predict the need for orthodontic extractions.

ObjectiveTo develop and explore the usefulness of an artificial intelligence system for the prediction of the need for dental extractions during orthodontic treatments based on gender, model variables, and cephalometric records.MethodsThe gender, model variables, and radiographic records of 214 patients were obtained from an anonymized data bank containing 314 cases treated by two experienced orthodontists. The data were processed using an automated machine learning software (Auto-WEKA) and used to predict the need for extractions.ResultsBy generating and comparing several prediction models, an accuracy of 93.9% was achieved for determining whether extraction is required or not based on the model and radiographic data. When only model variables were used, an accuracy of 87.4% was attained, whereas a 72.7% accuracy was achieved if only cephalometric information was used.ConclusionsThe use of an automated machine learning system allows the generation of orthodontic extraction prediction models. The accuracy of the optimal extraction prediction models increases with the combination of model and cephalometric data for the analytical process.

Forecasting conflict in africa with automated machine learning systems

ABSTRACT The ViEWS problem is to forecast changes in the level of state-based violence for each of the next six months at the PRIO-GRID and country level. For this competition and toward the goal of improving sub-national and country level forecasts, we experiment with combinations of automated machine learning (autoML) systems and limited datasets that emphasize the endogenous nature of conflict. Two core findings emerge: autoML improves predictive performance and the Dynamics model performs best. The data used for the Dynamics model is limited to measures of state-based violence built from the event-level violence data plus those describing the spatial and temporal structure of the data. The intent is to capture spatial and temporal conflict dynamics while not overfitting to exogenous factors, which is especially problematic with flexible autoML algorithms and the types of highly disaggregate data used here. At the PGM level, this model won the ViEWS competition for both “predictive accuracy” and “originality.” Beyond the ViEWS competition, we expect conflict forecasting models that couple advanced autoML systems with variables that reflect a diverse set of conflict dynamics to have high predictive performance, especially at sub-national and sub-annual aggregations.

Automated retention time prediction of new psychoactive substances in gas chromatography

New psychoactive substances (NPSs) are drugs that exhibit chemical structures and pharmacological actions similar to illegal drugs. Because the health hazards caused by these drugs of abuse have become a social problem, the quick identification of NPSs is necessary for rapid regulation. We developed an automated machine learning system to predict the retention time of the gas chromatography-mass spectrometry for NPSs. The use of an automated machine learning system and Mordred that calculates molecular descriptors enables accurate retention time prediction. Because the retention time prediction is crucial for identifying NPSs, our research can contribute to the prevention of health hazards.

Developing the platform model for problem solving of automated machine learning

The article discusses the current state of technologies for automated machine learning. The development trends and the nature of the distribution model - MLaaS - are defined. There is highlighted a number of problems of automating the machine learning process, such as: excessive simplification and specialization of tools, vagueness of implemented processes, lack of flexibility in the infrastructure hardware, using closed algorithms. As a partial or complete solution to them, we have proposed the architecture, consisting of separate modules: models, hybridizer, learning algorithms module, testing module, user support module, and a theoretical framework. The main feature of the given architecture is its modularity, transparency and encapsulation of components. Each module is described as a separate element, implemented as an independent microservice. The paper describes the benefits of applying the given approach to the implementation of automated machine learning systems, the need to implement the given or similar standards. For each of the modules, its purposes, the tasks it solves and the implemented functionality, as well as the data necessary for the functioning and their sources are described. A general diagram showing the flows of information exchange between modules is presented. The main scenarios for the resulting system operation, as well as ways of interacting with it and the result of its operation - the generated model - are described.

A Feature Driven Intelligent System for Neurodegenerative Disorder Detection: An Application on Speech Dataset for Diagnosis of Parkinson’s Disease

Parkinson’s disease (PD) is a prevalent, and progressive neurological disorder. Due to the motor and non-motor symptoms of the disease, it lowers life quality of the patients. Tremor, rigidity, depression, anxiety etc. are among the symptoms. Clinical diagnosis of PD is usually based on appearance of motor features. Additionally, different empirical tests were proposed by scholars for early detection of the disease. It is known that people with PD have speech impairments. Therefore, voice tests are used for early detection of the disease. In this study, an automated machine learning system was proposed for high accuracy classification of the speech signals of PD patients. The system includes feature reduction methods and classification algorithms. Feature reductions and classifications were performed for all participants, males, and females separately. Contributions of feature sets to classification accuracy were discussed. Experimental results were evaluated with different performance metrics. The proposed system obtained state of the art results in all categories. We acquired better performances for gender based classifications.

A Robust Automated Machine Learning System with Pseudoinverse Learning

Developing a robust deep neural network (DNN) for a specific task is not only time-consuming but also requires lots of experienced human experts. In order to make deep neural networks easier to apply or even take the human experts out of the design of network architecture completely, a growing number of researches focus on robust automated machine learning (AutoML). In this paper, we investigated the robustness problem of AutoML systems based on contractive pseudoinverse learners. In our proposed method, deep neural networks were built with stacked contractive pseudoinverse learners (CPILer). Each CPILer has a Jacobian regularized reconstruction loss function and is trained with pseudoinverse learning algorithm. When sigmoid activation function is adopted in the hidden layer, the graph Laplace regularizer is derived from square Frobenius norm of the Jacobian matrix. This learning scheme not only speeds up the training process dramatically but also reduces the effort of hyperparameter tuning. In addition, the graph Laplace regularization can improve the robustness of the learning systems by reducing the sensibility to noise. An ensemble network architecture consisting of several sub-networks was designed to build the AutoML systems. The architecture hyperparameters of the system were determined in an automated way which could be considered as a data-driven way. The proposed method shown good performance in the experiments in terms of efficiency and accuracy, and outperformed the baseline methods on a series of benchmark data sets. The robustness improvement of our proposed method was also demonstrated in the experiments.