Machine Learning Components Research Articles

A Two-Phase Machine Learning Approach for Predictive Maintenance of Low Voltage Industrial Motors

Predictive maintenance and sound operating industrial equipment are essential for nearly any production plant. The absence of a systematic maintenance program and data-driven mindset in making manufacturing decisions may result in serious safety risks, unexpected equipment damages, and financial strain. Condition monitoring and predictive maintenance management systems are commonly used in tandem with the Internet of Things, linking sensors on machines and transmitting the data through a wireless network to a data-logging center that will allow further analysis and support decision making. The system described in this paper measures vibrations using sensors attached to low voltage motors and then utilizes a two-phase machine learning approach for predictive maintenance. In the first phase, we conducted an analysis to look for any abnormal behavior, and in the second phase, we attempted to determine the type of specific faults that may occur. The proposed predictive maintenance system aims to reduce the fault detection time and assist with diagnosing the type of fault occurring. We utilized and tested three machine learning algorithms to detect abnormal motor behavior: support vector machine, backpropagation neural network, and random forest. For predicting the type of specific motor faults that may occur, we used a support vector machine. This two-phase machine learning approach demonstrated promising results in detecting abnormal behavior in low voltage motors. Therefore, integrating this machine learning component as a part of a predictive maintenance system can result in high confidence about the motor condition, reduce maintenance cost, and enhance the safety of the operators and the machines.

ICS: Total Freedom in Manual Text Classification Supported by Unobtrusive Machine Learning

We present the Interactive Classification System (ICS), a web-based application that supports the activity of manual text classification. The application uses machine learning to continuously fit automatic classification models that are in turn used to actively support its users with classification suggestions. The key requirement we have established for the development of ICS is to give its users total freedom of action: they can at any time modify any classification schema and any label assignment, possibly reusing any relevant information from previous activities. We investigate how this requirement challenges the typical scenarios faced in machine learning research, which instead give no active role to humans or place them into very constrained roles, e.g., on-demand labeling in active learning processes, and always assume some degree of batch processing of data. We satisfy the “total freedom” requirement by designing an unobtrusive machine learning model, i.e., the machine learning component of ICS as an unobtrusive observer of the users, that never interrupts them, continuously adapts and updates its models in response to their actions, and it is always available to perform automatic classifications. Our efficient implementation of the unobtrusive machine learning model combines various machine learning methods and technologies, such as hash-based feature mapping, random indexing, online learning, active learning, and asynchronous processing.

Machine-Learning Component for Multi-Start Metaheuristics to Solve the Capacitated Vehicle Routing Problem

Machine-Learning Component for Multi-Start Metaheuristics to Solve the Capacitated Vehicle Routing Problem

Adversarial Attacks and Defense Technologies on Autonomous Vehicles: A Review

Abstract In recent years, various domains have been influenced by the rapid growth of machine learning. Autonomous driving is an area that has tremendously developed in parallel with the advancement of machine learning. In autonomous vehicles, various machine learning components are used such as traffic lights recognition, traffic sign recognition, limiting speed and pathfinding. For most of these components, computer vision technologies with deep learning such as object detection, semantic segmentation and image classification are used. However, these machine learning models are vulnerable to targeted tensor perturbations called adversarial attacks, which limit the performance of the applications. Therefore, implementing defense models against adversarial attacks has become an increasingly critical research area. The paper aims at summarising the latest adversarial attacks and defense models introduced in the field of autonomous driving with machine learning technologies up until mid-2021.

Introducing Various Semantic Models for Amharic: Experimentation and Evaluation with Multiple Tasks and Datasets

The availability of different pre-trained semantic models has enabled the quick development of machine learning components for downstream applications. However, even if texts are abundant for low-resource languages, there are very few semantic models publicly available. Most of the publicly available pre-trained models are usually built as a multilingual version of semantic models that will not fit well with the need for low-resource languages. We introduce different semantic models for Amharic, a morphologically complex Ethio-Semitic language. After we investigate the publicly available pre-trained semantic models, we fine-tune two pre-trained models and train seven new different models. The models include Word2Vec embeddings, distributional thesaurus (DT), BERT-like contextual embeddings, and DT embeddings obtained via network embedding algorithms. Moreover, we employ these models for different NLP tasks and study their impact. We find that newly-trained models perform better than pre-trained multilingual models. Furthermore, models based on contextual embeddings from FLAIR and RoBERTa perform better than word2Vec models for the NER and POS tagging tasks. DT-based network embeddings are suitable for the sentiment classification task. We publicly release all the semantic models, machine learning components, and several benchmark datasets such as NER, POS tagging, sentiment classification, as well as Amharic versions of WordSim353 and SimLex999.

Classification of alcoholic EEG signals using wavelet scattering transform-based features

Following the research question and the relevant dataset, feature extraction is the most important component of machine learning and data science pipelines. The wavelet scattering transform (WST) is a recently developed knowledge-based feature extraction technique and is structurally like a convolutional neural network (CNN). It preserves information in high-frequency, is insensitive to signal deformations, and generates low variance features of real-valued signals generally required in classification tasks. With data from a publicly-available UCI database, we investigated the ability of WST-based features extracted from multichannel electroencephalogram (EEG) signals to discriminate 1.0-s EEG records of 20 male subjects with alcoholism and 20 male healthy subjects. Using record-wise 10-fold cross-validation, we found that WST-based features, inputted to a support vector machine (SVM) classifier, were able to correctly classify all alcoholic and normal EEG records. Similar performances were achieved with 1D CNN. In contrast, the highest independent-subject-wise mean 10-fold cross-validation performance was achieved with WST-based features fed to a linear discriminant (LDA) classifier. The results achieved with two 10-fold cross-validation approaches suggest that the WST together with a conventional classifier is an alternative to CNN for classification of alcoholic and normal EEGs. WST-based features from occipital and parietal regions were the most informative at discriminating between alcoholic and normal EEG records.

Do Datasets Have Politics? Disciplinary Values in Computer Vision Dataset Development

Data is a crucial component of machine learning. The field is reliant on data to train, validate, and test models. With increased technical capabilities, machine learning research has boomed in both academic and industry settings, and one major focus has been on computer vision. Computer vision is a popular domain of machine learning increasingly pertinent to real-world applications, from facial recognition in policing to object detection for autonomous vehicles. Given computer vision's propensity to shape machine learning research and impact human life, we seek to understand disciplinary practices around dataset documentation - how data is collected, curated, annotated, and packaged into datasets for computer vision researchers and practitioners to use for model tuning and development. Specifically, we examine what dataset documentation communicates about the underlying values of vision data and the larger practices and goals of computer vision as a field. To conduct this study, we collected a corpus of about 500 computer vision datasets, from which we sampled 114 dataset publications across different vision tasks. Through both a structured and thematic content analysis, we document a number of values around accepted data practices, what makes desirable data, and the treatment of humans in the dataset construction process. We discuss how computer vision datasets authors value efficiency at the expense of care; universality at the expense of contextuality; impartiality at the expense of positionality; and model work at the expense of data work. Many of the silenced values we identify sit in opposition with social computing practices. We conclude with suggestions on how to better incorporate silenced values into the dataset creation and curation process.

Reshaping Human Factors Education in Times of Big Data: Practitioner Perspectives

Human Factors Engineering (HFE) is an applied discipline that uses a wide range of methodologies to better the design of systems and devices for human use. Underpinning all human factors design is the maxim to fit the human to the task/machine/system rather than vice versa. While some HFE methods such as task analysis and anthropometrics remain relatively fixed over time, areas such as human-technology interaction are strongly influenced by the fast-evolving technological trend. In times of big data, human factors engineers need to have a good understanding of topics like machine learning, advanced data analytics, and data visualization so that they can design data-driven products that involve big data sets. There is a natural lag between industrial trends and HFE curricula, leading to gaps between what people are taught and what they will need to know. In this paper, we present the results of a survey involving HFE practitioners (N=101) and we demonstrate the need for including data science and machine learning components in HFE curricula.

A General Cooperative Optimization Approach for Distributing Service Points in Mobility Applications

This article presents a cooperative optimization approach (COA) for distributing service points for mobility applications, which generalizes and refines a previously proposed method. COA is an iterative framework for optimizing service point locations, combining an optimization component with user interaction on a large scale and a machine learning component that learns user needs and provides the objective function for the optimization. The previously proposed COA was designed for mobility applications in which single service points are sufficient for satisfying individual user demand. This framework is generalized here for applications in which the satisfaction of demand relies on the existence of two or more suitably located service stations, such as in the case of bike/car sharing systems. A new matrix factorization model is used as surrogate objective function for the optimization, allowing us to learn and exploit similar preferences among users w.r.t. service point locations. Based on this surrogate objective function, a mixed integer linear program is solved to generate an optimized solution to the problem w.r.t. the currently known user information. User interaction, refinement of the matrix factorization, and optimization are iterated. An experimental evaluation analyzes the performance of COA with special consideration of the number of user interactions required to find near optimal solutions. The algorithm is tested on artificial instances, as well as instances derived from real-world taxi data from Manhattan. Results show that the approach can effectively solve instances with hundreds of potential service point locations and thousands of users, while keeping the user interactions reasonably low. A bound on the number of user interactions required to obtain full knowledge of user preferences is derived, and results show that with 50% of performed user interactions the solutions generated by COA feature optimality gaps of only 1.45% on average.

Machine learning for materials discovery: Two-dimensional topological insulators

One of the main goals and challenges of materials discovery is to find the best candidates for each interest property or application. Machine learning rises in this context to efficiently optimize this search, exploring the immense materials space, consisting of simultaneously the atomic, compositional, and structural spaces. Topological insulators, presenting symmetry-protected metallic edge states, are a promising class of materials for different applications. However, further development is limited by the scarcity of viable candidates. Here we present and discuss machine learning–accelerated strategies for searching the materials space for two-dimensional topological materials. We show the importance of detailed investigations of each machine learning component, leading to different results. Using recently created databases containing thousands of ab initio calculations of 2D materials, we train machine learning models capable of determining the electronic topology of materials, with an accuracy of over 90%. We can then generate and screen thousands of novel materials, efficiently predicting their topological character without the need for a priori structural knowledge. We discover 56 non-trivial materials, of which 17 are novel insulating candidates for further investigation, for which we corroborate their topological properties with density functional theory calculations. This strategy is 10× more efficient than the trial-and-error approach while a few orders of magnitude faster and is a proof of concept for guiding improved materials discovery search strategies.

Machine learning techniques to construct patched analog ensembles for data assimilation

Using generative models from the machine learning literature to create artificial ensemble members for use within data assimilation schemes has been introduced in Grooms (2021) [1] as constructed analog ensemble optimal interpolation (cAnEnOI). Specifically, we study general and variational autoencoders for the machine learning component of this method, and combine the ideas of constructed analogs and ensemble optimal interpolation in the data assimilation piece. To extend the scalability of cAnEnOI for use in data assimilation on complex dynamical models, we propose using patching schemes to divide the global spatial domain into digestible chunks. Using patches makes training the generative models possible and has the added benefit of being able to exploit parallelism during the generative step. Testing this new algorithm on a 1D toy model, we find that larger patch sizes make it harder to train an accurate generative model (i.e. a model whose reconstruction error is small), while conversely the data assimilation performance improves at larger patch sizes. There is thus a sweet spot where the patch size is large enough to enable good data assimilation performance, but not so large that it becomes difficult to train an accurate generative model. In our tests the new patched cAnEnOI method outperforms the original (unpatched) cAnEnOI, as well as the ensemble square root filter results from [1].

Technically Representing Clinical Knowledge for Rehabilitation Care.

Providing a suitable rehabilitation after an acute episode or a chronic disease helps people to live independently and enhance their quality of life. However, the continuity of care is often interrupted in the transition from hospital to home. Virtual coaches (VCs) could help these patients to engage in personalized home rehabilitation programs. These coaching systems need also to be fed with procedural precepts in order to work as intended. This, in turn, relates both to properly represent the clinical knowledge (as the VC somehow replaces the formal caregivers that cannot be fully present) as well guide the patient correctly (in order to follow the medically desired procedures given the need for personalisation according to individual needs). Therefore, we outline our technical approach to deal with this. In particular, clinical pathways in terms of semi-formal procedure models in combination with machine learning components processing and powerful user interfaces providing these pathway information and feeding the VC are presented. The system is currently under testing in a participatory design phase called Living Lab. Thus, initial user feedback for further improvements is about to come.

Safety Assurance for Systems with Machine Learning Components

The use of machine learning components in safety-critical systems creates reliability concerns. My thesis focuses on developing algorithms to address these concerns. Because the assurance of a safety-critical system generally requires multiple types of validation, my research takes three directions: safe deep learning algorithms, formal verification of neural networks, and adaptive testing methods.

Multilayered review of safety approaches for machine learning-based systems in the days of AI

The unprecedented advancement of artificial intelligence (AI) in recent years has altered our perspectives on software engineering and systems engineering as a whole. Nowadays, software-intensive intelligent systems rely more on a learning model than thousands of lines of codes. Such alteration has led to new research challenges in the engineering process that can ensure the safe and beneficial behavior of AI systems. This paper presents a literature survey of the significant efforts made in the last fifteen years to foster safety in complex intelligent systems. This survey covers relevant aspects of AI safety research including safety requirements engineering, safety-driven design at both system and machine learning (ML) component level, validation and verification from the perspective of software and system engineers. We categorize these research efforts based on a three-layered conceptual framework for developing and maintaining AI systems. We also perform a gap analysis to emphasize the open research challenges in ensuring safe AI. Finally, we conclude the paper by providing future research directions and a road map for AI safety.

Machine learning based simulation optimisation for urban routing problems

Many real world routing problems, including those in tourism and surveillance, can be formulated as team orienteering problems. The goal in such problems is to maximise the rewards collected by a fleet of vehicles whose routes must be completed within a time limit. This work considers a team orienteering problem set within a traffic simulation. In the stochastic environment of a road network, travel times depend on network structure, the demands of road users, driver behaviour and the congestion that arises from these. As a result travel times are difficult to predict. In this work a learnheuristic solution approach is proposed. Learnheuristics integrate machine learning and optimisation for solving combinatorial problems with inherent parameter learning problems—in this case travel times. The machine learning component is used to predict travel times based on data obtained from a limited budget of traffic simulation runs, a budget that is used within the run-time learnheuristic algorithm. In each iteration of the learnheuristic, the optimisation component utilises the travel time predictions of the machine learning algorithm to rapidly generate candidate solutions. The strongest candidate is tested in the traffic simulator, and the results of which are used to train the machine learning component. In a range of test instances, the effectiveness of different combinations of machine learning and optimisation components are tested. Experiments reveal that different combinations of machine learning and optimisation components produce solutions with different characteristics in terms of total reward and reliability. Local search followed by biased randomisation combined with a neural network was found to be effective in multiple instances. The question of how best to use the run-time of a learnheuristic is also addressed.

STATegra: Multi-Omics Data Integration - A Conceptual Scheme With a Bioinformatics Pipeline.

Technologies for profiling samples using different omics platforms have been at the forefront since the human genome project. Large-scale multi-omics data hold the promise of deciphering different regulatory layers. Yet, while there is a myriad of bioinformatics tools, each multi-omics analysis appears to start from scratch with an arbitrary decision over which tools to use and how to combine them. Therefore, it is an unmet need to conceptualize how to integrate such data and implement and validate pipelines in different cases. We have designed a conceptual framework (STATegra), aiming it to be as generic as possible for multi-omics analysis, combining available multi-omic anlaysis tools (machine learning component analysis, non-parametric data combination, and a multi-omics exploratory analysis) in a step-wise manner. While in several studies, we have previously combined those integrative tools, here, we provide a systematic description of the STATegra framework and its validation using two The Cancer Genome Atlas (TCGA) case studies. For both, the Glioblastoma and the Skin Cutaneous Melanoma (SKCM) cases, we demonstrate an enhanced capacity of the framework (and beyond the individual tools) to identify features and pathways compared to single-omics analysis. Such an integrative multi-omics analysis framework for identifying features and components facilitates the discovery of new biology. Finally, we provide several options for applying the STATegra framework when parametric assumptions are fulfilled and for the case when not all the samples are profiled for all omics. The STATegra framework is built using several tools, which are being integrated step-by-step as OpenSource in the STATegRa Bioconductor package.1

Entropy Analysis of a Dataset using Machine Learning Approach

Artificial Intelligence and Machine Learning's component to recognise communication between machines and individual (real) languages is a natural language processing. Natural Language Processing is the identification of emotion mostly used to interpret terms to provide strong, slightly bad emotions due to people's reading patterns. Shannon's entropy helps me know whether or not people more like Zomato is a ranking program for restaurants. The assessment involves a restaurant review that can be used for entropy assessment. On this basis, the authors want to respond to the expected view of the analysis. The method used to pre-process the research is to minimise all terms, monitor access, remove quantities, sentence structure, stop words and compile. The latent semantic document frequency (TF-IDF) is then constructed from word to vector. The data we are gathering is 1,50,000 reviews. Great responses are rated 3 and above, poor comments are rated 3 and below, glowing reviews are rated 3 and above. The author uses split Evaluation, 80% full and 20% Data Screening. Accuracy, recall and precision is the criteria used to evaluate random forest classifiers. The reliability of this analysis is 92 percent. 92 %, 93 %, 96 % is the consistency of each selection's thoughts and feelings. 99%, 89%, 73% are a reminder of positive, pessimistic and constructive views. 93 % and 87 % are the average accuracy and recall. "Poor", "great", "fair", "better", "location", "care", "request", "food", "seek" and "pleasant" are the 10 terms that influence the results.

A survey of machine learning in credit risk

Machine learning algorithms have come to dominate several industries. After decades of resistance from examiners and auditors, machine learning is now moving from the research desk to the application stack for credit scoring and a range of other applications in credit risk. This migration is not without novel risks and challenges. Much of the research is now shifting from how best to make the models to how best to use the models in a regulator-compliant business context. This paper surveys the impressively broad range of machine learning methods and application areas for credit risk. In the process of that survey, we create a taxonomy to think about how different machine learning components are matched to create specific algorithms. The reasons for where machine learning succeeds over simple linear methods are explored through a specific lending example. Throughout, we highlight open questions, ideas for improvements and a framework for thinking about how to choose the best machine learning method for a specific problem.

Towards Risk-aware Machine Learning Supported Model Predictive Control and Open-loop Optimization for Repetitive Processes

Abstract Many processes operate repetitively, for example batch processes in biotechnology or chemical engineering. We propose a method for risk-aware run-to-run optimization and model predictive control of repetitive processes with uncertain models. The goal is to increase the performance as the number of runs increases by improving the model despite limited measurements while considering model uncertainty and avoiding uncertain areas. The method uses a gray-box model, i.e. a model formed by a first principle and a machine learning component, in this case an artificial neural network. The model uncertainty might be large, particularly in the first runs, where only a few measurements are available. We propose to quantify this uncertainty using Bayesian inference. This is in turn reflected by a risk measure entering an open-loop optimal control problem and a shrinking-horizon Model Predictive Controller as a constraint to limit control and exploitation in high risk areas. We show that using this risk measure we are able to efficiently reach high process performance. The proposed method is tested in simulations on two biotechnological fed-batch processes.

A Computationally Efficient Reconstruction Algorithm for Circular Cone-Beam Computed Tomography Using Shallow Neural Networks.

Circular cone-beam (CCB) Computed Tomography (CT) has become an integral part of industrial quality control, materials science and medical imaging. The need to acquire and process each scan in a short time naturally leads to trade-offs between speed and reconstruction quality, creating a need for fast reconstruction algorithms capable of creating accurate reconstructions from limited data. In this paper, we introduce the Neural Network Feldkamp–Davis–Kress (NN-FDK) algorithm. This algorithm adds a machine learning component to the FDK algorithm to improve its reconstruction accuracy while maintaining its computational efficiency. Moreover, the NN-FDK algorithm is designed such that it has low training data requirements and is fast to train. This ensures that the proposed algorithm can be used to improve image quality in high-throughput CT scanning settings, where FDK is currently used to keep pace with the acquisition speed using readily available computational resources. We compare the NN-FDK algorithm to two standard CT reconstruction algorithms and to two popular deep neural networks trained to remove reconstruction artifacts from the 2D slices of an FDK reconstruction. We show that the NN-FDK reconstruction algorithm is substantially faster in computing a reconstruction than all the tested alternative methods except for the standard FDK algorithm and we show it can compute accurate CCB CT reconstructions in cases of high noise, a low number of projection angles or large cone angles. Moreover, we show that the training time of an NN-FDK network is orders of magnitude lower than the considered deep neural networks, with only a slight reduction in reconstruction accuracy.