Challenges In Machine Learning Research Articles

OMCOKE: A Machine Learning Outlier-based Overlapping Clustering Technique for Multi-Label Data Analysis

Clustering is one of the challenging machine learning techniques due to its unsupervised learning nature. While many clustering algorithms constrain objects to single clusters, K-means overlapping partitioning clustering methods assign objects to multiple clusters by relaxing the constraints and allowing objects to belong to more than one cluster to better fit hidden structures in the data. However, when datasets contain outliers, they can significantly influence the mean distance of the data objects to their respective clusters, which is a drawback. Therefore, most researchers address this problem by simply removing the outliers. This can be problematic especially in applications such as fraud detection or cybersecurity attacks risk analysis. In this study, an alternative solution to this problem is proposed that captures outliers and stores them on-the-fly within a new cluster, instead of discarding. The new algorithm is named Outlier-based Multi-Cluster Overlapping K-Means Extension (OMCOKE). Empirical results on real-life multi-label datasets were derived to compare OMCOKE’s performance with other common overlapping clustering techniques. The results show that OMCOKE produced a better precision rate compared to the considered clustering algorithms. This method can benefit various stakeholders as these outliers could have real-life applications in cybersecurity, fraud detection, and the anti-phishing of websites.

Building interpretable predictive models with context-aware evolutionary learning

Building prediction models with the right balance between performance and interpretability is currently a great challenge in machine learning. A large number of recent studies have focused on either building intrinsically interpretable models or developing general explainers for blackbox models. Although these methods have been widely adopted, their interpretability or explanations are not always useful because of the lack of contexts considered in training machine learning models and producing explanations. This paper aims to tackle this significant challenge by developing a context-aware evolutionary learning algorithm (CELA) for building interpretable prediction models. A new context extraction method based on unsupervised self-structuring learning algorithms is developed to treat data in contexts. The proposed algorithm overcomes the limitations of existing evolutionary learning methods in handling a large number of features and large datasets by training specialised interpretable models based on the automatically extracted contexts. The new algorithm has been tested on complex regression datasets and a real-world building energy prediction task. The results suggest CELA can outperform well-known interpretable machine learning (IML) algorithms, the state-of-the-art evolutionary algorithm, and can produce predictions much closer to the results of blackbox algorithms such as XGBoost and artificial neural networks than the compared IML methods. Further analyses also demonstrate that the CELA’s prediction models are smaller and easier to interpret than those obtained by the evolutionary learning algorithm without context awareness.

Successful Applications and Future Challenges of Machine Learning for Power Systems: A Summary of Recent Activities by the IEEE WG on Machine Learning for Power Systems [What’s Popular

Presents a summary or recent activities by the IEEE WG on Machine Learning for Power Systems.

A Review on Machine Learning & It’s Algorithms

Machine learning is important because it gives us accurate predictions based on data. It can teach computers to perform complex tasks without any human intervention. Machine learning can analyze complex blocks of data. Machine learning enables entrepreneurs and businesses to quickly recognize potential business opportunities and risks. Businesses that rely solely on large amounts of data are using machine learning as the best way to analyze data and build models. Machine learning is not only considered as the backbone of artificial intelligence, but machine learning also plays a significant role in the development and advancement of artificial intelligence. Using algorithms to solve classification problems with different sets of parameters yields dramatically different classification accuracies. The machine learning challenge of finding the most appropriate parameter values for algorithms that best solve technical problems related to performance metrics. In this paper, the author discussed various types of machine learning such as supervised, unsupervised and reinforcement machine learning. The main emphasis is on supervised machine learning such as classification and regression using various machine learning algorithms such as Decision Tree, Naïve Bayes, K-Nearest Neighbor, Random Forest and SVM Classifier. The author explains all classification-based algorithms well with examples and diagrams. The authors also mention applications or domain areas where these classification algorithms can be used.

Open challenges for Machine Learning based Early Decision-Making research

More and more applications require early decisions, i.e. taken as soon as possible from partially observed data. However, the later a decision is made, the more its accuracy tends to improve, since the description of the problem to hand is enriched over time. Such a compromise between the earliness and the accuracy of decisions has been particularly studied in the field of Early Time Series Classification. This paper introduces a more general problem, called Machine Learning based Early Decision Making (ML-EDM), which consists in optimizing the decision times of models in a wide range of settings where data is collected over time. After defining the ML-EDM problem, ten challenges are identified and proposed to the scientific community to further research in this area. These challenges open important application perspectives, discussed in this paper.

The Impact of Cognitive Load on Students’ Academic Writing: An Authorship Verification Investigation

Automatic authorship verification is known to be a challenging machine learning task. In this paper, we examine the efficacy of an enhanced common n-gram profile-based approach to assist educational institutions to validate students' essays and assignments through their writing styles. We investigated the impact that essays with different cognitive load requirements have in students' writing styles, which may or may not impact authorship verification methods. A total of 46 undergraduate students completed six essays in a laboratory study. Although results showed small and mixed effects of the tasks differing in cognitive load on the different writing product metrics, students' essays and assignments texts contained features that remained stable across essays requiring different levels of cognitive load. These results suggest that our approach could be successfully used in authorship verification, potentially helping to address issues related to academic integrity in higher education settings.

A Review of Thermal Comfort in Primary Schools and Future Challenges in Machine Learning Based Prediction for Children

Children differ from adults in their physiology and cognitive ability. Thus, they are extremely vulnerable to classroom thermal comfort. However, very few reviews on the thermal comfort of primary school students are available. Further, children-focused surveys have not reviewed the state-of-the-art in thermal comfort prediction using machine learning (AI/ML). Consequently, there is a need for discussion on children-specific challenges in AI/ML-based prediction. This article bridges these research gaps. It presents a comprehensive review of thermal comfort studies in primary school classrooms since 1962. It considers both conventional (non-ML) studies and the recent AI/ML studies performed for children, classrooms, and primary students. It also underscores the importance of AI/ML prediction by analyzing adaptive opportunities for children/students in classrooms. Thereafter, a review of AI/ML-based prediction studies is presented. Through an AI/ML case-study, it demonstrates that model performance for children and adults differs markedly. Performance of classification models trained on ASHRAE-II database and a recent primary students’ dataset shows a 29% difference in thermal sensation and 86% difference in thermal preference, between adults and children. It then highlights three major children-specific AI/ML challenges, viz., “illogical votes”, “multiple comfort metrics”, and “extreme class imbalance”. Finally, it offers several technical solutions and discusses open problems.

Tiny Machine Learning for Resource-Constrained Microcontrollers

We use 250 billion microcontrollers daily in electronic devices that are capable of running machine learning models inside them. Unfortunately, most of these microcontrollers are highly constrained in terms of computational resources, such as memory usage or clock speed. These are exactly the same resources that play a key role in teaching and running a machine learning model with a basic computer. However, in a microcontroller environment, constrained resources make a critical difference. Therefore, a new paradigm known as tiny machine learning had to be created to meet the constrained requirements of the embedded devices. In this review, we discuss the resource optimization challenges of tiny machine learning and different methods, such as quantization, pruning, and clustering, that can be used to overcome these resource difficulties. Furthermore, we summarize the present state of tiny machine learning frameworks, libraries, development environments, and tools. The benchmarking of tiny machine learning devices is another thing to be concerned about; these same constraints of the microcontrollers and diversity of hardware and software turn to benchmark challenges that must be resolved before it is possible to measure performance differences reliably between embedded devices. We also discuss emerging techniques and approaches to boost and expand the tiny machine learning process and improve data privacy and security. In the end, we form a conclusion about tiny machine learning and its future development.

A Machine Learning Challenge: Detection of Cardiac Amyloidosis Based on Bi-Atrial and Right Ventricular Strain and Cardiac Function.

This study challenges state-of-the-art cardiac amyloidosis (CA) diagnostics by feeding multi-chamber strain and cardiac function into supervised machine (SVM) learning algorithms. Forty-three CA (32 males; 79 years (IQR 71; 85)), 20 patients with hypertrophic cardiomyopathy (HCM, 10 males; 63.9 years (±7.4)) and 44 healthy controls (CTRL, 23 males; 56.3 years (IQR 52.5; 62.9)) received cardiovascular magnetic resonance imaging. Left atrial, right atrial and right ventricular strain parameters and cardiac function generated a 41-feature matrix for decision tree (DT), k-nearest neighbor (KNN), SVM linear and SVM radial basis function (RBF) kernel algorithm processing. A 10-feature principal component analysis (PCA) was conducted using SVM linear and RBF. Forty-one features resulted in diagnostic accuracies of 87.9% (AUC = 0.960) for SVM linear, 90.9% (0.996; Precision = 94%; Sensitivity = 100%; F1-Score = 97%) using RBF kernel, 84.9% (0.970) for KNN, and 78.8% (0.787) for DT. The 10-feature PCA achieved 78.9% (0.962) via linear SVM and 81.8% (0.996) via RBF SVM. Explained variance presented bi-atrial longitudinal strain and left and right atrial ejection fraction as valuable CA predictors. SVM RBF kernel achieved competitive diagnostic accuracies under supervised conditions. Machine learning of multi-chamber cardiac strain and function may offer novel perspectives for non-contrast clinical decision-support systems in CA diagnostics.

AdaCC: cumulative cost-sensitive boosting for imbalanced classification

Class imbalance poses a major challenge for machine learning as most supervised learning models might exhibit bias towards the majority class and under-perform in the minority class. Cost-sensitive learning tackles this problem by treating the classes differently, formulated typically via a user-defined fixed misclassification cost matrix provided as input to the learner. Such parameter tuning is a challenging task that requires domain knowledge and moreover, wrong adjustments might lead to overall predictive performance deterioration. In this work, we propose a novel cost-sensitive boosting approach for imbalanced data that dynamically adjusts the misclassification costs over the boosting rounds in response to model’s performance instead of using a fixed misclassification cost matrix. Our method, called AdaCC, is parameter-free as it relies on the cumulative behavior of the boosting model in order to adjust the misclassification costs for the next boosting round and comes with theoretical guarantees regarding the training error. Experiments on 27 real-world datasets from different domains with high class imbalance demonstrate the superiority of our method over 12 state-of-the-art cost-sensitive boosting approaches exhibiting consistent improvements in different measures, for instance, in the range of [0.3–28.56%] for AUC, [3.4–21.4%] for balanced accuracy, [4.8–45%] for gmean and [7.4–85.5%] for recall.

Simplifying VGG-16 for Plant Species Identification

Plant species identification represents an extraordinary challenge for machine learning due to visual interspecies similarities and large intraspecies variations. Furthermore, research literature reports that plant species identification usually lacks sufficiently large datasets for training classification models. In this paper, we address this problem with a model that simplifies the VGG-16 architecture, the N-VGG model. The idea behind N-VGG is to reduce experimentally observed overfitting on VGG-16 by using as few trainable parameters as possible. To do this, we substitute the flattening layer on the VGG architecture with a global average pooling layer. This reduces the size of the feature vector. In addition, we eliminate one of the two fully-connected layers and use a new hyper-parameter, N, to indicate the number of nodes on the remaining layer. To show the robustness of the N-VGG model, we conducted extensive experimentation. We trained N-VGG on five datasets for plant species identification. Four of these datasets are publicly available and have been widely used as benchmarks for plant identification models. For all datasets, we compare the accuracy of N-VGG to that of the VGG-16, Inception-v4, and EfficienNet-B3 models. The experimental results show that the N-VGG model achieved the best classification performance for all but one datasets, whereas all the models showed a remarkable performance for the remaining dataset. This evidence supports our initial idea that, for plant species classification, some accuracy might be lost due to overfitting and that having fewer trainable parameters helps in producing a more robust model.

Application of Meta-Heuristic Algorithms for Training Neural Networks and Deep Learning Architectures: A Comprehensive Review.

The learning process and hyper-parameter optimization of artificial neural networks (ANNs) and deep learning (DL) architectures is considered one of the most challenging machine learning problems. Several past studies have used gradient-based back propagation methods to train DL architectures. However, gradient-based methods have major drawbacks such as stucking at local minimums in multi-objective cost functions, expensive execution time due to calculating gradient information with thousands of iterations and needing the cost functions to be continuous. Since training the ANNs and DLs is an NP-hard optimization problem, their structure and parameters optimization using the meta-heuristic (MH) algorithms has been considerably raised. MH algorithms can accurately formulate the optimal estimation of DL components (such as hyper-parameter, weights, number of layers, number of neurons, learning rate, etc.). This paper provides a comprehensive review of the optimization of ANNs and DLs using MH algorithms. In this paper, we have reviewed the latest developments in the use of MH algorithms in the DL and ANN methods, presented their disadvantages and advantages, and pointed out some research directions to fill the gaps between MHs and DL methods. Moreover, it has been explained that the evolutionary hybrid architecture still has limited applicability in the literature. Also, this paper classifies the latest MH algorithms in the literature to demonstrate their effectiveness in DL and ANN training for various applications. Most researchers tend to extend novel hybrid algorithms by combining MHs to optimize the hyper-parameters of DLs and ANNs. The development of hybrid MHs helps improving algorithms performance and capable of solving complex optimization problems. In general, the optimal performance of the MHs should be able to achieve a suitable trade-off between exploration and exploitation features. Hence, this paper tries to summarize various MH algorithms in terms of the convergence trend, exploration, exploitation, and the ability to avoid local minima. The integration of MH with DLs is expected to accelerate the training process in the coming few years. However, relevant publications in this way are still rare.

Deploying deep learning models on unseen medical imaging using adversarial domain adaptation.

The fundamental challenge in machine learning is ensuring that trained models generalize well to unseen data. We developed a general technique for ameliorating the effect of dataset shift using generative adversarial networks (GANs) on a dataset of 149,298 handwritten digits and dataset of 868,549 chest radiographs obtained from four academic medical centers. Efficacy was assessed by comparing area under the curve (AUC) pre- and post-adaptation. On the digit recognition task, the baseline CNN achieved an average internal test AUC of 99.87% (95% CI, 99.87-99.87%), which decreased to an average external test AUC of 91.85% (95% CI, 91.82-91.88%), with an average salvage of 35% from baseline upon adaptation. On the lung pathology classification task, the baseline CNN achieved an average internal test AUC of 78.07% (95% CI, 77.97-78.17%) and an average external test AUC of 71.43% (95% CI, 71.32-71.60%), with a salvage of 25% from baseline upon adaptation. Adversarial domain adaptation leads to improved model performance on radiographic data derived from multiple out-of-sample healthcare populations. This work can be applied to other medical imaging domains to help shape the deployment toolkit of machine learning in medicine.

Systematic Analysis of Ovarian Cancer Empowered with Machine and Deep Learning: A Taxonomy and Future Challenges

Machine and Deep learning has witnessed an exceptional amount of admiration in recent years. ML has ability to learn data itself by predicting uncertain conditions or future and classify categories with minimum intervention of human. While in DL, computers are able to automatically, learn useful features and representation precisely from raw data. ML and DL potentially a disruptive technology in predictive healthcare analysis. A detailed understanding and evaluation of the applications and principles of radiomics, machine and deep learning is an important task to construct possible solutions that are capable of accomplish compulsory and ethical requirements, which can enhance efficiency, quality and outcomes. Machine and deep learning extensively being use in medical image analysis, medical diagnostics and medical image technology. The wide scope and sudden progress of ML and DL has remarkably change the ways of diagnosis, prediction, classification and analyzing ovarian, lungs, brain, skin and various other types of cancer. In view of multiple applications of ML and DL, in this article a review conducted on ovarian cancer. This review comprises the detailed analysis of OC (prognosis, diagnosis, classification, evaluation) by covering all the major contributions of machine and deep learning. Furthermore, a literature taxonomy of the research conferred and emerging aspects analyzed. A section of discussion also signified to elaborate the limitations and future challenges of machine and deep learning comprises with ovarian cancer.

Towards high entropy alloy with enhanced strength and ductility using domain knowledge constrained active learning

The simultaneous optimization of competing properties is a challenge of machine learning based materials design. We proposed a domain knowledge constrained active learning loop for the design of high entropy alloys with optimized strength and ductility, by narrowing down the unexplored space using the valence electron concentration criterion. The active learning loop iterated six times and one alloy with an ultimate strength of 1258 MPa and an elongation of 17.3 % was synthesized. To uncover the underlying mechanism for synergetic optimization, we characterized the phase structure and eutectic microstructure, and discussed the possible origins from the view of strain hardening and crack initiation. The proposed framework that combines domain knowledge with machine learning can facilitate the design of target materials with coordinating optimization of competing properties.

Challenges of Machine Learning for eVTOL Reliability and Safety

The increasing number of requests for type certification received by the European Union Safety Agency on Vertical Takeoff and Landing (VTOL) aircraft attests to the expansion of frontiers in Urban Air Mobility (UAM). In addition, it has revealed the interest of traditional airplane and helicopter manufacturers in this new technology, all the while highlighting the emergence of new players developing their respective versions of electric-powered VTOLs (eVTOL). The perspective of eVTOLs going into service in the coming years for the transport of passengers raises new safety concerns. Indeed, it is necessary to ensure the reliability and safety aspects of those aircraft systems that will be flying under new operational missions, differing from current fixed wing (airplanes) and rotorcraft (helicopters) aircraft. At the same time, the evolution of aircraft systems monitoring technology is making it possible to acquire increasing amounts of data. The high complexity of new systems, combined with the huge amount of data provided, can make the decision-making process more difficult for pilots. Machine learning makes it possible to evaluate this data and improve reliability and safety.
 Even as the number of aeronautical accidents has decreased over the last years, 60–80% of those accidents are the result of human failure. In the initial implementation and operation stages of eVTOLs, machine learning (ML) can support pilots by using aircraft data to predict system failures and contribute to improve reliability and safety. Then, at an advanced stage of eVTOL operation, ML may help reduce human interaction with the aircraft, paving the way toward fully autonomous aircraft. The association of ML with technologies such as Digital Twins and 6G networks has the potential to enable safe and reliable autonomous flight. However, the introduction of eVTOLs will also increase air traffic in highly populated areas and thus needs to be studied to support the incorporation of the future autonomous aircraft. This paper addresses the main challenges for the incorporation of ML in the upcoming eVTOL fleet and its potential contribution to aircraft systems reliability and safety. It also explores the need for the use of ML techniques in a more autonomous air traffic management systems the face of increased air traffic.

A Two-Stream Continual Learning System With Variational Domain-Agnostic Feature Replay.

Learning in nonstationary environments is one of the biggest challenges in machine learning. Nonstationarity can be caused by either task drift, i.e., the drift in the conditional distribution of labels given the input data, or the domain drift, i.e., the drift in the marginal distribution of the input data. This article aims to tackle this challenge with a modularized two-stream continual learning (CL) system, where the model is required to learn new tasks from a support stream and adapted to new domains in the query stream while maintaining previously learned knowledge. To deal with both drifts within and across the two streams, we propose a variational domain-agnostic feature replay-based approach that decouples the system into three modules: an inference module that filters the input data from the two streams into domain-agnostic representations, a generative module that facilitates the high-level knowledge transfer, and a solver module that applies the filtered and transferable knowledge to solve the queries. We demonstrate the effectiveness of our proposed approach in addressing the two fundamental scenarios and complex scenarios in two-stream CL.

Machine Learning Approaches, Technologies, Recent Applications, Advantages and Challenges on Manufacturing and Industry 4.0 applications

Abstract: This research is based at creatively employing machine learning to generate new ways, evaluate approaches, and communicate outcomes for difficulties encountered in the manufacturing business. Because manufacturers aim to increase profits with the least amount of capital expenditure, a model has been developed in which machine learning is integrated into manufacturing. It is critical to use all available means in order to meet the demand for high-quality products in an effective manner. Machine learning is one topic that has seen rapid advancement in terms of not just promising outcomes but also applicability. Machine learning is being widely explored by both academics and professionals as a potential solution to many of manufacturing's old and new obstacles. Unfortunately, the field is highly diverse and even perplexing, posing a difficulty and a barrier to widespread implementation. The linear regression algorithm was used to predict the outcome for the provided input, and gradient descent was used to optimise the output. It can assist the firm maximise output and profit by anticipating the outcome so that the company can take the necessary action or foresee the problem ahead of time. This study helps to structure this relatively intricate field by providing an overview of existing machine learning approaches. A particular emphasis is placed on the prospective benefit, as well as instances of successful implementations in a manufacturing context.

Addressing modern and practical challenges in machine learning: a survey of online federated and transfer learning

Online federated learning (OFL) and online transfer learning (OTL) are two collaborative paradigms for overcoming modern machine learning challenges such as data silos, streaming data, and data security. This survey explores OFL and OTL throughout their major evolutionary routes to enhance understanding of online federated and transfer learning. Practical aspects of popular datasets and cutting-edge applications for online federated and transfer learning are also highlighted in this work. Furthermore, this survey provides insight into potential future research areas and aims to serve as a resource for professionals developing online federated and transfer learning frameworks.

A review on the application of machine learning for combustion in power generation applications

Abstract Although the world is shifting toward using more renewable energy resources, combustion systems will still play an important role in the immediate future of global energy. To follow a sustainable path to the future and reduce global warming impacts, it is important to improve the efficiency and performance of combustion processes and minimize their emissions. Machine learning techniques are a cost-effective solution for improving the sustainability of combustion systems through modeling, prediction, forecasting, optimization, fault detection, and control of processes. The objective of this study is to provide a review and discussion regarding the current state of research on the applications of machine learning techniques in different combustion processes related to power generation. Depending on the type of combustion process, the applications of machine learning techniques are categorized into three main groups: (1) coal and natural gas power plants, (2) biomass combustion, and (3) carbon capture systems. This study discusses the potential benefits and challenges of machine learning in the combustion area and provides some research directions for future studies. Overall, the conducted review demonstrates that machine learning techniques can play a substantial role to shift combustion systems towards lower emission processes with improved operational flexibility and reduced operating cost.