Challenges In Machine Learning Research Articles

Mitigating Label Noise through Data Ambiguation

Label noise poses an important challenge in machine learning, especially in deep learning, in which large models with high expressive power dominate the field. Models of that kind are prone to memorizing incorrect labels, thereby harming generalization performance. Many methods have been proposed to address this problem, including robust loss functions and more complex label correction approaches. Robust loss functions are appealing due to their simplicity, but typically lack flexibility, while label correction usually adds substantial complexity to the training setup. In this paper, we suggest to address the shortcomings of both methodologies by "ambiguating" the target information, adding additional, complementary candidate labels in case the learner is not sufficiently convinced of the observed training label. More precisely, we leverage the framework of so-called superset learning to construct set-valued targets based on a confidence threshold, which deliver imprecise yet more reliable beliefs about the ground-truth, effectively helping the learner to suppress the memorization effect. In an extensive empirical evaluation, our method demonstrates favorable learning behavior on synthetic and real-world noise, confirming the effectiveness in detecting and correcting erroneous training labels.

Generalisation through Negation and Predicate Invention

The ability to generalise from a small number of examples is a fundamental challenge in machine learning. To tackle this challenge, we introduce an inductive logic programming (ILP) approach that combines negation and predicate invention. Combining these two features allows an ILP system to generalise better by learning rules with universally quantified body-only variables. We implement our idea in NOPI, which can learn normal logic programs with predicate invention, including Datalog programs with stratified negation. Our experimental results on multiple domains show that our approach can improve predictive accuracies and learning times.

PDE+: Enhancing Generalization via PDE with Adaptive Distributional Diffusion

The generalization of neural networks is a central challenge in machine learning, especially concerning the performance under distributions that differ from training ones. Current methods, mainly based on the data-driven paradigm such as data augmentation, adversarial training, and noise injection, may encounter limited generalization due to model non-smoothness. In this paper, we propose to investigate generalization from a Partial Differential Equation (PDE) perspective, aiming to enhance it directly through the underlying function of neural networks, rather than focusing on adjusting input data. Specifically, we first establish the connection between neural network generalization and the smoothness of the solution to a specific PDE, namely transport equation. Building upon this, we propose a general framework that introduces adaptive distributional diffusion into transport equation to enhance the smoothness of its solution, thereby improving generalization. In the context of neural networks, we put this theoretical framework into practice as PDE+ (PDE with Adaptive Distributional Diffusion) which diffuses each sample into a distribution covering semantically similar inputs. This enables better coverage of potentially unobserved distributions in training, thus improving generalization beyond merely data-driven methods. The effectiveness of PDE+ is validated through extensive experimental settings, demonstrating its superior performance compared to state-of-the-art methods. Our code is available at https://github.com/yuanyige/pde-add.

Exploiting Data Geometry in Machine Learning

A key challenge in Machine Learning (ML) is the identification of geometric structure in high-dimensional data. Most algorithms assume that data lives in a high-dimensional vector space; however, many applications involve non-Euclidean data, such as graphs, strings and matrices, or data whose structure is determined by symmetries in the underlying system. Here, we discuss methods for identifying geometric structure in data and how leveraging data geometry can give rise to efficient ML algorithms with provable guarantees.

Neural Oscillators for Generalization of Physics-Informed Machine Learning

A primary challenge of physics-informed machine learning (PIML) is its generalization beyond the training domain, especially when dealing with complex physical problems represented by partial differential equations (PDEs). This paper aims to enhance the generalization capabilities of PIML, facilitating practical, real-world applications where accurate predictions in unexplored regions are crucial. We leverage the inherent causality and temporal sequential characteristics of PDE solutions to fuse PIML models with recurrent neural architectures based on systems of ordinary differential equations, referred to as neural oscillators. Through effectively capturing long-time dependencies and mitigating the exploding and vanishing gradient problem, neural oscillators foster improved generalization in PIML tasks. Extensive experimentation involving time-dependent nonlinear PDEs and biharmonic beam equations demonstrates the efficacy of the proposed approach. Incorporating neural oscillators outperforms existing state-of-the-art methods on benchmark problems across various metrics. Consequently, the proposed method improves the generalization capabilities of PIML, providing accurate solutions for extrapolation and prediction beyond the training data.

Deep learning for sustainable agriculture: Weed classification model to optimize herbicide application

<p>Herbicides, chemical substances designed to eliminate weeds, find widespread use in agriculture to eradicate unwanted plants and enhance crop productivity, despite their adverse impacts on both human health and the environment. The study involves the construction of a neural network classifier employing a Convolutional Neural Network (CNN) through Keras to categorize images with corresponding labels. This research paper introduces two distinct neural networks: a basic neural network and a hybrid variant combining CNN with Keras. Both networks undergo training and testing, yielding an accuracy of 30% for the basic neural network, whereas the hybrid neural network achieves an impressive 97% accuracy. Consequently, this model significantly diminishes the need for herbicide spraying over crops such as fruits, vegetables, and sugarcane, aiming to safeguard humans, animals, birds, and the environment from the detrimental effects of harmful chemicals. Functioning as the elevated API within the TensorFlow framework, Keras furnishes a user-friendly and immensely efficient interface tailored to address machine learning (ML) challenges, particularly in the realm of contemporary deep learning. Encompassing all facets of the machine learning process, from data manipulation to fine-tuning hyper parameters to deployment, Keras was meticulously crafted to expedite rapid experimentation.</p>

Deep convolutional neural networks with genetic algorithm-based synthetic minority over-sampling technique for improved imbalanced data classification

Imbalanced data classification presents a challenge in machine learning, inducing biased model learning. Moreover, data dimensionality poses another challenge as it highly impacts classifier performance. This paper proposes a new deep-learning method that combines feature selection with oversampling to address these challenges. The proposed approach, GA-SMOTE-DCNN, integrates a genetic algorithm (GA) for feature selection, SMOTE for oversampling, and a deep 1D-convolutional neural network (DCNN) for classification. This study reveals that pre-splitting the data into training and testing sets before applying SMOTE results in higher accuracy, showing an improvement in accuracy ranging between 1.94% and 3.98% compared to post-SMOTE splitting for each dataset. This method achieved accuracy rates of 86.81% for the Balance Scale dataset, 86.15% for the Oil Spill dataset, 89.21% for the Yeast dataset, 91.32% for the Mammography dataset, 88.23% for the Australian credit dataset, and 89.53% for the German Credit dataset when compared with benchmark methods, underscoring its significance in tackling high-dimensional and imbalanced data classification problems. This method demonstrates scalability in effectively addressing challenges associated with high-dimensional and imbalanced data classification across various domains.

Dynamic selection of machine learning models for time-series data

Adapting to changes in time-series data is a major challenge in machine learning. This problem is particularly acute in the case of limited computing power (e.g., edge devices) that does not enable the independent training of new models. While multiple studies attempt to adequately train a machine learning model on rapidly shifting data (i.e., concept drift), the challenge of dynamically selecting the most effective machine learning model for future time steps has been largely overlooked. In this study, we propose Adaptive machine learning for Dynamic ENvironments (ADEN ), a method for analyzing future trends in time-series data and selecting the most suitable ML model. Our approach models multiple aspects of the analyzed data and analyzes the behavior of multiple machine learning models on earlier time steps. By training ADEN on multiple datasets, we can deploy a zero-shot model that does not require additional training when applied to new datasets. Our evaluation, conducted on 46 time-series classification datasets, shows that ADEN not only outperforms all baselines in terms of average performance but is also capable of avoiding sudden drops in performance that characterize all other evaluated algorithms.

Few-shot meta-learning for pre-symptomatic detection of Covid-19 from limited health tracker data

Detecting (or screening for) Covid-19 even before symptoms fully manifest, could enable patients to receive timely and life-saving treatment. Prior work has demonstrated that heart rate and step data from low-end wearables analyzed using deep learning models can detect Covid-19 reliably. However, significant individual differences in vital sign manifestation (high inter-subject variability) present a challenge to the generalization of deep learning models across diverse users. The limited amount of data in many medical scenarios further exacerbates this issue. Consequently, neural network models that can learn from limited vital sign data and varied inter-subject patterns are compelling. Meta-learning has emerged as a powerful technique for tackling various machine learning challenges, including insufficient data, domain shifts across datasets, and issues with generalization. This study proposes MetaCovid, a deep adaptation framework that employs meta-learning to address the variability of vital sign manifestation between subjects using only two days of data in order to detect Covid-19 before symptoms manifest. MetaCovid leverages heart rate and step measurements collected from consumer-grade health trackers over the preceding 2 days, extracts 45 digital bio-markers (features), which along with raw data, are fed into a deep GRU-based network with an attention mechanism, followed by uncertainty filtering. MetaCovid is trained using OC-MAML, a one-class few-shot MAML variant that adapts to the target distribution/user using only samples from the majority class. MetaCovid generalized well across two relatively small, publicly available Covid-19 datasets, achieving a recall of 0.81 and 0.92, and detecting 61% (14 out of 23) and 50% (17 out of 34) of users infected with Covid-19 before symptom onset. When OC-MAML was excluded from MetaCovid in an ablation study, the F2 score dropped by 36%, highlighting that meta-learning indeed facilitates adaptation of deep sensing models to varying vital sign patterns. Notably, MetaCovid outperforms the current state-of-art method by predicting Covid-19 early on day N using heart rate and step measurements from only the preceding 2 days compared to 28 days, reducing data requirements by 93%. To the best of our knowledge, our study is the first to propose utilizing meta-learning to mitigate vital sign variability with limited data for Covid-19 screening. We believe that MetaCovid will pave the way for innovative Covid-19 interventions that are accurate even with limited data and help contain the spread of infectious diseases in the future.

An end-to-end machine learning approach with explanation for time series with varying lengths

AbstractAn accurate prediction of complex product quality parameters from process time series by an end-to-end learning approach remains a significant challenge in machine learning. A special difficulty is the application of industrial batch process data because many batch processes generate variable length time series. In the industrial application of such methods, explainability is often desired. In this study, a 1D convolutional neural network (CNN) algorithm with a masking layer is proposed to solve the problem for time series of variable length. In addition, a novel combination of 1D CNN and class activation mapping (CAM) technique is part of this study to better understand the model results and highlight some regions of interest in the time series. As a comparative state-of-the-art unsupervised machine learning method, the One-Nearest Neighbours (1NN) algorithm combined with dynamic time warping (DTW) was used. Both methods are investigated as end-to-end learning methods with balanced and unbalanced class distributions and with scaled and unscaled input data, respectively. The FastDTW and DTAIDistance algorithms were investigated for the DTW calculation. The data set is made up of sensor signals that was collected during the production of plastic parts. The objective was to predict a quality parameter of plastic parts during production. For this research, the quality parameter will be a difficult or only destructively measurable parameter and both methods will be investigated for their applicability to this prediction task. The application of the proposed approach to an industrial facility for producing plastic products shows a prediction accuracy of 83.7%. It can improve the reverence method by approximately 1.4%. In addition to the slight increase in accuracy, the CNN training time was significantly reduced compared to the DTW calculation.

Advances in Machine Learning Processing of Big Data from Disease Diagnosis Sensors.

Exploring accurate, noninvasive, and inexpensive disease diagnostic sensors is a critical task in the fields of chemistry, biology, and medicine. The complexity of biological systems and the explosive growth of biomarker data have driven machine learning to become a powerful tool for mining and processing big data from disease diagnosis sensors. With the development of bioinformatics and artificial intelligence (AI), machine learning models formed by data mining have been able to guide more sensitive and accurate molecular computing. This review presents an overview of big data collection approaches and fundamental machine learning algorithms and discusses recent advances in machine learning and molecular computational disease diagnostic sensors. More specifically, we highlight existing modular workflows and key opportunities and challenges for machine learning to achieve disease diagnosis through big data mining.

Evolving pathway activation from cancer gene expression data using nature-inspired ensemble optimization

Class-imbalanced biological datasets pose significant challenges in machine learning and data analysis tasks. Prior methods to handle imbalance rely on data oversampling, which increases computational costs and overfitting. While feature selection and ensemble learning are promising techniques, current applications in imbalanced contexts are limited. To address these challenges, we present a novel framework called Hybrid Sampling Nature-Inspired Optimization Ensemble (HSNOE) to enhance the identification of hidden responders in imbalanced biological datasets. Our contributions are three-fold: 1) A hybrid undersampling and oversampling technique to mitigate class-imbalance; 2) Integrate an ant colony optimization-based feature selection that identifies informative feature subsets; 3) An ensemble classifier integrating diverse models trained on optimized features to improve performance. The experiments conducted on the five biological datasets demonstrate that HSNOE exhibits more stable comprehensive performance across six evaluation metrics compared to ten benchmark methods. We also conducted a biological analysis specifically on the Pan-cancer dataset. Moreover, the HSNOE method has been made publicly available on GitHub.11https://github.com/wangxb96/HSNOE.

Kmeans-SMOTE Integration for Handling Imbalance Data in Classifying Financial Distress Companies using SVM and Naïve Bayes

Imbalanced data presents significant challenges in machine learning, leading to biased classification outcomes that favor the majority class. This issue is especially pronounced in the classification of financial distress, where data imbalance is common due to the scarcity of such instances in real-world datasets. This study aims to mitigate data imbalance in financial distress companies using the Kmeans-SMOTE method by combining Kmeans clustering and the synthetic minority oversampling technique (SMOTE). Various classification approaches, including Nave Bayes and support vector machine (SVM), are implemented on a Kaggle financial distress data set to evaluate the effectiveness of Kmeans-SMOTE. Experimental results show that SVM outperforms Nave Bayes with impressive accuracy (99.1%), f1-score (99.1%), area under precision recall (AUPRC) (99.1%), and geometric mean (Gmean) (98.1%). On the basis of these results, Kmeans-SMOTE can balance the data effectively, leading to a quite significant improvement in performance.

Feature importance feedback with Deep Q process in ensemble-based metaheuristic feature selection algorithms

Feature selection is an indispensable aspect of modern machine learning, especially for high-dimensional datasets where overfitting and computational inefficiencies are common concerns. Traditional methods often employ either filter, wrapper, or embedded approaches, which have limitations in terms of robustness, computational load, or capability to capture complex interactions among features. Despite the utility of metaheuristic algorithms like Particle Swarm Optimization (PSO), Firefly Algorithm (FA), and Whale Optimization (WOA) in feature selection, there still exists a gap in efficiently incorporating feature importance feedback into these processes. This paper presents a novel approach that integrates the strengths of PSO, FA, and WOA algorithms into an ensemble model and further enhances its performance by incorporating a Deep Q-Learning framework for relevance feedbacks. The Deep Q-Learning module intelligently updates feature importance based on model performance, thereby fine-tuning the selection process iteratively. Our ensemble model demonstrates substantial gains in effectiveness over traditional and individual metaheuristic approaches. Specifically, the proposed model achieved a 9.5% higher precision, an 8.5% higher accuracy, an 8.3% higher recall, a 4.9% higher AUC, and a 5.9% higher specificity across multiple software bug prediction datasets and samples. By resolving some of the key issues in existing feature selection methods and achieving superior performance metrics, this work paves the way for more robust and efficient machine learning models in various applications, from healthcare to natural language processing scenarios. This research provides an innovative framework for feature selection that promises not only superior performance but also offers a flexible architecture that can be adapted for a variety of machine learning challenges.

STAVOS: A Medaka Larval Cardiac Video Segmentation Method Based on Deep Learning

Medaka (Oryzias latipes), as a crucial model organism in biomedical research, holds significant importance in fields such as cardiovascular diseases. Currently, the analysis of the medaka ventricle relies primarily on visual observation under a microscope, involving labor-intensive manual operations and visual assessments that are cumbersome and inefficient for biologists. Despite attempts by some scholars to employ machine learning methods, limited datasets and challenges posed by the blurred edges of the medaka ventricle have constrained research to relatively simple tasks such as ventricle localization and heart rate statistics, lacking precise segmentation of the medaka ventricle edges. To address these issues, we initially constructed a video object segmentation dataset comprising over 7000 microscopic images of medaka ventricles. Subsequently, we proposed a semi-supervised video object segmentation model named STAVOS, incorporating a spatial-temporal attention mechanism. Additionally, we developed an automated system capable of calculating various parameters and visualizing results for a medaka ventricle using the provided video. The experimental results demonstrate that STAVOS has successfully achieved precise segmentation of medaka ventricle contours. In comparison to the conventional U-Net model, where a mean accuracy improvement of 0.392 was achieved, our model demonstrates significant progress. Furthermore, when compared to the state-of-the-art Tackling Background Distraction (TBD) model, there is an additional enhancement of 0.038.

OEBench: Investigating Open Environment Challenges in Real-World Relational Data Streams

How to get insights from relational data streams in a timely manner is a hot research topic. Data streams can present unique challenges, such as distribution drifts, outliers, emerging classes, and changing features, which have recently been described as open environment challenges for machine learning. While existing studies have been done on incremental learning for data streams, their evaluations are mostly conducted with synthetic datasets. Thus, a natural question is how those open environment challenges look like and how existing incremental learning algorithms perform on real-world relational data streams. To fill this gap, we develop an Open Environment Benchmark named OEBench to evaluate open environment challenges in real-world relational data streams. Specifically, we investigate 55 real-world relational data streams and establish that open environment scenarios are indeed widespread, which presents significant challenges for stream learning algorithms. Through benchmarks with existing incremental learning algorithms, we find that increased data quantity may not consistently enhance the model accuracy when applied in open environment scenarios, where machine learning models can be significantly compromised by missing values, distribution drifts, or anomalies in real-world data streams. The current techniques are insufficient in effectively mitigating these challenges brought by open environments. More researches are needed to address real-world open environment challenges. All datasets and code are open-sourced in https://github.com/Xtra-Computing/OEBench.

Burnt-in Text Recognition from Medical Imaging Modalities: Existing Machine Learning Practices

In recent times, medical imaging has become a significant component of clinical diagnosis and examinations to detect and evaluate various medical conditions. The interpretation of these medical examinations and the patient’s demographics are usually textual data, which is burned in on the pixel content of medical imaging modalities (MIM). Example of these MIM includes ultrasound and X-ray imaging. As artificial intelligence advances for medical applications, there is a high demand for the accessibility of these burned-in textual data for various needs. This article aims to review the significance of burned-in textual data recognition in MIM and recent research regarding the machine learning approach, challenges, and open issues for further investigation on this application. The review describes the significant problems in this study area as low resolution and background interference of textual data. Finally, the review suggests applying more advanced deep learning ensemble algorithms as possible solutions.

The Challenges of Machine Learning: A Critical Review

The concept of learning has multiple interpretations, ranging from acquiring knowledge or skills to constructing meaning and social development. Machine Learning (ML) is considered a branch of Artificial Intelligence (AI) and develops algorithms that can learn from data and generalize their judgment to new observations by exploiting primarily statistical methods. The new millennium has seen the proliferation of Artificial Neural Networks (ANNs), a formalism able to reach extraordinary achievements in complex problems such as computer vision and natural language recognition. In particular, designers claim that this formalism has a strong resemblance to the way the biological neurons operate. This work argues that although ML has a mathematical/statistical foundation, it cannot be strictly regarded as a science, at least from a methodological perspective. The main reason is that ML algorithms have notable prediction power although they cannot necessarily provide a causal explanation about the achieved predictions. For example, an ANN could be trained on a large dataset of consumer financial information to predict creditworthiness. The model takes into account various factors like income, credit history, debt, spending patterns, and more. It then outputs a credit score or a decision on credit approval. However, the complex and multi-layered nature of the neural network makes it almost impossible to understand which specific factors or combinations of factors the model is using to arrive at its decision. This lack of transparency can be problematic, especially if the model denies credit and the applicant wants to know the specific reasons for the denial. The model’s “black box” nature means it cannot provide a clear explanation or breakdown of how it weighed the various factors in its decision-making process. Secondly, this work rejects the belief that a machine can simply learn from data, either in supervised or unsupervised mode, just by applying statistical methods. The process of learning is much more complex, as it requires the full comprehension of a learned ability or skill. In this sense, further ML advancements, such as reinforcement learning and imitation learning denote encouraging similarities to similar cognitive skills used in human learning.

Overview of the 8th Social Media Mining for Health Applications (#SMM4H) shared tasks at the AMIA 2023 Annual Symposium.

The aim of the Social Media Mining for Health Applications (#SMM4H) shared tasks is to take a community-driven approach to address the natural language processing and machine learning challenges inherent to utilizing social media data for health informatics. In this paper, we present the annotated corpora, a technical summary of participants' systems, and the performance results. The eighth iteration of the #SMM4H shared tasks was hosted at the AMIA 2023 Annual Symposium and consisted of 5 tasks that represented various social media platforms (Twitter and Reddit), languages (English and Spanish), methods (binary classification, multi-class classification, extraction, and normalization), and topics (COVID-19, therapies, social anxiety disorder, and adverse drug events). In total, 29 teams registered, representing 17 countries.In general, the top-performing systems used deep neural network architectures based on pre-trained transformer models. In particular, the top-performing systems for the classification tasks were based on single models that were pre-trained on social media corpora. To facilitate future work, the datasets-a total of 61353 posts-will remain available by request, and the CodaLab sites will remain active for a post-evaluation phase.

Leveraging open science machine learning challenges for data constrained planetary mission instruments

Abstract We set up two open-science machine learning (ML) challenges focusing on building models to automatically analyse mass spectrometry (MS) data for Mars exploration. ML challenges provide an excellent way to engage a diverse set of experts with benchmark training data, explore a wide range of ML and data science approaches, and identify promising models based on empirical results, as well as to get independent external analyses to compare with those of the internal team. These two challenges were proof-of-concept projects to analyse the feasibility of combining data collected from different instruments in a single ML application. We selected MS data from (1) commercial instruments and (2) the Sample Analysis at Mars (an instrument suite that includes a mass spectrometer subsystem onboard the Curiosity rover) testbed. These challenges, organized with DrivenData, gathered more than 1150 unique participants from all over the world, and obtained more than 600 solutions contributing powerful models to the analysis of rock and soil samples relevant to planetary science using various MS data sets. These two challenges demonstrated the suitability and value of multiple ML approaches to classifying planetary analogue data sets from both commercial and flight-like instruments. We present the processes from the problem identification, challenge set-ups, and challenge results that gathered creative and diverse solutions from worldwide participants, in some cases with no backgrounds in MS. We also present the potential and limitations of these solutions for ML application in future planetary missions. Our longer term goal is to deploy these powerful methods onboard the spacecraft to autonomously guide space operations and reduce ground-in-the-loop reliance.