Deconstructing deep imbalance regression: a comprehensive review and experimental evaluation

Clustering is a fundamental problem in machine learning. To address this, a large number of algorithms have been developed. Some of these algorithms, such as K-means, handle the original data directly, while others, such as spectral clustering, apply linear transformation to the data. Still others, such as kernel-based algorithms, use nonlinear transformation. Since the performance of the clustering depends strongly on the quality of the data representation, representation learning approaches have been extensively researched. With the recent advances in deep learning, deep neural networks are being increasingly utilized to learn clustering-friendly representation. We provide here a review of existing algorithms that are being used to jointly optimize deep neural networks and clustering methods.

Antimicrobial peptides (AMPs) have garnered significant attention from researchers as effective alternatives to antibiotics. In recent years, deep learning has demonstrated unique advantages in AMP prediction, surpassing traditional machine learning methods and offering new avenues to address the issue of antibiotic resistance. This review introduces the research foundations of deep learning in AMP prediction, covering data set status, processing methods, and representation learning approaches. It particularly focuses on the application of basic models, language models, graph-related models, and other mixed and multimodal models for AMP prediction from the perspective of algorithmic models. Additionally, this review provides a comparative validation using classic deep learning models, offering guidance for subsequent research. Finally, it discusses the challenges and opportunities faced by deep learning algorithms in AMP prediction, particularly in terms of data balance, data augmentation, cyclic peptides, and interpretability, providing a comprehensive perspective and reference for further research in this field.

Learning an expressive representation from multi-view data is a key step in various real-world applications. In this paper, we propose a semi-supervised multi-view deep discriminant representation learning (SMDDRL) approach. Unlike existing joint or alignment multi-view representation learning methods that cannot simultaneously utilize the consensus and complementary properties of multi-view data to learn inter-view shared and intra-view specific representations, SMDDRL comprehensively exploits the consensus and complementary properties as well as learns both shared and specific representations by employing the shared and specific representation learning network. Unlike existing shared and specific multi-view representation learning methods that ignore the redundancy problem in representation learning, SMDDRL incorporates the orthogonality and adversarial similarity constraints to reduce the redundancy of learned representations. Moreover, to exploit the information contained in unlabeled data, we design a semi-supervised learning framework by combining deep metric learning and density clustering. Experimental results on three typical multi-view learning tasks, i.e., webpage classification, image classification, and document classification demonstrate the effectiveness of the proposed approach.

Cyber-physical systems (CPS) have gained a lot of attention recently in design and control and have many applications including healthcare, transportation, etc. Over the past few decades, the adoption of machine learning (ML)-enabled cyber-physical systems are becoming prevalent in various sectors of modern society. Although machine (deep) learning systems have been very beneficial in many applications and made significant improvements possible in CPS, it is important to investigate the resilience and scalability of these systems as they are being used extensively in various industries and have shown failures under malicious attacks or when they are scaled up to be run at large scales. Recent focus on the robustness of deep learning algorithms to adversarial attacks produced a large variety of algorithms for training robust models. Most of the effective algorithms involve solving the min-max optimization problem for training robust models (min step) under worst-case attacks (max step). However, they often suffer from high computational costs. Therefore, it becomes difficult to readily apply such algorithms for moderate to large size real-world data sets. To alleviate this, this report proposes a novel discrete-time dynamical system-based algorithm that aims to find the saddle point of a min-max optimization problem in the presence of uncertainties. Based on that, a fast robust training algorithm is devised which is applicable to deep neural networks. Although such training involves highly non-convex robust optimization problems, empirical results show that the algorithm can achieve significant robustness compared to other state-of-the-art robust models on benchmark data sets. The effect of adversarial attacks in Reinforcement Learning (RL) environments is also explored in this thesis. As this area has become a new center of attention for research in machine (deep) learning, it is imperative to study the performance of RL systems under malicious state and actuator attacks. In this thesis, projected gradient descent (PGD) attacks are crafted and applied to the action-space of fully trained Deep RL agents. This work shows that a well-performing agent that is initially susceptible to action space perturbations (e.g., actuator attacks) can be robustified against similar perturbations through adversarial training. Another key element in using deep learning models is to make them capable of scalability in order to make them functional in large-scale systems. To achieve this, distributed centralized learning has emerged as a class of machine (deep) learning algorithms that enables a group of collaborative learning agents to train models using a dataset distributed among the agents with the aid of a central parameter server. Recently, decentralized learning algorithms have demonstrated state-of-the-art results, comparable with centralized algorithms which makes them independent from the central parameter server. However, a key requirement to achieve such performance has been balanced distribution (among classes) of data among the agents, also referred to as IID data. In real-life applications, having a precise IID distribution of data among the agents is often not feasible. To address this, a decentralized learning algorithm is proposed, where each agent collects the gradient information from its neighboring agents and updates its model with a projected gradient. it is demonstrated in this work that this algorithm is effective on both IID and non-IID data distributions and comparisons are made against the state-of-the-art algorithms analytically and experimentally. As a real-world application, being able to use deep learning models to detect anomalies, anticipate a certain incident, and find similar trends from real-world datasets have been explored by researchers. Generally, these data can come from various sensor types installed to capture different features of the data e.g. cameras installed in agricultural fields to capture different sections of the field from various angles. The viability of distributed learning algorithms to learn from these sensors in a decentralized fashion is explored in this thesis. These algorithms are used to train autoencoders to learn from $300$ cameras installed in agricultural fields. The trained models are then used to conduct downstream tasks such as anomaly detection and image retrieval. Experimental results show that by distributing the learning tasks among sensors not only accurate models can be achieved, but learning from large datasets connected with different graph topologies would be feasible too. In Summary, this dissertation attempts to bring robustness and scalability qualities to deep learning algorithms in various settings including supervised, unsupervised, and reinforcement learning.

Speaker recognition is the process of identifying an individual from their voices, and it has been widely applied in many real-world applications. Recently, deep learning has instigated a revolutionary high success rate in speaker recognition. The major advantage of deep learning over conventional methods for speaker recognition is attributed to its representation ability, and the ability to produce highly abstract embedding features from utterances. Recent researches had revealed that deep learning method in learning speaker features from raw data, is strongly depending on a speaker's language. However, only minimal researches had done on deep learning over Vietnamese speaker recognition to present. Nevertheless, this paper has proposed a deep transfer learning method which integrates both transfer learning and deep learning to build models for Vietnamese speaker recognition. Our experimental results indicated that the proposed method is able to build accurate models for Vietnamese speaker recognition.

Deep long-tailed learning, one of the most challenging problems in visual recognition, aims to train well-performing deep models from a large number of images that follow a long-tailed class distribution. In the last decade, deep learning has emerged as a powerful recognition model for learning high-quality image representations and has led to remarkable breakthroughs in generic visual recognition. However, long-tailed class imbalance, a common problem in practical visual recognition tasks, often limits the practicality of deep network based recognition models in real-world applications, since they can be easily biased towards dominant classes and perform poorly on tail classes. To address this problem, a large number of studies have been conducted in recent years, making promising progress in the field of deep long-tailed learning. Considering the rapid evolution of this field, this article aims to provide a comprehensive survey on recent advances in deep long-tailed learning. To be specific, we group existing deep long-tailed learning studies into three main categories (i.e., class re-balancing, information augmentation and module improvement), and review these methods following this taxonomy in detail. Afterward, we empirically analyze several state-of-the-art methods by evaluating to what extent they address the issue of class imbalance via a newly proposed evaluation metric, i.e., relative accuracy. We conclude the survey by highlighting important applications of deep long-tailed learning and identifying several promising directions for future research.

The combination of deep learning with whole-brain computational models reveals the low-dimensional representation of neurodegenerative diseases, which emerges from a highly multidimensional brain, providing valuable insight into pathological states' diagnostic, prognosis, and treatment response.

The combination of deep learning with whole-brain computational models reveals the low-dimensional representation of neurodegenerative diseases, which emerges from a highly multidimensional brain, providing valuable insight into pathological states' diagnostic, prognosis, and treatment response.

Facial age recognition has been widely used in real-world applications. Most of current facial age recognition methods use deep learning to extract facial features to identify age. However, due to the high dimension features of faces, deep learning methods might extract a lot of redundant features, which is not beneficial for facial age recognition. To improve facial age recognition effectively, this paper proposed the deep manifold learning (DML), a combination of deep learning and manifold learning. In DML, deep learning was used to extract high-dimensional facial features, and manifold learning selected age-related features from these high-dimensional facial features for facial age recognition. Finally, we validated the DML on Multivariate Observations of Reactions and Physical Health (MORPH) and Face and Gesture Recognition Network (FG-NET) datasets. The results indicated that the mean absolute error (MAE) of MORPH is 1.60 and that of FG-NET is 2.48. Moreover, compared with the state of the art facial age recognition methods, the accuracy of DML has been greatly improved.

dipped into the urine.When 5 ml. of this was applied to a 3-in.x3-in.(7.5-cm.X7.5cm.)doubled piece of towel, a much reduced

Although plant disease recognition has witnessed a significant improvement with deep learning in recent years, a common observation is that current deep learning methods with decent performance tend to suffer in real-world applications. We argue that this illusion essentially comes from the fact that current plant disease recognition datasets cater to deep learning methods and are far from real scenarios. Mitigating this illusion fundamentally requires an interdisciplinary perspective from both plant disease and deep learning, and a core question arises. What are the characteristics of a desired dataset? This paper aims to provide a perspective on this question. First, we present a taxonomy to describe potential plant disease datasets, which provides a bridge between the two research fields. We then give several directions for making future datasets, such as creating challenge-oriented datasets. We believe that our paper will contribute to creating datasets that can help achieve the ultimate objective of deploying deep learning in real-world plant disease recognition applications. To facilitate the community, our project is publicly available at https://github.com/xml94/PPDRD with the information of relevant public datasets.

Security concerns for Internet of Things (IoT) applications have been alarming because of their widespread use in different enterprise systems. The potential threats to these applications are constantly emerging and changing, and, therefore, sophisticated and dependable defense solutions are necessary against such threats. With the rapid development of IoT networks and evolving threat types, the traditional machine learning based IDS must update to cope with the security requirements of the current sustainable IoT environment. In recent years, deep learning and deep transfer learning have progressed and experienced great success in different fields and have emerged as a potential solution for dependable network intrusion detection. However, new and emerging challenges have arisen related to the accuracy, efficiency, scalability, and dependability of the traditional IDS in a heterogeneous IoT setup. This manuscript proposes a deep transfer learning based dependable IDS model that outperforms several existing approaches. The unique contributions include effective attribute selection, which is best suited to identify normal and attack scenarios for a small amount of labeled data, designing a dependable deep transfer learning based ResNet model and evaluating considering real-world data. To this end, a comprehensive experimental performance evaluation has been conducted. Extensive analysis and performance evaluation show that the proposed model is robust, more efficient, and has demonstrated better performance, ensuring dependability.

A survey of deep learning applications in cryptocurrency

Ensuring data reliability and mitigating failures are critical challenges in large-scale cloud infrastructures, given their complexity, dynamic nature, and the increasing demand for real-time data processing. Traditional approaches often struggle with scalability, adaptability, and predictive accuracy, necessitating innovative solutions. Deep learning, with its ability to model complex patterns and predict outcomes, has emerged as a transformative tool for addressing these challenges. This article explores the application of deep learning techniques to enhance data reliability and failure mitigation in large-scale cloud systems. It examines methods such as anomaly detection using auto-encoders and convolutional neural networks (CNNs), predictive maintenance through recurrent neural networks (RNNs) and long short-term memory (LSTM) models, and fault localization enabled by deep reinforcement learning. Additionally, intelligent resource allocation, adaptive scaling, and data recovery processes are highlighted as critical areas where deep learning delivers significant advancements. Through real-world case studies and experimental evaluations, the research demonstrates the superiority of deep learning approaches over traditional methods in terms of accuracy, scalability, and efficiency. While the findings underscore deep learning's potential, the discussion also addresses limitations, ethical considerations, and integration challenges. This study not only establishes a framework for leveraging deep learning in cloud reliability and resilience but also outlines future directions for research, emphasizing model interpret-ability, federated learning, and sustainable AI practices.

Consumer-grade EEG devices have the potential for widespread brain-computer interface deployment but pose significant challenges for emotion recognition due to reduced spatial coverage and the variable signal quality encountered in uncontrolled deployment environments. While deep learning approaches have employed increasingly complex architectures, their efficacy in noisy consumer-grade signals and cross-system generalizability remains unexplored. We present a comprehensive systematic comparison of EEGNet architecture, which has become a benchmark model for consumer-grade EEG analysis versus traditional machine learning, examining when and why domain-specific feature engineering outperforms end-to-end learning in resource constrained scenarios. We conducted comprehensive within-dataset evaluation using the DREAMER dataset (23 subjects, Emotiv EPOC 14-channel) and challenging cross-dataset validation (DREAMER→SEED-VII transfer). Traditional ML employed domain-specific feature engineering (statistical, frequency-domain, and connectivity features) with random forest classification. Deep learning employed both optimized and enhanced EEGNet architectures, specifically designed for low channel consumer EEG systems. For cross-dataset validation, we implemented progressive domain adaptation combining anatomical channel mapping, CORAL adaptation, and TCA subspace learning. Statistical validation included 345 comprehensive evaluations with fivefold cross-validation × 3 seeds × 23 subjects, Wilcoxon signed-rank tests, and Cohen's d effect size calculations. Traditional ML achieved superior within-dataset performance (F1 = 0.945 ± 0.034 versus 0.567 for EEGNet architectures, p < 0.000001, Cohen's d = 3.863, 67% improvement) across 345 evaluations. Cross-dataset validation demonstrated good performance (F1 = 0.619 versus 0.007) through systematic domain adaptation. Progressive improvements included anatomical channel mapping (5.8× improvement), CORAL domain adaptation (2.7× improvement), and TCA subspace learning (4.5× improvement). Feature analysis revealed inter-channel connectivity patterns contributed 61% of the discriminative power. Traditional ML demonstrated superior computational efficiency (95% faster training, 10× faster inference) and excellent stability (CV = 0.036). Fairness validation experiments supported the advantage of traditional ML in its ability to persist even with minimal feature engineering (F1 = 0.842 vs. 0.646 for enhanced EEGNet), and robustness analysis revealed that deep learning degrades more under consumer-grade noise conditions (17% vs. <1% degradation). These findings challenge the assumption that architectural complexity universally improves biosignal processing performance in consumer-grade applications. Through the comparison of traditional ML against the EEGNet consumer-grade architecture, we highlight the potential that domain-specific feature engineering and lightweight adaptation techniques can provide superior accuracy, stability, and practical deployment capabilities for consumer-grade EEG emotion recognition. While our empirical comparison focused on EEGNet, the underlying principles regarding data efficiency, noise robustness, and the value of domain expertise could extend to comparisons with other complex architectures facing similar constraints in further research. This comprehensive domain adaptation framework enables robust cross-system deployment, addressing critical gaps in real-world BCI applications.