Imbalanced Domains Research Articles

Wasserstein bi-classifier adversarial learning network for machinery fault diagnostics

In industrial applications of machinery fault diagnostics, deep learning has been widely adopted to process large amounts of monitoring data. Unfortunately, due to the domain discrepancy, diagnostic models trained with source domain data suffer from degraded diagnostic performance on the target domain. To address this problem, a Wasserstein bi-classifier adversarial learning network (WBALN) is proposed. Specifically, WBALN consists of a feature extractor, two classifiers, and a discrepancy metric based on Wasserstein distance. A two-stream optimization strategy is used in the training process, which involves jointly performing bi-classifier adversarial learning and Wasserstein generative adversarial network (WGAN)-based adversarial learning. In the bi-classifier training stream, a min-max game is conducted between a discrepancy detector composed of two classifiers and the feature extractor to reduce the disparity between these classifiers. In the WGAN-based training stream, a Wasserstein adversarial discrepancy (WAD) is applied in combination with the original classifier as a domain discriminator, which achieves fault diagnosis and distribution alignment through a unified objective. This WAD enables WBALN to achieve sufficient feature alignment using the predicted discriminative information. In addition, the utilization of the nuclear-norm is useful for ensuring the determinacy and diversity of predictions. Except for ordinary domain adaptation, WBALN is also extended for challenging problems about inter-class imbalanced domain adaptation. The performance of the proposed WBALN is verified through multiple experiments on two bearing datasets.

CentroIDA: Cross-domain class discrepancy minimization based on accumulative class-centroids for Imbalanced Domain Adaptation

Unsupervised Domain Adaptation (UDA) is a critical task in transfer learning, which seeks to apply the knowledge acquired in one domain (the source domain) to a different but related domain (the target domain). In this context, the two domains exhibit Out-Of-Distribution(OOD), and the label information of the target domain is unavailable. Although numerous UDA methods have made significant strides in mitigating the discrepancies in feature distributions across domains, they often neglect a more prevalent real-world scenario. This scenario, referred to as Imbalanced Domain Adaptation (IDA) task, is characterized by the coexistence of differences in both feature and label distributions. Our empirical analysis suggests that overlooking the disparities in label distributions can severely compromise the performance of the model, thereby limiting the generalizability of domain adaptation models. Consequently, our objective is to introduce a model capable of concurrently addressing differences in both feature and label distributions, thereby fostering a broader application of domain adaptation methodologies. In this paper, we propose a cross-domain class discrepancy minimization method based on accumulative class-centroids for IDA, termed as centroIDA. The strategies of accumulative class-centroids alignment and class-wise feature alignment pay heed to the relationship between cross-domain intra-class features and inter-class features, thereby facilitating conditional feature alignment to overcome both feature and label distribution differences simultaneously. Our experimental evaluation on OfficeHome and DomainNet datasets reveals that, in comparison to TIToK (a novel method also designed to tackle the IDA problem), our method enhances the average accuracy by 5.6% and 2.7%, respectively. Furthermore, as the label shift intensifies, our method consistently outperforms other methods, demonstrating a robust resistance to label shift.

Cross-modality facial attribute translation with feature space consistency between imbalanced domains

Cross-modality facial attribute translation with feature space consistency between imbalanced domains

Deep Domain Adaptation With Max-Margin Principle for Cross-Project Imbalanced Software Vulnerability Detection

Software vulnerabilities (SVs) have become a common, serious, and crucial concern due to the ubiquity of computer software. Many AI-based approaches have been proposed to solve the software vulnerability detection (SVD) problem to ensure the security and integrity of software applications (in both the development and testing phases). However, there are still two open and significant issues for SVD in terms of i) learning automatic representations to improve the predictive performance of SVD, and ii) tackling the scarcity of labeled vulnerability datasets that conventionally need laborious labeling effort by experts. In this paper, we propose a novel approach to tackle these two crucial issues. We first exploit the automatic representation learning with deep domain adaptation for SVD. We then propose a novel cross-domain kernel classifier leveraging the max-margin principle to significantly improve the transfer learning process of SVs from imbalanced labeled into imbalanced unlabeled projects. Our approach is the first work that leverages solid body theories of the max-margin principle, kernel methods, and bridging the gap between source and target domains for imbalanced domain adaptation (DA) applied in cross-project SVD . The experimental results on real-world software datasets show the superiority of our proposed method over state-of-the-art baselines. In short, our method obtains a higher performance on F1-measure, one of the most important measures in SVD, from 1.83% to 6.25% compared to the second highest method in the used datasets.

Intelligent fault diagnosis under imbalanced multivariate working conditions leveraging dynamic unsupervised domain adaptation with sample and margin regularization

Utilizing unsupervised domain adaptation for intelligent fault diagnosis (IFD) has demonstrated significant potential for ensuring the security of machinery systems. Nonetheless, the inherent imbalance attribute of collected data affects the performance of diagnostic model. Especially, for machines working under varied conditions, the acquired unlabeled data frequently exhibits diverse degrees of distributional deviations, thus further undermining the transferable model’s generalization capability. To address this challenge, we introduce a method termed Dynamic Unsupervised Imbalanced Domain Adaptation (DUIDA) for IFD. Employment of class rebalancing and label-dependent margin regularization strategies optimizes the selection of decision boundaries which counteract the distributional deviations introduced by the imbalance. In addition, by integrating a dynamic weighting mechanism, encompassing both adversarial-based and MMD-based domain adaptation, our model becomes versatile across varied UIDA tasks, assigning higher weights to fundamental faulty features. Finally, our empirical analyses on two faulty bearing datasets substantiate the efficacy and superior performance of the proposed framework across diverse operational scenarios.

Single imbalanced domain generalization network for intelligent fault diagnosis of compressors in HVAC systems under unseen working conditions

Effective fault diagnosis of compressors in heating, ventilation, and air conditioning (HVAC) systems is critical to ensure service reliability and boost energy efficiency. HVAC compressors are distributed in different areas and work under heterogeneous conditions, which poses emerging challenges to their data-driven modeling. Most existing methods assume multiple category-balanced source domains for model training. Although domain adaptation and generalization methods have emerged to address the data distribution discrepancies in cross-domain fault diagnosis, limited source domains and imbalanced fault categories across domains still constrain the real-world applicability of data-driven models in HVAC compressor fault diagnosis under unseen working conditions. Therefore, this paper studies a significant fault diagnosis problem named single imbalanced domain generalization (SIDG) and proposes a corresponding network (SIDGNet) for intelligent HVAC compressor fault diagnosis. Specifically, a rare fault diagnosis module combined with focal loss is introduced to tackle the class imbalance problem. To achieve better diagnostic boundaries and resist unknown data distribution discrepancies, joint supervised contrastive learning and adversarial learning with specialized data augmentation are introduced as auxiliary modules to improve the robustness and generalizability of SIDGNet. An improved uncertainty-based dynamic weighting mechanism is developed to intelligently balance the weights of module-specific losses during training, which ensures an efficient and stable optimization process. Extensive SIDG fault diagnosis experiments conducted on HVAC compressors demonstrate the superiority of SIDGNet over existing models in SIDG scenarios.

Class overlap handling methods in imbalanced domain: A comprehensive survey

Class overlap handling methods in imbalanced domain: A comprehensive survey

Imbalanced Domain Adaptation for Automatic Modulation Classification

Imbalanced Domain Adaptation for Automatic Modulation Classification

Imbalanced domain generalization via Semantic-Discriminative augmentation for intelligent fault diagnosis

Domain generalization-based fault diagnosis (DGFD) has garnered significant attention due to its ability to generalize prior diagnostic knowledge to unseen working conditions or machines. However, existing DGFD methods are generally implemented under the premise of class balance, which may not accurately reflect real-world diagnosis scenarios since fault data collected in practical engineering often exhibits severe class imbalance. To address this challenge, this paper proposes a semantic-discriminative augmentation-driven network for imbalanced domain generalization fault diagnosis. A semantic regularization-based mixup strategy is devised to synthesize sufficient reliable samples to compensate for minority classes. Subsequently, discriminative representations are acquired by minimizing the triplet loss, thereby enhancing the model's generalization capabilities. Extensive evaluations, including cross-working condition and cross-machine tasks, demonstrate the effectiveness and superiority of the proposed method.

Self-Adaptive Imbalanced Domain Adaptation With Deep Sparse Autoencoder

Domain adaptation aims to transfer knowledge between different domains to develop an effective hypothesis in the target domain with scarce labeled data, which is an effective method for remedying the problem of labeled data requirement in deep learning. In reality, it is unavoidable that the dataset has a large gap in the number of positive and negative instances across different categories in source and target domains, which is the imbalanced domain adaptation problem. However, since the imbalanced degree always varies greatly in different source and target domains datasets, most existing imbalanced domain adaptation models fix the imbalanced parameters which can not adapt to the change of the proportion between positive and negative instances in different domains. To address this problem, in this paper, we propose a self-adaptive imbalanced domain adaptation method via deep sparse autoencoder, which can adjust the model automatically according to the imbalanced extent for bridging the chasm of domains. More specifically, the self-adaptive imbalanced cross entropy loss is designed for emphasizing more on minority categories and compensating the bias of training loss automatically. In addition, to alleviate the deficient problem of labeled data, we further propose the unlabeled information incorporating method by minimizing the distribution discrepancy of high-level representation space between the source and target domains. Experiments on several real-world datasets demonstrate the effectiveness of our method compared to other state-of-the-art methods.

SIDA

The coronavirus disease 2019 (COVID-19) pneumonia still persists and its chief complaint is dry cough. Physicians design wireless stethoscopes to facilitate diagnosis, however, lung sounds could be easily interfered with by external noises. To achieve lung sound enhancement, prior researches mostly assume the amount of clean and noisy data are the same. This assumption is hardly met due to extensive labor effort for data collection and annotation. The data imbalance across domains widely happens in real-world IoT systems, e.g. sound enhancement and WiFi-based human sensing. In this paper, we propose SIDA, a self-supervised imbalanced domain adaptation framework for sound enhancement and WiFi sensing, which makes it a generic time series domain adaptation solution for IoT systems. SIDA proposes a self-supervised imbalanced domain adaptation model that separately learns the representation of time series signals in a minority domain with limited samples, a majority domain with rich samples, and their mapping relations. For lung sound enhancement, we further proposes a phase correction model to sanitize the phase and a SNR prediction algorithm to recursively perform domain adaptation in an imbalanced noisy and clean lung sound dataset. Extensive experiments demonstrate SIDA increases noisy samples' SNR by 16.49dB and 4.06dB on a synthetic and a realistic imbalanced lung sound dataset, respectively. For WiFi-based human sensing, SIDA designs a cross-domain WiFi-based human identification model irrespective of walking trajectory. A specific trajectory where a group of people walks along in a realistic testing environment is considered the minority domain, and several other trajectories are stored at a server as the majority domain. Extensive experiments show SIDA could recognize individuals with an average accuracy of 94.72% and significantly outperform baselines on highly imbalanced WiFi dataset in cross-domain human identification tasks.

Advancing Imbalanced Domain Adaptation: Cluster-Level Discrepancy Minimization With a Comprehensive Benchmark.

Unsupervised domain adaptation methods have been proposed to tackle the problem of covariate shift by minimizing the distribution discrepancy between the feature embeddings of source domain and target domain. However, the standard evaluation protocols assume that the conditional label distributions of the two domains are invariant, which is usually not consistent with the real-world scenarios such as long-tailed distribution of visual categories. In this article, the imbalanced domain adaptation (IDA) is formulated for a more realistic scenario where both label shift and covariate shift occur between the two domains. Theoretically, when label shift exists, aligning the marginal distributions may result in negative transfer. Therefore, a novel cluster-level discrepancy minimization (CDM) is developed. CDM proposes cross-domain similarity learning to learn tight and discriminative clusters, which are utilized for both feature-level and distribution-level discrepancy minimization, palliating the negative effect of label shift during domain transfer. Theoretical justifications further demonstrate that CDM minimizes the target risk in a progressive manner. To corroborate the effectiveness of CDM, we propose two evaluation protocols according to the real-world situation and benchmark existing domain adaptation approaches. Extensive experiments demonstrate that negative transfer does occur due to label shift, while our approach achieves significant improvement on imbalanced datasets, including Office-31, Image-CLEF, and Office-Home.

Evaluating active learning for blind imbalanced domains

Deep learning, which has been very successful in recent years, requires a large amount of data. Active learning has been widely studied and used for decades to reduce annotation costs and now attracts lots of attention in deep learning. Many real-world deep learning applications use active learning to select the informative data to be annotated. In this paper, we first investigate laboratory settings for active learning. We show significant gaps between the results from different laboratory settings and describe our practical laboratory setting that reasonably reflects the active learning use cases in real-world applications. Then, we introduce a problem setting of blind imbalanced domains. Any data set includes multiple domains, e.g., individuals in handwritten character recognition with different social attributes. Major domains have many samples, and minor domains have few samples in the training set. However, we must accurately infer both major and minor domains in the test phase. We experimentally compare different methods of active learning for blind imbalanced domains in our practical laboratory setting. We show that a simple active learning method using softmax margin and a model training method using distance-based sampling with center loss, both working in the deep feature space, perform well.

Generative Inference Network for Imbalanced Domain Generalization.

Domain generalization (DG) aims to learn transferable knowledge from multiple source domains and generalize it to the unseen target domain. To achieve such expectation, the intuitive solution is to seek domain-invariant representations via generative adversarial mechanism or minimization of cross-domain discrepancy. However, the widespread imbalanced data scale problem across source domains and category in real-world applications becomes the key bottleneck of improving generalization ability of model due to its negative effect on learning the robust classification model. Motivated by this observation, we first formulate a practical and challenging imbalance domain generalization (IDG) scenario, and then propose a straightforward but effective novel method generative inference network (GINet), which augments reliable samples for minority domain/category to promote discriminative ability of the learned model. Concretely, GINet utilizes the available cross-domain images from the identical category and estimates their common latent variable, which derives to discover domain-invariant knowledge for unseen target domain. According to these latent variables, our GINet further generates more novel samples with optimal transport constraint and deploys them to enhance the desired model with more robustness and generalization ability. Considerable empirical analysis and ablation studies on three popular benchmarks under normal DG and IDG setups suggests the advantage of our method over other DG methods on elevating model generalization. The source code is available in GitHub https://github.com/HaifengXia/IDG.

NI-UDA: Graph Contrastive Domain Adaptation for Nonshared-and-Imbalanced Unsupervised Domain Adaptation

With the technology development, information networks continuously generate a large amount of integrated labeled Big Data. Some types of labeled data in real scenes are scarce and difficult to obtain, such as some aerospace data. It is important to address the problem of nonshared and imbalanced unsupervised domain adaptation (NI-UDA) from the labeled Big Data with nonshared and long-tail distribution to unlabeled specified small and imbalanced space applications, where nonshared classes mean the label space out of the target domain. Previous methods proposed to integrate the semantic knowledge of Big Data to help the unsupervised domain adaptation for sparse data. However, they have the challenges of limited effect of knowledge sharing for long-tail Big Data and the imbalanced domain adaptation. To solve them, our goal is to leverage priori hierarchy knowledge to enhance domain contrastive aligned feature representation with graph reasoning. Our method consists of hierarchy graph reasoning (HGR) layer and K-positive contrastive domain adaptation (K-CDA). Our HGR contributes to learn direct semantic patterns for sparse classes by hierarchy attention in self-attention, nonlinear mapping, and graph normalization. For alleviating imbalanced domain adaptation, we proposed K-CDA, which explores k-positive instances for each class to every mini-batch with contrastive learning to align imbalanced feature representations. Compared with the previous contrastive UDA, our K-CDA alleviates the problems of large memory consumption and high computational cost. Experiments on three benchmark datasets shows our methods consistently improve the state-of-the-art contrastive UDA algorithms.

Deep imbalanced domain adaptation for transfer learning fault diagnosis of bearings under multiple working conditions

The tremendous success of deep learning and transfer learning broadened the scope of fault diagnosis, especially the latter further improved the diagnosis accuracy under multiple working conditions. However, most existing attempts assume that label distribution is domain-invariant despite taking into account the different feature distributions. This does not accommodate the diversity of fault mode distributions under different operating conditions and weakens the generalization to imbalanced domain adaptation (IDA) scenarios. Therefore, this work proposed a novel deep imbalanced domain adaptation (DIDA) framework for fault diagnosis of bearings, aiming at the challenging scenario where feature shift and label shift exist simultaneously under different working conditions. Specifically, DIDA overcomes the class-imbalanced label shift and achieves a fine-grained latent space matching by cost-sensitive learning and categorical alignment. Besides, margin loss regularization is introduced to further optimize classification boundaries and improve cross-domain generalization for IDA fault diagnosis tasks. Finally, we simulated the IDA protocols on experimental datasets and conducted case studies under multiple working conditions, thus validating the effectiveness and superiority of the proposed framework.

Occupancy estimation in smart buildings using predictive modeling in imbalanced domains

Occupancy estimation in smart buildings using predictive modeling in imbalanced domains

A unifying view of class overlap and imbalance: Key concepts, multi-view panorama, and open avenues for research

The combination of class imbalance and overlap is currently one of the most challenging issues in machine learning. While seminal work focused on establishing class overlap as a complicating factor for classification tasks in imbalanced domains, ongoing research mostly concerns the study of their synergy over real-word applications. However, given the lack of a well-formulated definition and measurement of class overlap in real-world domains, especially in the presence of class imbalance, the research community has not yet reached a consensus on the characterisation of both problems. This naturally complicates the evaluation of existing approaches to address these issues simultaneously and prevents future research from moving towards the devise of specialised solutions. In this work, we advocate for a unified view of the problem of class overlap in imbalanced domains. Acknowledging class overlap as the overarching problem – since it has proven to be more harmful for classification tasks than class imbalance – we start by discussing the key concepts associated to its definition, identification, and measurement in real-world domains, while advocating for a characterisation of the problem that attends to multiple sources of complexity. We then provide an overview of existing data complexity measures and establish the link to what specific types of class overlap problems these measures cover, proposing a novel taxonomy of class overlap complexity measures. Additionally, we characterise the relationship between measures, the insights they provide, and discuss to what extent they account for class imbalance. Finally, we systematise the current body of knowledge on the topic across several branches of Machine Learning (Data Analysis, Data Preprocessing, Algorithm Design, and Meta-learning), identifying existing limitations and discussing possible lines for future research.

Empirical investigation of the domain knowledge and team advertising creativity typology: The case of Nescafe coffee

This study proposes and tests a typology of domain knowledge and team creativity by empirically assessing the effects of varying levels of domain knowledge on the creative outcomes of the team members. Two separate studies were conducted to address this inquiry. Study one aimed at determining the level of domain knowledge of each team member in the teams. Eleven groups comprising of thirty-three business students designed eleven advertisements for the products of their own choices. Utilizing the situation judgment test and the grade earned in the advertising course, four teams were formed comprising two balanced and two imbalanced domain knowledge teams. To test the hypotheses of the study, these teams were asked to develop a print advertisement for Nescafe for the summer season (Study Two). Upon creativity assessment of the final ads by twenty-six independent creative personnel in a total of seven advertising agencies in Pakistan, the results revealed that a balanced team with low domain knowledge outperformed the other balanced team with high domain knowledge. Further, unexpectedly, one of the imbalanced domain knowledge teams also outperformed the balanced high domain knowledge team. The study in the light of extant literature presents worthwhile implications for academia and practitioners.

Cross-Domain Empirical Risk Minimization for Unbiased Long-Tailed Classification

We address the overlooked unbiasedness in existing long-tailed classification methods: we find that their overall improvement is mostly attributed to the biased preference of "tail" over "head", as the test distribution is assumed to be balanced; however, when the test is as imbalanced as the long-tailed training data---let the test respect Zipf's law of nature---the "tail" bias is no longer beneficial overall because it hurts the "head" majorities. In this paper, we propose Cross-Domain Empirical Risk Minimization (xERM) for training an unbiased test-agnostic model to achieve strong performances on both test distributions, which empirically demonstrates that xERM fundamentally improves the classification by learning better feature representation rather than the "head vs. tail" game. Based on causality, we further theoretically explain why xERM achieves unbiasedness: the bias caused by the domain selection is removed by adjusting the empirical risks on the imbalanced domain and the balanced but unseen domain.