Class Prototype Research Articles

A Pseudo-Labeling Multi-Screening-Based Semi-Supervised Learning Method for Few-Shot Fault Diagnosis.

In few-shot fault diagnosis tasks in which the effective label samples are scarce, the existing semi-supervised learning (SSL)-based methods have obtained impressive results. However, in industry, some low-quality label samples are hidden in the collected dataset, which can cause a serious shift in model training and lead to the performance of SSL-based method degradation. To address this issue, the latest prototypical network-based SSL techniques are studied. However, most prototypical network-based scenarios consider that each sample has the same contribution to the class prototype, which ignores the impact of individual differences. This article proposes a new SSL method based on pseudo-labeling multi-screening for few-shot bearing fault diagnosis. In the proposed work, a pseudo-labeling multi-screening strategy is explored to accurately screen the pseudo-labeling for improving the generalization ability of the prototypical network. In addition, the AdaBoost adaptation-based weighted technique is employed to obtain accurate class prototypes by clustering multiple samples, improving the performance that deteriorated by low-quality samples. Specifically, the squeeze and excitation block technique is used to enhance the useful feature information and suppress non-useful feature information for extracting accuracy features. Finally, three well-known bearing datasets are selected to verify the effectiveness of the proposed method. The experiments illustrated that our method can receive better performance than that of the state-of-the-art methods.

Multi-reference source aircraft classification network based on unknown class prototype estimation

Abstract Gathering complete aircraft types for close-set tasks is challenging and costly in fine-grained aircraft classification, resulting in encountering unknown class aircraft images in real-world models. To address this problem, we propose a network called a multi-reference source fine-grained aircraft classification network (MRSN) to explicitly and implicitly distinguish known class boundaries and embed unknown classes into the sample space. Specifically, we propose an embedding module (UCPE) to synthesize an unknown class prototype to facilitate the modeling of unknown class data distribution. Besides, we introduce a concatenated multi-reference source module (CMS) that combines distance measurement and probability estimation to classify whether testing images are of unknown class or not. Our proposed MRSN innovatively extends the open-set learning paradigm to the fine-grained classification of geospatial objects rather than simple scene classification. Besides, it shows excellent performance in our constructed dataset, which demonstrates the effectiveness and progressiveness of our method.

Enhanced Prototypical Network with Customized Region-Aware Convolution for Few-Shot SAR ATR

With the prosperous development and successful application of deep learning technologies in the field of remote sensing, numerous deep-learning-based methods have emerged for synthetic aperture radar (SAR) automatic target recognition (ATR) tasks over the past few years. Generally, most deep-learning-based methods can achieve outstanding recognition performance on the condition that an abundance of labeled samples are available to train the model. However, in real application scenarios, it is difficult and costly to acquire and to annotate abundant SAR images due to the imaging mechanism of SAR, which poses a big challenge to existing SAR ATR methods. Therefore, SAR target recognition in the situation of few-shot, where only a scarce few labeled samples are available, is a fundamental problem that needs to be solved. In this paper, a new method named enhanced prototypical network with customized region-aware convolution (CRCEPN) is put forward to specially tackle the few-shot SAR ATR tasks. To be specific, a feature-extraction network based on a customized and region-aware convolution is first developed. This network can adaptively adjust convolutional kernels and their receptive fields according to each SAR image’s own characteristics as well as the semantical similarity among spatial regions, thus augmenting its capability to extract more informative and discriminative features. To achieve accurate and robust target identity prediction under the few-shot condition, an enhanced prototypical network is proposed. This network can improve the representation ability of the class prototype by properly making use of training and test samples together, thus effectively raising the classification accuracy. Meanwhile, a new hybrid loss is designed to learn a feature space with both inter-class separability and intra-class tightness as much as possible, which can further upgrade the recognition performance of the proposed method. Experiments performed on the moving and stationary target acquisition and recognition (MSTAR) dataset, the OpenSARShip dataset, and the SAMPLE+ dataset demonstrate that the proposed method is competitive with some state-of-the-art methods for few-shot SAR ATR tasks.

FedCPG: A class prototype guided personalized lightweight federated learning framework for cross-factory fault detection

Industrial equipment condition monitoring and fault detection are crucial to ensure the reliability of industrial production. Recently, data-driven fault detection methods have achieved significant success, but they all face challenges due to data fragmentation and limited fault detection capabilities. Although centralized data collection can improve detection accuracy, the conflicting interests brought by data privacy issues make data sharing between different devices impractical, thus forming the problem of industrial data silos. To address these challenges, this paper proposes a class prototype guided personalized lightweight federated learning framework(FedCPG). This framework decouples the local network, only uploading the backbone model to the server for model aggregation, while employing the head model for local personalized updates, thereby achieving efficient model aggregation. Furthermore, the framework incorporates prototype constraints to steer the local personalized update process, mitigating the effects of data heterogeneity. Finally, a lightweight feature extraction network is designed to reduce communication overhead. Multiple complex industrial data distribution scenarios were simulated on two benchmark industrial datasets. Extensive experiments have demonstrated that FedCPG can achieve an average detection accuracy of 95% in complex industrial scenarios, while simultaneously reducing memory usage and the number of parameters by 82%, surpassing existing methods in most average metrics. These findings offer novel perspectives on the application of personalized federated learning in industrial fault detection.

A federated learning method based on class prototype guided classifier for long-tailed data

A federated learning method based on class prototype guided classifier for long-tailed data

Reserving embedding space for new fault types: A new continual learning method for bearing fault diagnosis

In complex operating environments, rotating equipment may continually generate new fault categories, affecting the safety of equipment operation, and the number of collected fault samples is limited, which make it difficult to establish a reliable diagnostic model. Few-shot continual learning (FSCL) can continuously learn and summarize fault knowledge from limited samples. These traditional FSCL models, when learning from limited samples of new types of bearing faults, exhibit model overfitting and catastrophic forgetting issues after self-updating (new model). This phenomenon significantly limits the reliability of traditional models. This paper presents a new approach coined as reserving embedding space for new fault types (RESNFT) for continual learning of newly occurred bearing faults for machinery fault diagnosis, which includes several key and new ideas. Preliminary, a class prototype (prototype vector) is derived to replace a classifier weight. Then, embedding space reserved by forcibly assigning real fault samples to both real and virtual categories can mitigate catastrophic forgetting. Finally, virtual faults formed by sample mixing guide new fault samples to the reserved embedding space, and realize continuous classification of new faults. Results show that the RESNFT can effectively alleviate the catastrophic forgetting and overfitting problems, and it is superior to other existing methods.

Partial label learning via identifying outlier features

Partial label learning (PLL) is a classification problem where each training instance is ambiguously annotated with a set of candidate labels, among which only one is the ground truth label. The topological structure in the feature space is frequently used by existing PLL approaches to clarify if a candidate label is the ground truth label for a training example. However, these techniques frequently experience cumulative errors brought on by the error-prone labeling confidence estimate throughout the topological structure because of the redundant and noisy features occurring naturally in the feature space. In this paper, we propose a novel approach through Identifying Outlier Features (PL-IOF) to address this challenge. The feature space is decomposed into class prototype features and outlier features, where the former captures the discriminative features of each class prototype, and the latter captures the outlier features in each training instance. A unified framework is specifically suggested to simultaneously optimize class prototypes and outlier features, as well as to estimate labeling confidences over incomplete label training instances. This framework ensures the high quality of the extracted class prototypes by recognizing the outliers. Experiments on both synthetic and real-world datasets demonstrate the out-performance of PL-IOF over the state-of-the-art.

Noise-Tolerant Hybrid Prototypical Learning with Noisy Web Data

We focus on the challenging problem of learning an unbiased classifier from a large number of potentially relevant but noisily labeled web images given only a few clean labeled images. This problem is particularly practical, because it reduces the expensive annotation costs by utilizing freely accessible web images with noisy labels. Typically, prototypes are representative images or features used to classify or identify other images. However, in the few clean and many noisy scenarios, the class prototype can be severely biased due to the presence of irrelevant noisy images. The resulting prototypes are less compact and discriminative, as previous methods do not take into account the diverse range of images in the noisy web image collections. On the other hand, the relation modeling between noisy and clean images is not learned for the class prototype generation in an end-to-end manner, which results in a suboptimal class prototype. In this paper, we introduce a similarity maximization loss named the SimNoiPro. Our SimNoiPro first generates noise-tolerant hybrid prototypes composed of clean and noise-tolerant prototypes, and then pulls them closer to each other. Our approach considers the diversity of noisy images by explicit division and overcomes the optimization discrepancy issue. This enables better relation modeling between clean and noisy images and helps extract judicious information from the noisy image set. The evaluation results on two extended few-shot classification benchmarks confirm that our SimNoiPro outperforms prior methods in measuring image relations and cleaning noisy data.

Unsupervised domain adaptation with hard-sample dividing and processing strategy

In Unsupervised Domain Adaptation(UDA), the hard samples cause serious negative transfer to model performance. However, the existing UDA methods don’t fully pay attention to the fine-grained processing of hard samples in target and source domains. To address these problems, the UDA framework with Hard-sample Dividing and Processing Strategy (HDPS) is proposed in this paper. In HDPS, we define sample division criteria in target domain and source domain respectively and divide the samples into easy samples, hard samples with low confidence and hard samples with high confidence. We further design systematic processing strategy of the three types of samples. Specifically, we firstly define geometric metrics based on the class prototype to divide the hard samples and eliminate them in source domain. We further design two weight allocation strategies based on sample classification entropy in target domain. Finally, we build a teacher-student guidance mechanism to learn the discriminative features of hard samples. The HDPS framework can be easily loaded to other methods as a plug-in, can promote the diversity feature learning of samples, and significantly improve the discriminant performance of the model. Extensive experiments on six benchmark datasets verify that HDPS is effective and superior to most existing UDA methods.

DP-DDCL: A discriminative prototype with dual decoupled contrast learning method for few-shot object detection

Few-shot object detection (FSOD) can effectively improve object detection performance with limited training data, attracting increasing interest from researchers. Due to the limited sample size, there is often a large variance in the learned features, resulting in deviating from the class center and being confused with other classes. People are often able to accurately identify few-shot objects based on discriminative information. Motivated by it, we propose a discriminative prototype with dual decoupled contrast learning (DP-DDCL) method to address the issue of limited training data in FSOD. By introducing domain knowledge comprising the CLIP and attribute knowledge graph to obtain explicit and implicit semantic and visual information, we construct a novel discriminative prototype that enhances the representation of different classes (especially few-shot samples). Simultaneously, dual decoupled contrast learning consists of the decoupled of positive and negative samples, and the decoupled of discriminative prototype and instance contrast learning is proposed. It makes samples of the same class cluster around their respective class prototype while maintaining a clear semantic boundary between different classes. Extensive experiments demonstrate the efficacy of our method, surpassing the current state-of-the-art in any-shot and all-data splits, with maximum performance gains of up to +9.15% on the standard PASCAL VOC benchmark and +2.5% on the challenging COCO benchmark.

Discovery of an Accretion Streamer and a Slow Wide-angle Outflow around FU Orionis

We present Atacama Large Millimeter/submillimeter Array 12-m, 7-m, and Total Power Array observations of the FU Orionis outbursting system, covering spatial scales ranging from 160 to 25,000 au. The high-resolution interferometric data reveal an elongated 12CO(2–1) feature previously observed at lower resolution in 12CO(3–2). Kinematic modeling indicates that this feature can be interpreted as an accretion streamer feeding the binary system. The mass infall rate provided by the streamer is significantly lower than the typical stellar accretion rates (even in quiescent states), suggesting that this streamer alone is not massive enough to sustain the enhanced accretion rates characteristic of the outbursting class prototype. The observed streamer may not be directly linked to the current outburst, but rather a remnant of a previous, more massive streamer that may have contributed enough to the disk mass to render it unstable and trigger the FU Orionis outburst. The new data detect, for the first time, a vast, slow-moving carbon monoxide molecular outflow emerging from this object. To accurately assess the outflow properties (mass, momentum, and kinetic energy), we employ 13CO(2–1) data to correct for optical depth effects. The analysis indicates that the outflow corresponds to swept-up material not associated with the current outburst, similar to the slow molecular outflows observed around other FUor and Class I protostellar objects.

Beyond Prototypes: Semantic Anchor Regularization for Better Representation Learning

One of the ultimate goals of representation learning is to achieve compactness within a class and well-separability between classes. Many outstanding metric-based and prototype-based methods following the Expectation-Maximization paradigm, have been proposed for this objective. However, they inevitably introduce biases into the learning process, particularly with long-tail distributed training data. In this paper, we reveal that the class prototype is not necessarily to be derived from training features and propose a novel perspective to use pre-defined class anchors serving as feature centroid to unidirectionally guide feature learning. However, the pre-defined anchors may have a large semantic distance from the pixel features, which prevents them from being directly applied. To address this issue and generate feature centroid independent from feature learning, a simple yet effective Semantic Anchor Regularization (SAR) is proposed. SAR ensures the inter-class separability of semantic anchors in the semantic space by employing a classifier-aware auxiliary cross-entropy loss during training via disentanglement learning. By pulling the learned features to these semantic anchors, several advantages can be attained: 1) the intra-class compactness and naturally inter-class separability, 2) induced bias or errors from feature learning can be avoided, and 3) robustness to the long-tailed problem. The proposed SAR can be used in a plug-and-play manner in the existing models. Extensive experiments demonstrate that the SAR performs better than previous sophisticated prototype-based methods. The implementation is available at https://github.com/geyanqi/SAR.

Contrastive regularization guided label refurbishment for fault diagnosis under label noise

Conventional intelligent fault diagnosis studies require precise labels by default. However, label noise is inevitable in real industrial environments due to human errors, measurement biases, data transmission and storage errors. The performance of existing data-driven methods may be severely affected by mislabels. In this paper, we investigate the fault diagnosis issue of rotating machinery under noisy labels. A generalized framework contrastive regularization guided label refurbishment (CRLR) is proposed, which utilizes contrastive regularization function extracting class prototype to guide label refurbishment. The learned contrastive representations can suppress the misleading effect of noisy labels. The theoretical analysis from an information theoretic perspective also ensures the effectiveness of CRLR. Validation experiments have been conducted on public and private collected datasets using the ResNet-50 as the backbone network. The experiments demonstrate that our proposed framework outperforms the current state-of-the-art methods at different noise rates.

An adaptive class prototype generation framework for partial label learning

Partial label learning involves instances with multiple potential labels, of which only one is correct. This process poses challenges when managing limited-size datasets and ambiguity arising from overlapping characteristics across various categories. This paper proposes a novel framework based on class prototype generation to address these issues. The framework integrates a generator that creates label-specific prototypes, a discriminator that assigns weights reflecting the representativeness of examples, and a classifier learning with the assistance of the generator and discriminator. We introduce an adaptive loss adjustment method intended to dynamically balance each model’s weights and mitigate the potential negative impact of inferior examples. Experimental results demonstrate our method’s superiority over state-of-the-art techniques on widely used real-world datasets, with various visualizations illustrating its effectiveness.

A small-sample time-series signal augmentation and analysis method for quantitative assessment of bradykinesia in Parkinson's disease

Patients with Parkinson's disease (PD) usually have varying degrees of bradykinesia, and the current clinical assessment is mainly based on the Movement Disorder Society Unified PD Rating Scale, which can hardly meet the needs of objectivity and accuracy. Therefore, this paper proposed a small-sample time series classification method (DTW-TapNet) based on dynamic time warping (DTW) data augmentation and attentional prototype network. Firstly, for the problem of small sample sizes of clinical data, a DTW-based data merge method is used to achieve data augmentation. Then, the time series are dimensionally reorganized using random grouping, and convolutional operations are performed to learn features from multivariate time series. Further, attention mechanism and prototype learning are introduced to optimize the distance of the class prototype to which each time series belongs to train a low-dimensional feature representation of the time series, thus reducing the dependency on data volume. Clinical experiments were conducted to collect motion capture data of upper and lower limb movements from 36 patients with PD and eight healthy controls. For the upper limb movement data, the proposed method improved the classification accuracy, weighted precision, and kappa coefficient by 8.89%-15.56%, 9.22%-16.37%, and 0.13-0.23, respectively, compared with support vector machines, long short-term memory, and convolutional prototype network. For the lower limb movement data, the proposed method improved the classification accuracy, weighted precision, and kappa coefficient by 8.16%-20.41%, 10.01%-23.73%, and 0.12-0.28, respectively. The experiments and results show that the proposed method can objectively and accurately assess upper and lower limb bradykinesia in PD.

A small sample bearing fault diagnosis method based on ConvGRU relation network

Considering that in the fault diagnosis of bearing based on relation network, using the sample mean value as the class prototype for each class is susceptible to outliers, resulting in inaccurate class prototypes, this paper proposes a convolutional gate recurrent unit (ConvGRU) relation network fault diagnosis model; firstly, the model utilizes a embedding module to extract bearing fault features, and then uses the ConvGRU as a learnable class prototype generator to generate class prototypes for each class. Secondly, a relation module is utilized to measure the similarity between class prototypes and the sample features of the query set, obtaining relation scores, and ultimately achieving fault diagnosis. In order to test the validity and advantages of the model, experimental verification and analysis were conducted on the case western storage rolling bearing dataset. The results of the experiment show that the class prototypes generated by the ConvGRU class prototype generation module have better discrimination and accuracy compared to the class prototypes generated by the relation network. In the 10-way 5-shot experiment, the accuracy of the model proposed in this paper reaches 99.60%, which increases by 6.63%, 5.10%, 4.80%, and 1.75% compared to k-nearest neighbor, convolutional neural network, prototypical network, and relation network. The method proposed in this paper helps to generate more accurate class prototypes and has a certain effect on improving the accuracy of model fault diagnosis.

Domain adaptive remote sensing image semantic segmentation with prototype guidance

Current unsupervised domain adaptation (UDA) techniques in semantic segmentation effectively decrease the domain discrepancy between the labeled source domain and unlabeled target domain, thereby enhancing the model’s pixel-wise discriminative capability for target domain data. However, in remote sensing images (RSIs), our study uncovers that these approaches perform poorly in the presence of class distribution inconsistencies between the source and target domains. In this work, we propose a one-stage mean teacher framework with a novel auxiliary prototype classifier, named MTA, to address this issue. Specifically, the teacher model assigns pseudo labels at pixel level for target samples and captures knowledge from the student model via exponential moving average (EMA). With labeled source samples and target samples that have pseudo labels, the student model can alleviate the divergence between the source and target domains. In addition, the auxiliary prototype classifier (APC) reduces the performance degradation in the parametric softmax classifier of the student model caused by class distribution divergence. We also propose a prototype computation scheme to obtain each class prototype in the APC. Specifically, we build a memory bank for each class of the two domains to store feature embeddings dynamically. Then, we compute the class prototype by applying the clustering algorithm on memory banks corresponding to the class. Meanwhile, the APC reduces the intra-class domain discrepancy by optimizing the cross-entropy loss, which brings each class feature distribution of the two domains closer to the class prototype. The experimental results on RSIs UDA semantic segmentation tasks show the superiority of our approach over comparative methods.

Class‐Relation Reasoning with Knowledge‐Transfer for Few‐Shot Object Detection

Few‐Shot Object Detection (FSOD) task involves accurately identifying target object classes using only a small set of labeled samples. Most of the current FSOD tasks independently predict class prototype features without considering class relationships and only rely on visual information. To address these challenges, we propose a novel Class‐relational Reasoning Method with Knowledge‐transfer (CRK‐Net), built on the meta‐learning‐based framework. Although data may be scarce, the semantic relationship between classes is invariant, Joint‐feature Fusion Module (JFM) are hence proposed to transfers the semantic information of different categories in the natural language world to integrate with visual information and produce multi‐modality embeddings. Some base classes and novel classes have similar features, so this can be borrowed by modeling the relationship between classes feature. Building upon the observation, we propose a Class‐relational Reasoning Module (CRM) to establish the correlations between categories and enhance prototype representations for each category. After passing through the JFM and CRM modules, a high‐quality class prototype is finally produced for subsequent regression and classification. Extensive experiments on PASCAL VOC demonstrate the effectiveness of our proposed method and provide a new scheme for fusing semantic and visual information. © 2024 Institute of Electrical Engineers of Japan. Published by Wiley Periodicals LLC.

Few-shot learning based on prototype rectification with a self-attention mechanism

Few-shot learning (FSL) is a challenging problem. Prototype-based methods are simple and effective methods for addressing few-shot learning. Due to the lack of labeled samples, the learned class prototype in the existing prototype-based few-shot learning methods has a great deviation and cannot express the representative and discriminant characteristics of its corresponding class well. To address this problem, in this work we propose few-shot learning based on prototype rectification with a self-attention mechanism(FSL-PRS). To learn more unbiased and discriminative class prototypes, FSL-PRS takes the support set and the query set as a whole and learns task-related features from the features extracted from pretrained backbone networks with a self-attention mechanism. Then, the learned task-related features are utilized to compute the original class prototypes and predict a pseudo label and confidence for each query sample. The query samples with high confidence are incorporated into the support set to rectify the class prototypes. We hope that the learned class prototype can better highlight the class significance. Therefore, a class significance learning module is designed for making the learned class prototypes more discriminative. Different from prior works, we take the support set and the query set as a whole to learn task-related features with a self-attention mechanism, which not only alleviates the negative effect of distribution differences between the support set and query set but also fuses global context information to enhance features for FSL. We conduct comprehensive experiments on four benchmark datasets widely adopted in few-shot learning. The experimental results demonstrate that the FSL-PRS achieves state-of-the-art performance, which validates its effectiveness.

Semi-supervised medical image classification based on class prototype matching for soft pseudo labels with consistent regularization

Semi-supervised medical image classification based on class prototype matching for soft pseudo labels with consistent regularization