Prototype Representation Research Articles

Prototype Relationship Optimization Network for Few‐Shot Learning

AbstractFew‐shot learning (FSL) aims to infer labels for new samples based on a few labeled samples. FSL based on prototype metrics emphasizes the distance between unlabeled sample features and prototypes generated from a few labeled samples. However, due to the limited number of samples and the uncertainty of samples, the discrepancy between prototypes and the center of the real features may lead to metric error. This paper proposes the Prototype Relationship Optimization Network (PRON) to alleviate the prototype bias issue by utilizing prototype relationship optimization. The PRON involves three fundamental processes: (i) Constructing a multi‐layer prototype through multi‐layer feature embedding to enhance prototype representation capabilities; (ii) Utilizing prototype relationship interaction network of multi‐layer prototype forms multi‐layer prototype relationships to weaken the influence of uncertainty and strengthen the discrimination; (iii) Introduces a prototype refinement mechanism in the cascade structure to optimize prototype relationship representation. The inductive standard experimental results on three popular benchmark datasets show that the performance of the proposed method is very competitive and comparable to the state‐of‐the‐art FSL methods. © 2024 Institute of Electrical Engineers of Japan and Wiley Periodicals LLC.

Intuitive-K-prototypes: A mixed data clustering algorithm with intuitionistic distribution centroid

Data sets are usually mixed with numerical and categorical attributes in the real world. Data mining of mixed data makes a lot of sense. This paper proposes an Intuitive-K-prototypes clustering algorithm with improved prototype representation and attribute weights. The proposed algorithm defines intuitionistic distribution centroid for categorical attributes. In our approach, a heuristic search for initial prototypes is performed. Then, we combine the mean of numerical attributes and intuitionistic distribution centroid to represent the cluster prototype. In addition, intra-cluster complexity and inter-cluster similarity are used to adjust attribute weights, with higher priority given to those with lower complexity and similarity. The membership and non-membership distance are calculated using the intuitionistic distribution centroid. These distances are then combined parametrically to obtain the composite distance. The algorithm is judged for its clustering effectiveness on the real UCI data set, and the results show that the proposed algorithm outperforms the traditional clustering algorithm in most cases.

Identifying crystal structures beyond known prototypes from x-ray powder diffraction spectra

The large amount of powder diffraction data for which the corresponding crystal structures have not yet been identified suggests the existence of numerous undiscovered, physically relevant crystal structure prototypes. In this paper, we present a scheme to resolve powder diffraction data into crystal structures with precise atomic coordinates by screening the space of all possible atomic arrangements, i.e., structural prototypes, including those not previously observed, using a pre-trained machine learning (ML) model. This involves (i) enumerating all possible symmetry-confined ways in which a given composition can be accommodated in a given space group, (ii) ranking the element-assigned prototype representations using energies predicted using the Wyckoff representation regression ML model [Goodall , ], (iii) assigning and perturbing atoms along the degree of freedom allowed by the Wyckoff positions to match the experimental diffraction data, and (iv) validating the thermodynamic stability of the material using density-functional theory. An advantage of the presented method is that it does not rely on a database of previously observed prototypes and is, therefore capable of finding crystal structures with entirely new symmetric arrangements of atoms. We demonstrate the workflow on unidentified x-ray diffraction spectra from the ICDD database and identify a number of stable structures, where a majority turns out to be derivable from known prototypes. However, at least two are found not to be part of our prior structural data sets. Published by the American Physical Society 2024

Multivariate prototype representation for domain-generalized incremental learning

Deep learning models often suffer from catastrophic forgetting when fine-tuned with samples of new classes. This issue becomes even more challenging when there is a domain shift between training and testing data. In this paper, we address the critical yet less explored Domain-Generalized Class-Incremental Learning (DGCIL) task. We propose a DGCIL approach designed to memorize old classes, adapt to new classes, and reliably classify objects from unseen domains. Specifically, our loss formulation maintains classification boundaries while suppressing domain-specific information for each class. Without storing old exemplars, we employ knowledge distillation and estimate the drift of old class prototypes as incremental training progresses. Our prototype representations are based on multivariate Normal distributions, with means and covariances continually adapted to reflect evolving model features, providing effective representations for old classes. We then sample pseudo-features for these old classes from the adapted Normal distributions using Cholesky decomposition. Unlike previous pseudo-feature sampling strategies that rely solely on average mean prototypes, our method captures richer semantic variations. Experiments on several benchmarks demonstrate the superior performance of our method compared to the state of the art.

Teacher-Student Prototype Enhancement Network for a Few-Shot Remote Sensing Scene Classification

Few-shot remote sensing scene classification identifies new classes from limited labeled samples where the great challenges are intraclass diversity, interclass similarity, and limited supervision. To alleviate these problems, a teacher-student prototype enhancement network is proposed for a few-shot remote sensing scene classification. Instead of introducing an attentional mechanism in mainstream studies, a prototype enhancement module is recommended to adaptively select high-confidence query samples, which can enhance the support prototype representations to emphasize intraclass and interclass relationships. The construction of a few-shot teacher model generates more discriminative predictive representations with inputs from many labeled samples, thus providing a strong supervisory signal to the student model and encouraging the network to achieve accurate classification with a limited number of labeled samples. Extensive experiments of four public datasets, including NWPU-remote sens ing image scene classification (NWPU-RESISC45), aerial image dataset (AID), UC Merced, and WHU-RS19, demonstrate that this method achieves superior competitive performance than the state-of-the-art methods on five-way, one-shot, and five-shot classifications.

GNN-based multi-source domain prototype representation for cross-subject EEG emotion recognition

Emotion recognition based on electroencephalography (EEG) signals is a major area of affective computing. However, the existence of distributional differences between subjects has greatly hindered the large-scale application of EEG emotion recognition techniques. Most of the existing cross-subject methods primarily concentrate on treating multiple subjects as a single source domain. These methods lead to significant distributional differences within the source domain, which hinder the model’s ability to generalise effectively to target subjects. In this paper, we propose a new method that combines graph neural network-based prototype representation of multiple source domains with clustering similarity loss. It consists of three parts: multi-source domain prototype representation, graph neural network and loss. Multi-source domain prototype representation treats different subjects in the source domain as sub-source domains and extracts prototype features, which learns a more fine-grained feature representation. Graph neural network can better model the association properties between prototypes and samples. In addition, we propose a similarity loss based on clustering idea. The loss makes maximum use of similarity between samples in the target domain while ensuring that the classification performance does not degrade. We conduct extensive experiments on two benchmark datasets, SEED and SEED IV. The experimental results validate the effectiveness of the proposed multi-source domain fusion approach and indicate its superiority over existing methods in cross-subject classification tasks.

Advanced Global Prototypical Segmentation Framework for Few-Shot Hyperspectral Image Classification.

With the advancement of deep learning, related networks have shown strong performance for Hyperspectral Image (HSI) classification. However, these methods face two main challenges in HSI classification: (1) the inability to capture global information of HSI due to the restriction of patch input and (2) insufficient utilization of information from limited labeled samples. To overcome these challenges, we propose an Advanced Global Prototypical Segmentation (AGPS) framework. Within the AGPS framework, we design a patch-free feature extractor segmentation network (SegNet) based on a fully convolutional network (FCN), which processes the entire HSI to capture global information. To enrich the global information extracted by SegNet, we propose a Fusion of Lateral Connection (FLC) structure that fuses the low-level detailed features of the encoder output with the high-level features of the decoder output. Additionally, we propose an Atrous Spatial Pyramid Pooling-Position Attention (ASPP-PA) module to capture multi-scale spatial positional information. Finally, to explore more valuable information from limited labeled samples, we propose an advanced global prototypical representation learning strategy. Building upon the dual constraints of the global prototypical representation learning strategy, we introduce supervised contrastive learning (CL), which optimizes our network with three different constraints. The experimental results of three public datasets demonstrate that our method outperforms the existing state-of-the-art methods.

Cluster prototype earth mover’s distance adapters and alignment-guided prompt learning for vision–language models

Vision–language pre-trained models have focused on fine-tuning methods to enhance generalization in downstream tasks, such as CLIP. Recent research proposes fine-tuning these models using prompting and adapter techniques. However, prompting methods tend to overfit class-specific data distributions, and most adapters ignore the prototype representations as the weights, resulting in poor generalization. To tackle these challenges, we propose a novel method that integrates the cluster prototype Earth Mover’s Distance adapters and alignment-guided prompt learning (CPAAP) for vision–language models. Specifically, our adapter designs three components: graph convolutional network-based clusters, hierarchy prototype representations, and Earth Mover’s Distance similarity, providing a robust metric for measuring feature representations. Additionally, alignment-guided prompt learning enforces a constraint in the prediction of trainable and frozen models, enabling effective adaptation to downstream tasks. Extensive experiments are conducted on few-shot classification, base-to-novel generalization, cross-dataset evaluation, and domain generalization. In practice, CPAAP outperforms state-of-the-art methods on zero-shot tasks, achieving an absolute gain of 0.46% on the harmonic mean across 11 popular datasets.

Enhancing Image Classification Using Few-Shot Learning Prototypical Networks with ResNet-18: Detection, Accuracy Enhancement, and Optimization

Image classification, especially in scenarios with limited data, presents significant challenges. Few shot learning (FSL) aims to address these challenges by training models that can generalize from a few examples. This paper explores the integration of prototypical networks with ResNet-18 for feature extraction to enhance image classification accuracy. Prototypical networks are designed to create a prototype representation for each class, which can then be used to classify new examples based on their distance to these prototypes. By leveraging ResNet-18's powerful feature extraction capabilities, we aim to improve the quality of these prototypes, thereby enhancing classification performance.We propose various methods for accuracy enhancement and optimization, including hyperparameter tuning, regularization techniques, and advanced methods like attention mechanisms and metric learning. Hyperparameter tuning involves adjusting the model's parameters to find the optimal settings that yield the best performance. Regularization techniques, such as dropout and weight decay, help prevent overfitting and improve the model's generalization capabilities. Advanced methods like attention mechanisms can focus on the most relevant parts of the image, while metric learning aims to learn a distance metric that better reflects the similarities between images.Our experiments on datasets like Mini-ImageNet and Omniglot demonstrate significant improvements in classification performance. These datasets are commonly used benchmarks in the few-shot learning community, allowing us to compare our results with existing methods. The integration of prototypical networks with ResNet-18, along with the proposed optimization techniques, provides a robust approach for tackling the challenges of image classification in few-shot learning scenarios. Key Words: Few-shot learning, ResNet-18, Prototypical Networks.

DP-MP: a novel cross-subject fatigue detection framework with DANN-based prototypical representation and mix-up pairwise learning.

Electroencephalography (EEG) is widely recognized as an effective method for detecting fatigue. However, practical applications of EEG for fatigue detection in real-world scenarios are often challenging, particularly in cases involving subjects not included in the training datasets, owing to bio-individual differences and noisy labels. This study aims to develop an effective framework for cross-subject fatigue detection by addressing these challenges. In this study, we propose a novel framework, termed DP-MP, for cross-subject fatigue detection, which utilizes a Domain-Adversarial Neural Network (DANN)-based prototypical representation in conjunction with Mix-up pairwise learning. Our proposed DP-MP framework aims to mitigate the impact of bio-individual differences by encoding fatigue-related semantic structures within EEG signals and exploring shared fatigue prototype features across individuals. Notably, to the best of our knowledge, this work is the first to conceptualize fatigue detection as a pairwise learning task, thereby effectively reducing the interference from noisy labels. Furthermore, we propose the Mix-up pairwise learning (MixPa) approach in the field of fatigue detection, which broadens the advantages of pairwise learning by introducing more diverse and informative relationships among samples. Cross-subject experiments were conducted on two benchmark databases, SEED-VIG and FTEF, achieving state-of-the-art performance with average accuracies of 88.14% and 97.41%, respectively. These promising results demonstrate our model's effectiveness and excellent generalization capability. This is the first time EEG-based fatigue detection has been conceptualized as a pairwise learning task, offering a novel perspective to this field. Moreover, our proposed DP-MP framework effectively tackles the challenges of bio-individual differences and noisy labels in the fatigue detection field and demonstrates superior performance. Our work provides valuable insights for future research, promoting the application of brain-computer interfaces for fatigue detection in real-world scenarios.&#xD.

Dual-Channel Prototype Network for Few-Shot Pathology Image Classification.

In the field of pathology, the scarcity of certain diseases and the difficulty of annotating images hinder the development of large, high-quality datasets, which in turn affects the advancement of deep learning-assisted diagnostics. Few-shot learning has demonstrated unique advantages in modeling tasks with limited data, yet explorations of this method in the field of pathology remain in the early stages. To address this issue, we present a dual-channel prototype network (DCPN), a novel few-shot learning approach for efficiently classifying pathology images with limited data. The DCPN leverages self-supervised learning to extend the pyramid vision transformer (PVT) to few-shot classification tasks and combines it with a convolutional neural network to construct a dual-channel network for extracting multi-scale, high-precision pathological features, thereby substantially enhancing the generalizability of prototype representations. Additionally, we design a soft voting classifier based on multi-scale features to further augment the discriminative power of the model in complex pathology image classification tasks. We constructed three few-shot classification tasks with varying degrees of domain shift using three publicly available pathological datasets-CRCTP, NCTCRC, and LC25000-to emulate real-world clinical scenarios. The results demonstrated that the DCPN outperformed the prototypical network across all metrics, achieving the highest accuracies in same-domain tasks-70.86% for 1-shot, 82.57% for 5-shot, and 85.2% for 10-shot setups-corresponding to improvements of 5.51%, 5.72%, and 6.81%, respectively, over the prototypical network. Notably, in the same-domain 10-shot setting, the accuracy of the DCPN (85.2%) surpassed that of the PVT-based supervised learning model (85.15%), confirming its potential to diagnose rare diseases within few-shot learning frameworks.

Proto-Adapter: Efficient Training-Free CLIP-Adapter for Few-Shot Image Classification.

Large vision-language models, such as Contrastive Vision-Language Pre-training (CLIP), pre-trained on large-scale image-text datasets, have demonstrated robust zero-shot transfer capabilities across various downstream tasks. To further enhance the few-shot recognition performance of CLIP, Tip-Adapter augments the CLIP model with an adapter that incorporates a key-value cache model constructed from the few-shot training set. This approach enables training-free adaptation and has shown significant improvements in few-shot recognition, especially with additional fine-tuning. However, the size of the adapter increases in proportion to the number of training samples, making it difficult to deploy in practical applications. In this paper, we propose a novel CLIP adaptation method, named Proto-Adapter, which employs a single-layer adapter of constant size regardless of the amount of training data and even outperforms Tip-Adapter. Proto-Adapter constructs the adapter's weights based on prototype representations for each class. By aggregating the features of the training samples, it successfully reduces the size of the adapter without compromising performance. Moreover, the performance of the model can be further enhanced by fine-tuning the adapter's weights using a distance margin penalty, which imposes additional inter-class discrepancy to the output logits. We posit that this training scheme allows us to obtain a model with a discriminative decision boundary even when trained with a limited amount of data. We demonstrate the effectiveness of the proposed method through extensive experiments of few-shot classification on diverse datasets.

Prototype or Exemplar Representations in the 5/5 Category Learning Task.

Theories of category learning have typically focused on how the underlying category structure affects the category representations acquired by learners. However, there is limited research as to how other factors affect what representations are learned and utilized and how representations might change across the time course of learning. We used a novel "5/5" categorization task developed from the well-studied 5/4 task with the addition of one more stimulus to clarify an ambiguity in the 5/4 prototypes. We used multiple methods including computational modeling to identify whether participants categorized on the basis of exemplar or prototype representations. We found that, overall, for the stimuli we used (schematic robot-like stimuli), learning was best characterized by the use of prototypes. Most importantly, we found that relative use of prototype and exemplar strategies changed across learning, with use of exemplar representations decreasing and prototype representations increasing across blocks.

Few-shot intent detection with self-supervised pretraining and prototype-aware attention

Few-shot intent detection is a more challenging application. However, traditional prototypical networks based on averaging often suffer from issues such as missing key information, poor generalization capabilities. In previous work, using three-dimensional convolutional neural networks (3DCNN) to generate prototype representations faces challenges with long-distance dependencies. Furthermore, a pretrained encoder’s performance in a specific domain is often suboptimal because its knowledge of the specific domain is fragmented. Therefore, in this paper, we propose a simple yet effective two-stage learning strategy to address these issues. In the first stage, we propose a self-supervised multi-task pretraining (SMTP) strategy. SMTP utilizes unlabeled data from the current domain to help the pretrained encoder learn the semantic information of the text and implicitly distinguish semantically similar text representations without using any labels. SMTP aims to enhance the representation capability of the pretrained encoder in a specific domain. In the second stage, we propose a prototype-aware attention (PaAT) model to generate prototype representations of the same class. PaAT generates prototype representations by calculating the attention between class texts, which can effectively solve the long-distance dependence problem of 3DCNN. PaAT is a siamese architecture that can simultaneously generate prototype representations and sentence-level representations of unseen data. In addition, to prevent overfitting in few-shot learning, we introduce an unsupervised contrastive regularization term to constrain PaAT. Our method achieves state-of-the-art performance on four public datasets.11Our code is available: https://github.com/YS19999/SMTP-PaAT. .

Supervised Cross-Momentum Contrast: Aligning representations with prototypical examples to enhance financial sentiment analysis

Financial sentiment analysis plays a pivotal role in understanding market dynamics and investor sentiment. In this paper, we propose the Supervised Cross-Momentum Contrast (SuCroMoCo) framework, a novel approach for financial sentiment analysis. SuCroMoCo leverages supervised contrastive learning and cross-momentum contrast to align financial text representations with prototypical representations based on sentiment categories. This alignment greatly improves classification performance, addressing the limitations of pre-trained language models (PLMs) in fully grasping the intricate nature of financial text. Through extensive experiments, we demonstrate that SuCroMoCo outperforms existing PLMs-based approaches and Large Language Models (LLMs) on diverse benchmark datasets.

PR-PL: A Novel Prototypical Representation Based Pairwise Learning Framework for Emotion Recognition Using EEG Signals

Affective brain-computer interface based on electroencephalography (EEG) is an important branch in the field of affective computing. However, the individual differences in EEG emotional data and the noisy labeling problem in the subjective feedback seriously limit the effectiveness and generalizability of existing models. To tackle these two critical issues, we propose a novel transfer learning framework with Prototypical Representation based Pairwise Learning ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">PR-PL</i> ). The discriminative and generalized EEG features are learned for emotion revealing across individuals and the emotion recognition task is formulated as pairwise learning for improving the model tolerance to the noisy labels. More specifically, a prototypical learning is developed to encode the inherent emotion-related semantic structure of EEG data and align the individuals' EEG features to a shared common feature space under consideration of the feature separability of both source and target domains. Based on the aligned feature representations, pairwise learning with an adaptive pseudo labeling method is introduced to encode the proximity relationships among samples and alleviate the label noises effect on modeling. Extensive results on two benchmark databases (SEED and SEED-IV) under four different cross-validation evaluation protocols validate the model reliability and stability across subjects and sessions. Compared to the literature, the average enhancement of emotion recognition across four different evaluation protocols is 2.04% (SEED) and 2.58% (SEED-IV). The source code is available at <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><uri>https://github.com/KAZABANA/PR-PL</uri></i> .

Uncertainty-aware prototypical learning for anomaly detection in medical images

Anomalous object detection (AOD) in medical images aims to recognize the anomalous lesions, and is crucial for early clinical diagnosis of various cancers. However, it is a difficult task because of two reasons: (1) the diversity of the anomalous lesions and (2) the ambiguity of the boundary between anomalous lesions and their normal surroundings. Unlike existing single-modality AOD models based on deterministic mapping, we constructed a probabilistic and deterministic AOD model. Specifically, we designed an uncertainty-aware prototype learning framework, which considers the diversity and ambiguity of anomalous lesions. A prototypical learning transformer (Pformer) is established to extract and store the prototype features of different anomalous lesions. Moreover, Bayesian neural uncertainty quantizer, a probabilistic model, is designed to model the distributions over the outputs of the model to measure the uncertainty of the model’s detection results for each pixel. Essentially, the uncertainty of the model’s anomaly detection result for a pixel can reflect the anomalous ambiguity of this pixel. Furthermore, an uncertainty-guided reasoning transformer (Uformer) is devised to employ the anomalous ambiguity, encouraging the proposed model to focus on pixels with high uncertainty. Notably, prototypical representations stored in Pformer are also utilized in anomaly reasoning that enables the model to perceive diversities of the anomalous objects. Extensive experiments on five benchmark datasets demonstrate the superiority of our proposed method. The source code will be available in github.com/umchaohuang/UPformer.

The Ideality of the Kiai's Role in Practical Political Constellations

This research aims to find a philosophical basis for the ideal position of kiai in practical political constellations. This study is a qualitative research with a literature study design and descriptive method focusing on finding the ideal concept of kiai in politics based on its reality dimensions. The research results show that kiai and politics in Islam are two things that are synergistic, kiai is a prototypical representation of the Prophet's heirs who appear as an active agent of social change, and the kiai's ideality in politics ultimately returns to the kiai's own consistency. In politics which is filled with intrigue and conflict, political leaders must be able to minimize conflict and act as problem solvers. If not, then religious positions can become instruments of power. Kiai must be able to realize his sociological role as agent of change consistently because politics is not just a tool of power but the science of how to build universal benefit.

Class‐Relation Reasoning with Knowledge‐Transfer for Few‐Shot Object Detection

Few‐Shot Object Detection (FSOD) task involves accurately identifying target object classes using only a small set of labeled samples. Most of the current FSOD tasks independently predict class prototype features without considering class relationships and only rely on visual information. To address these challenges, we propose a novel Class‐relational Reasoning Method with Knowledge‐transfer (CRK‐Net), built on the meta‐learning‐based framework. Although data may be scarce, the semantic relationship between classes is invariant, Joint‐feature Fusion Module (JFM) are hence proposed to transfers the semantic information of different categories in the natural language world to integrate with visual information and produce multi‐modality embeddings. Some base classes and novel classes have similar features, so this can be borrowed by modeling the relationship between classes feature. Building upon the observation, we propose a Class‐relational Reasoning Module (CRM) to establish the correlations between categories and enhance prototype representations for each category. After passing through the JFM and CRM modules, a high‐quality class prototype is finally produced for subsequent regression and classification. Extensive experiments on PASCAL VOC demonstrate the effectiveness of our proposed method and provide a new scheme for fusing semantic and visual information. © 2024 Institute of Electrical Engineers of Japan. Published by Wiley Periodicals LLC.

Multi-channels Prototype Contrastive Learning with Condition Adversarial Attacks for Few-shot Event Detection

Few-shot Event Detection (FSED) is a sub-task of Event Detection that aims to accurately identify event types with limited training instances and enable smooth transfer to newly-emerged event types. Recently, the dominant works have used the prototypical network to accomplish this task and employ contrastive learning to alleviate the issue of semantically-close categories. Nevertheless, these methods still suffer from two serious problems: (1) inadequate learning of prototype representations resulting from limited training data; (2) hard-easy sample imbalance and categories imbalance caused by the large number of non-trigger word("O" tags) in the token-level classification task. To address the problems, this paper proposes the Multi-channels Prototype and Contrastive learning method with Conditional Adversarial attack, which introduces the improved multi-channels prototype and contrastive networks to alleviate the categories and hard-easy samples imbalance. Moreover, we devise a constrained adversarial attack to improve the problem of limited training data. Extensive experimental results show that our model performs better than other FSED methods. All the code and data will be available for online public access.