Prototype Learning Research Articles

Revealing the Proximate Long-Tail Distribution in Compositional Zero-Shot Learning

Compositional Zero-Shot Learning (CZSL) aims to transfer knowledge from seen state-object pairs to novel unseen pairs. In this process, visual bias caused by the diverse interrelationship of state-object combinations blurs their visual features, hindering the learning of distinguishable class prototypes. Prevailing methods concentrate on disentangling states and objects directly from visual features, disregarding potential enhancements that could arise from a data viewpoint. Experimentally, we unveil the results caused by the above problem closely approximate the long-tailed distribution. As a solution, we transform CZSL into a proximate class imbalance problem. We mathematically deduce the role of class prior within the long-tailed distribution in CZSL. Building upon this insight, we incorporate visual bias caused by compositions into the classifier's training and inference by estimating it as a proximate class prior. This enhancement encourages the classifier to acquire more discernible class prototypes for each composition, thereby achieving more balanced predictions. Experimental results demonstrate that our approach elevates the model's performance to the state-of-the-art level, without introducing additional parameters.

SurgicalSAM: Efficient Class Promptable Surgical Instrument Segmentation

The Segment Anything Model (SAM) is a powerful foundation model that has revolutionised image segmentation. To apply SAM to surgical instrument segmentation, a common approach is to locate precise points or boxes of instruments and then use them as prompts for SAM in a zero-shot manner. However, we observe two problems with this naive pipeline: (1) the domain gap between natural objects and surgical instruments leads to inferior generalisation of SAM; and (2) SAM relies on precise point or box locations for accurate segmentation, requiring either extensive manual guidance or a well-performing specialist detector for prompt preparation, which leads to a complex multi-stage pipeline. To address these problems, we introduce SurgicalSAM, a novel end-to-end efficient-tuning approach for SAM to effectively integrate surgical-specific information with SAM’s pre-trained knowledge for improved generalisation. Specifically, we propose a lightweight prototype-based class prompt encoder for tuning, which directly generates prompt embeddings from class prototypes and eliminates the use of explicit prompts for improved robustness and a simpler pipeline. In addition, to address the low inter-class variance among surgical instrument categories, we propose contrastive prototype learning, further enhancing the discrimination of the class prototypes for more accurate class prompting. The results of extensive experiments on both EndoVis2018 and EndoVis2017 datasets demonstrate that SurgicalSAM achieves state-of-the-art performance while only requiring a small number of tunable parameters. The source code is available at https://github.com/wenxi-yue/SurgicalSAM.

A small-sample time-series signal augmentation and analysis method for quantitative assessment of bradykinesia in Parkinson's disease

Patients with Parkinson's disease (PD) usually have varying degrees of bradykinesia, and the current clinical assessment is mainly based on the Movement Disorder Society Unified PD Rating Scale, which can hardly meet the needs of objectivity and accuracy. Therefore, this paper proposed a small-sample time series classification method (DTW-TapNet) based on dynamic time warping (DTW) data augmentation and attentional prototype network. Firstly, for the problem of small sample sizes of clinical data, a DTW-based data merge method is used to achieve data augmentation. Then, the time series are dimensionally reorganized using random grouping, and convolutional operations are performed to learn features from multivariate time series. Further, attention mechanism and prototype learning are introduced to optimize the distance of the class prototype to which each time series belongs to train a low-dimensional feature representation of the time series, thus reducing the dependency on data volume. Clinical experiments were conducted to collect motion capture data of upper and lower limb movements from 36 patients with PD and eight healthy controls. For the upper limb movement data, the proposed method improved the classification accuracy, weighted precision, and kappa coefficient by 8.89%-15.56%, 9.22%-16.37%, and 0.13-0.23, respectively, compared with support vector machines, long short-term memory, and convolutional prototype network. For the lower limb movement data, the proposed method improved the classification accuracy, weighted precision, and kappa coefficient by 8.16%-20.41%, 10.01%-23.73%, and 0.12-0.28, respectively. The experiments and results show that the proposed method can objectively and accurately assess upper and lower limb bradykinesia in PD.

Smart System on Two-dimensional Shapes Recognition Application for Kindergarten Students

Abstract. Kindergarten-aged children are going through an important period of cognitive development, such as the ability to think concretely, including recognizing simple geometric shapes such as circles, triangles, and squares. However, many children find it difficult to understand the basic concepts of two-dimensional shapes.Purpose: It is necessary to develop prototype learning aids in the form of intelligent systems in two-dimensional shapes applications for kindergarten students, which utilize information technology and object visualization directly through cameras on smartphones. This is expected to increase children's learning motivation and help strengthen their understanding of two-dimensional shapes.Methods: The research combines Waterfall and Agile methodologies, tailoring them to four stages: plan and discovery, analysis and design, application development, and testing. Testing gathers accuracy with 120 smartphone-collected data points for square, triangle, circle, and pentagon shapes. Also, usability testing based on learnability, efficiency, memorability, error handling, and satisfaction, was obtained from six kindergarten teacher questionnaires and quantitatively processed.Results: The application achieves an accuracy rate of approximately 79%. Notably, accuracy decreases with fewer corners, mainly due to low resolution or lack of focus, resulting in simplified detected shapes. Regarding usability, it is evident that the application has received positive feedback from users, particularly kindergarten teachers, who have given it an average score of 78.83.Novelty: Distinguished from previous research, the novelty of this study resides in its ability to capture objects through a camera, eliminating the need for predefined shapes within the application, and innovating by creating an educational tool aligned with the kindergarten curriculum to recognize two-dimensional shapes. The research contribution is the creation of an innovative learning tool for kindergarteners, merging smartphone technology with real-world objects to teach two-dimensional shapes, thus integrating technology into early childhood education effectively, which has urgency in efforts to improve the quality of learning.

ScEVOLVE: cell-type incremental annotation without forgetting for single-cell RNA-seq data.

The evolution in single-cell RNA sequencing (scRNA-seq) technology has opened a new avenue for researchers to inspect cellular heterogeneity with single-cell precision. One crucial aspect of this technology is cell-type annotation, which is fundamental for any subsequent analysis in single-cell data mining. Recently, the scientific community has seen a surge in the development of automatic annotation methods aimed at this task. However, these methods generally operate at a steady-state total cell-type capacity, significantly restricting the cell annotation systems'capacity for continuous knowledge acquisition. Furthermore, creating a unified scRNA-seq annotation system remains challenged by the need to progressively expand its understanding of ever-increasing cell-type concepts derived from a continuous data stream. In response to these challenges, this paper presents a novel and challenging setting for annotation, namely cell-type incremental annotation. This concept is designed to perpetually enhance cell-type knowledge, gleaned from continuously incoming data. This task encounters difficulty with data stream samples that can only be observed once, leading to catastrophic forgetting. To address this problem, we introduce our breakthrough methodology termed scEVOLVE, an incremental annotation method. This innovative approach is built upon the methodology of contrastive sample replay combined with the fundamental principle of partition confidence maximization. Specifically, we initially retain and replay sections of the old data in each subsequent training phase, then establish a unique prototypical learning objective to mitigate the cell-type imbalance problem, as an alternative to using cross-entropy. To effectively emulate a model that trains concurrently with complete data, we introduce a cell-type decorrelation strategy that efficiently scatters feature representations of each cell type uniformly. We constructed the scEVOLVE framework with simplicity and ease of integration into most deep softmax-based single-cell annotation methods. Thorough experiments conducted on a range of meticulously constructed benchmarks consistently prove that our methodology can incrementally learn numerous cell types over an extended period, outperforming other strategies that fail quickly. As far as our knowledge extends, this is the first attempt to propose and formulate an end-to-end algorithm framework to address this new, practical task. Additionally, scEVOLVE, coded in Python using the Pytorch machine-learning library, is freely accessible at https://github.com/aimeeyaoyao/scEVOLVE.

An ensemble neural network model for predicting the energy utility in individual houses

With the rapid increase in the world's human population and the advancement of technology, energy utilization has substantially increased. To sustain a stable stream of energy, it is vital to anticipate power utilization beforehand. This issue makes predicting short-term electrical time series use difficult. In this research, the user proposed a neural network with a deep learning prototype that performs energy prediction by combining. Long Short-Term Memory (LSTM) and Feed Forward Neural Networks (FFNN). In order to enhance the performance, the ensemble FFNN-LSTM network and the Stationary Wavelet Transform (SWT) method are combined to create a hybrid deep learning algorithm. The SWT's reduction of instability and expansion of data dimensionality may improve the FFNN-LSTM forecasting accuracy. The suggested technique's predicting effectiveness is even further improved by the ensemble Long short-term neural network. On the basis of a real-world household energy usage database gathered by the "UK-DALE" project, validation studies were carried out. Researchers determined that the LSTM-FFNN based SWT produced a better result in terms of accuracy and energy utilization than other existing results after contrasting the experimental findings to the baseline, which is the conventional LSTM and Moving Average (MA) based upon the Root Mean Squared Error (RMSE) score.

Event diagnosis method for a nuclear power plant using meta-learning

Artificial intelligence (AI) techniques are now being considered in the nuclear field, but application faces with the lack of actual plant data. For this reason, most previous studies on AI applications in nuclear power plants (NPPs) have relied on simulators or thermal-hydraulic codes to mimic the plants. However, it remains uncertain whether an AI model trained using a simulator can properly work in an actual NPP. To address this issue, this study suggests the use of metadata, which can give information about parameter trends. Referred to here as robust AI, this concept started with the idea that although the absolute value of a plant parameter differs between a simulator and actual NPP, the parameter trend is identical under the same scenario. Based on the proposed robust AI, this study designs an event diagnosis algorithm to classify abnormal and emergency scenarios in NPPs using prototypical learning. The algorithm was trained using a simulator referencing a Westinghouse 990 MWe reactor and then tested in different environments in Advanced Power Reactor 1400 MWe simulators. The algorithm demonstrated robustness with 100 % diagnostic accuracy (117 out of 117 scenarios). This indicates the potential of the robust AI–based algorithm to be used in actual plants.

PERANCANGAN UI/UX LEARNING MANAGEMENT SYSTEM (LMS) APLIKASI MOBILE EDU-LEARN MENGGUNAKAN METODE DESIGN THINKING

Distance learning during the Covid-19 pandemic has the potential to reduce student enthusiasm for learning. The problems faced by Learning Management System (LMS) users are the lack of features that support distance learning, many students are left behind in their learning and teachers who find it difficult to control students and the Learning Management System whose features are not yet complete enough. Based on this problem, the Mobile Edu-Learn application was built which provides easy distance learning with features that support teachers in controlling students and reduces the decline in student enthusiasm for learning with features needed by users. This research uses the Design Thinking method which goes through 5 stages : Emphatize, Define, Ideate, Prototype, Testing. The results of this research are a prototype Learning Management System for the Mobile Edu-Learn application by implementing features that have been suggested by users such as integrated features with the Zoom application, downloading student grades, academic reminders (attendance, assignments and tests), and discussion groups. At the testing stage, using two methods, namely task scenarios with maze tools and usability testing, resulted in an average score of 80.83 with a grade scale of B, adjective rating excellent and included in the Acceptable category with two design iterations.

Prototype-Augmented Self-Supervised Generative Network for Generalized Zero-Shot Learning.

Generalized Zero-Shot Learning (GZSL) aims at recognizing images from both seen and unseen classes by constructing correspondences between visual images and semantic embedding. However, existing methods suffer from a strong bias problem, where unseen images in the target domain tend to be recognized as seen classes in the source domain. To address this issue, we propose a Prototype-augmented Self-supervised Generative Network by integrating self-supervised learning and prototype learning into a feature generating model for GZSL. The proposed model enjoys several advantages. First, we propose a Self-supervised Learning Module to exploit inter-domain relationships, where we introduce anchors as a bridge between seen and unseen categories. In the shared space, we pull the distribution of the target domain away from the source domain and obtain domain-aware features. To our best knowledge, this is the first work to introduce self-supervised learning into GZSL as learning guidance. Second, a Prototype Enhancing Module is proposed to utilize class prototypes to model reliable target domain distribution in finer granularity. In this module, a Prototype Alignment mechanism and a Prototype Dispersion mechanism are combined to guide the generation of better target class features with intra-class compactness and inter-class separability. Extensive experimental results on five standard benchmarks demonstrate that our model performs favorably against state-of-the-art GZSL methods.

Cross-Domain Sentiment Analysis via Disentangled Representation and Prototypical Learning

Cross-Domain Sentiment Analysis via Disentangled Representation and Prototypical Learning

Electricity Theft Detection Method Based on Ensemble Learning and Prototype Learning

Electricity Theft Detection Method Based on Ensemble Learning and Prototype Learning

Ensemble Prototype Network For Weakly Supervised Temporal Action Localization.

Weakly supervised temporal action localization (TAL) aims to localize the action instances in untrimmed videos using only video-level action labels. Without snippet-level labels, this task should be hard to distinguish all snippets with accurate action/background categories. The main difficulties are the large variations brought by the unconstraint background snippets and multiple subactions in action snippets. The existing prototype model focuses on describing snippets by covering them with clusters (defined as prototypes). In this work, we argue that the clustered prototype covering snippets with simple variations still suffers from the misclassification of the snippets with large variations. We propose an ensemble prototype network (EPNet), which ensembles prototypes learned with consensus-aware clustering. The network stacks a consensus prototype learning (CPL) module and an ensemble snippet weight learning (ESWL) module as one stage and extends one stage to multiple stages in an ensemble learning way. The CPL module learns the consensus matrix by estimating the similarity of clustering labels between two successive clustering generations. The consensus matrix optimizes the clustering to learn consensus prototypes, which can predict the snippets with consensus labels. The ESWL module estimates the weights of the misclassified snippets using the snippet-level loss. The weights update the posterior probabilities of the snippets in the clustering to learn prototypes in the next stage. We use multiple stages to learn multiple prototypes, which can cover the snippets with large variations for accurate snippet classification. Extensive experiments show that our method achieves the state-of-the-art weakly supervised TAL methods on two benchmark datasets, that is, THUMOS'14, ActivityNet v1.2, and ActivityNet v1.3 datasets.

HOPE: Hybrid-Granularity Ordinal Prototype Learning for Progression Prediction of Mild Cognitive Impairment.

Mild cognitive impairment (MCI) is often at high risk of progression to Alzheimer's disease (AD). Existing works to identify the progressive MCI (pMCI) typically require MCI subtype labels, pMCI vs. stable MCI (sMCI), determined by whether or not an MCI patient will progress to AD after a long follow-up. However, prospectively acquiring MCI subtype data is time-consuming and resource-intensive; the resultant small datasets could lead to severe overfitting and difficulty in extracting discriminative information. Inspired by that various longitudinal biomarkers and cognitive measurements present an ordinal pathway on AD progression, we propose a novel Hybrid-granularity Ordinal PrototypE learning (HOPE) method to characterize AD ordinal progression for MCI progression prediction. First, HOPE learns an ordinal metric space that enables progression prediction by prototype comparison. Second, HOPE leverages a novel hybrid-granularity ordinal loss to learn the ordinal nature of AD via effectively integrating instance-to-instance ordinality, instance-to-class compactness, and class-to-class separation. Third, to make the prototype learning more stable, HOPE employs an exponential moving average strategy to learn the global prototypes of NC and AD dynamically. Experimental results on the internal ADNI and the external NACC datasets demonstrate the superiority of the proposed HOPE over existing state-of-the-art methods as well as its interpretability. Source code is made available at https://github.com/thibault-wch/HOPE-for-mild-cognitive-impairment.

Effective Semi-Supervised Medical Image Segmentation with Probabilistic Representations and Prototype Learning.

Label scarcity, class imbalance and data uncertainty are three primary challenges that are commonly encountered in the semi-supervised medical image segmentation. In this work, we focus on the data uncertainty issue that is overlooked by previous literature. To address this issue, we propose a probabilistic prototype-based classifier that introduces uncertainty estimation into the entire pixel classification process, including probabilistic representation formulation, probabilistic pixel-prototype proximity matching, and distribution prototype update, leveraging principles from probability theory. By explicitly modeling data uncertainty at the pixel level, model robustness of our proposed framework to tricky pixels, such as ambiguous boundaries and noises, is greatly enhanced when compared to its deterministic counterpart and other uncertainty-aware strategy. Empirical evaluations on three publicly available datasets that exhibit severe boundary ambiguity show the superiority of our method over several competitors. Moreover, our method also demonstrates a stronger model robustness to simulated noisy data. Code is available at https://github.com/IsYuchenYuan/PPC.

Enhanced Implicit Sentiment Understanding With Prototype Learning and Demonstration for Aspect-Based Sentiment Analysis

Enhanced Implicit Sentiment Understanding With Prototype Learning and Demonstration for Aspect-Based Sentiment Analysis

Predicting Clinical Anticancer Drug Response of Patients by using Domain Alignment and Prototypical Learning.

Anticancer drug response prediction is crucial in developing personalized treatment plans for cancer patients. However, High-quality patient anticancer drug response data are scarce and cell line data and patient data have different distributions, models trained solely on cell line data perform poorly. Some existing methods predict anticancer drug response by transferring knowledge from the cell line domain to the patient domain using transfer learning. However, the robustness of these classifiers is affected by anomalies in the cell line data, and they do not utilize the knowledge in the unlabeled target domain data. To this end, we proposed a model called DAPL to predict patient responses to anticancer drugs. The model extracts domain-invariant features from cell lines and patients by constructing multiple VAEs and extracts drug features using GNNs. These features are then combined for prototypical learning to train a classifier, resulting in better predictions of patient anticancer drug response. We used the cell line datasets CCLE and GDSC as source domains and the patient datasets TCGA and PDTC as target domains and conducted experiments. The results indicate that DAPL shows excellent performance in predicting patient anticancer drug response compared to other state-of-the-art methods.

Adaptive Prototype Learning for Weakly-supervised Temporal Action Localization.

Weakly-supervised Temporal Action Localization (WTAL) aims to localize action instances with only video-level labels during training, where two primary issues are localization incompleteness and background interference. To relieve these two issues, recent methods adopt an attention mechanism to activate action instances and simultaneously suppress background ones, which have achieved remarkable progress. Nevertheless, we argue that these two issues have not been well resolved yet. On the one hand, the attention mechanism adopts fixed weights for different videos, which are incapable of handling the diversity of different videos, thus deficient in addressing the problem of localization incompleteness. On the other hand, previous methods only focus on learning the foreground attention and the attention weights usually suffer from ambiguity, resulting in difficulty of suppressing background interference. To deal with the above issues, in this paper we propose an Adaptive Prototype Learning (APL) method for WTAL, which includes two key designs: (1) an Adaptive Transformer Network (ATN) to explicitly model background and learn video-adaptive prototypes for each specific video, (2) an OT-based Collaborative (OTC) training strategy to guide the learning of prototypes and remove the ambiguity of the foreground-background separation by introducing an Optimal Transport (OT) algorithm into the collaborative training scheme between RGB and FLOW streams. These two key designs can work together to learn video-adaptive prototypes and solve the above two issues, achieving robust localization. Extensive experimental results on two standard benchmarks (THUMOS14 and ActivityNet) demonstrate that our proposed APL performs favorably against state-of-the-art methods.

Separable Deep Graph Convolutional Network Integrated With CNN and Prototype Learning for Hyperspectral Image Classification

Separable Deep Graph Convolutional Network Integrated With CNN and Prototype Learning for Hyperspectral Image Classification

Class-Wise Contrastive Prototype Learning for Semi-Supervised Classification Under Intersectional Class Mismatch

Class-Wise Contrastive Prototype Learning for Semi-Supervised Classification Under Intersectional Class Mismatch

Learning Transferable Conceptual Prototypes for Interpretable Unsupervised Domain Adaptation.

Despite the great progress of unsupervised domain adaptation (UDA) with the deep neural networks, current UDA models are opaque and cannot provide promising explanations, limiting their applications in the scenarios that require safe and controllable model decisions. At present, a surge of work focuses on designing deep interpretable methods with adequate data annotations and only a few methods consider the distributional shift problem. Most existing interpretable UDA methods are post-hoc ones, which cannot facilitate the model learning process for performance enhancement. In this paper, we propose an inherently interpretable method, named Transferable Conceptual Prototype Learning (TCPL), which could simultaneously interpret and improve the processes of knowledge transfer and decision-making in UDA. To achieve this goal, we design a hierarchically prototypical module that transfers categorical basic concepts from the source domain to the target domain and learns domain-shared prototypes for explaining the underlying reasoning process. With the learned transferable prototypes, a self-predictive consistent pseudo-label strategy that fuses confidence, predictions, and prototype information, is designed for selecting suitable target samples for pseudo annotations and gradually narrowing down the domain gap. Comprehensive experiments show that the proposed method can not only provide effective and intuitive explanations but also outperform previous state-of-the-arts. Code is available at https://drive.google.com/file/d/1b1EHFghiF1ExD-Cn1HYg75VutfkXWp60/view?usp=sharing.