Prototype Generation Research Articles

Exploration of an intrinsically explainable self-attention based model for prototype generation on single-channel EEG sleep stage classification

Prototype-based methods in deep learning offer interpretable explanations for decisions by comparing inputs to typical representatives in the data. This study explores the adaptation of SESM, a self-attention-based prototype method successful in electrocardiogram (ECG) tasks, for electroencephalogram (EEG) signals. The architecture is evaluated on sleep stage classification, exploring its efficacy in predicting stages with single-channel EEG. The model achieves comparable test accuracy compared to EEGNet, a state-of-the-art black-box architecture for EEG classification. The generated prototypical components are exaimed qualitatively and using the area over the perterbation curve (AOPC) indicate some alignment with expected bio-markers for different sleep stages such as alpha spindles and slow waves in non-REM sleep, but the results are severely limited by the model’s ability to only extract and present information in the time-domain. Ablation studies are used to explore the impact of kernel size, number of heads, and diversity threshold on model performance and explainability. This study represents the first application of a self-attention based prototype method to EEG data and provides a step forward in explainable AI for EEG data analysis.

Study of High-Frequency Pulsating Flow Control Valves

Pulsating flow signals are widely used in the fields of dynamic calibration of flow meters and reliability testing of fluid transmission pipelines. A high-frequency pulsating flow control valve is proposed, its structure and working principle are introduced, and its mathematical model is also established. Finally, an experimental platform is built to experimentally verify the performance of the high-frequency pulsating flow signal generation of this flow control valve prototype. When applying sinusoidal flow signals of 1–25 Hz, the amplitude of the output flow starts to decay after 2 Hz, and the frequency corresponding to the decay to −3 dB is about 23.5 Hz. The experimental results show that the valve can effectively generate unidirectional sinusoidal flow with a maximum amplitude of 25 L/min and a frequency of 23.5 Hz.

Boosting Zero-Shot Learning via Contrastive Optimization of Attribute Representations.

Zero-shot learning (ZSL) aims to recognize classes that do not have samples in the training set. One representative solution is to directly learn an embedding function associating visual features with corresponding class semantics for recognizing new classes. Many methods extend upon this solution, and recent ones are especially keen on extracting rich features from images, e.g., attribute features. These attribute features are normally extracted within each individual image; however, the common traits for features across images yet belonging to the same attribute are not emphasized. In this article, we propose a new framework to boost ZSL by explicitly learning attribute prototypes beyond images and contrastively optimizing them with attribute-level features within images. Besides the novel architecture, two elements are highlighted for attribute representations: a new prototype generation module (PM) is designed to generate attribute prototypes from attribute semantics; a hard-example-based contrastive optimization scheme is introduced to reinforce attribute-level features in the embedding space. We explore two alternative backbones, CNN-based and transformer-based, to build our framework and conduct experiments on three standard benchmarks, Caltech-UCSD Birds-200-2011 (CUB), SUN attribute database (SUN), and animals with attributes 2 (AwA2). Results on these benchmarks demonstrate that our method improves the state of the art by a considerable margin. Our codes will be available at https://github.com/dyabel/CoAR-ZSL.git.

Noise-Tolerant Hybrid Prototypical Learning with Noisy Web Data

We focus on the challenging problem of learning an unbiased classifier from a large number of potentially relevant but noisily labeled web images given only a few clean labeled images. This problem is particularly practical because it reduces the expensive annotation costs by utilizing freely accessible web images with noisy labels. Typically, prototypes are representative images or features used to classify or identify other images. However, in the few clean and many noisy scenarios, the class prototype can be severely biased due to the presence of irrelevant noisy images. The resulting prototypes are less compact and discriminative, as previous methods do not take into account the diverse range of images in the noisy web image collections. On the other hand, the relation modeling between noisy and clean images is not learned for the class prototype generation in an end-to-end manner, which results in a suboptimal class prototype. In this article, we introduce a similarity maximization loss named SimNoiPro. Our SimNoiPro first generates noise-tolerant hybrid prototypes composed of clean and noise-tolerant prototypes and then pulls them closer to each other. Our approach considers the diversity of noisy images by explicit division and overcomes the optimization discrepancy issue. This enables better relation modeling between clean and noisy images and helps extract judicious information from the noisy image set. The evaluation results on two extended few-shot classification benchmarks confirm that our SimNoiPro outperforms prior methods in measuring image relations and cleaning noisy data.

Decoupling Deep Learning for Enhanced Image Recognition Interpretability

The quest for enhancing the interpretability of neural networks has become a prominent focus in recent research endeavors. Prototype-based neural networks have emerged as a promising avenue for imbuing models with interpretability by gauging the similarity between image components and category prototypes to inform decision-making. However, these networks face challenges as they share similarity activations during both the inference and explanation processes, creating a tradeoff between accuracy and interpretability. To address this issue and ensure that a network achieves high accuracy and robust interpretability in the classification process, this article introduces a groundbreaking prototype-based neural network termed the “Decoupling Prototypical Network” (DProtoNet). This novel architecture comprises encoder, inference, and interpretation modules. In the encoder module, we introduce decoupling feature masks to facilitate the generation of feature vectors and prototypes, enhancing the generalization capabilities of the model. The inference module leverages these feature vectors and prototypes to make predictions based on similarity comparisons, thereby preserving an interpretable inference structure. Meanwhile, the interpretation module advances the field by presenting a novel approach: a “multiple dynamic masks decoder” that replaces conventional upsampling similarity activations. This decoder operates by perturbing images with mask vectors of varying sizes and learning saliency maps through consistent activation. This methodology offers a precise and innovative means of interpreting prototype-based networks. DProtoNet effectively separates the inference and explanation components within prototype-based networks. By eliminating the constraints imposed by shared similarity activations during the inference and explanation phases, our approach concurrently elevates accuracy and interpretability. Experimental evaluations on diverse public natural datasets, including CUB-200-2011, Stanford Cars, and medical datasets like RSNA and iChallenge-PM, corroborate the substantial enhancements achieved by our method compared to previous state-of-the-art approaches. Furthermore, ablation studies are conducted to provide additional evidence of the effectiveness of our proposed components.

Label-Guided relation prototype generation for Continual Relation Extraction.

Continual relation extraction (CRE) aims to extract relations towards the continuous and iterative arrival of new data. To address the problem of catastrophic forgetting, some existing research endeavors have focused on exploring memory replay methods by storing typical historical learned instances or embedding all observed relations as prototypes by averaging the hidden representation of samples and replaying them in the subsequent training process. However, this prototype generation method overlooks the rich semantic information within the label namespace and are also constrained by memory size, resulting in inadequate descriptions of relation semantics by relation prototypes. To this end, we introduce an approach termed Label-Guided Relation Prototype Generation. Initially, we enhance the representations of label embeddings through a technique named label knowledge infusion. Following that, we utilize the multi-head attention mechanism to form relation prototypes, allowing them to capture diverse aspects of typical instances. The embeddings of relation labels are utilized at this stage, leveraging their contained semantics. Additionally, we propose a feature-based distillation loss function called multi-similarity distillation, to ensure the model retains prior knowledge after learning new tasks. The experimental results indicate that our method has achieved competitive performance compared to the state-of-the-art baseline models in CRE.

Semantic Guided prototype learning for Cross-Domain Few-Shot hyperspectral image classification

Hyperspectral image(HSI) classification has found extensive application in the field of remote sensing. However, traditional HSI classification methods typically rely on a large number of labeled samples, incurring significant costs and requiring considerable time and effort for acquisition. To address this issue, this paper proposes a novel cross-domain few-shot learning method termed Semantic Guided Prototype Learning (SGPL), which aims to optimize prototype learning through the incorporation semantic information, thereby enhancing the classification accuracy of hyperspectral images in few-shot scenarios. The core idea of the SGPL method is to integrate semantic information into the prototype learning framework, thereby enhancing the expressive capacity of each category prototype. Specifically, HSI category labels are encoded via one-hot encoding, and the corresponding HSI features are concatenated with these encoded vectors to construct an augmented feature representation. These augmented feature vectors are subsequently fed into a prototype generator, which is designed to learn the generation of category prototypes from the concatenated features. Through the application of one-hot encoding and feature concatenation, SGPL effectively captures and leverages discriminative information between classes. Moreover, the proposed model exhibits superior performance on three different HSI datasets: Salinas, the University of Pavia(UP), and Indian Pines(IP) with overall classification accuracies of 91.27%, 85.84%, and 69.33%, respectively, outperforming other methods. The source code is available at https://github.com/AIYAU/SGPL.

Prototype-Guided Graph Reasoning Network for Few-Shot Medical Image Segmentation.

Few-shot semantic segmentation (FSS) is of tremendous potential for data-scarce scenarios, particularly in medical segmentation tasks with merely a few labeled data. Most of the existing FSS methods typically distinguish query objects with the guidance of support prototypes. However, the variances in appearance and scale between support and query objects from the same anatomical class are often exceedingly considerable in practical clinical scenarios, thus resulting in undesirable query segmentation masks. To tackle the aforementioned challenge, we propose a novel prototype-guided graph reasoning network (PGRNet) to explicitly explore potential contextual relationships in structured query images. Specifically, a prototype-guided graph reasoning module is proposed to perform information interaction on the query graph under the guidance of support prototypes to fully exploit the structural properties of query images to overcome intra-class variances. Moreover, instead of fixed support prototypes, a dynamic prototype generation mechanism is devised to yield a collection of dynamic support prototypes by mining rich contextual information from support images to further boost the efficiency of information interaction between support and query branches. Equipped with the proposed two components, PGRNet can learn abundant contextual representations for query images and is therefore more resilient to object variations. We validate our method on three publicly available medical segmentation datasets, namely CHAOS-T2, MS-CMRSeg, and Synapse. Experiments indicate that the proposed PGRNet outperforms previous FSS methods by a considerable margin and establishes a new state-of-the-art performance.

Designing a Data Dashboard for Health Departments and Overdose Fatality Review Teams

IntroductionOverdose Fatality Review (OFR) teams within health departments rely on data from multiple isolated sources to study the opioid crisis. Recently, we linked electronic health record data with state agency data to form a regional Substance Misuse Data Commons (SMDC) using privacy-preserving record linkage. Our goal in this study was to use human factors design principles to design a data dashboard for OFR teams using the linked datasets that will overcome current barriers within OFR workflows. MethodsWe utilized NASA task load surveys, semi-structured interviews, and design sessions with end users to identify data needs for an optimal dashboard design. We assessed current workloads, data collection processes, and desired future state. We subsequently performed iterative design sessions for the generation and evaluation of low-fidelity prototypes. To overcome issues with privacy and security, we used synthetic data in a cloud-based platform to represent the SMDC for simulation. ResultsEleven OFR organizers participated. Pre-dashboard surveys on existing workflow showed high mental workload associated with data aggregation and case review, identifying a need for more accessible, comprehensive data. In our low-fidelity dashboard demo with synthetic data, iterative design adjustments were made in data visualization, storyline organizations, and theme-based data aggregation across pre-hospital and hospital data. ConclusionsWe refined the data dashboard prototype into a high-fidelity version, set for further usability and human factors evaluation. We addressed privacy and security concerns through synthetic data use while the real-world data is maintained in a HIPAA-secure Azure cloud environment with access for approved users.

Individualized generation of women's prototype based on the classification of body shape

The traditional prototype method only relies on the basic dimensions of the human body such as bust girth and back length to make the prototype pattern, ignoring the differences of human body details dimensions. In order to improve the fit of garments, the research on intelligent generation technology of individualized garment patterns has become one of the hot spots in the field of clothing. Based on the 3D body scanning technology, we proposed a method for generating prototype pattern based on body shape classification. The proposed method takes as inputs the body parameters (width, thickness and angle), while the output of the method is one of the four pattern generation rules-"Y" shape, "V" shape, "I" shape, or "X" shape. 207 subjects were divided into four categories based on the body angle parameters, namely "Y" shape, "V" shape, "I" shape and "X" shape, and a shape recognition model was built. The pattern database is required to obtain pattern generation rules by 3D point cloud reconstruction and flattening. Combined with the human body shape parameters, the mathematical formulas of the feature points on the pattern landmarks are built. The accuracy of body shape recognition model is 97.4%. The error analysis of the predicted pattern parameters shows that 90% of the pattern feature points have a goodness of fit above 0.7. In the 5 main landmarks, the proportion of pattern within the absolute error range is more than 80%, indicating that the prediction effect of the pattern is good. This method can be applied to the process of automatic pattern generation system based 2D measurement to improve work efficiency.

Initial experimental testing of a hybrid solar-dish Brayton cycle for combined heat and power (ST-CHP)

The Solar Turbo Combined Heat and Power (ST-CHP) project developed a novel solar-gas hybrid prototype for combined heat and power generation in Pretoria, South Africa. A vacuum-membrane faceted parabolic dish with a large-pipe open-cavity receiver was coupled to a counterflow recuperator, a combustion chamber, a micro gas turbine with air bearings and a phase change thermal energy storage unit containing solar salts. This study aimed to validate the electrical output of the full-scale dish-Brayton prototype, addressing the literature gap on micro-scale dish-Brayton plants' in-field power generation. Solar hybridization of micro gas turbine technology can significantly reduce combustion fuel consumption. A performance analysis under relevant conditions revealed a late afternoon micro gas turbine output intermittently peaking at 0.4kWe and subsequently stabilizing to a steady state of 0.145kWe at 130krpm. A SimuPact numerical model and an analytical model supplemented the telemetry data to reduce interference in the experimental setup and fully characterize the ST-CHP prototype performance, estimating a steady-state turbine isentropic efficiency of 57%, a compressor efficiency of 71% and a collector efficiency of 17% due to the late afternoon steady-state point. Analytical case studies revealed that fuel savings of between 12% and 33% at the combustion chamber were achievable from the solarized preheating. A subsequent test of the micro gas turbine without solar hybridization or a recuperator resulted in 1.05kWe being generated. The study confirms the dish-Brayton prototype's viability for combined heat and power generation, producing an initial full-scale performance characterisation during in-field testing, and highlighting the impact of solar hybridization on turbine electrical output. Optical efficiency, insulation effectiveness, pressure losses, and turbine operating conditions were identified as critical areas requiring optimization to improve electrical output. The lessons learned, and the calibrated numerical model may be used to optimize the performance of the dish-Brayton plant in future work. The successful in-field full-scale power generation of the ST-CHP prototype adds to the available literature on dish-Brayton technology and brings the technology closer to a commercial product.

The Impact of AI on Human Roles in the User Interface & User Experience Design Industry

The ever-evolving field of User Interface & User Experience UI/UX design prioritizes creating user-friendly interfaces. Traditionally, this has been a human-centred process. However, Artificial Intelligence AI offers new possibilities for automating design tasks and leveraging data for user insights. This research paper delves into the potential impact of AI on UI/UX design. The core question is whether AI tools can effectively replace human designers in crafting user experiences. Alternatively, can AI work alongside designers to enhance their capabilities? This paper explores these questions through a review of existing research and proposes a research methodology to further investigate the topic. The literature review analyses how AI can be used for tasks like user behaviour analysis, A/B testing, and prototype generation. However, it also acknowledges AI's limitations in understanding user emotions, implementing creative solutions, and adapting to unforeseen user needs. The suggested research methodology will give a combined approach, making a review of existing literature with interviews from UI/UX professionals and a case study analyzing a design project utilising AI tools. By investigating the strengths and limitations of AI in design tasks, how AI can be integrated into the workflow, and the ethical considerations surrounding AI-driven design decisions, this research helps to provide a better understanding of the relationship between AI and human designers in the UI/UX field. The findings can inform design education, industry practices, and the development of future AI tools specifically tailored for UI/UX design applications. Keywords: UI/UX Design, Artificial Intelligence, Human-Computer Interaction, User Experience, User Interface, Design Automation

Relevant Intrinsic Feature Enhancement Network for Few-Shot Semantic Segmentation

For few-shot semantic segmentation, the primary task is to extract class-specific intrinsic information from limited labeled data. However, the semantic ambiguity and inter-class similarity of previous methods limit the accuracy of pixel-level foreground-background classification. To alleviate these issues, we propose the Relevant Intrinsic Feature Enhancement Network (RiFeNet). To improve the semantic consistency of foreground instances, we propose an unlabeled branch as an efficient data utilization method, which teaches the model how to extract intrinsic features robust to intra-class differences. Notably, during testing, the proposed unlabeled branch is excluded without extra unlabeled data and computation. Furthermore, we extend the inter-class variability between foreground and background by proposing a novel multi-level prototype generation and interaction module. The different-grained complementarity between global and local prototypes allows for better distinction between similar categories. The qualitative and quantitative performance of RiFeNet surpasses the state-of-the-art methods on PASCAL-5i and COCO benchmarks.

An adaptive class prototype generation framework for partial label learning

Partial label learning involves instances with multiple potential labels, of which only one is correct. This process poses challenges when managing limited-size datasets and ambiguity arising from overlapping characteristics across various categories. This paper proposes a novel framework based on class prototype generation to address these issues. The framework integrates a generator that creates label-specific prototypes, a discriminator that assigns weights reflecting the representativeness of examples, and a classifier learning with the assistance of the generator and discriminator. We introduce an adaptive loss adjustment method intended to dynamically balance each model’s weights and mitigate the potential negative impact of inferior examples. Experimental results demonstrate our method’s superiority over state-of-the-art techniques on widely used real-world datasets, with various visualizations illustrating its effectiveness.

A small sample bearing fault diagnosis method based on ConvGRU relation network

Considering that in the fault diagnosis of bearing based on relation network, using the sample mean value as the class prototype for each class is susceptible to outliers, resulting in inaccurate class prototypes, this paper proposes a convolutional gate recurrent unit (ConvGRU) relation network fault diagnosis model; firstly, the model utilizes a embedding module to extract bearing fault features, and then uses the ConvGRU as a learnable class prototype generator to generate class prototypes for each class. Secondly, a relation module is utilized to measure the similarity between class prototypes and the sample features of the query set, obtaining relation scores, and ultimately achieving fault diagnosis. In order to test the validity and advantages of the model, experimental verification and analysis were conducted on the case western storage rolling bearing dataset. The results of the experiment show that the class prototypes generated by the ConvGRU class prototype generation module have better discrimination and accuracy compared to the class prototypes generated by the relation network. In the 10-way 5-shot experiment, the accuracy of the model proposed in this paper reaches 99.60%, which increases by 6.63%, 5.10%, 4.80%, and 1.75% compared to k-nearest neighbor, convolutional neural network, prototypical network, and relation network. The method proposed in this paper helps to generate more accurate class prototypes and has a certain effect on improving the accuracy of model fault diagnosis.

Learning self-target knowledge for few-shot segmentation

Few-shot semantic segmentation uses a few annotated data of a specific class in the support set to segment the target of the same class in the query set. Most existing approaches fail to perform well when there are significant intra-class variances. This paper alleviates the problem by concentrating on mining the query image and using the support set as supplementary information. First, it proposes a Query Prototype Generation Module to generate a query foreground prototype from the query features. Specifically, we use both prototype-level and pixel-level similarity matching to generate two complementary initial prototypes, which we then integrate to create a discriminative query foreground prototype. Second, we propose a Support Auxiliary Refinement Module to further guide the final precise prediction of the query image by leveraging the target category information of the support set through step-by-step mining. Specifically, we generate a query-support mixture prototype based on the support prototype representation obtained using the attention mechanism. Then we generate a support supplement prototype to complement the missing information by encoding over the foreground regions that the query-support mixture prototype fails to segment out. Extensive experiments on PASCAL-5i and COCO-20i demonstrate that our model outperforms the prior works of few-shot segmentation.

Enhancing Sample Utilization in Noise-robust Deep Metric Learning with Subgroup-based Positive-pair Selection.

The existence of noisy labels in real-world data negatively impacts the performance of deep learning models. Although much research effort has been devoted to improving the robustness towards noisy labels in classification tasks, the problem of noisy labels in deep metric learning (DML) remains under-explored. Existing noisy label learning methods designed for DML mainly discard suspicious noisy samples, resulting in a waste of the training data. To address this issue, we propose a noise-robust DML framework with SubGroup-based Positive-pair Selection (SGPS), which constructs reliable positive pairs for noisy samples to enhance the sample utilization. Specifically, SGPS first effectively identifies clean and noisy samples by a probability-based clean sample selectionstrategy. To further utilize the remaining noisy samples, we discover their potential similar samples based on the subgroup information given by a subgroup generation module and then aggregate them into informative positive prototypes for each noisy sample via a positive prototype generation module. Afterward, a new contrastive loss is tailored for the noisy samples with their selected positive pairs. SGPS can be easily integrated into the training process of existing pair-wise DML tasks, like image retrieval and face recognition. Extensive experiments on multiple synthetic and real-world large-scale label noise datasets demonstrate the effectiveness of our proposed method. Without any bells and whistles, our SGPS framework outperforms the state-of-the-art noisy label DML methods.

Instance Segmentation Evaluation For Traffic Signs

This research paper focuses on developing a traffic sign recognition system based on the You Only Look At Coefficients (YOLACT) model, a one-stage instance segmentation model that offers high performance in terms of accuracy and reliability. However, the performance of YOLACT is influenced by various conditions such as day/night and different angles of objects. Therefore, this study aims to evaluate the impact of different angles and environments on the performance of the system. The paper discusses the framework, backbone structure, prototype generation branch, mask coefficient, and mask assembly used in the system. ResNet-101 and ResNet-50 were used as the backbone structure to extract feature maps of objects in the input image. The prototype generation branch generates prototype masks using fully convolutional networks (FCN), and the mask coefficient branch generates the Mask assembly using the sigmoid nonlinearity. Two models, YOLACT and Mask-RCNN, were evaluated in terms of mean precision (mAP) and frames per second (FPS) with the front view dataset. The results show that YOLACT outperforms Mask-RCNN in terms of accuracy and speed. For an image resolution of 550x550, YOLACT with Resnet-101 was considered the best model in this article since it achieves over 80% precision, recall, specificity, and accuracy in various conditions such as day, night, left and right, and forward-looking angles.

Column generation-based prototype learning for optimizing area under the receiver operating characteristic curve

The traditional classification algorithms focus on the maximization of classification accuracy which might lead to poor performance in practice by forcing classifiers to overfit to the majority class. In order to overcome this issue, various approaches focus on the optimization of alternative loss functions such as the Area Under the Curve (AUC). AUC is a Receiver Operating Characteristics (ROC) metric that has been widely used to measure classification performance, especially when there are class imbalances. In this work, we propose a column generation (CG)-based algorithm called Ranking-CG, which learns a model, similar to the popular Ranking SVM, through approximate maximization of the AUC. Unlike the Ranking SVM, our algorithm utilizes a column generation method that iteratively adds features to control the model complexity effectively working as an internal feature selection procedure. Our experiments show that column generation can be an important tool to prevent overfitting. We extend the Ranking-CG by proposing a prototype generation method, denoted by Ranking-CG Prototype, that constructs reference points by solving a non-linear optimization problem. Based on the extensive experiments conducted on 74 binary classification problems, the Ranking-CG Prototype yields the best average test AUC among all competing methods by using significantly few features than other benchmarks.

Prototype Selection for Multilabel Instance-Based Learning

Reducing the size of the training set, which involves replacing it with a condensed set, is a widely adopted practice to enhance the efficiency of instance-based classifiers while trying to maintain high classification accuracy. This objective can be achieved through the use of data reduction techniques, also known as prototype selection or generation algorithms. Although there are numerous algorithms available in the literature that effectively address single-label classification problems, most of them are not applicable to multilabel data, where an instance can belong to multiple classes. Well-known transformation methods cannot be combined with a data reduction technique due to different reasons. The Condensed Nearest Neighbor rule is a popular parameter-free single-label prototype selection algorithm. The IB2 algorithm is the one-pass variation of the Condensed Nearest Neighbor rule. This paper proposes variations of these algorithms for multilabel data. Through an experimental study conducted on nine distinct datasets as well as statistical tests, we demonstrate that the eight proposed approaches (four for each algorithm) offer significant reduction rates without compromising the classification accuracy.