Inter-class Relationships Research Articles

Teacher-Student Prototype Enhancement Network for a Few-Shot Remote Sensing Scene Classification

Few-shot remote sensing scene classification identifies new classes from limited labeled samples where the great challenges are intraclass diversity, interclass similarity, and limited supervision. To alleviate these problems, a teacher-student prototype enhancement network is proposed for a few-shot remote sensing scene classification. Instead of introducing an attentional mechanism in mainstream studies, a prototype enhancement module is recommended to adaptively select high-confidence query samples, which can enhance the support prototype representations to emphasize intraclass and interclass relationships. The construction of a few-shot teacher model generates more discriminative predictive representations with inputs from many labeled samples, thus providing a strong supervisory signal to the student model and encouraging the network to achieve accurate classification with a limited number of labeled samples. Extensive experiments of four public datasets, including NWPU-remote sens ing image scene classification (NWPU-RESISC45), aerial image dataset (AID), UC Merced, and WHU-RS19, demonstrate that this method achieves superior competitive performance than the state-of-the-art methods on five-way, one-shot, and five-shot classifications.

Emotion knowledge-based fine-grained facial expression recognition

Existing facial expression recognition (FER) techniques rely primarily on seven coarse-grained emotions as emotional labels, which are insufficient to cover the subtle changes in human emotions in the real world. We use 135 fine-grained emotions as emotional benchmarks to address the problem of highly semantically similar fine-grained emotion recognition. In this work, we propose a robust emotion knowledge-based fine-grained (EK-FG) emotion recognition network that captures inter-class relationships and discriminative representations of fine-grained emotions through two prior-based losses: coarse-grained hierarchical loss and fine-grained semantic loss. Specifically, the coarse-grained hierarchical loss obtains a structured semantic representation of fine-grained emotions based on prior knowledge, and captures inter-class relationships through effective category-level push-pull to obtain discriminative representations. The fine-grained semantic loss provides more accurate measurement information for semantic features based on prior knowledge, and enhances the model’s discriminative ability for subtle facial expression differences through regression constraints. Extensive experimental results on the Emo135 dataset demonstrate that EK-FG can effectively overcome the class ambiguity of fine-grained emotion.

Curriculum reform and stereotype threat in medical education: Implications for student well-being.

Altering the curriculum of a program can have negative repercussions for the student experience, including peer mentorships and interclass relationships. This study investigated the effect of curriculum reform on students' emotional and social well-being in a predoctoral dental program. We explored if any of these consequences could be related to stereotype threat. We utilized a quasi-experimental design with two different treatments, New Curriculum Treatment (New-CT, n=44) and Past Curriculum Treatment (Past-CT, n=43). Quantitative data were collected through surveys to assess students' perceptions of curriculum changes and their impacts on anxiety, confidence, and clinical performance. Qualitative data were gathered via semi-structured interviews to explore personal experiences of stereotype threat and its implications on peer relationships and mentorship dynamics. The findings suggest significant effects of curriculum changes on interpersonal relationships. Past-CT viewed New-CT as overconfident, while New-CT felt heightened performance pressure. Thematic and interview analyses revealed deep-rooted tensions, with New-CT feeling mistrusted and Past-CT resenting New-CT's perceived accelerated competence. Stereotype threat was identified as a key factor worsening these inter-group tensions and affecting clinical performance and relationships. Curriculum changes in dental education can significantly affect students' well-being, with stereotype threat playing a critical role in these dynamics. When making changes to the structure, sequencing, or content of a program, administrators need to be aware of the potential ramifications these changes could have on students' relationships with their peers.

A General Framework for Comparing Embedding Visualizations Across Class-Label Hierarchies.

Projecting high-dimensional vectors into two dimensions for visualization, known as embedding visualization, facilitates perceptual reasoning and interpretation. Comparing multiple embedding visualizations drives decision-making in many domains, but traditional comparison methods are limited by a reliance on direct point correspondences. This requirement precludes comparisons without point correspondences, such as two different datasets of annotated images, and fails to capture meaningful higher-level relationships among point groups. To address these shortcomings, we propose a general framework for comparing embedding visualizations based on shared class labels rather than individual points. Our approach partitions points into regions corresponding to three key class concepts-confusion, neighborhood, and relative size-to characterize intra- and inter-class relationships. Informed by a preliminary user study, we implemented our framework using perceptual neighborhood graphs to defne these regions and introduced metrics to quantify each concept. We demonstrate the generality of our framework with usage scenarios from machine learning and single-cell biology, highlighting our metrics' ability to draw insightful comparisons across label hierarchies. To assess the effectiveness of our approach, we conducted an evaluation study with fve machine learning researchers and six single-cell biologists using an interactive and scalable prototype built with Python, JavaScript, and Rust. Our metrics enable more structured comparisons through visual guidance and increased participants' confdence in their fndings.

The Power of Diversity: Class, Networks and Attitudes Towards Inequality

We examine how inter-class relationships shape attitudes towards inequality. Our theoretical frame defines attitudes towards inequality as ethical dispositions and hypothesises that class-based network diversity may favour the development of what has been termed a ‘reflexive habitus’ that likely pushes for more egalitarian dispositions. The focus of our enquiry is on Chile – one of the most unequal countries in the world. We use large-scale representative survey data collected in 2016 (N = 2927). Results of our regression analyses indicate that class diversity in social networks strongly increases awareness of inequality and more egalitarian preferences while reducing belief in meritocracy and the perception that the current distribution is fair. Additional analyses suggest that network diversity influences views on inequality to a similar degree among individuals from different social classes, except for meritocratic beliefs, where network diversity especially affects the upper class. We conclude by discussing the importance of reducing socio-economic segregation.

Cross-View Gait Recognition Method Based on Multi-Teacher Joint Knowledge Distillation.

Aiming at challenges such as the high complexity of the network model, the large number of parameters, and the slow speed of training and testing in cross-view gait recognition, this paper proposes a solution: Multi-teacher Joint Knowledge Distillation (MJKD). The algorithm employs multiple complex teacher models to train gait images from a single view, extracting inter-class relationships that are then weighted and integrated into the set of inter-class relationships. These relationships guide the training of a lightweight student model, improving its gait feature extraction capability and recognition accuracy. To validate the effectiveness of the proposed Multi-teacher Joint Knowledge Distillation (MJKD), the paper performs experiments on the CASIA_B dataset using the ResNet network as the benchmark. The experimental results show that the student model trained by Multi-teacher Joint Knowledge Distillation (MJKD) achieves 98.24% recognition accuracy while significantly reducing the number of parameters and computational cost.

Adaptive class augmented prototype network for few-shot relation extraction

Relation extraction is one of the most essential tasks of knowledge construction, but it depends on a large amount of annotated data corpus. Few-shot relation extraction is proposed as a new paradigm, which is designed to learn new relationships between entities with merely a small number of annotated instances, effectively mitigating the cost of large-scale annotation and long-tail problems. To generalize to novel classes not included in the training set, existing approaches mainly focus on tuning pre-trained language models with relation instructions and developing class prototypes based on metric learning to extract relations. However, the learned representations are extremely sensitive to discrepancies in intra-class and inter-class relationships and hard to adaptively classify the relations due to biased class features and spurious correlations, such as similar relation classes having closer inter-class prototype representation. In this paper, we introduce an adaptive class augmented prototype network with instance-level and representation-level augmented mechanisms to strengthen the representation space. Specifically, we design the adaptive class augmentation mechanism to expand the representation of classes in instance-level augmentation, and class augmented representation learning with Bernoulli perturbation context attention to enhance the representation of class features in representation-level augmentation and explore adaptive debiased contrastive learning to train the model. Experimental results have been demonstrated on FewRel and NYT-25 under various few-shot settings, and the proposed model has improved accuracy and generalization, especially for cross-domain and different hard tasks.

Metric Learning as a Service With Covariance Embedding

Metric learning as a service (MLaaS) represents one of the main learning streams to handle complex datasets in service computing research communities and industries. A common approach for dealing with high-dimensional and complex datasets is employing a feature embedding algorithm to compress data through dimension reduction while optimizing intra-class distance. To create generalizable MLaaS for high-performance artificial intelligence applications with high-dimensional Big Data, a robust and meaningful embedding space representation by efficiently optimizing both intra-class and inter-class relationships is required. We developed a novel MLaaS methodology that incorporates covariance to signify the direction of the linear relationship between data points in an embedding space. Our covariance-based feature embedding architecture introduces three different yet complementary mapping functions: inner-class mapping, intra-class with semi-inter-class mapping, and intra- and inter-class mapping. Unlike conventional metric learning, our covariance-embedding-enhanced approach is more expressive and explainable for computing similar or dissimilar measures and can capture positive, negative, or neutral relationships. Our MLaaS framework ensures efficient, composable, and extensible metric learning by supporting the selection of dimension reduction and data compression methods. Experiments conducted using various benchmark datasets demonstrate that the proposed model can obtain higher-quality, more separable, and more expressive embedding representations than existing models.

Hybrid Contrastive Learning of Tri-Modal Representation for Multimodal Sentiment Analysis

The wide application of smart devices enables the availability of multimodal data, which can be utilized in many tasks. In the field of multimodal sentiment analysis, most previous works focus on exploring intra- and inter-modal interactions. However, training a network with cross-modal information (language, audio and visual) is still challenging due to the modality gap. Besides, while learning dynamics within each sample draws great attention, the learning of inter-sample and inter-class relationships is neglected. Moreover, the size of datasets limits the generalization ability of the models. To address the afore-mentioned issues, we propose a novel framework HyCon for hybrid contrastive learning of tri-modal representation. Specifically, we simultaneously perform intra-/inter-modal contrastive learning and semi-contrastive learning, with which the model can fully explore cross-modal interactions, learn inter-sample and inter-class relationships, and reduce the modality gap. Besides, refinement term and modality margin are introduced to enable a better learning of unimodal pairs. Moreover, we devise pair selection mechanism to identify and assign weights to the informative negative and positive pairs. HyCon can naturally generate many training pairs for better generalization and reduce the negative effect of limited datasets. Extensive experiments demonstrate that our method outperforms baselines on multimodal sentiment analysis and emotion recognition.

Unsupervised domain adaptation via reliable pseudolabeling based memory module and dynamic distance threshold learning

An important challenge in unsupervised domain adaptation (UDA) is how to capture domain-specific discriminative information in the target domain data. For the labels in the target domain are unavailable, some methods uncover the intrinsic discriminative information by clustering the target domain features in the embedding space. However, most of existing works only consider filtering reliable target domain samples by setting a fixed distance threshold, ignoring the potential intraclass and interclass relationships and cannot set appropriate distance thresholds for different transfer tasks and different classes. To this end, we propose a new pseudolabeling framework named the dynamic distance threshold learning framework (DDT) for UDA. In our framework, we construct distance memory module to encode intraclass and interclass distribution information, and select reliable target domain samples satisfying intraclass compactness and interclass independence via dynamic intraclass and interclass distance thresholds. Specifically, we define class centroid memory modules, intraclass and interclass distance memory modules in the embedding space. Then, we retrieve the intraclass compactness distance threshold and interclass independence distance threshold from the distance memory based on the class and classpair information of target domain sample. Finally, we use reliable target domain samples to update the class centroid and the distance memory module, so that the distance memory module is dynamically updated following the model training. Benefiting from DDT, the UDA model can be updated with reliable target domain samples for training, and the joint distribution can be further aligned at the category level. As a lightweight technique, DDT constructs class center and distance memory modules to transfer the intraclass and interclass distribution relationships from the source domain to the target domain, and sets dynamic intraclass and interclass distances for different classes in different transfer tasks. Without any additional training parameters, the framework can be easily plugged into various domain adaptation paradigms. Experiments on benchmark datasets show that our method can filter the noise in pseudolabels and confers a competitive advantage compared with existing state-of-the-art domain adaptation methods.

Underwater acoustic target recognition based on smoothness-inducing regularization and spectrogram-based data augmentation

Underwater acoustic target recognition is a challenging task owing to the intricate underwater environments and limited data availability. Insufficient data can hinder the ability of recognition systems to support complex modeling, thus impeding their advancement. To improve the generalization capacity of recognition models, techniques such as data augmentation have been employed to simulate underwater signals and diversify data distribution. However, the complexity of underwater environments can cause the simulated signals to deviate from real scenarios, resulting in biased models that are misguided by non-true data. In this study, we propose two strategies to enhance the generalization ability of models in the case of limited data while avoiding the risk of performance degradation. First, as an alternative to traditional data augmentation, we utilize smoothness-inducing regularization, which only incorporates simulated signals in the regularization term. Additionally, we propose a specialized spectrogram-based data augmentation strategy, namely local masking and replicating (LMR), to capture inter-class relationships. Our experiments and visualization analysis demonstrate the superiority of our proposed strategies.

Toward Facial Expression Recognition in the Wild via Noise-Tolerant Network

Facial Expression Recognition (FER) has recently emerged as a crucial area in Human-Computer Interaction (HCI) system for understanding the user’s inner state and intention. However, feature- and label-noise constitute the major challenge for FER in the wild due to the ambiguity of facial expressions worsened by low-quality images. To deal with this problem, in this paper, we propose a simple but effective <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Facial Expression Noise-tolerant Network (FENN)</i> which explores the inter-class correlations for mitigating ambiguity that usually happens between morphologically similar classes. Specifically, FENN leverages a multivariate normal distribution to model such correlations at the final hidden layer of the neural network to suppress the heteroscedastic uncertainty caused by inter-class label noise. Furthermore, the discriminative ability of deep features is weakened by the subtle differences between expressions and the presence of feature noise. FENN utilizes a feature-noise mitigation module to extract compact intra-class feature representations under feature noise while preserving the intrinsic inter-class relationships. We conduct extensive experiments to evaluate the effectiveness of FENN on both original annotated images and synthetic noisy annotated images from RAF-DB, AffectNet, and FERPlus in-the-wild facial expression datasets. The results show that FENN significantly outperforms state-of-the-art FER methods.

2DHeadPose: A simple and effective annotation method for the head pose in RGB images and its dataset

Head pose estimation is one of the essential tasks in computer vision, which predicts the Euler angles of the head in an image. In recent years, CNN-based methods for head pose estimation have achieved excellent performance. Their training relies on RGB images providing facial landmarks or depth images from RGBD cameras. However, labeling facial landmarks is complex for large angular head poses in RGB images, and RGBD cameras are unsuitable for outdoor scenes. We propose a simple and effective annotation method for the head pose in RGB images. The novelty method uses a 3D virtual human head to simulate the head pose in the RGB image. The Euler angle can be calculated from the change in coordinates of the 3D virtual head. We then create a dataset using our annotation method: 2DHeadPose dataset, which contains a rich set of attributes, dimensions, and angles. Finally, we propose Gaussian label smoothing to suppress annotation noises and reflect inter-class relationships. A baseline approach is established using Gaussian label smoothing. Experiments demonstrate that our annotation method, datasets, and Gaussian label smoothing are very effective. Our baseline approach surpasses most current state-of-the-art methods. The annotation tool, dataset, and source code are publicly available at https://github.com/youngnuaa/2DHeadPose.

GoMIC: Multi-view image clustering via self-supervised contrastive heterogeneous graph co-learning

AbstractGraph learning is being increasingly applied to image clustering to reveal intra-class and inter-class relationships in data. However, existing graph learning-based image clustering focuses on grouping images under a single view, which under-utilises the information provided by the data. To address that, we propose a self-supervised multi-view image clustering technique under contrastive heterogeneous graph learning. Our method computes a heterogeneous affinity graph for multi-view image data. It conducts Local Feature Propagation (LFP) for reasoning over the local neighbourhood of each node and executes an Influence-aware Feature Propagation (IFP) from each node to its influential node for learning the clustering intention. The proposed framework pioneeringly employs two contrastive objectives. The first targets to contrast and fuse multiple views for the overall LFP embedding, and the second maximises the mutual information between LFP and IFP representations. We conduct extensive experiments on the benchmark datasets for the problem, i.e. COIL-20, Caltech7 and CASIA-WebFace. Our evaluation shows that our method outperforms the state-of-the-art methods, including the popular techniques MVGL, MCGC and HeCo.

Semantic guided knowledge graph for large-scale zero-shot learning

Zero-shot learning has received growing attention, which aims to improve generalization to unseen concepts. The key challenge in zero-shot tasks is to precisely model the relationship between seen and unseen classes. Most existing zero-shot learning methods capture inter-class relationships via a shared embedding space, leading to inadequate use of relationships and poor performance. Recently, knowledge graph-based methods have emerged as a new trend of zero-shot learning. These methods use a knowledge graph to accurately model the inter-class relationships. However, the currently dominant method for zero-shot learning directly extracts the fixed connection from off-the-shelf WordNet, which will inherit the inherent noise in WordNet. In this paper, we propose a novel method that adopts class-level semantic information as a guidance to construct a new semantic guided knowledge graph (SG-KG), which can correct the errors in the existing knowledge graph and accurately model the inter-class relationships. Specifically, our method includes two main steps: noise suppression and semantic enhancement. Noise suppression is used to eliminate noise edges in the knowledge graph, and semantic enhancement is used to connect two classes with strong relations. To promote high efficient information propagation among classes, we develop a novel multi-granularity fusion network (MGFN) that integrates discriminative information from multiple GCN branches. Extensive experiments on the large-scale ImageNet-21K dataset and AWA2 dataset demonstrate that our method consistently surpasses existing methods and achieves a new state-of-the-art result.

Iterative Zero-Shot Localization via Semantic-Assisted Location Network

This paper considers zero-shot localization problem where the images used for localization are taken from new locations that are not included in the training dataset. We propose the Semantic-Assisted Location Network (SLN), which considers a new location essentially as a new combination of certain semantic classes. Moreover, we propose an iterative zero-shot learning method based on Expectation-Maximization (EM) algorithm to deal with the problem that the inter-class relationships of class representations in image embedding space and class embedding space are inconsistent. Experiments show that the proposed iterative zero-shot learning method outperforms start-of-the-art zero-shot localization methods by a large margin.

Supervised Contrastive Representation Embedding Based on Transformer for Few-Shot Classification

Few-shot learning as an emerging problem has attracted intensive attention in the data-limited situations of Computer Vision. Recently, massive works concentrate on exploiting meta learning to improve performance of recognition. However, most of them overlook the intra-class and inter-class relationships within the support set during episodic training, which are significant and important for the tasks of downstream. To tackle this problem, we present an efficient supervised contrastive representation embedding for few-shot classification. Specially, (1) we employ Swin Transformer as the backbone to replace CNN architecture in order to explore the huge potential of transformer-based backbone for the field of few-shot learning. (2) We introduce supervised contrastive loss to meta learning to take good advantage of extremely limited relations for the first time. Combined the classifier based on metric learning, extensive experiments have demonstrated the efficiency of the representation embedding. We conduct experiments and establish competitive results on two widely-used few-shot classification benchmarks: Fewshot-CIFAR 100 (FC-100) and MiniImageNet.

Deep learning based multi-source localization with source splitting and its effectiveness in multi-talker speech recognition

Multi-source localization is an important and challenging technique for multi-talker conversation analysis. This paper proposes a novel supervised learning method using deep neural networks to estimate the direction of arrival (DOA) of all the speakers simultaneously from the audio mixture. At the heart of the proposal is a source splitting mechanism that creates source-specific intermediate representations inside the network. This allows our model to give source-specific posteriors as the output unlike the traditional multi-label classification approach. Existing deep learning methods perform a frame level prediction, whereas our approach performs an utterance level prediction by incorporating temporal selection and averaging inside the network to avoid post-processing. We also experiment with various loss functions and show that a variant of earth mover distance (EMD) is very effective in classifying DOA at a very high resolution by modeling inter-class relationships. In addition to using the prediction error as a metric for evaluating our localization model, we also establish its potency as a frontend with automatic speech recognition (ASR) as the downstream task. We convert the estimated DOAs into a feature suitable for ASR and pass it as an additional input feature to a strong multi-channel and multi-talker speech recognition baseline. This added input feature drastically improves the ASR performance and gives a word error rate (WER) of 6.3% on the evaluation data of our simulated noisy two speaker mixtures, while the baseline which does not use explicit localization input has a WER of 11.5%. We also perform ASR evaluation on real recordings with the overlapped set of the MC-WSJ-AV corpus in addition to simulated mixtures.

Graph-Embedding Balanced Transfer Subspace Learning for Hyperspectral Cross-Scene Classification

Hyperspectralcross-scene classification utilizes the prior knowledge of source scenes with known labels to classify unlabeled target scenes via transfer learning. The existing methods did not properly balance the contribution of marginal and conditional distribution to transfer learning. They did not fully exploit the neighborhood information of intraclass/interclass in the shared transfer subspace. Regarding the two problems, first, by using maximum mean discrepancy and class weights, a direct estimation method was proposed for the balance parameter that adjusted the contribution of the marginal and conditional distribution. It could avoid the inefficiency problem of the traditional indirect estimation method, without introducing extra parameters. Two different strategies (supervised and unsupervised) were developed to comprehensively represent the feature relationship. The difference between source data and the target data was significantly reduced. At the same time, the intraclass and interclass relationships, represented by graph construction in the original high-dimensional space under graph-embedding framework, were maximally maintained. Experimental results on two classical cross-scene data sets (HyRank and Pavia) showed that the proposed method performed more efficiently and accurately in estimating the balance parameter and achieved better cross-scene classification performance by integrating with the graph construction strategies.

Fine-Grained Recognition for Oriented Ship Against Complex Scenes in Optical Remote Sensing Images

Remote sensing ship recognition is widely used in civil and military applications, such as national defense construction, fishery management, and navigation supervision. Unfortunately, limited by the lack of public datasets for fine-grained ship recognition, current studies mainly focus on ship detection or coarse-grained ship recognition, whereas fine-grained recognition task is left out. The main challenges for fine-grained ship recognition include: 1) complex scenes and 2) ship characteristics of arbitrary orientation, dense distribution, and huge scale and appearance variation. Aiming at the challenges above, we propose a novel efficient information reuse network (EIRNet) and establish a public 20-class Dataset for Oriented Ship Recognition (DOSR). In our EIRNet, considering recognition robustness to multiscale ships, a dense feature fusion network (DFF-Net) with two fusion directions is designed to maximize the utilization of multilayer information and reduce information redundancy. Then, the fused feature maps are refined by a dual-mask attention module (DMAM) to improve performance in dense and clutter scenes by enhancing the distinction between ships and suppressing clutter. Furthermore, Mask-RPN improves the efficiency of generating proposals by reusing the attention mask. Finally, we introduce a concept of upper level class to mine interclass relationships, which further improves the recognition accuracy. Extensive experiments demonstrate that our EIRNet achieves state-of-the-art performance on DOSR and another popular public dataset called HRSC2016.