Intra-class Information Research Articles

FedCCL: Federated dual-clustered feature contrast under domain heterogeneity

Federated learning (FL) facilitates a privacy-preserving neural network training paradigm through collaboration between edge clients and a central server. One significant challenge is that the distributed data is not independently and identically distributed (non-IID), typically including both intra-domain and inter-domain heterogeneity. However, recent research is limited to simply using averaged signals as a form of regularization and only focusing on one aspect of these non-IID challenges. Given these limitations, this paper clarifies these two non-IID challenges and attempts to introduce cluster representation to address them from both local and global perspectives. Specifically, we propose a dual-clustered feature contrast-based FL framework with dual focuses. First, we employ clustering on the local representations of each client, aiming to capture intra-class information based on these local clusters at a high level of granularity. Then, we facilitate cross-client knowledge sharing by pulling the local representation closer to clusters shared by clients with similar semantics while pushing them away from clusters with dissimilar semantics. Second, since the sizes of local clusters belonging to the same class may differ for each client, we further utilize clustering on the global side and conduct averaging to create a consistent global signal for guiding each local training in a contrastive manner. Experimental results on multiple datasets demonstrate that our proposal achieves comparable or superior performance gain under intra-domain and inter-domain heterogeneity.

Improving deep metric learning via self-distillation and online batch diffusion process

Currently, deep metric learning (DML) methods typically rely on class labels to keep positive samples as closely clustered as possible, while distancing them from negative samples. However, this approach tends to disregard some vital information inherent in the data, such as intra-class information variation, which can hinder the generalization of the trained model. To address this issue, we propose an online batch diffusion-based self-distillation method (OBD-SD), which consists of a progressive self-distillation (PSD) technique and an online batch diffusion process (OBDP). Specifically, PSD is a simple yet effective self-distillation technique that is beneficial for the diversity of embedding representations. OBDP uses the diffusion process to reveal the intrinsic relationships among samples in the mini-batch and produce better soft distance targets. Our approach combines PSD with OBDP, which can extract richer relational information among samples by capturing the local geometric structure of manifolds in each mini-batch. OBD-SD is a highly flexible framework that can be integrated into state-of-the-art DML methods. Our experimental results on CUB200, CARS196, and Stanford Online Products datasets demonstrate that our OBD-SD consistently enhances the performance of existing DML methods, without requiring additional training time, resulting in competitive results.

Feature fusion network based on few-shot fine-grained classification.

The objective of few-shot fine-grained learning is to identify subclasses within a primary class using a limited number of labeled samples. However, many current methodologies rely on the metric of singular feature, which is either global or local. In fine-grained image classification tasks, where the inter-class distance is small and the intra-class distance is big, relying on a singular similarity measurement can lead to the omission of either inter-class or intra-class information. We delve into inter-class information through global measures and tap into intra-class information via local measures. In this study, we introduce the Feature Fusion Similarity Network (FFSNet). This model employs global measures to accentuate the differences between classes, while utilizing local measures to consolidate intra-class data. Such an approach enables the model to learn features characterized by enlarge inter-class distances and reduce intra-class distances, even with a limited dataset of fine-grained images. Consequently, this greatly enhances the model's generalization capabilities. Our experimental results demonstrated that the proposed paradigm stands its ground against state-of-the-art models across multiple established fine-grained image benchmark datasets.

From Instance to Metric Calibration: A Unified Framework for Open-World Few-Shot Learning.

Robust few-shot learning (RFSL), which aims to address noisy labels in few-shot learning, has recently gained considerable attention. Existing RFSL methods are based on the assumption that the noise comes from known classes (in-domain), which is inconsistent with many real-world scenarios where the noise does not belong to any known classes (out-of-domain). We refer to this more complex scenario as open-world few-shot learning (OFSL), where in-domain and out-of-domain noise simultaneously exists in few-shot datasets. To address the challenging problem, we propose a unified framework to implement comprehensive calibration from instance to metric. Specifically, we design a dual-networks structure composed of a contrastive network and a meta network to respectively extract feature-related intra-class information and enlarged inter-class variations. For instance-wise calibration, we present a novel prototype modification strategy to aggregate prototypes with intra-class and inter-class instance reweighting. For metric-wise calibration, we present a novel metric to implicitly scale the per-class prediction by fusing two spatial metrics respectively constructed by the two networks. In this way, the impact of noise in OFSL can be effectively mitigated from both feature space and label space. Extensive experiments on various OFSL settings demonstrate the robustness and superiority of our method. Our source codes is available at https://github.com/anyuexuan/IDEAL.

Local Correlation Ensemble with GCN Based on Attention Features for Cross-domain Person Re-ID

Person re-identification (Re-ID) has achieved great success in single-domain. However, it remains a challenging task to adapt a Re-ID model trained on one dataset to another one. Unsupervised domain adaption (UDA) was proposed to migrate a model from a labeled source domain to an unlabeled target domain. The main difference in the cross-domain is different background styles. Although the style transfer approach effectively reduces inter-domain gaps, it ignores the reduction of intra-class differences. Clustering-based pipelines maintain state-of-the-art performance for UDA by learning domain-independent features; however, most existing models do not sufficiently exploit the rich unlabeled samples in target domains due to unsatisfactory clustering. Thus, we propose a novel local correlation ensemble model that focuses on the diversity of intra-class information and the reliability of class centers. Specifically, a pedestrian attention module is proposed to enable the encoder to pay more attention to the person’s features to relieve interference caused by the shared background style. Furthermore, we propose a priority-distance graph convolutional network (PDGCN) module that employs a graph convolutional network network to predict the priority of a node as a class center and then calculates the distance between nodes with high priority values to screen out the class center nodes. Finally, the encoder features (local) and PDGCN features (context-aware) are combined to perform person Re-ID. The results of experiments on the large-scale public Re-ID datasets verified the effectiveness of the proposed method.

Supervised Regularized Multidimensional Scaling Using Weighted Stress Measure

Supervised Regularized Multidimensional Scaling (SRMDS), a non-linear variant of classical Multi-Dimensional Scaling (cMDS) is proposed recently which involves Radial basis function. The method is focused on the effective selection of centers of the radial basis functions in transforming data from a higher dimensional space to a lower dimension. The transformation matrix is determined by minimizing stress. Weights of components of the stress that are of great importance for classification of data got less focus in Supervised Regularized Multidimensional Scaling. In this article, we have investigated several forms of non-linear functions which may be used as weights of the stress measure. A new form of Z-shape weight function dependent on intraclass information of the dataset is introduced which prefers to preserve global structure of the dataset. The efficiency of the proposed approach is illustrated on several benchmarking datasets which shows that the weighted Supervised Regularized Multidimensional Scaling (WSRMDS) outperforms the base method and some other state of the art approaches such as Linear Discriminant Analysis (LDA), and Supervised Principal Component Analysis (SPCA). Observing the finding of this research, among different weight functions, Z-shape weight function is recommended to use since it works better then any other weight functions for most of the data sets.

Multi-view learning with privileged weighted twin support vector machine

By using inter-class and intra-class K-Nearest Neighbors (KNNs), weighted twin support vector machine (WLTSVM) mines as much potential similarity information in samples as possible to improve the common short-coming of nonparallel hyperplane classifiers. Multi-view learning (MVL) has a lot of potential due to the multi-modal datasets that are becoming available. In this paper, we propose a new multi-view learning with privileged weighted twin support vector machine (MPWTSVM). It not only inherits the advantages of WLTSVM but also has its characteristics. Firstly, it enhances generalization ability by mining intra-class information from the same perspective. Secondly, it reduces the redundant constraints with the help of inter-class information, thus improving the running speed. Most importantly, it can follow both the consensus and the complementary principles simultaneously. The consensus principle is realized by minimizing the coupling items of different views in the original objective function. The complementary principle is achieved by establishing privileged information paradigms and MVL. Compared with state-of-the-art MVL methods: SVM-2K, MVTSVM, MCPK, PSVM-2V, MVRDTSVM and MVTHSVM-2C, our model has better accuracy and classification efficiency. Experimental results on numerous datasets prove the effectiveness of the proposed algorithm.

Identification of Epileptic EEG Signals Through TSK Transfer Learning Fuzzy System.

We propose a new model to identify epilepsy EEG signals. Some existing intelligent recognition technologies require that the training set and test set have the same distribution when recognizing EEG signals, some only consider reducing the marginal distribution distance of the data while ignoring the intra-class information of data, and some lack of interpretability. To address these deficiencies, we construct a TSK transfer learning fuzzy system (TSK-TL) based on the easy-to-interpret TSK fuzzy system the transfer learning method. The proposed model is interpretable. By using the information contained in the source domain and target domains more effectively, the requirements for data distribution are further relaxed. It realizes the identification of epilepsy EEG signals in data drift scene. The experimental results show that compared with the existing algorithms, TSK-TL has better performance in EEG recognition of epilepsy.

Graph-Based Dimensionality Reduction for Hyperspectral Imagery: A Review

Hyperspectral image (HSI) contains a wealth of spectral information, which makes fine classification of ground objects possible. In the meanwhile, overly redundant information in HSI brings many challenges. Specifically, the lack of training samples and the high computational cost are the inevitable obstacles in the design of classifier. In order to solve these problems, dimensionality reduction is usually adopted. Recently, graph-based dimensionality reduction has become a hot topic. In this paper, the graph-based methods for HSI dimensionality reduction are summarized from the following aspects. 1) The traditional graph-based methods employ Euclidean distance to explore the local information of samples in spectral feature space. 2) The dimensionality-reduction methods based on sparse or collaborative representation regard the sparse or collaborative coefficients as graph weights to effectively reduce reconstruction errors and represent most important information of HSI in the dictionary. 3) Improved methods based on sparse or collaborative graph have made great progress by considering global low-rank information, local intra-class information and spatial information. In order to compare typical techniques, three real HSI datasets were used to carry out relevant experiments, and then the experimental results were analysed and discussed. Finally, the future development of this research field is prospected.

Sentiment analysis of Japanese text and vocabulary learning based on natural language processing and SVM

The traditional Japanese text sentiment classification mainly relies on some domain experts to classify text sentiment according to their knowledge, which is time-consuming and laborious. In order to improve the effect of text sentiment classification, this study combines intelligent machine algorithms to study Japanese text sentiment classification, which provides a theoretical reference for subsequent natural language research. Moreover, this study combines the TF-IDF algorithm with SVM to construct a Japanese text sentiment classification model and proposes a chi-square statistic that combines word frequency factor, inter-class concentration coefficient, and correction coefficient. Aiming at the problem of traditional feature weighting method TFIDF ignoring the distribution of feature items inter-class and intra-class when calculating the weight of feature items, chi-square statistics is introduced to improve the effect that TFIDF ignores the distribution of feature items in inter-class, and intra-class information entropy is introduced to improve the effect that TFIDF ignores the distribution of feature items in the intra-class. In addition, this study designs a control experiment to analyze the performance of the model proposed in this study. Through various comparative analysis, it shows that the research model has good comprehensive performance, meets the needs of sentiment classification system, and can provide theoretical reference for related research.

Joint Feature Disentanglement and Hallucination for Few-Shot Image Classification.

Few-shot learning (FSL) refers to the learning task that generalizes from base to novel concepts with only few examples observed during training. One intuitive FSL approach is to hallucinate additional training samples for novel categories. While this is typically done by learning from a disjoint set of base categories with sufficient amount of training data, most existing works did not fully exploit the intra-class information from base categories, and thus there is no guarantee that the hallucinated data would represent the class of interest accordingly. In this paper, we propose Feature Disentanglement and Hallucination Network (FDH-Net), which jointly performs feature disentanglement and hallucination for FSL purposes. More specifically, our FDH-Net is able to disentangle input visual data into class-specific and appearance-specific features. With both data recovery and classification constraints, hallucination of image features for novel categories using appearance information extracted from base categories can be achieved. We perform extensive experiments on two fine-grained datasets (CUB and FLO) and two coarse-grained ones (mini-ImageNet and CIFAR-100). The results confirm that our framework performs favorably against state-of-the-art metric-learning and hallucination-based FSL models.

ArcVein-Arccosine Center Loss for Finger Vein Verification

This article proposes a new loss function termed arccosine center loss, which can learn interclass and intraclass information simultaneously, to improve the discriminative ability of convolutional neural networks for finger vein verification. Specifically, the purpose of arccosine center loss is to reduce intraclass distance and increase the interclass distance through network model training. With the combination of softmax loss and arccosine center loss, the proposed network model can extract features with the interclass dispensation and intraclass compactness, which improves the discriminative ability of learned features. Experimental studies have confirmed the effectiveness and efficiency of the proposed method for finger vein verification.

SAR Target Recognition via Joint Sparse and Dense Representation of Monogenic Signal

Synthetic aperture radar (SAR) target recognition under extended operating conditions (EOCs) is a challenging problem due to the complex application environment, especially for insufficient target variations and corrupted SAR images in the training samples. This paper proposes a new strategy to solve these problems for target recognition. The SAR images are firstly characterized by multi-scale components of monogenic signal. The generated monogenic features are decomposed to learn a class dictionary and a shared dictionary, which represent the possible intraclass variations information and the common information, respectively. Moreover, a sparse representation of the class dictionary and a dense representation of the shared dictionary are jointly employed to represent a query sample for classification. The validity of the proposed strategy is demonstrated with multiple comparative experiments on moving and stationary target acquisition and recognition (MSTAR) database.

Class Information-Based Band Selection for Hyperspectral Image Classification

This paper presents a class information (CI)-based band selection (BS) approach to hyperspectral image classification (HSIC). It introduces a new concept from an information theory point of view, CI which can be used to determine an appropriate weight imposed on each class of interest. Specifically, two types of criteria, intraclass information criterion (IC) and interclass IC are derived as CI probabilities to measure CI that can be used to determine the number of training samples required to be selected for each class. With such CI-calculated probabilities, another new concept called class self-information (CSI) is also defined for each class that can be further used to define the class entropy (CE) so that CSI and CE can be used to determine the number of bands required for BS, $n_{\text {BS}}$ . In order to find desired $n_{\text {BS}}$ bands, two types of BS methods based on CSI and CE are custom-designed, called single class signature-constrained BS (SCSC-BS) which utilizes the constrained energy minimization (CEM) to constrain each individual class signature to select bands for a particular class according to its CSI-determined $n_{\text {BS}}$ and a multiple class signatures-constrained BS (MCSC-BS) which takes advantage of linearly constrained minimum variance (LCMV) to constrain all class signatures to select CE-determined $n_{\text {BS}}$ bands for all classes. These SCSC-BS and MCSC-BS selected bands are then used to perform classification and evaluated by CI-weighted classification measures by real image experiments. The results show that HSIC using judiciously selected partial bands as well as CI-weighted measures can improve HSIC with using full bands.

Video Face Recognition Using Siamese Networks With Block-Sparsity Matching

Deep learning models for still-to-video FR typically provide a low level of accuracy because faces captured in unconstrained videos are matched against a reference gallery comprised of a single facial still per individual. For improved robustness to intra-class variations, deep Siamese networks have recently been used for pair-wise face matching. Although these networks can improve state-of-the-art accuracy, the absence of prior knowledge from the target domain means that many images must be collected to account for all possible capture conditions, which is not practical for many real-world surveillance applications. In this paper, we propose the deep SiamSRC network that employs block-sparsity for face matching, while the reference gallery is augmented with a compact set of domain-specific facial images. Prior to deployment, clustering based on row sparsity is performed on unlabelled faces captured in videos from the target domain. Cluster centers discovered in the capture condition space (defined by, e.g., pose, scale and illumination) are used as rendering parameters with an off-the-shelf 3D face model, and a compact set of synthetic faces are thereby generated for each reference still based on representative intra-class information from the target domain. For pair-wise similarity matching with query facial images, the SiamSRC exploits sparse representation-based classification with a block structure. Experimental results obtained with the videos from the Chokepoint and COX-S2V datasets indicate that the proposed SiamSRC network can outperform state-of-the-art methods for still-to-video FR with a single sample per person, with only a moderate increase in computational complexity.

Robust Distracter-Resistive Tracker via Learning a Multi-Component Discriminative Dictionary

Discriminative dictionary learning (DDL) provides an appealing paradigm for appearance modeling in visual tracking. However, most existing DDL-based trackers cannot handle drastic appearance changes, especially for scenarios with background cluster and/or similar object interference. One reason is that they often suffer from the loss of subtle visual information, which is critical to distinguish an object from distracters. In this paper, we explore the use of activations from the convolutional layer of a convolutional neural network to improve the object representation and then propose a robust distracter-resistive tracker via learning a multi-component discriminative dictionary. The proposed method exploits both the intra-class and inter-class visual information to learn shared atoms and the class-specific atoms. By imposing several constraints into the objective function, the learned dictionary is reconstructive, compressive, and discriminative, and thus can better distinguish an object from the background. In addition, our convolutional features have structural information for object localization and balance the discriminative power and semantic information of the object. Tracking is carried out within a Bayesian inference framework where a joint decision measure is used to construct the observation model. To alleviate the drift problem, the reliable tracking results obtained online are accumulated to update the dictionary. Both the qualitative and quantitative results on the CVPR2013 benchmark, the VOT2015 data set, and the SPOT data set demonstrate that our tracker achieves substantially better overall performance against the state-of-the-art approaches.

Weakly supervised segment annotation via expectation kernel density estimation

Since the labelling for the positive images/videos is ambiguous in weakly supervised segment annotation, negative mining-based methods that only use the intra-class information emerge. In these methods, negative instances are utilised to penalise unknown instances for ranking their likelihood of being an object, which can be considered as voting in terms of similarity. However, these methods (i) ignore the information contained in positive bags; (ii) only rank the likelihood but cannot generate an explicit decision function. In this study, the authors propose a voting scheme involving not only the definite negative instances but also the ambiguous positive instances to make use of the extra useful information in the weakly labelled positive bags. In the scheme, each instance votes for its label with a magnitude arising from the similarity, and the ambiguous positive instances are assigned soft labels that are iteratively updated during the voting. It overcomes the limitations of voting using only the negative bags. They also propose an expectation kernel density estimation algorithm to gain further insight into the voting mechanism. Experimental results demonstrate the superiority of the authors’ scheme beyond the baselines.

Discriminant Deep Feature Learning based on joint supervision Loss and Multi-layer Feature Fusion for heterogeneous face recognition

Heterogeneous face recognition (HFR) is still a challenging problem in computer vision community due to large appearance difference between near infrared (NIR) and visible light (VIS) modalities. Recently, breakthroughs have been made for traditional face recognition by applying deep learning on a huge amount of labeled VIS face samples. However, the same deep learning approach cannot be simply applied to HFR task due to large domain difference as well as insufficient pairwise images in different modalities during training. In general, the pooling layer of deep network can play the role of feature reduction, but also lead to the loss of useful face information, resulting in a decrease in the performance of HFR problem. It is important to eliminate modal-related information and retain more facial identity information. In this paper, we propose a novel method called Discriminant Deep Feature Learning Based on Joint Supervision Loss and Multi-layer Feature Fusion (DDFLJM) for HFR task. In most of the available CNNs, the softmax loss function is used as the supervision signal to train the deep model. In order to enhance the discriminative power of the deeply learned features, this paper proposes a new loss function called Scatter Loss (SL), which embeds both inter- and intra-class information for effectively training the deep model. To make full use of the various layers of the deep network, a Dimension Reduction Block (DRB) is designed to effectively extract the auxiliary features on multiple mid-level layers. An orthogonality constraint is introduced to the DRB block to reduce spectrum variations of two different modalities. The proposed SL is applied to multiple layers of network for joint supervision training, which enables multiple layers of the network to obtain discriminative identity features. Moreover, a Modified Gate Two-stream Neural Network (MGTNN) is adopted to fuse multiple-layer features. Extensive experiments are carried out on two challenging NIR-VIS HFR datasets CASIA NIR-VIS 2.0 and Oulu-CASIA NIR-VIS, demonstrating the superiority of the proposed method.

Block kernel nonnegative matrix factorization for face recognition

Nonnegative matrix factorization (NMF) is a linear approach for extracting localized feature of facial image. However, NMF may fail to process the data points that are nonlinearly separable. The kernel extension of NMF, named kernel NMF (KNMF), can model the nonlinear relationship among data points and extract nonlinear features of facial images. KNMF is an unsupervised method, thus it does not utilize the supervision information. Moreover, the extracted features by KNMF are not sparse enough. To overcome these limitations, this paper proposes a supervised KNMF called block kernel NMF (BKNMF). A novel objective function is established by incorporating the intra-class information. The algorithm is derived by making use of the block strategy and kernel theory. Our BKNMF has some merits for face recognition, such as highly sparse features and orthogonal features from different classes. We theoretically analyze the convergence of the proposed BKNMF. Compared with some state-of-the-art methods, our BKNMF achieves superior performance in face recognition.

Feature Learning Using Spatial-Spectral Hypergraph Discriminant Analysis for Hyperspectral Image.

Hyperspectral image (HSI) contains a large number of spatial-spectral information, which will make the traditional classification methods face an enormous challenge to discriminate the types of land-cover. Feature learning is very effective to improve the classification performances. However, the current feature learning approaches are mostly based on a simple intrinsic structure. To represent the complex intrinsic spatial-spectral of HSI, a novel feature learning algorithm, termed spatial-spectral hypergraph discriminant analysis (SSHGDA), has been proposed on the basis of spatial-spectral information, discriminant information, and hypergraph learning. SSHGDA constructs a reconstruction between-class scatter matrix, a weighted within-class scatter matrix, an intraclass spatial-spectral hypergraph, and an interclass spatial-spectral hypergraph to represent the intrinsic properties of HSI. Then, in low-dimensional space, a feature learning model is designed to compact the intraclass information and separate the interclass information. With this model, an optimal projection matrix can be obtained to extract the spatial-spectral features of HSI. SSHGDA can effectively reveal the complex spatial-spectral structures of HSI and enhance the discriminating power of features for land-cover classification. Experimental results on the Indian Pines and PaviaU HSI data sets show that SSHGDA can achieve better classification accuracies in comparison with some state-of-the-art methods.