Intra-class Similarity Research Articles

Rank-in-Rank Loss for Person Re-identification

Person re-identification (re-ID) is commonly investigated as a ranking problem. However, the performance of existing re-ID models drops dramatically, when they encounter extreme positive-negative class imbalance (e.g., very small ratio of positive and negative samples) during training. To alleviate this problem, this article designs a rank-in-rank loss to optimize the distribution of feature embeddings. Specifically, we propose a Differentiable Retrieval-Sort Loss (DRSL) to optimize the re-ID model by ranking each positive sample ahead of the negative samples according to the distance and sorting the positive samples according to the angle (e.g., similarity score). The key idea of the proposed DRSL lies in minimizing the distance between samples of the same category along with the angle between them. Considering that the ranking and sorting operations are non-differentiable and non-convex, the DRSL also performs the optimization of automatic derivation and backpropagation. In addition, the analysis of the proposed DRSL is provided to illustrate that the DRSL not only maintains the inter-class distance distribution but also preserves the intra-class similarity structure in terms of angle constraints. Extensive experimental results indicate that the proposed DRSL can improve the performance of the state-of-the-art re-ID models, thus demonstrating its effectiveness and superiority in the re-ID task.

Unsupervised domain adaptation with Joint Adversarial Variational AutoEncoder

Unsupervised domain adaptation techniques increase the classification performance of tasks from the target domain by utilizing the information in a related source domain. Since the target labeled samples are unavailable, matching similar samples across different domains effectively becomes increasingly hard, which retards the progress in this field. In this paper, we propose a Joint Adversarial Variational AutoEncoder (JVA2E) for unsupervised domain adaptation tasks. JVA2E chooses variational autoencoder as the basic framework to improve the generative ability. Both the marginal and conditional distributions are considered for joint distribution adaptation. The Wasserstein distance is chosen for improving the final performance. Multiple unique classifiers are carefully designed for generating pseudo labels which are utilized to increase intra-class similarity as well as narrow conditional distribution. Experiments are conducted on three publicly available datasets and the final results are compared with some state-of-the-art techniques. It illustrates that our proposed method yields better performances for most tasks against previous domain adaptation methods.

Investigating the impact of vulnerability datasets on deep learning-based vulnerability detectors.

Software vulnerabilities have led to system attacks and data leakage incidents, and software vulnerabilities have gradually attracted attention. Vulnerability detection had become an important research direction. In recent years, Deep Learning (DL)-based methods had been applied to vulnerability detection. The DL-based method does not need to define features manually and achieves low false negatives and false positives. DL-based vulnerability detectors rely on vulnerability datasets. Recent studies found that DL-based vulnerability detectors have different effects on different vulnerability datasets. They also found that the authenticity, imbalance, and repetition rate of vulnerability datasets affect the effectiveness of DL-based vulnerability detectors. However, the existing research only did simple statistics, did not characterize vulnerability datasets, and did not systematically study the impact of vulnerability datasets on DL-based vulnerability detectors. In order to solve the above problems, we propose methods to characterize sample similarity and code features. We use sample granularity, sample similarity, and code features to characterize vulnerability datasets. Then, we analyze the correlation between the characteristics of vulnerability datasets and the results of DL-based vulnerability detectors. Finally, we systematically study the impact of vulnerability datasets on DL-based vulnerability detectors from sample granularity, sample similarity, and code features. We have the following insights for the impact of vulnerability datasets on DL-based vulnerability detectors: (1) Fine-grained samples are conducive to detecting vulnerabilities. (2) Vulnerability datasets with lower inter-class similarity, higher intra-class similarity, and simple structure help detect vulnerabilities in the original test set. (3) Vulnerability datasets with higher inter-class similarity, lower intra-class similarity, and complex structure can better detect vulnerabilities in other datasets.

Hypertuned Deep Convolutional Neural Network for Sign Language Recognition.

Sign language plays a pivotal role in the lives of impaired people having speaking and hearing disabilities. They can convey messages using hand gesture movements. American Sign Language (ASL) recognition is challenging due to the increasing intra-class similarity and high complexity. This paper used a deep convolutional neural network for ASL alphabet recognition to overcome ASL recognition challenges. This paper presents an ASL recognition approach using a deep convolutional neural network. The performance of the DeepCNN model improves with the amount of given data; for this purpose, we applied the data augmentation technique to expand the size of training data from existing data artificially. According to the experiments, the proposed DeepCNN model provides consistent results for the ASL dataset. Experiments prove that the DeepCNN gives a better accuracy gain of 19.84%, 8.37%, 16.31%, 17.17%, 5.86%, and 3.26% as compared to various state-of-the-art approaches.

Fusing Attention Features and Contextual Information for Scene Recognition

Aiming to obtain more discriminative features in scene images and overcome the impacts of intra-class differences and inter-class similarities, the paper proposes a scene recognition method that combines attention and context information. First, we introduce the attention mechanism and build a multi-scale attention model. Discriminative information considers salient objects and regions by means of channel attention and spatial attention. Besides, the central loss function joint supervision strategy is introduced to further reduce the misjudgment of intra-class differences. Second, a model based on multi-level context information is proposed to describe the positional relationship between objects, which can effectively alleviate the influence of the similarity of objects between classes. Finally, the two models are merged to give full play to the compatibility of features, so that the final feature representation not only focuses on the effective discriminant information, but also manifests the relative position relationship between significant objects. Extensive experiments have proved that the method in this paper effectively solves the problem of insufficient feature representation in scene recognition tasks, and improves the accuracy of scene recognition.

MDCADNet: Multi dilated & context aggregated dense network for non-textual components classification in digital documents

Non-Textual images like charts and tables are unlike natural images in various aspects, including high inter-class similarities, low intra-class similarities, substantial textual component proportions, and lower resolutions. This paper proposes a novel Multi-Dilated Context Aggregation based Dense Network (MDCADNet) addressing the multi-resolution and larger receptive field modeling need for the non-textual component classification task. MDCADNet includes a densely connected convolutional network for the feature map computation as front-end with a multi-dilated Backend Context Module (BCM). The proposed BCM generates multi-scale features and provides a systematic context aggregation of both low and high-level feature maps through its densely connected layers. Additionally, the controlled multi-dilation scheme offers a more extensive scale range for better prediction performance. A thorough quantitative evaluation has been performed on seven benchmark datasets for demonstrating the generalization capability of MDCADNet. Experimental results show MDCADNet performs consistently better than the state-of-the-art models across all datasets.

Automatic segmentation of head and neck primary tumors on MRI using a multi-view CNN

Background Accurate segmentation of head and neck squamous cell cancer (HNSCC) is important for radiotherapy treatment planning. Manual segmentation of these tumors is time-consuming and vulnerable to inconsistencies between experts, especially in the complex head and neck region. The aim of this study is to introduce and evaluate an automatic segmentation pipeline for HNSCC using a multi-view CNN (MV-CNN).MethodsThe dataset included 220 patients with primary HNSCC and availability of T1-weighted, STIR and optionally contrast-enhanced T1-weighted MR images together with a manual reference segmentation of the primary tumor by an expert. A T1-weighted standard space of the head and neck region was created to register all MRI sequences to. An MV-CNN was trained with these three MRI sequences and evaluated in terms of volumetric and spatial performance in a cross-validation by measuring intra-class correlation (ICC) and dice similarity score (DSC), respectively.ResultsThe average manual segmented primary tumor volume was 11.8±6.70 cm3 with a median [IQR] of 13.9 [3.22-15.9] cm3. The tumor volume measured by MV-CNN was 22.8±21.1 cm3 with a median [IQR] of 16.0 [8.24-31.1] cm3. Compared to the manual segmentations, the MV-CNN scored an average ICC of 0.64±0.06 and a DSC of 0.49±0.19. Improved segmentation performance was observed with increasing primary tumor volume: the smallest tumor volume group (<3 cm3) scored a DSC of 0.26±0.16 and the largest group (>15 cm3) a DSC of 0.63±0.11 (p<0.001). The automated segmentation tended to overestimate compared to the manual reference, both around the actual primary tumor and in false positively classified healthy structures and pathologically enlarged lymph nodes.ConclusionAn automatic segmentation pipeline was evaluated for primary HNSCC on MRI. The MV-CNN produced reasonable segmentation results, especially on large tumors, but overestimation decreased overall performance. In further research, the focus should be on decreasing false positives and make it valuable in treatment planning.

Clustering Countries in the Context of the Pandemic and Underlying Conditions

The COVID-19 pandemic has revealed the state of underlying conditions of countries in terms of health system, sanitary infrastructure, governance, among others. This study aims to classify countries using COVID-19-related variables such as the lethality rate, the contagion growth rate, the stringency index, and underlying conditions of countries directly related to COVID-19 such as access to clean water, hospital beds per 10000 inhabitants, government effectiveness index, population older than 65 years old and economic growth rate. To determine the clusters of a set of countries from all continents (29 from Africa, 35 from Asia, 35 from Europe, 11 from North America, 2 from Oceania and 8 from South America), the k-means partitioning method is used. This approach consists in constructing partitions and evaluate their intra-class and inter-class similarity. Based on the results, three clusters are identified: i. Severely affected countries with high stringency and moderate capacity, ii. Moderately affected countries with moderate stringency and high capacity and iii. Severely affected countries with low stringency but low capacity.

Fine-Grained Feature Enhancement for Object Detection in Remote Sensing Images

Recently, object detection in aerial images has ushered in a new challenge—a new benchmark for fine-grained object recognition in high-resolution remote sensing imagery called FAIR1M has been proposed. Fine-grained categories usually have smaller inter class differences and intra-class similarities, which is more difficult to classify with existing object detectors. To address this problem, we propose two enhanced strategies on the current two-stage object detection algorithm. The first strategy uses attention-based group feature enhancement called group enhance module (GEM). By extending and grouping feature channels, the model can improve the ability to extract various discriminative features. The second strategy is to emphasize the sub-saliency feature learning, avoiding the network only focusing on the most significant part of the feature and ignoring the other parts. Our method is easy to implement and effective, and experiments show that our method can improve the Oriented regions with convolutional neural networks features (R-CNN) by about 1.45 mAP on the FAIR1M benchmark.

Multisensor Fusion and Explicit Semantic Preserving-Based Deep Hashing for Cross-Modal Remote Sensing Image Retrieval

Cross-modal hashing is an important tool for retrieving useful information from very-high-resolution (VHR) optical images and synthetic aperture radar (SAR) images. Dealing with the intermodal discrepancies, including both spatial–spectral and visual semantic aspects, between VHR and SAR images is extremely vital to generate high-quality common hash codes in the Hamming space. However, existing cross-modal hashing methods ignore the spatial–spectral discrepancy when representing VHR and SAR images. Moreover, existing methods employ derived supervised signals, such as pairwise training images, to implicitly guide hashing learning, which fails to effectively deal with the visual semantic discrepancy, i.e., cannot adequately preserve the intraclass similarity and interclass discrimination between VHR and SAR images. To address these drawbacks, this article proposes a multisensor fusion and explicit semantic preserving-based deep Hashing method, termed as MsEspH, which can effectively deal with the discrepancies. Specifically, we design a novel cross-modal hashing network to eliminate the spatial–spectral discrepancies by fusing extra multispectral images (MSIs), which are generated in real time by a generative adversarial network. Then, we propose an explicit semantic preserving-based objective function by analyzing the connection between classification and hash learning. The objective function can preserve the intraclass similarity and interclass discrimination with class labels directly. Moreover, we theoretically verify that hash learning and classification can be unified into a learning framework under certain conditions. To evaluate our method, we construct and release a large-scale VHR-SAR image dataset. Extensive experiments on the dataset demonstrate that our method outperforms various state-of-the-art cross-modal hashing methods.

Deep Classified Autoencoder for Lithofacies Identification

Lithofacies classification is an indispensable procedure in well logging and seismic data interpretation. We propose a novel deep classified autoencoder learning approach to identify lithofacies for high-dimensional data and complex problems. Deep autoencoder (DAE) is an unsupervised learning method via layerwise pretraining multiple autoencoders. It can learn deep data features automatically and reconstruct the original data with a small error. Introducing sparse constraint (i.e., sparse autoencoder) potentiates the learning ability of autoencoder. On this foundation, additional regularization terms constructed by labeled samples are considered in the new DAE approach in order to boost the performance. The new method can adaptively preserve the most significant input features and remove insensitive properties to decrease computational complexity. At the same time, we embed the class information into the loss function of autoencoder to measure intraclass similarity and improve the classification accuracy. Several experiments on well data and seismic data show that the proposed method achieves promising results. Compared with the traditional deep autoencoder (DAE), the proposed method is more competitive in terms of classification accuracy and robustness.

Divergent Angular Representation for Open Set Image Recognition.

Open set recognition (OSR) models need not only discriminate between known classes but also detect unknown class samples unavailable during training. One promising approach is to learn discriminative representations over known classes with strong intra-class similarity and inter-class discrepancy. Then, the powerful class discrimination learned from the known classes can be extended to known and unknown classes. Without appropriate regularization, however, the model may learn representations trivially, collapsing unknown class representations to the known class ones. To resolve this problem, we propose Divergent Angular Representation (DivAR) based on two approaches. Firstly, DivAR maximizes its representational discrimination between known classes via a highly discriminative loss. Secondly, to ensure separation between known and unknown classes in the representation space, DivAR boosts the directional variation of representations over global samples. In addition, self-supervision is leveraged to improve the representation's robustness and extend DivAR to one-class classification. Moreover, unlike other OSR methods that require an extra machinery for inference, DivAR learns and infers in a single module. Extensive experiments on generic image datasets demonstrate the plausibility and effectiveness of DivAR for both OSR and One-Class Classification (OCC) problems.

Mixed Loss Graph Attention Network for Few-Shot SAR Target Classification

Restricted by the observation condition, synthetic aperture radar (SAR) automatic target classification based on deep learning usually suffers from insufficient training samples. To tackle this problem, a novel few-shot learning (FSL) framework for SAR target classification, i.e., the mixed loss graph attention network (MGA-Net), is proposed. The classification procedure of the MGA-Net consists of three main stages. In the first stage, the task set is expanded by the data augmentation module to increase diversity. In the second stage, the embedding network is designed to map samples to the embedding space with strong intra-class similarity and inter-class divergence. In the third stage, the multilayer graph attention network (GAT) is constructed and updated according to a novel mixed loss to obtain the classification result. In particular, the data augmentation module alleviates the desire of training samples under large model capacity and enhances the robustness to noise and viewing angle variation; the multilayer GAT accurately captures relations between samples by the attention mechanism; and the mixed loss increases the inter-class separability and accelerates convergence. Experimental results under various few-shot observation settings of the MSTAR and the OpenSARShip benchmark datasets demonstrate that the MGA-Net obtains higher accuracy than typical FSL methods and exhibits robustness to large depression angle variation.

Attention Regularized Laplace Graph for Domain Adaptation.

In leveraging manifold learning in domain adaptation (DA), graph embedding-based DA methods have shown their effectiveness in preserving data manifold through the Laplace graph. However, current graph embedding DA methods suffer from two issues: 1). they are only concerned with preservation of the underlying data structures in the embedding and ignore sub-domain adaptation, which requires taking into account intra-class similarity and inter-class dissimilarity, thereby leading to negative transfer; 2). manifold learning is proposed across different feature/label spaces separately, thereby hindering unified comprehensive manifold learning. In this paper, starting from our previous DGA-DA, we propose a novel DA method, namely A ttention R egularized Laplace G raph-based D omain A daptation (ARG-DA), to remedy the aforementioned issues. Specifically, by weighting the importance across different sub-domain adaptation tasks, we propose the A ttention R egularized Laplace Graph for class aware DA, thereby generating the attention regularized DA. Furthermore, using a specifically designed FEEL strategy, our approach dynamically unifies alignment of the manifold structures across different feature/label spaces, thus leading to comprehensive manifold learning. Comprehensive experiments are carried out to verify the effectiveness of the proposed DA method, which consistently outperforms the state of the art DA methods on 7 standard DA benchmarks, i.e., 37 cross-domain image classification tasks including object, face, and digit images. An in-depth analysis of the proposed DA method is also discussed, including sensitivity, convergence, and robustness.

Focal Cosine Metric and Adaptive Attention Module for Remote Sensing Scene Classification With Siamese Convolutional Neural Networks

Convolutional neural networks (CNNs) have been widely used in remote sensing (RS) scene classification tasks due to their remarkable feature representation and inference capability. The complexity of RS images not only brings the challenges of high inter-class similarity and large intra-class diversity, but also introduces the problem that category-relevant regions are insufficiently prominent in feature extraction. Siamese CNNs with feature similarity measurement are chosen in some applications to overcome the former issue, but most ignore the randomness of input sample pairs. This makes the Siamese CNNs not focus enough on challenging samples, which limits the training efficiency. We propose the focal cosine metric (FCM) block that combines the cosine similarity metric and the threshold control to achieve sample selection, thereby completing network learning more efficiently. FCM only permits the misclassified focal samples to participate in similarity measurement based on Siamese CNN. It flexibly mitigates the misclassification caused by the high inter-class similarity and large intra-class diversity. Moreover, the adaptive attention (AA) module is designed to stress the pivotal target regions and assist in the similarity measurement of Siamese CNN. This is realized by adaptively assigning high weights to key targets with learnable guided vectors. It enables the model to focus on the details of intra-class similarities or inter-class differences in sample pairs, and thus reduces the difficulty of model optimization. Encouraging experimental results on three public data sets demonstrate the effectiveness of the novel Siamese CNN-based method with FCM and AA and show its superiority compared to other state-of-the-art scene classification methods.

Few-Shot Segmentation of Remote Sensing Images Using Deep Metric Learning

Current convolutional neural network (CNN)-based methods for remote sensing image segmentation require a large number of densely annotated images for model training and have limited generalization abilities for unseen object categories. In this letter, we propose a novel few-shot learning-based method for the semantic segmentation of remote sensing images. Our method can perform semantic labeling for unseen object categories with only a few annotated samples. More specifically, our model starts by using a deep CNN to extract high-level semantic features. The prototype representation of each class is then generated by using a masked average pooling on the feature embeddings of the support images with ground truth masks. Finally, our model performs semantic labeling over the query images by matching the feature embedding of each pixel to its nearest prototypes in the embedding space. Our model is optimized with a nonparametric metric learning-based loss function to maximize the intra-class similarity of learned prototypes while minimizing the inter-class similarity. Experiments on International Society for Photogrammetry and Remote Sensing (ISPRS) 2-D semantic labeling dataset demonstrate satisfying in-domain and cross-domain transferring abilities of our model.

FSL-EGNN: Edge-Labeling Graph Neural Network for Hyperspectral Image Few-Shot Classification

The existing hyperspectral image (HSI) classification encounters the obstacle of improving the classification accuracy with limited labeled samples. In this context, as a typical implementation of meta-learning, few-shot learning (FSL) makes the model learn by episodic training on source HSI, which has achieved significant improvements in small sample classification of target HSI. However, the existing FSL methods lack explicit consideration and exploration of the association between pixels, especially the intraclass association and interclass association between pixels in the support set and query set. To mitigate these issues, an FSL method based on edge-labeling graph neural network (FSL-EGNN) is proposed for small sample classification of HSI, which is the first attempt to explicitly quantify the associations between pixels by exploiting EGNN in HSI few-shot classification (FSC). Specifically, based on graph construction of HSI, episodic training is performed on the existing source HSI. During training, EGNN is used to predict the edge labels on the graph, thereby explicitly modeling the intraclass similarity and interclass dissimilarity between pixels of HSI. After the trained model is fine-tuned, it can realize FSC on the unseen target HSI. Experiments conducted on three benchmark HSI datasets demonstrate that the proposed FSL-EGNN outperforms the existing state-of-the-art methods with limited labeled samples.

SIL-LAND: Segmentation Incremental Learning in Aerial Imagery via LAbel Number Distribution Consistency

Segmentation incremental learning has received a lot of attention in recent years due to the ability to overcome the problem of catastrophic forgetting. Our study found that differences in label number distribution affect the performance of segmentation incremental learning. Because the labels for pixels of the old category are marked as background when the model is trained on the new tasks, the label number distribution is inconsistent with static learning that is considered to be the upper bound on incremental learning, which hinders the mitigation of the catastrophic forgetting problem. In response to the above problems, we propose an incremental learning method named SIL-LAND, which improves the accuracy by making the label number distribution of our method close to that of static learning. From the perspective of high-level semantic labels, we propose the prototype update mechanism for the problem that non-adaptive representative prototypes ignore the sample diversity of semantic categories in remote sensing images. By compensating for the difference in label number distribution at the feature level, the distance between the prototype and the actual class center is reduced; Aiming at the lack of semantic consistency between feature vectors and prototypes, we propose a similarity measure module to increase the intra-class similarity between the prototype and corresponding feature vectors. From the perspective of one-hot labels, we propose label reconstruction, including foreground screening and background padding to make the number distribution of one-hot labels as close as possible to that of static learning. A series of experimental results demonstrate the effectiveness of our method.

Meta-Learning Based Hyperspectral Target Detection Using Siamese Network

When predicting data for which limited supervised information is available, hyperspectral target detection methods based on deep transfer learning expect that the network will not require considerable retraining to generalize to unfamiliar application contexts. Meta-learning is an effective and practical framework for solving this problem in deep learning. This article proposes a new meta-learning based hyperspectral target detection using Siamese network (MLSN). First, a deep residual convolution feature embedding module is designed to embed spectral vectors into the Euclidean feature space. Then, the triplet loss is used to learn the intraclass similarity and interclass dissimilarity between spectra in embedding feature space by using the known labeled source data on the designed three-channel Siamese network for meta-training. The learned meta-knowledge is updated with the prior target spectrum through a designed two-channel Siamese network to quickly adapt to the new detection task. It should be noted that the parameters and structure of the deep residual convolution embedding modules of each channel in the Siamese network are identical. Finally, the spatial information is combined, and the detection map of the two-channel Siamese network is processed by the guiding image filtering and morphological closing operation, and a final detection result is obtained. Based on the experimental analysis of six real hyperspectral image datasets, the proposed MLSN has shown its excellent comprehensive performance.

A Deep Learning to Distinguish COVID-19 from Others Pneumonia Cases

A new virus called SARS-CoV-2 appeared at the end of space 2019 in Wuhan, China. This virus immediately spread throughout the world due to its highly contagious nature. Moreover, SARS-CoV-2 has changed the way of our life and has caused a huge economic and public health disaster. Therefore, it is urgent to identify positive cases as soon as possible and treat them as isolated. Automatic detection of viruses using computer vision and machine learning will be a valuable contribution to detecting and limiting the spread of this epidemic. The delay introduction of X-ray technology as diagnostic tool limited our ability to distinguish COVID-19 from other pneumonia cases. These images can be feed into a machine learning system that can be trained to detect lung infections, which can arise in acute pneumonia. However, some efforts attempt to detect SARS-CoV-2 using binary data and such combinations, which may leading to confusion. Indeed, in this case, the classifier’s training minimizes the intra-class similarity between pneumonia in COVID-19 and typical pneumonia. Due to the fact that in addition to SARS-CoV-2 pneumonia, there are two other common types: viral and bacterial pneumonia. They all appear in similar shapes, thus making them difficult to distinguish. This research proposed a deep multi-layered classification of pneumonia based on a neural network using chest X-ray images. The experiment was conducted using the combination of two images from an open-source dataset collected by Cohen GP and Kagel. The data consists of 4 categories: normal pneumonia, bacterial pneumonia, viral pneumonia, and COVID19, for 2,433 images. We considered two architectures: Xception and ResNet50. In addition, a comparison was made between the two models. The pre-trained Xception model in 20 epochs provided a classification accuracy of 86%.