Inter-class Similarity Research Articles

A deep learning‐basedx‐rayimaging diagnosis system for classification of tuberculosis,COVID‐19, and pneumonia traits using evolutionary algorithm

AbstractTo aid in detection of tuberculosis, researchers have concentrated on developing computer‐aided diagnostic technologies based on x‐ray imaging. Since it generates noninvasive standard‐of‐care data, a chest x‐ray image is one of the most often used diagnostic imaging modalities in computer‐aided solutions. Due to their significant interclass similarities and low intra‐class variation abnormalities, chest x‐ray pictures continue to pose difficulty for proper diagnosis. In this paper, a novel automated framework is proposed for the classification of tuberculosis, COVID‐19, and pneumonia from chest x‐ray images using deep learning and improved optimization technique. Two pre‐trained convolutional neural network models such as EfficientB0 and ResNet50 have been utilized and fine‐tuned based on the additional layers. Both models are trained with fixed hyperparameters on the selected datasets and obtained newly trained models. A novel feature selection technique has been proposed that selects the best features. In the novel version, distance and update position formulation has been modified. The selected features are further fused using a novel technique that is based on the serial and standard deviation threshold function. The experimental process of the proposed framework is conducted on three datasets and obtained an accuracy of 98.2%, 99.0%, and 98.7%, respectively. In addition, a detailed Wilcoxon signed‐rank analysis is conducted and shows the proposed method significance performance. Based on the results, it is concluded that the proposed method accuracy is improved after the fusion process. In addition, the comparison with recent techniques shows the proposed method as more significant in terms of accuracy and precision rate.

Early diagnosis and meta-agnostic model visualization of tuberculosis based on radiography images

Despite being treatable and preventable, tuberculosis (TB) affected one-fourth of the world population in 2019, and it took the lives of 1.4 million people in 2019. It affected 1.2 million children around the world in the same year. As it is an infectious bacterial disease, the early diagnosis of TB prevents further transmission and increases the survival rate of the affected person. One of the standard diagnosis methods is the sputum culture test. Diagnosing and rapid sputum test results usually take one to eight weeks in 24 h. Using posterior-anterior chest radiographs (CXR) facilitates a rapid and more cost-effective early diagnosis of tuberculosis. Due to intraclass variations and interclass similarities in the images, TB prognosis from CXR is difficult. We proposed an early TB diagnosis system (tbXpert) based on deep learning methods. Deep Fused Linear Triangulation (FLT) is considered for CXR images to reconcile intraclass variation and interclass similarities. To improve the robustness of the prognosis approach, deep information must be obtained from the minimal radiation and uneven quality CXR images. The advanced FLT method accurately visualizes the infected region in the CXR without segmentation. Deep fused images are trained by the Deep learning network (DLN) with residual connections. The largest standard database, comprised of 3500 TB CXR images and 3500 normal CXR images, is utilized for training and validating the recommended model. Specificity, sensitivity, Accuracy, and AUC are estimated to determine the performance of the proposed systems. The proposed system demonstrates a maximum testing accuracy of 99.2%, a sensitivity of 98.9%, a specificity of 99.6%, a precision of 99.6%, and an AUC of 99.4%, all of which are pretty high when compared to current state-of-the-art deep learning approaches for the prognosis of tuberculosis. To lessen the radiologist’s time, effort, and reliance on the level of competence of the specialist, the suggested system named tbXpert can be deployed as a computer-aided diagnosis technique for tuberculosis.

Learn by Yourself: A Feature-Augmented Self-Distillation Convolutional Neural Network for Remote Sensing Scene Image Classification

In recent years, with the rapid development of deep learning technology, great progress has been made in remote sensing scene image classification. Compared with natural images, remote sensing scene images are usually more complex, with high inter-class similarity and large intra-class differences, which makes it difficult for commonly used networks to effectively learn the features of remote sensing scene images. In addition, most existing methods adopt hard labels to supervise the network model, which makes the model prone to losing fine-grained information of ground objects. In order to solve these problems, a feature-augmented self-distilled convolutional neural network (FASDNet) is proposed. First, ResNet34 is adopted as the backbone network to extract multi-level features of images. Next, a feature augmentation pyramid module (FAPM) is designed to extract and fuse multi-level feature information. Then, auxiliary branches are constructed to provide additional supervision information. The self-distillation method is utilized between the feature augmentation pyramid module and the backbone network, as well as between the backbone network and auxiliary branches. Finally, the proposed model is jointly supervised using feature distillation loss, logits distillation loss, and cross-entropy loss. A lot of experiments are conducted on four widely used remote sensing scene image datasets, and the experimental results show that the proposed method is superior to some state-ot-the-art classification methods.

Oracle character recognition using unsupervised discriminative consistency network

Ancient history relies on the study of ancient characters. However, real-world scanned oracle characters are difficult to collect and annotate, posing a major obstacle for oracle character recognition (OrCR). Besides, serious abrasion and inter-class similarity also make OrCR more challenging. In this paper, we propose a novel unsupervised domain adaptation method for OrCR, which enables to transfer knowledge from labeled handprinted oracle characters to unlabeled scanned data. We leverage pseudo-labeling to incorporate the semantic information into adaptation and constrain augmentation consistency to make the predictions of scanned samples consistent under different perturbations, leading to the model robustness to abrasion, stain and distortion. Simultaneously, an unsupervised transition loss is proposed to learn more discriminative features on the scanned domain by optimizing both between-class and within-class transition probability. Extensive experiments show that our approach achieves state-of-the-art result on Oracle-241 dataset and substantially outperforms the recently proposed structure-texture separation network by 15.1%.

SPEED:Semantic Prior and Extremely Efficient Dilated Convolution Network for Real-Time Metal Surface Defects Detection

Automatic defect detection on the metal surface is a vital task for product inspection in industrial assembly lines or production processes. Owing to miscellaneous patterns of defects, inter-class similarity, intra-class difference, and fewer defect samples, achieving accurate and automatic detection remains a big challenge. What's more, since the rising demand for production efficiency, real-time detection is increasingly desirable. This article proposes a semantic prior and extremely efficient dilated convolution network, named SPEED, for pixel-wise detection on the metal surface, which aims to address the aforementioned issues. The architecture of SPEED involves the following: (i) a semantic prior (SP) branch, with shallow layer and prior mapping module to capture low-level details; and (ii) an extremely efficient dilation (EED) branch, with lightweight bottleneck to obtain high-level context. Furthermore, an aggregation module is designed to fuse both types of feature representation. Additionally, different level features of bottleneck are fused to improve the segmentation performance. Experimental results on three metal surface defect datasets indicate that the proposed method outperforms the state-of-the-art approaches in terms of the mean intersection of union, model parameters, FLOPs, and FPS. More specifically, SPEED achieves 92.34% mIoU on NEU-Seg, 88.65% mIoU on Severstal Strip Steel, and 63.91% mIoU on MT Defect.

Satellite Imagery-Based Cloud Classification Using Deep Learning

A significant amount of satellite imaging data is now easily available due to the continued development of remote sensing (RS) technology. Enabling the successful application of RS in real-world settings requires efficient and scalable solutions to extend their use in multidisciplinary areas. The goal of quick analysis and precise classification in Remote Sensing Imaging (RSI) is often accomplished by utilizing approaches based on deep Convolution Neural Networks (CNNs). This research offers a unique snapshot-based residual network (SnapResNet) that consists of fully connected layers (FC-1024), batch normalization (BN), L2 regularization, dropout layers, dense layer, and data augmentation. Architectural changes overcome the inter-class similarity problem while data augmentation resolves the problem of imbalanced classes. Moreover, the snapshot ensemble technique is utilized to prevent over-fitting, thereby further improving the network’s performance. The proposed SnapResNet152 model employs the most challenging Large-Scale Cloud Images Dataset for Meteorology Research (LSCIDMR), having 10 classes with thousands of high-resolution images and classifying them into respective classes. The developed model outperforms the existing deep learning-based algorithms (e.g., AlexNet, VGG-19, ResNet101, and EfficientNet) and achieves an overall accuracy of 97.25%.

Cross-Level Multi-Modal Features Learning With Transformer for RGB-D Object Recognition

Object recognition, one of the main goals of robot vision, is a vital prerequisite for service robots to perform domestic tasks. Thanks to the rich sense of information provided by RGB-D sensors, RGB-D-based object recognition has received increasing attention. However, the existing works focus on collaborative RGB and depth data for object recognition, while ignoring the influence of depth image quality on recognition performance. Moreover, in real-world scenarios, there are many objects with strong similarity from certain observation angles, which poses a challenge for the service robot to recognize objects accurately. In this paper, we propose CNN-TransNet, a novel end-to-end Transformer-based architecture with convolutional neural networks (CNNs) for RGB-D object recognition. In order to deal with the effect of high inter-class similarity, discriminative multi-modal feature representations are generated by learning and relating multi-modal features at multiple levels. Besides, we employ a multi-modal fusion and projection (MMFP) module to reweight the contribution of each modality to address the problem of poor-quality depth image. Our proposed approach achieves state-of-the-art performance on three datasets (including Washington RGB-D Object Dataset, JHUIT-50, and Object Clutter Indoor Dataset), with accuracy of 95.4%, 98.1%, and 94.7%, respectively. The results demonstrate the effectiveness and superiority of the proposed model in RGB-D object recognition task.

PCA: Progressive class-wise attention for skin lesions diagnosis

Skin cancer is the most prevalent type of cancer worldwide. Early detection is essential as it could be fatal at later stages. The classification of skin lesions is challenging since there are many variations, including changes in color, shape, size, high intra-class variation, and high inter-class similarity. In this paper, a unique class-wise attention method is proposed that considers each class equally while extracting additional discriminative information of skin lesions. The proposed attention mechanism is employed in a progressive manner to incorporate discriminative feature information from multiple scales. The proposed approach obtained competitive performance against more than 15 state-of-the-art methods including HAM1000 and ISIC 2019 leaderboard winners. The proposed method achieved 97.40% accuracy on the HAM10000 and 94.9% accuracy on the ISIC 2019 dataset.

Image patch-based deep learning approach for crop and weed recognition

Accurate classification of weed species in crop plants plays a crucial role in precision agriculture by enabling targeted treatment. Recent studies show that artificial intelligence deep learning (DL) models achieve promising solutions. However, several challenging issues, such as lack of adequate training data, inter-class similarity between weed species and intra-class dissimilarity between the images of the same weed species at different growth stages or for other reasons (e.g., variations in lighting conditions, image capturing mechanism, agricultural field environments) limit their performance. In this research, we propose an image based weed classification pipeline where a patch of the image is considered at a time to improve the performance. We first enhance the images using generative adversarial networks. The enhanced images are divided into overlapping patches, a subset of which are used for training the DL models. For selecting the most informative patches, we use the variance of Laplacian and the mean frequency of Fast Fourier Transforms. At test time, the model's outputs are fused using a weighted majority voting technique to infer the class label of an image. The proposed pipeline was evaluated using 10 state-of-the-art DL models on four publicly available crop weed datasets: DeepWeeds, Cotton weed, Corn weed, and Cotton Tomato weed. Our pipeline achieved significant performance improvements on all four datasets. DenseNet201 achieved the top performance with F1 scores of 98.49%, 99.83% and 100% on Deepweeds, Corn weed and Cotton Tomato weed datasets, respectively. The highest F1 score on the Cotton weed dataset was 98.96%, obtained by InceptionResNetV2. Moreover, the proposed pipeline addressed the issues of intra-class dissimilarity and inter-class similarity in the DeepWeeds dataset and more accurately classified the minority weed classes in the Cotton weed dataset. This performance indicates that the proposed pipeline can be used in farming applications.

Lithological mapping of geological remote sensing via adversarial semi-supervised segmentation network

Geological remote sensing interpretation (GRSI), which aims to recognize multiple geological elements based on their characteristics on satellite remote sensing images, is vital in large-scale regional lithological mapping. However, due to the influence of long-term geological movements, the spatial distribution of geological elements (such as lithology, glaciers, and soils) on the image is often complex and highly fragmented. In addition, the characteristics of high inter-class similarity and severe homogenization make the annotation of geological element samples require significant cost and expert knowledge. These lead to insufficient interpretation accuracy and limited annotation samples in GRSI. To alleviate the dependence of labeled samples and promote the performance of GRSI, we propose an adversarial semi-supervised segmentation network with object-context and global-attention (AdvSemi-OCGNet) for the GRSI, which achieves effective segmentation results in the case of limited labeled samples. Under the architecture of adversarial learning, a proposed baseline network OCGNet and a full convolution discriminator (FCD) are integrated to conduct semi-supervised segmentation. OCGNet, as the generator, aims to confuse FCD by predicting probability maps. Then FCD selects trustworthy regions with high-confidence of unlabeled sample to generate pseudo-labels for semi-supervised segmentation of OCGNet. Iterative adversarial learning is employed to simulate the process of expert interpretation of geological elements through continuous discrimination and correction. Finally, a fully connected conditional random field eliminates holes and isolated areas of segmented results caused by misclassification. Two study areas are selected, which include various geological elements such as multiple lithologies, soils, rock glaciers, and surface water. Numerous experiments have revealed the superiority of the proposed model in GRSI with limited annotation samples.

Skin lesion recognition via global-local attention and dual-branch input network

Owing to the deficiency of training data, the variety of skin lesion shapes for different patients, inter-class similarity, and intra-class variation, the recognition accuracy and speed in skin lesion recognition based on deep learning remain challenging. To solve the above issues, we propose a Dual-Branch and Global-Local Attention network based on ResNet50 (DGLA-ResNet50) to reduce model parameters and improve identification accuracy. For inter-class and intra-class problems, a global-local attention mechanism is designed to obtain the dependency information of query points in horizontal and vertical directions in turn for obtaining the global information indirectly. And we choose to obtain the local information through multiple convolutions simultaneously. Aiming at insufficient samples, the attention mechanism is lightened from the perspectives of reducing query points and the points associated with them, so as to reduce parameter redundancy. In addition, we present a dual-branch input network to deal with the problem of image variety, in which two branches are used to extract image features with different resolutions for fusion, so as to expand the network receptive field. We evaluated DGLA-ResNet50 on ISIC2018 and ISIC2019. The experimental results on ISIC2018 demonstrate that DGLA-ResNet50 has a recognition accuracy of 90.71%, while the parameters of the model are only 104.2M and the FLOPs value is 15.6G. Compared with the common models, our model obtains significantly better performance. The results indicate that DGLA-ResNet50 can improve the accuracy well while ensuring the lightweight of the model, and can prospectively assist doctors in the rapid diagnosis of skin lesions.

HiFuse: Hierarchical multi-scale feature fusion network for medical image classification

Effective fusion of global and local multi-scale features is crucial for medical image classification. Medical images have many noisy, scattered features, intra-class variations, and inter-class similarities. Many studies have shown that global and local features are helpful to reduce noise interference in medical images. It is difficult to capture the global features of images due to the fixed size of the receptive domain of the convolution kernel. Although the self-attention-based Transformer can model long-range dependencies, it has high computational complexity and lacks local inductive bias. In this paper, we propose a three-branch hierarchical multi-scale feature fusion network structure termed as HiFuse, which can fuse multi-scale global and local features without destroying the respective modeling, thus improving the classification accuracy of various medical images. There are two key characteristics: (i) a parallel hierarchical structure consisting of global and local feature blocks; (ii) an adaptive hierarchical feature fusion block (HFF block) and inverted residual multi-layer perceptron(IRMLP). The advantage of this network structure lies in that the resulting representation is semantically richer and the local features and global representations can be effectively extracted at different semantic scales. Our proposed model’s ACC and F1 values reached 85.85% and 75.32% on the ISIC2018 dataset, 86.12% and 86.13% on the Kvasir dataset, 76.88% and 76.31% on the Covid-19 dataset, 92.31% and 88.81% on the esophageal cancer pathology dataset. The HiFuse model performs the best compared to other advanced models. Our code is open source and available from https://github.com/huoxiangzuo/HiFuse.

Semantic Segmentation of Remote Sensing Images Using Multiway Fusion Network

To effectively solve the problems of intra-class dissimilarity and inter-class similarity, this study proposes a deep learning semantic segmentation model that fuses multiple path features. It utilizes Multipath Fusion Module (MFM) to extract input image features, and dynamically fuses the features extracted from each input path. In the fusion process, the segmentation model dynamically adjusts the fuse on ratio and feature threshold of each path according to the input image, enables highly accurate image segmentation. In the upsampling stage, a guided upsampling strategy helps to avoid edge classification errors due to bilinear interpolation. The proposed network was trained and tested on the Potsdam dataset with good results, with mean intersection over union (mIoU) of 83.38%, overall accuracy (OA) of 90.21% and an F1 score of 90.86%.

ARM-Net: Attention-guided residual multiscale CNN for multiclass brain tumor classification using MR images

Brain tumor is the deadliest type of cancer and has the lowest survival rate when compared with other cancers. Hence, timely detection of brain tumor is indispensable for patients to make better treatment plans, leading to improved life expectancy. However, accurate classification of different brain tumor types from MR images is challenging due to high inter-class similarities. Though deep learning architectures, mainly CNNs, have shown promising performance compared to traditional approaches, such models often demand huge parameters and lead to overfitting while dealing with limited training samples. Further, the state-of-the-art CNN models cannot capture the subtle lesion size and shape variations among different classes. To cope with these issues, in this paper, we propose an attention-based residual multiscale CNN called ARM-Net for multiclass brain tumor classification. In particular, we propose a lightweight residual multiscale CNN dubbed RM-Net to capture high-level feature representations at different receptive fields. Further, a lightweight global attention module (LGAM) is proposed to selectively learn more discriminative features. The LGAM is placed on the top of RM-Net and is introduced to capture wide-range feature dependencies. Experimental results on two benchmark datasets indicate the superiority of our ARM-Net over the state-of-the-art CNN architectures and existing methods. The ARM-Net achieves an accuracy of 96.64% and 97.11% on MBTD and BraTS 2020 dataset, respectively. The ablation studies, Grad-CAM, and Grad-CAM++ visualization results confirm the effectiveness of our proposed LGAM. In addition, our ARM-Net is lightweight, end-to-end learnable, and hence more suitable for real-time brain tumor classification.

Toward efficient deep learning system for in-situ plankton image recognition

Plankton is critical for the structure and function of marine ecosystems. In the past three decades, various underwater imaging systems have been developed to collect in-situ plankton images and image processing has been a major bottleneck that hinders the deployment of plankton imaging systems. In recent years, deep learning methods have greatly enhanced our ability of processing in-situ plankton images, but high-computational demands and longtime consumption still remain problematic. In this study, we used knowledge distillation as a framework for model compression and improved computing efficiency while maintaining original high accuracy. A novel inter-class similarity distillation algorithm based on feature prototypes was proposed and enabled the student network (small scale) to acquire excellent ability for plankton recognition after being guided by the teacher network (large scale). To identify the suitable teacher network, we compared emerging Transformer neural networks and convolution neural networks (CNNs), and the best performing deep learning model, Swin-B, was selected. Utilizing the proposed knowledge distillation algorithm, the feature extraction ability of Swin-B was transferred to five more lightweight networks, and the results had been evaluated in taxonomic dataset of in-situ plankton images. Subsequently, the chosen lightweight model and the Bilateral–Sobel edge enhancement were tested to process in-situ images with high level of noises captured from coastal waters of Guangdong, China and achieved an overall recall rate of 91.73%. Our work contributes to effective deep learning models and facilitates the deployment of underwater plankton imaging systems by promoting both accuracy and speed in recognition of plankton targets.

A Novel Discriminative Enhancement Method for Few-Shot Remote Sensing Image Scene Classification

Remote sensing image scene classification (RSISC) has garnered significant attention in recent years. Numerous methods have been put forward in an attempt to tackle this issue, particularly leveraging deep learning methods that have shown promising performance in classifying remote sensing image (RSI). However, it is widely recognized that deep learning methods typically require a substantial amount of labeled data to effectively converge. Acquiring a sufficient quantity of labeled data often necessitates significant human and material resources. Hence, few-shot RSISC has become highly meaningful. Fortunately, the recently proposed deep nearest neighbor neural network based on the attention mechanism (DN4AM) model incorporates episodic training and class-related attention mechanisms, effectively reducing the impact of background noise regions on classification results. Nevertheless, the DN4AM model does not address the problem of significant intra-class variability and substantial inter-class similarities observed in RSI scenes. Therefore, the discriminative enhanced attention-based deep nearest neighbor neural network (DEADN4) is proposed to address the few-shot RSISC task. Our method makes three contributions. Firstly, we introduce center loss to enhance the intra-class feature compactness. Secondly, we utilize the deep local-global descriptor (DLGD) to increase inter-class feature differentiation. Lastly, we modify the Softmax loss by incorporating cosine margin to amplify the inter-class feature dissimilarity. Experiments are conducted on three diverse RSI datasets to gauge the efficacy of our approach. Through comparative analysis with various cutting-edge methods including MatchingNet, RelationNet, MAML, Meta-SGD, DN4, and DN4AM, our approach showcases promising outcomes in the few-shot RSISC task.

How to identify pollen like a palynologist: A prior knowledge-guided deep feature learning for real-world pollen classification

Airborne pollen identification is crucial to help patients prevent pollinosis symptoms. Existing data-driven methods rely on large-scale pollen images with simple backgrounds. In real scenarios, the background is complex and the data scale is small. Therefore, these methods suffer from two challenges: (1) Irrelevant information interference; (2) Incomplete feature attention. To overcome these challenges, we propose a prior knowledge-guided deep feature learning (PK-DFL) for real-world optical microscope image classification. Its main steps are as follows: Pollen location is designed to locate pollen grains based on color features, aiming to boost the accuracy of shape and texture prior feature extraction. Shape-texture awareness helps to extract the shape and texture of pollen grains via predefined feature extractors (i.e., a set of shape descriptors and an improved SFTA). These features are used to construct two types of prior knowledge, namely shape-texture attention maps (STA maps) and shape-texture feature vectors (STF vectors). Pollen classification uses a deep network (CNN) to classify pollen via imitating the pollen identification procedure of palynologists. It uses STA maps to weight pollen images and convolutional feature maps for instructing the CNN to focus on critical areas of pollen images (for the first challenge). STF vectors are employed to obtain the inter-class similarity of pollen via template matching. This information is further converted to soft targets that are used to supervise the CNN attending to comprehensive key features (for the second challenge). Extensive experiments on real-world datasets demonstrate the effectiveness of our PK-DFL (with accuracy and F1-score over 88%).

SkinNet-INIO: Multiclass Skin Lesion Localization and Classification Using Fusion-Assisted Deep Neural Networks and Improved Nature-Inspired Optimization Algorithm.

Background: Using artificial intelligence (AI) with the concept of a deep learning-based automated computer-aided diagnosis (CAD) system has shown improved performance for skin lesion classification. Although deep convolutional neural networks (DCNNs) have significantly improved many image classification tasks, it is still difficult to accurately classify skin lesions because of a lack of training data, inter-class similarity, intra-class variation, and the inability to concentrate on semantically significant lesion parts. Innovations: To address these issues, we proposed an automated deep learning and best feature selection framework for multiclass skin lesion classification in dermoscopy images. The proposed framework performs a preprocessing step at the initial step for contrast enhancement using a new technique that is based on dark channel haze and top-bottom filtering. Three pre-trained deep learning models are fine-tuned in the next step and trained using the transfer learning concept. In the fine-tuning process, we added and removed a few additional layers to lessen the parameters and later selected the hyperparameters using a genetic algorithm (GA) instead of manual assignment. The purpose of hyperparameter selection using GA is to improve the learning performance. After that, the deeper layer is selected for each network and deep features are extracted. The extracted deep features are fused using a novel serial correlation-based approach. This technique reduces the feature vector length to the serial-based approach, but there is little redundant information. We proposed an improved anti-Lion optimization algorithm for the best feature selection to address this issue. The selected features are finally classified using machine learning algorithms. Main Results: The experimental process was conducted using two publicly available datasets, ISIC2018 and ISIC2019. Employing these datasets, we obtained an accuracy of 96.1 and 99.9%, respectively. Comparison was also conducted with state-of-the-art techniques and shows the proposed framework improved accuracy. Conclusions: The proposed framework successfully enhances the contrast of the cancer region. Moreover, the selection of hyperparameters using the automated techniques improved the learning process of the proposed framework. The proposed fusion and improved version of the selection process maintains the best accuracy and shorten the computational time.

Adaptive Adjacent Layer Feature Fusion for Object Detection in Remote Sensing Images

Object detection in remote sensing images faces the challenges of a complex background, large object size variations, and high inter-class similarity. To address these problems, we propose an adaptive adjacent layer feature fusion (AALFF) method, which is developed on the basis of RTMDet. Specifically, the AALFF method incorporates an adjacent layer feature fusion enhancement (ALFFE) module, designed to capture high-level semantic information and accurately locate object spatial positions. ALFFE also effectively preserves small objects by fusing adjacent layer features and employs involution to aggregate contextual information in a wide spatial range for object essential features extraction in complex backgrounds. Additionally, the adaptive spatial feature fusion (ASFF) module is introduced to guide the network to select and fuse the crucial features to improve the adaptability to objects with different sizes. The proposed method achieves mean average precision (mAP) values of 77.1%, 88.9%, and 95.7% on the DIOR, HRRSD, and NWPU VHR-10 datasets, respectively. Notably, our approach achieves mAP75 values of 60.8% and 79.0% on the DIOR and HRRSD datasets, respectively, surpassing the state-of-the-art performance on the DIOR dataset.

Consistent penalizing field loss for zero-shot image retrieval

Zero-shot image retrieval involves retrieving images of unseen classes using a query image of the same class. To determine whether a given image is of the same class as the query image, a universal threshold of similarity measures is needed, as class-specific thresholds are not feasible for unseen classes. However, existing methods for zero-shot image retrieval focus on pushing a margin between intra-class and inter-class similarities for each class during the training phase. This approach can result in varying decision boundaries between intra- and inter-class similarities across classes, which could compromise performance when a universal threshold is used in the inference stage. Additionally, for classes with low intra-class variances or inter-class correlations, the pushing force of the margin-pushing approach might be too weak to learn high-quality embeddings. To address these issues, we propose a novel Consistent Penalizing Field (CPF) Loss for zero-shot image retrieval. The proposed method has a single consistent penalizing field for all classes, resulting in similar decision boundaries across classes. By penalizing samples outside the penalizing field, CPF Loss can better utilize the information of samples with highly unbalanced intra-class and inter-class correlations, and improve the discriminative power of DML learning for zero-shot image retrieval. Extensive experiments are conducted on the challenging Shopee Product Matching dataset and other established benchmarks, and the results demonstrate that the proposed method consistently outperforms the state-of-the-art methods. The code is available at https://github.com/cloudlc/CPF.