Few-shot Detection Research Articles

Few-shot fine-grained recognition in remote sensing ship images with global and local feature aggregation

Remote sensing ship image detection methods have broad application prospects in areas such as maritime traffic and fisheries management. However, previous detection methods relied heavily on a large amount of accurately annotated training data. When the number of remote sensing ship targets is scarce, the detection performance of previous methods is unsatisfactory. To address this issue, this paper proposes a few-shot detection method based on global and local feature aggregation. Specifically, we introduce global and local feature aggregation. We aggregate query-image global and local features with support features. This encourages the model to learn invariant features under varying global feature conditions which enhances the model’s performance in training and inference. Building upon this, we propose combined feature aggregation, where query features are aggregated with all support features in the same batch, further reducing the confusion of target features caused by the imbalance between base-class samples and novel-class samples, improving the model’s learning effectiveness for novel classes. Additionally, we employ an adversarial autoencoder to reconstruct support features, enhancing the model’s generalization performance. Finally, the model underwent extensive experiments on the publicly available remote sensing ship dataset HRSC-2016. The results indicate that compared to the baseline model, our model achieved new state-of-the-art performance under various dataset settings. This model presented in this paper will provide new insights for few-shot detection work based on meta-learning.

MetaNet: Interpretable unknown mobile malware identification with a novel meta-features mining algorithm

The continuous emergence of malware has threatened to the Android platform and user privacy. With the evolution of the Android system and malware, it is challenging to design a method that can accurately identify the categories of sophisticated malware, including known and unknown families, as well as their obfuscated variants, given that they may be newly emerging and lack available detection knowledge. Although some methods try to use anomaly detection and zero-shot technology to identify unseen applications, they are limited to binary classification or lack the robustness, stability, universality, and interpretability in multi-class identification. To this end, we first propose a generic meta-features mining algorithm, which can discover the potential relationships between samples belonging to the same category. Then we present metaNet, a novel method leveraging meta-features to identify sophisticated Android malware. Specifically, metaNet is mainly powered by four components: (i) mExtractor is a feature collector to obtain the static and dynamic features. (ii) mProcessor is taking unique meta-features of each category from extracted features. (iii) mLearner is a machine learning suite that leverages features and meta-features to design and train a classifier called HSU-Net. (iv) mEnforcer is a flexible deployer that identifies categories of malware families in the real world. We implement a prototype of metaNet with 15K lines of Python code and compare it with state-of-the-art (SOTA) methods. The results show that it can not only achieve superior performance in terms of known families (99.52% of accuracy) and unknown families (99.31% of accuracy trained with 80% known families) for binary classification, but also perform well in multi-class identification, i.e., 99.05% and 93.45% of accuracy for known and unknown families, respectively. Furthermore, we deploy and evaluate metaNet in the real world. It can identify applications over an acceptable time and memory overheads, i.e., average of 11.8 s and 56 MB per sample with a size of 8 MB. Also, the few-shot detection and feature perturbation experiments reflect its robustness and stability benefiting from meta-features. Finally, we collect the traffic of 112 decentralized applications (DApps) belonging to 16 categories, such as finance and health, and evaluated metaNet in DApp identification. The results illustrate its applicability across various tasks. That is, it can accurately classify 94.6% and 81.36% of DApp flows in all-known and 80%-known DApp scenarios, respectively, outperforming the SOTA methods.

In defense of local descriptor-based few-shot object detection.

State-of-the-art image object detection computational models require an intensive parameter fine-tuning stage (using deep convolution network, etc). with tens or hundreds of training examples. In contrast, human intelligence can robustly learn a new concept from just a few instances (i.e., few-shot detection). The distinctive perception mechanisms between these two families of systems enlighten us to revisit classical handcraft local descriptors (e.g., SIFT, HOG, etc.) as well as non-parametric visual models, which innately require no learning/training phase. Herein, we claim that the inferior performance of these local descriptors mainly results from a lack of global structure sense. To address this issue, we refine local descriptors with spatial contextual attention of neighbor affinities and then embed the local descriptors into discriminative subspace guided by Kernel-InfoNCE loss. Differing from conventional quantization of local descriptors in high-dimensional feature space or isometric dimension reduction, we actually seek a brain-inspired few-shot feature representation for the object manifold, which combines data-independent primitive representation and semantic context learning and thus helps with generalization. The obtained embeddings as pattern vectors/tensors permit us an accelerated but non-parametric visual similarity computation as the decision rule for final detection. Our approach to few-shot object detection is nearly learning-free, and experiments on remote sensing imageries (approximate 2-D affine space) confirm the efficacy of our model.

Few-shot detection of surface roughness of workpieces processed by different machining techniques

The traditional deep learning method for detecting workpiece surface roughness relies heavily on a large number of training samples. Also, when detecting the surface roughness of workpieces processed by different machining techniques, it requires a large number of samples of that workpiece to rebuild the model. To address these problems, this paper proposes a few-sample visual detection method for the surface roughness of workpieces processed by different techniques. This method first trains a base model using a relatively large amount of samples from one machining technique, then fine-tunes the model using small amounts of samples from workpieces of different techniques. By introducing contrastive proposal encoding into Faster R-CNN, the model’s ability to learn surface features from small amounts of workpiece samples is enhanced, thus improving the detection accuracy of surface roughness of workpieces processed by different techniques. Experiments show that this method reduces the model’s dependence on training samples and the cost of data preparation. It also demonstrates higher accuracy in surface roughness detection tasks of workpieces processed by different techniques, providing a new approach and insights for few-sample surface roughness detection.

A Biological Immunity-Based Neuro Prototype for Few-Shot Anomaly Detection with Character Embedding.

Anomaly detection has wide applications to help people recognize false, intrusion, flaw, equipment failure, etc. In most practical scenarios, the amount of the annotated data and the trusted labels is low, resulting in poor performance of the detection. In this paper, we focus on the anomaly detection for the text type data and propose a detection network based on biological immunity for few-shot detection, by imitating the working mechanism of the immune system of biological organisms. This network enabling the protected system to distinguish the aggressive behavior of "nonself" from the legitimate behavior of "self" by embedding characters. First, it constructs episodic task sets and extracts data representations at the character level. Then, in the pretraining phase, Word2Vec is used to embed the representations. In the meta-learning phase, a dynamic prototype containing encoder, routing, and relation is designed to identify the data traffic. Compare to the mean-based prototype, the proposed prototype applies a dynamic routing algorithm that assigns different weights to samples in the support set through multiple iterations to obtain a prototype that combines the distribution of samples. The proposed method is validated on 2 real traffic datasets. The experimental results indicate that (a) the proposed anomaly detection prototype outperforms state-of-the-art few-shot techniques with 1.3% to 4.48% accuracy and 0.18% to 4.55% recall; (b) under the premise of ensuring the accuracy and recall, the number of training samples is reduced to 5 or 10; (c) ablation experiments are designed for each module, and the results show that more accurate prototypes can be obtained by using the dynamic routing algorithm.

Towards Generalized Few-Shot Open-Set Object Detection.

Open-set object detection (OSOD) aims to detect the known categories and reject unknown objects in a dynamic world, which has achieved significant attention. However, previous approaches only consider this problem in data-abundant conditions, while neglecting the few-shot scenes. In this paper, we seek a solution for the generalized few-shot open-set object detection (G-FOOD), which aims to avoid detecting unknown classes as known classes with a high confidence score while maintaining the performance of few-shot detection. The main challenge for this task is that few training samples induce the model to overfit on the known classes, resulting in a poor open-set performance. We propose a new G-FOOD algorithm to tackle this issue, named Few-shOt Open-set Detector (FOOD), which contains a novel class weight sparsification classifier (CWSC) and a novel unknown decoupling learner (UDL). To prevent over-fitting, CWSC randomly sparses parts of the normalized weights for the logit prediction of all classes, and then decreases the co-adaptability between the class and its neighbors. Alongside, UDL decouples training the unknown class and enables the model to form a compact unknown decision boundary. Thus, the unknown objects can be identified with a confidence probability without any threshold, prototype, or generation. We compare our method with several state-of-the-art OSOD methods in few-shot scenes and observe that our method improves the F-score of unknown classes by 4.80%-9.08% across all shots in VOC-COCO dataset settings.

TKR-FSOD: Fetal Anatomical Structure Few-Shot Detection Utilizing Topological Knowledge Reasoning.

Fetal multi-anatomical structure detection in Ultrasound (US) images can clearly present the relationship and influence between anatomical structures, providing more comprehensive information about fetal organ structures and assisting sonographers in making more accurate diagnoses, widely used in structure evaluation. Recently, deep learning methods have shown superior performance in detecting various anatomical structures in ultrasound images but still have the potential for performance improvement in categories where it is difficult to obtain samples, such as rare diseases. Few-shot learning has attracted a lot of attention in medical image analysis due to its ability to solve the problem of data scarcity. However, existing few-shot learning research in medical image analysis focuses on classification and segmentation, and the research on object detection has been neglected. In this paper, we propose a novel fetal anatomical structure fewshot detection method in ultrasound images, TKR-FSOD, which learns topological knowledge through a Topological Knowledge Reasoning Module to help the model reason about and detect anatomical structures. Furthermore, we propose a Discriminate Ability Enhanced Feature Learning Module that extracts abundant discriminative features to enhance the model's discriminative ability. Experimental results demonstrate that our method outperforms the state-of-the-art baseline methods, exceeding the second best method with a maximum margin of 4.8% on 5-shot of split 1 under 4CC. The code is publicly available at: https://github.com/lixi92/TKR-FSOD.

Dual-Metric Neural Network With Attention Guidance for Surface Defect Few-Shot Detection in Smart Manufacturing

Abstract As an important application of human–robot collaboration, intelligent detection of surface defects is crucial for production quality control, which also helps in relieving the workload of technical staff in human-centric smart manufacturing. To accurately detect defects with limited samples in industrial practice, a dual-metric neural network with attention-guided is proposed. First, an attention-guided recognition network with channel attention and position attention module is designed to efficiently learn representative defect features with limited samples. Second, aiming to detect defects with confusing surface images, a dual-metric function is presented to learn the classification boundary by controlling the distance of samples in feature space from intraclass and interclass. The experiment results on the fabric defect dataset demonstrate that the proposed approach outperforms state-of-the-art methods in accuracy, recall, precision, F1-score, and few-shot accuracy. Further comparative experiments reveal that the dual-metric function is superior in improving the few-shot detection accuracy.

GDE model: A variable intrusion detection model for few-shot attack

With the formation and popularity of the Internet of Things(IoT), the difficulty of protecting IoT infrastructure and smart devices from a few-shot of ever-changing malicious attacks has increased significantly. Traditional intrusion detection models in static mode cannot defend against intelligent attacks that change in real time and are good at reconnaissance, and it is difficult to achieve effective detection of few-shot attacks. Therefore, to solve the above problems, this paper proposes a variable few-shot intrusion detection model GDE Model for IoT, which contains a data processing module consisting of an improved Gramian Angular Fields(GAF), a data augmentation module consisting of an improved Denoising Diffusion Probabilistic Models(DDPM), and an image classification module consisting of a variable network ETNet V2 obtained from a neural architecture search based on efficient net v2. The one-dimensional network traffic data collected from a real network environment is converted into two-dimensional images and fed into the above model for learning and training, so as to construct a variable intrusion detection model that combines accuracy and inference speed. To evaluate the work of this paper, different categories of evaluation metrics were selected and experiments were conducted on each module and the overall model separately. The experimental results show that the proposed intrusion detection model achieves variable functionality with an accuracy of up to 99.20% for few-shot detection and an average inference time as low as 23ms, which can be used not only for IoT but also for different network environments and attack types.

Cross-domain mechanism for few-shot object detection on Urine Sediment Image

Deep learning object detection networks require a large amount of box annotation data for training, which is difficult to obtain in the medical image field. The few-shot object detection algorithm is significant for an unseen category, which can be identified and localized with a few labeled data. For medical image datasets, the image style and target features are incredibly different from the knowledge obtained from training on the original dataset. We propose a background suppression attention(BSA) and feature space fine-tuning module (FSF) for this cross-domain situation where there is a large gap between the source and target domains. The background suppression attention reduces the influence of background information in the training process. The feature space fine-tuning module adjusts the feature distribution of the interest features, which helps to make better predictions. Our approach improves detection performance by using only the information extracted from the model without maintaining additional information, which is convenient and can be easily plugged into other networks. We evaluate the detection performance in the in-domain situation and cross-domain situation. In-domain experiments on the VOC and COCO datasets and the cross-domain experiments on the VOC to medical image dataset UriSed2K show that our proposed method effectively improves the few-shot detection performance.

Incremental Few-Shot Object Detection with scale- and centerness-aware weight generation

Our work targets the task of Incremental Few-Shot Detection (iFSD), which requires quickly detecting objects of novel classes after seeing only a few training examples, while maintaining the ability to detect old classes. Different from re-training a new detector to incorporate few-shot novel categories, we tackle iFSD in a direct weight generation way. Specifically, we propose to reorganize base weights by leveraging responses of novel region features, hence transferring learned information from base weights to the novel but relevant ones. To obtain representative region features for the novel weight generation, two strategies are further presented. The scale-aware strategy respects the scale variation in object detection, and assigns each object to its corresponding pyramid to obtain the scale-specific region feature. The centerness-aware strategy reorganizes region features in the spatial range, since the localization quality of each pixel on the feature map varies. Extensive experiments are conducted on the challenging MS COCO and PASCAL VOC datasets, and our method achieves promising performances under various settings with a direct weight generation fashion. Furthermore, our generated weights can be used as good initialization for fine-tuning to induce faster convergence, thus mitigating the performance drop on old classes and improving overall results in the incremental setting.

Online visual end-to-end detection monitoring on surface defect of aluminum strip under the industrial few-shot condition

Surface defect detection systems based on deep learning are employed in the manufacturing system, and their good detection performance largely relies on abundant annotated data. Nevertheless, industrial datasets are often difficult to obtain, which hinders the development of defect detection systems to some extent. In an effort to address the issue that the scarcity of training data leads to poor performance, this paper proposes a novel end to end few-shot detection method for industrial real-time detection on aluminum strips. The meta-learning theory is introduced into the multi-scale structure of the YOLOv4 framework, which means that multi-scale meta-feature maps are extracted through the backbone network and each meta-feature map is recalibrated by reweighted vectors to strengthen class-specific features. Subsequently, a multi-scale prediction module with a new parsing strategy is designed to locate and classify the defects at different scales for the reweighted meta-features. All experiments are performed using data collected from the aluminum strip production line in a cold rolling workshop. In addition, the dataset required for training is constructed based on the few-shot training strategy. Pre-training is conducted on a large amount of labeled data from base classes in advance, and the pre-trained weights are then loaded to train the model on a small amount of labeled data from all classes to rapidly deploy the model in multi-task scenarios. The proposed method shows excellent performance, improving the accuracy for novel classes by more than 14.6%, which enhances the industrial practicality of detection systems.

Few-shot Object Detection Based on Sample Balance Correction

We propose a few-shot object detection algorithm based on sample balance correction, which introduces categorical sample balance correction and positioning sample balance correction on the basis of the Faster R-CNN network to improve the performance of few-shot detection, and categorical sample balance correction increases the passing probability of the new type pre-selection box by changing the selection rules of the pre-selection box. And the similarity calculation branch is added to the detection head to help improve the classification ability, and the positioning sample balance correction solves the sample balance in the training process and the balance problem of correcting difficult samples by improving the positioning loss function. Experiments on the PASCAL VOC dataset show that the proposed algorithm improves the detection effect when the sample size is small, up to 3.2 percentage points higher than the comparison algorithm, and has good robustness and generalization ability.

ITFD: an instance-level triplet few-shot detection network under weighted pair-resampling

ITFD: an instance-level triplet few-shot detection network under weighted pair-resampling

FSVM: A Few-Shot Threat Detection Method for X-ray Security Images.

In recent years, automatic detection of threats in X-ray baggage has become important in security inspection. However, the training of threat detectors often requires extensive, well-annotated images, which are hard to procure, especially for rare contraband items. In this paper, a few-shot SVM-constraint threat detection model, named FSVM is proposed, which aims at detecting unseen contraband items with only a small number of labeled samples. Rather than simply finetuning the original model, FSVM embeds a derivable SVM layer to back-propagate the supervised decision information into the former layers. A combined loss function utilizing SVM loss is also created as the additional constraint. We have evaluated FSVM on the public security baggage dataset SIXray, performing experiments on 10-shot and 30-shot samples under three class divisions. Experimental results show that compared with four common few-shot detection models, FSVM has the highest performance and is more suitable for complex distributed datasets (e.g., X-ray parcels).

Prior knowledge auxiliary for few-shot pest detection in the wild.

One of the main techniques in smart plant protection is pest detection using deep learning technology, which is convenient, cost-effective, and responsive. However, existing deep-learning-based methods can detect only over a dozen common types of bulk agricultural pests in structured environments. Also, such methods generally require large-scale well-labeled pest data sets for their base-class training and novel-class fine-tuning, and these significantly hinder the further promotion of deep convolutional neural network approaches in pest detection for economic crops, forestry, and emergent invasive pests. In this paper, a few-shot pest detection network is introduced to detect rarely collected pest species in natural scenarios. Firstly, a prior-knowledge auxiliary architecture for few-shot pest detection in the wild is presented. Secondly, a hierarchical few-shot pest detection data set has been built in the wild in China over the past few years. Thirdly, a pest ontology relation module is proposed to combine insect taxonomy and inter-image similarity information. Several experiments are presented according to a standard few-shot detection protocol, and the presented model achieves comparable performance to several representative few-shot detection algorithms in terms of both mean average precision (mAP) and mean average recall (mAR). The results show the promising effectiveness of the proposed few-shot detection architecture.

Decoupled Metric Network for Single-Stage Few-Shot Object Detection.

Within the last few years, great efforts have been made to study few-shot learning. Although general object detection is advancing at a rapid pace, few-shot detection remains a very challenging problem. In this work, we propose a novel decoupled metric network (DMNet) for single-stage few-shot object detection. We design a decoupled representation transformation (DRT) and an image-level distance metric learning (IDML) to solve the few-shot detection problem. The DRT can eliminate the adverse effect of handcrafted prior knowledge by predicting objectness and anchor shape. Meanwhile, to alleviate the problem of representation disagreement between classification and location (i.e., translational invariance versus translational variance), the DRT adopts a decoupled manner to generate adaptive representations so that the model is easier to learn from only a few training data. As for a few-shot classification in the detection task, we design an IDML tailored to enhance the generalization ability. This module can perform metric learning for the whole visual feature, so it can be more efficient than traditional DML due to the merit of parallel inference for multiobjects. Based on the DRT and IDML, our DMNet efficiently realizes a novel paradigm for few-shot detection, called single-stage metric detection. Experiments are conducted on the PASCAL VOC dataset and the MS COCO dataset. As a result, our method achieves state-of-the-art performance in few-shot object detection. The codes are available at https://github.com/yrqs/DMNet.

PVEL-AD: A Large-Scale Open-World Dataset for Photovoltaic Cell Anomaly Detection

The anomaly detection in photovoltaic (PV) cell electroluminescence (EL) image is of great significance for the vision-based fault diagnosis. Many researchers are committed to solving this problem, but a large-scale open-world dataset is required to validate their novel ideas. We build a PV EL Anomaly Detection (PVEL-AD <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1, 2, 3</sup> ) dataset for polycrystalline solar cell, which contains 36 543 near-infrared images with various internal defects and heterogeneous background. This dataset contains anomaly free images and anomalous images with ten different categories. Moreover, 37 380 ground truth bounding boxes are provided for eight types of defects. We also carry out a comprehensive evaluation of the state-of-the-art object detection methods based on deep learning. The evaluation results on this dataset provide the initial benchmark, which is convenient for follow-up researchers to conduct experimental comparisons. To the best of our knowledge, this is the first public dataset for PV solar cell anomaly detection that provides box-wise ground truth. Furthermore, this dataset can also be used for the evaluation of many computer vision tasks such as few-shot detection, one-class classification, and anomaly generation.

Few-Shot Aircraft Detection in Satellite Videos Based on Feature Scale Selection Pyramid and Proposal Contrastive Learning

To date, few-shot object detection methods have received extensive attention in the field of remote sensing, and no relevant research has been conducted using satellite videos. It is difficult to identify foreground objects in satellite videos duo to their small size and low contrast and the domain differences between base and novel classes under few-shot conditions. In this paper, we propose a few-shot aircraft detection method with a feature scale selection pyramid and proposal contrastive learning for satellite videos. Specifically, a feature scale selection pyramid network (FSSPN) is constructed to replace the traditional feature pyramid network (FPN), which alleviates the limitation of the inconsistencies in gradient computation between different layers for small-scale objects. In addition, we add proposal contrastive learning items to the loss function to achieve more robust representations of objects. Moreover, we expand the freezing parameters of the network in the fine-tuning stage to reduce the interference of visual differences between the base and novel classes. An evaluation of large-scale experimental data showed that the proposed method makes full use of the advantages of the two-stage fine-tuning strategy and the characteristics of satellite video to enhance the few-shot detection performance.

Joint Stance and Rumor Detection in Hierarchical Heterogeneous Graph

Recently, large volumes of false or unverified information (e.g., fake news and rumors) appear frequently in emerging social media, which are often discussed on a large scale and widely disseminated, causing bad consequences. Many studies on rumor detection indicate that the stance distribution of posts is closely related to the rumor veracity. However, these two tasks are generally considered separately or just using a shared encoder/layer via multitask learning, without exploring the more profound correlation between them. In particular, the performance of existing methods relies heavily on the quality of hand-crafted features and the quantity of labeled data, which is not conducive to early rumor detection and few-shot detection. In this article, we construct a hierarchical heterogeneous graph by associating posts containing the same high-frequency words to facilitate the feature cross-topic propagation and jointly formulate stance and rumor detection as multistage classification tasks. To realize the updating of node embeddings jointly driven by stance and rumor detection, we propose a multigraph neural network framework, which can more flexibly capture the attribute and structure information of the context. Experiments on real datasets collected from Twitter and Reddit show that our method outperforms state-of-the-art by a large margin on both stance and rumor detection. And the experimental results also show that our method has better interpretability and requires less labeled data.