miniImageNet Datasets Research Articles

Improved generalization performance of convolutional neural networks with LossDA

In recent years, convolutional neural networks (CNNs) have been used in many fields. Nowadays, CNNs have a high learning capability, and this learning capability is accompanied by a more complex model architecture. Complex model architectures allow CNNs to learn more data features, but such a learning process tends to reduce the training model’s ability to generalize to unknown data, and may be associated with problems of overfitting. Although many regularization methods have been proposed, such as data augmentation, batch normalization, and Dropout, research on improving generalization performance is still a common concern in the training process of robust CNNs. In this paper, we propose a dynamically controllable adjustment method, which we call LossDA, that embeds a disturbance variable in the fully-connected layer. The trend of this variable is kept consistent with the training loss, while the magnitude of the variable can be preset to adapt to the training process of different models. Through this dynamic adjustment, the training process of CNNs can be adaptively adjusted. The whole regularization process can improve the generalization performance of CNNs while helping to suppress overfitting. To evaluate this method, this paper conducts comparative experiments on MNIST, FashionMNIST, CIFAR-10, Cats_vs_Dogs, and miniImagenet datasets. The experimental results show that the method can improve the model performance of Light CNNs and Transfer CNNs (InceptionResNet, VGG19, ResNet50, and InceptionV3). The average maximum improvement in accuracy of Light CNNs is 4.62%, F1 is 3.99%, and Recall is 4.69%. The average maximum improvement accuracy of Transfer CNNs is 4.17%, F1 is 5.64%, and Recall is 4.05%.

FeFET Multi-Bit Content-Addressable Memories for In-Memory Nearest Neighbor Search

Nearest neighbor (NN) search computations are at the core of many applications such as few-shot learning, classification, and hyperdimensional computing. As such, efficient hardware support for NN search is highly desired. In-memory computing using emerging devices offers attractive solutions for NN search. Solutions based on ternary content-addressable memories (TCAMs) offer high energy and latency improvements for NN search at the expense of accuracy. In this work, we propose a novel distance function that can be natively evaluated with multi-bit content-addressable memories (MCAMs) based on ferroelectric FETs (FeFETs) to perform a single-step, in-memory NN search. We evaluate the efficacy of FeFET MCAMs in the context of few-shot learning applications with different datasets. As an example, we achieve a 78.54% accuracy for a 5-way, 5-shot classification task for the mini-ImageNet dataset (only 1.5% lower than software-based implementations) when using a 3-bit MCAM for NN search. We consider the effects of FeFET threshold voltage variations on the application accuracy and analyze the area and search energy requirements of FeFET MCAMs for accurate operations. Our results indicate that MCAMs require 2× lower area and search energy than TCAMs to achieve the same accuracy. Furthermore, we experimentally demonstrate a 2-bit implementation of FeFET MCAM using AND arrays from GLOBALFOUNDRIES to further validate the design concept.

SiSL-Net: Saliency-guided self-supervised learning network for image classification

The emergent studies and success on contrastive self-supervised learning have been well verified in the pretext task of instance discrimination, which learns visual representations by maximizing agreement between different augmented views of the same image sample (positive pairs). However, randomly cropping on original images may cause that the augmented view contains interference from a large proportion of the backgrounds, referred to as noisy data. Aiming to optimize the data augmentation and improve positive pairs, a Saliency-Augmented Module is proposed to obtain the augmented views, which only contain the ”latent” object area, referred to as clean data. Furthermore, a Saliency-Guided Self-Supervised Learning Network (SiSL-Net) is constructed as a new pattern of contrastive learning. A symmetric structure of trunk net and branch net is trained to learn a feature mapping from the clean data space and the noisy data space. Besides, a novel loss function is designed, including the embedding contrastive loss and distribution consistency loss, to optimize the feature representations during network training. The linear classification performance of our SiSL-Net is evaluated on the miniImageNet dataset with ResNet-50. Experiments show that our method achieves the top-1 accuracy from 64.67% to 69.02%, outperforming the state-of-the-art performance.

B-AT-KD: Binary attention map knowledge distillation

Convolutional neural networks (CNNs) have been extensively used in a number of applications and have shown to be quite effective. As the depth and width of the network expand, not only does prediction accuracy improve, but so does the network’s training complexity. To solve this problem, the knowledge distillation (KD) network, which consists of a teacher and a student network, is established. Furthermore, the widely accessible attention mechanism is important in the KD. Therefore, we build the Binary Attention Map Knowledge Distillation (B-AT-KD) model to enhance the performance of the student network by combining the output of the teacher network and associated attention maps with distinct semantics to supervise the student network. In addition, to improve training results, we propose a new global loss function called KDM-Loss and use hyperparameters search to assign suitable weights. Finally, we compare our B-AT-KD to state-of-the-art KD methods on the CIFAR 10, CIFAR 100, and Mini-ImageNet datasets. Experiments reveal that our proposed approach improves CIFAR 10 precision by 3.98%, Mini-ImageNet accuracy by 2.30%, and CIFAR 100 precision by 2.51% while reducing the number of parameters and computations by more than 50%.

A Compressed Model-Agnostic Meta-Learning Model Based on Pruning for Disease Diagnosis

Meta-learning has been widely used in medical image analysis. However, it requires a large amount of storage space and computing resources to train and use neural networks, especially model-agnostic meta-learning (MAML) models, making networks difficult to deploy on embedded systems and low-power devices for smart healthcare. Aiming at this problem, we explore to compress a MAML model with pruning methods for disease diagnosis. First, for each task, we find unimportant and redundant connections in MAML for its classification, respectively. Next, we find common unimportant connections for most tasks with intersections. Finally, we prune the common unimportant connections of the initial network. We conduct some experiments to assess the proposed model by comparison with MAML on Omniglot dataset and MiniImagenet dataset. The results show that our method reduces 40% parameters of the raw models, without incurring accuracy loss, demonstrating the potential of the proposed method for disease diagnosis.

Multi-layer noise reshaping and perceptual optimization for effective adversarial attack of images

Adversarial attack aims to fail the deep neural network by adding a small amount of perturbation to the input image, in which the attack success rate and resulting image quality are maximized under the lp norm perturbation constraint. However, the lp norm is not accurately correlated to human perception of image quality. Attack methods based on l0 norm constraint usually suffer from the high computational cost due to the iterative search for candidate pixels to modify. In this work, we explore how perceptual quality optimization can be incorporated into the adversarial attack design and propose a two-stage attack method to reshape the adversarial noise by an initial attack and optimize the visual quality of the attacked images without sacrificing the attack success rate. Specifically, we construct a visual attention network to generate a perceptual attention map to modulate the adversarial noise generated by a base attack method. The network is trained to maximize the visual quality in Structural Similarity Index Metric (SSIM) while achieving the same attack success rate. To improve the image perceptual quality further, we propose a fast search algorithm to perform an iterative block-wise pruning of the adversarial noise. We evaluate our method on the mini-ImageNet dataset against three different defense schemes. The results have demonstrated that our method can achieve better attack performance in image quality, attack success rate, and efficiency than the state-of-the-art attack methods.

Adaptive few-shot learning with a fair priori distribution

Few-shot learning is a special challenge in pattern recognition, which identifies unseen categories given only limited samples. In the past few years, various methods have been proposed to solve few-shot problems. Based on the predecessors, we discuss the prior influence of the training set bring into the model, the relationship between data quality and feature performance, as well as the method and reason to expand negative samples. We consequently improve the models and learning strategies in these aspects, then we obtain a novel framework named AFSL: Adaptive Few-Shot Learning, which can eliminate the impact of data sets on the model, and also performs well on several public datasets. Specifically, we achieved a percentage increase of verification accuracy of 7.8% and 1.5% for 1-shot and 5-shots tasks comparing to prototype network on mini-Imagenet dataset, 9.8% and 2.4% on tiered-Imagenet, and 28.4% and 14.0% on the CUB dataset.

Superclass-aware network for few-shot learning

Humans can learn to recognize a novel object by just going through its images a few times. It might because that they do not recognize the novel object purely on the visual information, but also based on their prior knowledge. Inspired from this, we propose a novel framework named Superclass-aware Network (Sup-Net) to tackle the few-shot learning problem. We first present a knowledge extraction schema in Sup-Net, which can acquire superclass information, and compute superclass semantic relations between different categories. We introduce a novel soft label supervised contrastive loss to help extract discriminative superclass features from images so that the superclass relation can be captured by these features. A novel model architecture that is jointly trained by images and prior knowledge has been proposed. The model encodes image features that minimize the cross-entropy loss at the category level, while it also extracts the superclass feature that minimizes the soft label contrastive loss at the superclass level. Experimental results demonstrate that Sup-Net achieves competitive results on miniImageNet datasets. In addition, we conduct experiments on a large-scale dataset tieredImageNet; the results further demonstrate the effectiveness of our Sup-Net.

Heterogeneous Few-Shot Learning for Hyperspectral Image Classification

Deep learning has achieved great success in hyperspectral image (HSI) classification. However, its success relies on the availability of sufficient training samples. Unfortunately, the collection of training samples is expensive, time-consuming, and even impossible in some cases. Natural image datasets that are different from HSI, such as Image Net and mini-ImageNet, have abundant texture and structure information. Effective knowledge transfer between two heterogeneous datasets can significantly improve the accuracy of HSI classification. In this letter, heterogeneous few-shot learning (HFSL) for HSI classification is proposed with only a few labeled samples per class. First, few-shot learning is performed on the mini-ImageNet datasets to learn the transferable knowledge. Then, to make full use of the spatial and spectral information, a spectral&#x2013;spatial fusion network is devised. Spectral information is obtained by the residual network with pure 1-D operators. Spatial information is extracted by a convolution network with pure 2-D operators, and the weights of the spatial network are initialized by the weights of the model trained on the mini-ImageNet datasets. Finally, few-shot learning is fine-tuned on HSI to extract discriminative spectral&#x2013;spatial features and individual knowledge, which can improve the classification performance of the new classification task. Experiments conducted on two public HSI datasets demonstrate that the HFSL outperforms the existing few-shot learning methods and supervised learning methods for HSI classification with only a few labeled samples. Our source code is available at <uri>https://github.com/Li-ZK/HFSL</uri>.

CDFKD-MFS: Collaborative Data-Free Knowledge Distillation via Multi-Level Feature Sharing

Recently, the compression and deployment of powerful deep neural networks (DNNs) on resource-limited edge devices to provide intelligent services have become attractive tasks. Although knowledge distillation (KD) is a feasible solution for compression, its requirement on the original dataset raises privacy concerns. In addition, it is common to integrate multiple pretrained models to achieve satisfactory performance. How to compress multiple models into a tiny model is challenging, especially when the original data are unavailable. To tackle this challenge, we propose a framework termed collaborative data-free knowledge distillation via multi-level feature sharing (CDFKD-MFS), which consists of a multi-header student module, an asymmetric adversarial data-free KD module, and an attention-based aggregation module. In this framework, the student model equipped with a multi-level feature-sharing structure learns from multiple teacher models and is trained together with a generator in an asymmetric adversarial manner. When some real samples are available, the attention module adaptively aggregates predictions of the student headers, which can further improve performance. We conduct extensive experiments on three popular computer visual datasets. In particular, compared with the most competitive alternative, the accuracy of the proposed framework is 1.18&#x0025; higher on the CIFAR-100 dataset, 1.67&#x0025; higher on the Caltech-101 dataset, and 2.99&#x0025; higher on the mini-ImageNet dataset.

P-PseudoLabel: Enhanced Pseudo-Labeling Framework With Network Pruning in Semi-Supervised Learning

Semi-supervised learning (SSL) methods for classification tasks exhibit a significant performance gain because they combine regularization and pseudo-labeling methods. General pseudo-labeling methods only depend on the model’s prediction when assigning pseudo-labels, but this approach often leads to the generation of incorrect pseudo-labels, due to the network being biased toward easy classes or to the presence of confusing samples in the training set, which further decreases model performance. To address this issue, we propose a novel pseudo-labeling framework that dramatically reduces the ambiguity of pseudo-labels for confusing samples in SSL. We operate our method, called Pruning for Pseudo-Label (P-PseudoLabel), using the Easy-to-Forget (ETF) Sample Finder, which compares the outputs of the model and the pruned model to identify confusing samples. Next, we perform negative learning using the confusing samples to decrease the risk of providing incorrect information and to improve performance. Our method achieves better performance than those of recent state-of-the-art SSL methods on the CIFAR-10, CIFAR-100, and Mini-ImageNet datasets, and is on par with the state-of-the-art methods on SVHN and STL-10.

Mutual Adversarial Training: Learning Together is Better Than Going Alone

Recent studies have shown that robustness to adversarial attacks can be transferred across deep neural networks. In other words, we can make a weak model more robust with the help of a strong teacher model. In this paper, we ask if models can &#x201C;learn together&#x201D; and &#x201C;teach each other&#x201D; to achieve better robustness instead of learning from a static teacher.We study how interactions among models affect robustness via knowledge distillation. We propose mutual adversarial training (MAT), in which multiple models are trained together and share the knowledge of adversarial examples to achieve improved robustness. MAT allows robust models to explore a larger space of adversarial samples and find more robust feature spaces and decision boundaries. Through extensive experiments on the CIFAR-10, CIFAR-100, and mini-ImageNet datasets, we demonstrate that MAT can effectively improve model robustness and outperform state-of-the-art methods under white-box attacks. In addition, we show that MAT can also mitigate the robustness trade-off among different perturbation types. Specially, we train specialist models that learn to defend a specific perturbation type and a generalist model that learns to defend multiple perturbation types by learning from the specialists, which brings as much as 13.4% accuracy gain to AT baselines against the union of <i>l</i>&#x221E;, <i>l</i>2, and <i>l</i>1 attacks. Our results show the superiority of the proposed method and demonstrate that collaborative learning is an effective strategy for designing robust models.

Efficient-PrototypicalNet with self knowledge distillation for few-shot learning

The focus of recent few-shot learning research has been on the development of learning methods that can quickly adapt to unseen tasks with small amounts of data and low computational cost. In order to achieve higher performance in few-shot learning tasks, the generalizability of the method is essential to enable it generalize well from seen tasks to unseen tasks with limited number of samples. In this work, we investigate a new metric-based few-shot learning framework which transfers the knowledge from another effective classification model to produce well generalized embedding and improve the effectiveness in handling unseen tasks. The idea of our proposed Efficient-PrototypicalNet involves transfer learning, knowledge distillation, and few-shot learning. We employed a pre-trained model as a feature extractor to obtain useful features from tasks and decrease the task complexity. These features reduce the training difficulty in few-shot learning and increase the performance. Besides that, we further apply knowledge distillation to our framework and achieve extra performance improvement. The proposed Efficient-PrototypicalNet was evaluated on five benchmark datasets, i.e., Omniglot, miniImageNet, tieredImageNet, CIFAR-FS, and FC100. The proposed Efficient-PrototypicalNet achieved the state-of-the-art performance on most datasets in the 5-way K-shot image classification task, especially on the miniImageNet dataset.

Few-shot learning approach for 3D defect detection in lithium battery

Detecting the surface defects in a lithium battery with an aluminium/steel shell is a difficult task. The effect of reflectivity, the limitation of acquiring the 3D information, and the shortage of massive amounts of labelled training data make the 2D detection method hard to classify surface defects. In this work, a few-shot learning approach for 3D defect detection in lithium batteries is proposed. The multi-exposure-based structured light method is introduced to reconstruct the 3D shape of the lithium battery. Then, the anomaly part of the 3D point cloud is transferred into 2D images by the height-gray transformation. The MiniImageNet datasets are used as the source domain to pretrain the Cross-Domain Few-Shot Learning (CD-FSL) model. The accuracy in our experiment result is 97.17%, which means that our method can be used to classify the surface defects of the lithium battery.

L2AE-D: Learning to Aggregate Embeddings for Few-shot Learning with Meta-level Dropout

Few-shot learning focuses on learning a new visual concept with very limited labelled examples. A successful approach to tackle this problem is to compare the similarity between examples in a learned metric space based on convolutional neural networks. However, existing methods typically suffer from meta-level overfitting due to the limited amount of training tasks and do not normally consider the importance of the convolutional features of different examples within the same channel. To address these limitations, we make the following two contributions: (a) We propose a novel meta-learning approach for aggregating useful convolutional features and suppressing noisy ones based on a channel-wise attention mechanism to improve class representations. The proposed model does not require fine-tuning and can be trained in an end-to-end manner. The main novelty lies in incorporating a shared weight generation module that learns to assign different weights to the feature maps of different examples within the same channel. (b) We also introduce a simple meta-level dropout technique that reduces meta-level overfitting in several few-shot learning approaches. In our experiments, we find that this simple technique significantly improves the performance of the proposed method as well as various state-of-the-art meta-learning algorithms. Applying our method to few-shot image recognition using Omniglot and miniImageNet datasets shows that it is capable of delivering a state-of-the-art classification performance.

Attribute-Guided Feature Learning for Few-Shot Image Recognition

Few-shot image recognition has become an essential problem in the field of machine learning and image recognition, and has attracted more and more research attention. Typically, most few-shot image recognition methods are trained across tasks. However, these methods are apt to learn an embedding network for discriminative representations of training categories, and thus could not distinguish well for novel categories. To establish connections between training and novel categories, we use attribute-related representations for few-shot image recognition and propose an attribute-guided two-layer learning framework, which is capable of learning general feature representations. Specifically, few-shot image recognition trained over tasks and attribute learning trained over images share the same network in a multi-task learning framework. In this way, few-shot image recognition learns feature representations guided by attributes, and is thus less sensitive to novel categories compared with feature representations only using category supervision. Meanwhile, the multi-layer features associated with attributes are aligned with category learning on multiple levels respectively. Therefore we establish a two-layer learning mechanism guided by attributes to capture more discriminative representations, which are complementary compared with a single-layer learning mechanism. Experimental results on CUB-200, AWA and MiniImageNet datasets demonstrate our method effectively improves the performance.

Multi-level Attention Feature Network for Few-shot Learning

Existing few-shot methods have problems that feature extraction scale is single, the learned class representations are inaccurate, the similarity calculation still relies on standard metrics. In order to solve the above problems, multi-level attention feature network is proposed. Firstly, the multiple scale images are obtained by scale processing, the features of multiple scale images are extracted and the image-level attention features are obtained by the image-level attention mechanism to fusion them. Then, class-level attention features are learned by using the class-level attention mechanism. Finally, the classification is performed by using the network to compute the similarity scores between features. The proposed method is evaluated on the Omniglot dataset and the MiniImagenet dataset. The experimental results show that multi-level attention feature network can further improve the classification accuracy under small sample conditions compared to the single-scale image features and average prototypes.

Multi-scale feature network for few-shot learning

Few-shot learning aims to learn a classifier that has good generalization performance in new classes, where each class only a small number of labeled examples are available. The existing few-shot classification methods use the single-scale image do not learn effective feature representation. Moreover, most of previous methods still depend on standard metrics to calculate visual similarities, such as Euclidean or cosine distance. Standard metrics are independent of data and lack nonlinear internal structure that captures the similarity between data. In this paper, we propose a new method for few-shot learning problem, which learns a multi-scale feature space, and classification is performed by computing similarities between the multi-scale representation of the image and the label feature of each class (i.e. class representation). Our method, called the Multi-Scale Feature Network (MSFN), is trained end-to-end from scratch. The proposed method improves 1-shot accuracy from 50.44% to 54.48% and 5-shot accuracy from 68.2% to 69.06% on MiniImagenet dataset compared to competing approaches. Experimental results on Omniglot, MiniImagenet, Cifar100, CUB200, and Caltech256 datasets demonstrate the effectiveness of the proposed method.

A Class Imbalance Loss for Imbalanced Object Recognition

The class imbalance problem exists widely in vision data. In these imbalanced datasets, the majority classes dominate the loss and influence the gradient. Hence, these datasets have a significantly negative impact on the performance of many state-of-the-art methods. In this article, we propose a class imbalance loss (CI loss) to handle this problem. To distinguish imbalanced datasets in accordance with the extent of imbalance, we also define an imbalance degree that works as a decision index factor in the CI loss. Because the minority classes with fewer samples probably lose chances in descending the gradient in the training process, CI loss is introduced to make these minority classes descend further than the majority classes. In view of the imbalanced distribution of data in few-shot learning, a method for generating an imbalanced few-shot learning dataset is presented in this article. We conducted a large number of experiments in the MiniImageNet dataset, which showed the effectiveness of an algorithm for model-agnostic metalearning for rapid adaptation with CI loss. In the problem of detecting 15 ship categories, our loss function is transplanted to a rotational region convolutional neural network detection method and a cascade network architecture and achieves higher mean average precision than focal loss and cross-entropy loss. In addition, the Mixed National Institute of Standards and Technology dataset and the Moving and Stationary Target Acquisition and Recognition dataset are sampled to imbalance datasets to verify the effectiveness of CI loss.

Multi-Scale Decision Network with Feature Fusion and Weighting for Few-Shot Learning

Learning from limited labelled examples is key a research hotspot with excellent scenarios and potential applications. Currently, most of metric learning-based few-shot models still have the problem of low recognition accuracy. This is mainly because that they only use the top-layer abstract feature with semantic information, which ignores the low-layer features that are also critical for the few-shot recognition. Therefore, the extracted features do not have abundant representation ability, and it is difficult to recognize easily confusing objects. Moreover, they usually adopt a fixed distance function or train a comparable network to measure features. These methods lack adaptability, cannot sufficiently fuse features, which leads to weaken the fitting ability of the metric function. And the same or different classes of images are treated equally, which makes the metric function have no emphasis point during training. To address these issues, we propose an end-to-end, metric learning-based model in this paper, called multi-scale decision network with feature fusion and weighting for few-shot learning (MSDN). Considering the importance of the low-layer features, we exploit a convolutional network to extract each layer feature. Then, we exploit a relation network to learn a non-linear metric between the support set and the query set features of each layer and classify the test images via a voting decision. During feature concatenation, we design a non-linear feature fusion item to improve the way of concatenation, so that the relation network can have a stronger function fitting ability to learn the relation score. Meanwhile, we introduce the attention mechanism by calculating the cosine similarity between the support set and the query set features as their weight, which makes the relation network pay more attention to the same class of images. Our model achieves the state-of-the-art accuracy result on Omniglot and miniImageNet datasets compared with popular few-shot recognition models.