Small Number Of Training Samples Research Articles

A Few-Shot Object Detection Method for Endangered Species

Endangered species detection plays an important role in biodiversity conservation and is significant in maintaining ecological balance. Existing deep learning-based object detection methods are overly dependent on a large number of supervised samples, and building such endangered species datasets is usually costly. Aiming at the problems faced by endangered species detection, such as low accuracy and easy loss of location information, an efficient endangered species detection method with fewer samples is proposed to extend the few-shot object detection technique to the field of endangered species detection, which requires only a small number of training samples to obtain excellent detection results. First, SE-Res2Net is proposed to optimize the feature extraction capability. Secondly, an RPN network with multiple attention mechanism is proposed. Finally, for the classification confusion problem, a weighted prototype-based comparison branch is introduced to construct weighted category prototype vectors, which effectively improves the performance of the original classifier. Under the setting of 30 samples in the endangered species dataset, the average detection accuracy value of the method, mAP50, reaches 76.54%, which is 7.98% higher than that of the pre-improved FSCE method. This paper also compares the algorithm on the PASCOL VOC dataset, which is optimal and has good generalization ability compared to the other five algorithms.

Fault diagnosis of drone motors driven by current signal data with few samples

Multi rotor unmanned aerial vehicles (UAVs) are extensively utilized across various domains, and the motor constitutes a pivotal element in the UAV power system. The majority of UAV failures and crashes stem from motor malfunctions, underscoring the imperative need for comprehensive research on fault diagnosis in UAV motors to ensure the stable and reliable execution of flight tasks. This study focuses on quadrotor UAVs as the research subject and devises targeted fault simulation experiments based on the structural features and operational characteristics of the DC brushless motor used in quadrotor UAVs, specifically examining the stator, rotor, and bearings. To address challenges related to the UAV’s own loads, limited space for redundant parts, and the high cost and difficulty associated with installing sensors for traditional fault diagnostic signals such as vibration and temperature, this study opts to use current signals as a substitute. This approach resolves the issue of challenging data collection for UAVs and investigates a current signal based fault diagnosis method for UAV motors. Lastly, in response to the limited training samples available for fault data due to the UAV’s highly sensitive characteristics regarding the health status of its components and flight stability, traditional machine learning and deep learning methods encounter difficulties in identifying representative features with a small number of training samples, leading to the risk of overfitting and reduced model accuracy in fault diagnosis. To overcome this challenge, we propose a hybrid neural network fault diagnosis model that incorporates a width learning system and a convolutional neural network (CNN). The width learning system eliminates temporal characteristics from the original current signal, capturing more comprehensive and representative sample features in the width feature space. Subsequently, the CNN is employed for feature extraction and classification tasks. In empirical small sample fault diagnosis experiments using current signal data for UAV motors, our proposed model outperforms other models used for comparison.

Semi-supervised vanishing point detection with contrastive learning

Vanishing point (VP) detection in road images plays an important role in driving scenes for advanced driver assistance systems (ADAS) and autonomous vehicles. Because it is still a challenging problem to obtain expensive annotated datasets in deep-learning-based supervised training, in this paper, we propose a semi-supervised VP detection method in road images. Our proposed model first extracts high-resolution VP heatmaps of the road images by fusing multi-level and global–local contrastive learning. Then, we apply the π model as a semi-supervised learning framework and feed the whole global network with a small number of training samples to detect VPs. In the experiment, we used Kong’s dataset for the unstructured road test and the KITTI-VP dataset for the structured road test. Compared with the state-of-the-art methods, our model achieved high accuracy and robustness in road VP detection.

Self-supervised learning backdoor defense mixed with self-attention mechanism

Recent studies have shown that Deep Neural Networks (DNNs) are vulnerable to backdoor attacks, where attackers embed hidden backdoors into the DNN models by poisoning a small number of training samples. The attacked models perform normally on benign samples, but when the backdoor is activated, their prediction results will be maliciously altered. To address the issues of suboptimal backdoor defense effectiveness and limited generality, a hybrid self-attention mechanism-based self-supervised learning method for backdoor defense is proposed. This method defends against backdoor attacks by leveraging the attack characteristics of backdoor threats, aiming to mitigate their impact. It adopts a decoupling approach, disconnecting the association between poisoned samples and target labels, and enhances the connection between feature labels and clean labels by optimizing the feature extractor. Experimental results on CIFAR-10 and CIFAR-100 datasets show that this method performs moderately in terms of Clean Accuracy (CA), ranking at the median level. However, it achieves significant effectiveness in reducing the Attack Success Rate (ASR), especially against BadNets and Blended attacks, where its defense capability is notably superior to other methods, with attack success rates below 2%.

Деформирующие преобразования изображений и их применение при аугментации данных для обучения глубоких нейронных сетей

The paper focuses on the improvement of the quality of learning for deep neural networks for a small data set in a classification task. One of the possible approaches to improve the quality of learning is researched which is based on the use of data augmentation (artificial reproduction of the data set) by image warping. The presented mathematical model and fast algorithm for warping make it possible to transform the original image while preserving its structural basis. The proposed algorithm is used to augment image data sets containing a small number of training samples. The augmentation consists of two stages including horizontal mirroring and warping of each of the samples. The effectiveness of such augmentation is tested through the training of neural networks of various types: convolutional neural networks (CNN) of a standard architecture and deep residual networks (DRN). A specific feature of the implemented approach for the solution of the problem under consideration consists in the refusal to use pre-trained neural networks with a large number of layers as well as further transfer learning, since their application incurs costs in terms of the computational resources. The paper shows that the efficiency of image classification when implementing the proposed method of augmenting training data on small and medium-sized data sets increases to statistically significant values of the metric used.

Does Few-Shot Learning Suffer from Backdoor Attacks?

The field of few-shot learning (FSL) has shown promising results in scenarios where training data is limited, but its vulnerability to backdoor attacks remains largely unexplored. We first explore this topic by first evaluating the performance of the existing backdoor attack methods on few-shot learning scenarios. Unlike in standard supervised learning, existing backdoor attack methods failed to perform an effective attack in FSL due to two main issues. Firstly, the model tends to overfit to either benign features or trigger features, causing a tough trade-off between attack success rate and benign accuracy. Secondly, due to the small number of training samples, the dirty label or visible trigger in the support set can be easily detected by victims, which reduces the stealthiness of attacks. It seemed that FSL could survive from backdoor attacks. However, in this paper, we propose the Few-shot Learning Backdoor Attack (FLBA) to show that FSL can still be vulnerable to backdoor attacks. Specifically, we first generate a trigger to maximize the gap between poisoned and benign features. It enables the model to learn both benign and trigger features, which solves the problem of overfitting. To make it more stealthy, we hide the trigger by optimizing two types of imperceptible perturbation, namely attractive and repulsive perturbation, instead of attaching the trigger directly. Once we obtain the perturbations, we can poison all samples in the benign support set into a hidden poisoned support set and fine-tune the model on it. Our method demonstrates a high Attack Success Rate (ASR) in FSL tasks with different few-shot learning paradigms while preserving clean accuracy and maintaining stealthiness. This study reveals that few-shot learning still suffers from backdoor attacks, and its security should be given attention.

A multi-view ensemble model based on semi-supervised feature learning for small sample classification of PolSAR images

ABSTRACT Classification with a few samples of training set has been a long-standing issue in the field of polarimetric synthetic aperture radar (PolSAR) image analysis and processing. In fact, one of the most important challenges of the PolSAR image classification task is the number of labelled samples. In essence, training network by utilizing just few numbers of training samples cannot be led to the reliable result since the neural network is so sensitive to the number of training samples. In addition, in the real PolSAR image classification task, the huge number of training samples is not accessible. On the other hand, classification performance using just small number of labelled samples is not an accurate result. So, in this paper, aiming at the small number of training samples of the PolSAR image classification task, a novel ensemble self-supervised feature-learning (ESSFL) model is designed. The designed ESSFL can automatically extract PolSAR features conducive to PolSAR image classification with a small number of training samples. In addition, it can significantly decrease the dependence of neural network algorithms on large labelled samples of training set. First, to utilize the spatial–polarimetric features of PolSAR data perfectly, the EfficientNet-B0 is presented and utilized as the main section of the deep learning (DL) model to extract DL features of PolSAR data. Then, using an optimization function that constrains the cross-correlation matrix of various distortions of each sample to the identity matrix, the designed deep learning model can obtain the effective features of homogeneous samples gathering together and heterogeneous samples separating from each other in a self-supervised manner. Moreover, two ensemble learning (EL) models, feature-level and view-level ensemble, are proposed to increase the feature extraction capability and classification result by jointly using spatial features at different scales and polarimetric information at different bands. Finally, the stack of the obtained features and the main polarimetric information of PolSAR data can be input into classifier for the classification of PolSAR data. In this paper we use two different classifiers, the first one is the Random Forest (RF) classifier and the second one is the Support Vector Machine (SVM) classifier. The advantages of SVM include that it can be used to avoid the difficulties of using linear functions in the high-dimensional feature space, and the optimization problem is transformed into dual convex quadratic programs that are so usable for an actual image classification task. In addition, among all the available classification methods, random forests provide one of the highest accuracies. The random forest technique can also handle big data with numerous variables running into thousands. In fact, it can automatically balance data sets when a class is more infrequent than other classes in the data that is so important for a real PolSAR image classification task. Experimental results on three well-known PolSAR data sets illustrate that the designed ESSFL can extract more discriminant features using the designed deep learning model. In the end, the experiments prove that the designed ESSFL has a significant classification performance compared with different deep learning models in the case of small number of training samples, and also it can achieve a better result in the case of large number of training samples by comparison with the most of the deep learning models.

The CRLB for target localization using Gaussian process regression with distance-dependent noise

The variance of distance-dependent noise exhibits different correlations with distance when the measurement method or device cost varies due to environmental interference. To efficiently model this correlation with less time cost, Gaussian Process Regression (GPR), which is characterized by its ability to learn nonlinear relationships with a small number of training samples, is utilized. Then, using the chain rule, the general CRLB can be applied under different measurement methods. It is represented by a function containing the GPR-based model and the relative position of the target and anchor. This means that the corresponding CRLB can be obtained quickly only by modeling the distance-dependent noise and determining measurement methods. Finally, to demonstrate the effectiveness of the GPR-based model and the validity of the derivation in this paper, two examples of time-of-arrival-based (TOA-based) and received signal strength-based (RSS-based) experiments are presented.

Deep learning with soft attention mechanism for small-scale ground roll attenuation

Ground roll is a type of coherent noise with low frequency, low velocity, and high amplitude, which masks useful signals and decreases the quality of subsequent seismic data processing. It is a challenge for traditional signal processing methods to separate useful signals effectively when the ground roll and useful reflected signals overlap seriously in the low-frequency band. We develop a supervised-learning-based framework with soft attention residual learning mechanisms for suppressing the ground roll noise. To reduce the cost of manual labeling, the 2D patching technique is used to segment large-scale seismic data into a large number of small-scale patches for training. Our network includes a multibranch attention block that uses multiple branches with different kernel sizes to extract waveform features at different scales from input noisy patches. Then, we use the soft attention mechanism to select and fuse the feature maps of different branches. Our network can achieve encouraging ground roll attenuation performance by using a small number of training samples, which is demonstrated by synthetic and field data examples. Compared with one traditional method and two advanced deep-learning frameworks, our network has better abilities in preserving low-frequency useful signals and removing ground roll.

Exploring visual representations of computer mouse movements for bot detection using deep learning approaches

Nowadays, bots can be seen everywhere on the Internet and are responsible for a large percentage of website traffic. The problem of bot detection has increasingly gained attention since more and more bots have been abused from click fraud in online advertisements to launching credential stuffing attacks for harvesting user accounts at a large scale. In this work, we present an end-to-end deep framework for bot detection based on computer mouse movements. Specifically, we propose a novel visual representation scheme that can simultaneously encode spatial and kinematic information in mouse movements into an image which can then be used as the input to Convolutional Neural Networks (CNN). Various strategies to encode kinematic features into images are investigated to obtain a better scheme of visual representation. Experimental results show that the proposed representation scheme in combination with CNN outperforms several baseline models with a TPR of 99.34% in detecting known bots and can be generalized to unknown bots with a highest accuracy of 99.20%. We also demonstrate that the proposed approach can reach acceptable performance levels even for models trained with a small number of training samples. This makes the deployment of our approach easier in real-world scenarios.

Galaxy mergers in Subaru HSC-SSP: A deep representation learning approach for identification, and the role of environment on merger incidence

Context.Galaxy mergers and interactions are an important process within the context of galaxy evolution, however, there is still no definitive method which identifies pure and complete merger samples is still not definitive. A method for creating such a merger sample is required so that studies can be conducted to deepen our understanding of the merger process and its impact on galaxy evolution.Aims.In this work, we take a deep-learning-based approach for galaxy merger identification in Subaru HSC-SSP, using deep representation learning and fine-tuning, with the aim of creating a pure and complete merger sample within the HSC-SSP survey. We can use this merger sample to conduct studies on how mergers affect galaxy evolution.Methods.We used Zoobot, a deep learning representation learning model pretrained on citizen science votes on Galaxy Zoo DeCALS images. We fine-tuned Zoobot for the purpose of merger classification of images of SDSS and GAMA galaxies in HSC-SSP public data release 3. Fine-tuning was done using ∼1200 synthetic HSC-SSP images of galaxies from the TNG simulation. We then found merger probabilities on observed HSC images using the fine-tuned model. Using our merger probabilities, we examined the relationship between merger activity and environment.Results.We find that our fine-tuned model returns an accuracy on the synthetic validation data of ∼76%. This number is comparable to those of previous studies in which convolutional neural networks were trained with simulation images, but with our work requiring a far smaller number of training samples. For our synthetic data, our model is able to achieve completeness and precision values of ∼80%. In addition, our model is able to correctly classify both mergers and non-mergers of diverse morphologies and structures, including those at various stages and mass ratios, while distinguishing between projections and merger pairs. For the relation between galaxy mergers and environment, we find two distinct trends. Using stellar mass overdensity estimates for TNG simulations and observations using SDSS and GAMA, we find that galaxies with higher merger scores favor lower density environments on scales of 0.5 to 8h−1Mpc. However, below these scales in the simulations, we find that galaxies with higher merger scores favor higher density environments.Conclusions.We fine-tuned a citizen-science trained deep representation learning model for purpose of merger galaxy classification in HSC-SSP, and make our merger probability catalog available to the public. Using our morphology-based catalog, we find that mergers are more prevalent in lower density environments on scales of 0.5–8h−1Mpc.

Prompt template construction by Average Gradient Search with External Knowledge for aspect sentimental analysis

Aspect-based Sentiment Analysis (ABSA) aims to predict the sentiment polarity towards a particular aspect in a sentence. Most of the existing methods construct neural networks or fine-tune the pre-trained language models. However, it is difficult to fully utilize the knowledge of language model learned during pretraining process, and it usually performs worse in few-shot experiment. Prompt learning can alleviate the above problems effectively. Manual prompt template construction methods are adopted usually but it leads to high costs and low efficiency, and there are seldom prompt templates applicable to ABSA task. We propose a method for ABSA prompt template construction, named Average Gradient Search with External Knowledge Based on KNN. The vocabulary list of the pre-trained language model is built as a KD-Tree, and the KNN algorithm is used to search the best prompt template on KD-Tree. For the sentiment polarity prediction, the best prompt template is used to wrap the ABSA data, converting the classification task to generative task, which enables the language model to fully leverage the learned knowledge from the pre-training stage. Moreover, the constructed verbalizer incorporates external knowledge to provide label words of each class with extensive semantic coverage and lower subjective bias. The comparison experimental results on SemEval2014 Restaurant and Laptop datasets demonstrate that the proposed method outperforms the existing SOTA models. Further, the few-shot experimental results indicate that, with only a small number of training samples, the proposed method achieves comparable or even better performance than the baseline model trained on the full dataset.

A novel self‐supervised ensemble learning framework for land use and land cover classification of polarimetric synthetic aperture radar images

AbstractClassification with a few samples of training set has been a longstanding issue in the field of polarimetric synthetic aperture radar (PolSAR) image analysis and processing. Aiming at the small number of training samples of the PolSAR image classification task, a novel Self‐Supervised Ensemble Learning Framework (SSELF) is designed. The designed SSELF can automatically extract PolSAR features conducive to PolSAR image classification with a small number of training samples. In addition, it can significantly decrease the dependence of neural network algorithms on large labelled samples of training set. First, utilise the spatial–polarimetric features of PolSAR data perfectly, the EfficientNet‐B0 is presented and utilised as the main section of the Deep Learning (DL) model to extract DL features of PolSAR data. Then, using an optimisation function that constrains the cross‐correlation matrix of various distortions of each sample to the identity matrix, the designed DL model can obtain the effective features of homogeneous samples gathering and heterogeneous samples separating from each other in a self‐supervised manner. Moreover, following the great success of curriculum learning in the area of machine learning, a novel deep curriculum‐learning model is proposed, entitled Deep Curriculum Learning (DCL), to train the DL network in our self‐supervised model. The proposed DCL utilises the entropy‐alpha target decomposition to estimate the degree of complexity of each PolSAR image patch before applying it to the EfficientNet‐B0. Also, an accumulative mini‐batch pacing function is used to introduce more difficult patches to EfficientNet‐B0 in the training process of the designed self‐supervised model. Furthermore, two EL models, feature‐level and view‐level ensemble, are proposed to increase the feature extraction capability and classification result of PolSAR data by jointly using spatial features at different scales and polarimetric information at different bands. In fact, in the proposed feature‐level ensemble strategy, to improve the classification result, the extracted features of the different scales are used as the input of the designed feature fusion algorithm. Therefore, the extracted features are mapped to the new space with lower dimension. In general, in the proposed view‐level ensemble strategy, first, the Mutual Information (MI) between each feature and the other features calculates, and then based‐on the calculated MI, different group of features are selected as various views. Next, the final classification result of the PolSAR image classification is obtained by using the designed majority vote of the result in each view. It should be noted that unlike existing traditional methods, both amplitude and phase information of SAR images to devolve the training process of the proposed model are used. In addition, all of the parameters of the proposed network are developed to a complex domain. In addition, a complex backpropagation algorithm by using the gradient‐based model is used for training the network. Finally, experimental results on three well‐known PolSAR data sets illustrate that the designed SSELF can extract more discriminant features using the designed DL model and can achieve better classification results than the other six novel DL models in terms of small training samples and adequate labelled samples.

Apple leaf disease recognition method based on Siamese dilated Inception network with less training samples

The application of existing deep learning networks may prove difficult when datasets are small. Siamese network can achieve better accuracy on small dataset. A recognition method of apple leaf disease based on Siamese dilated Inception network (SDINet) with few training samples is proposed. Dilated Inception module is introduced into AlexNet to construct two subnetworks for SDINet, and two subnetworks are responsible for extracting multi-scale features from image pair in the same layer, and the global pooling instead of the fully connected layers are utilized to reduce the number of model parameters and ensure that the features are not lost. SDINet is trained with image pairs, each pair consisting of two real diseased leaf images or one real and one healthy leaf image. SDINet makes full use of the advantages of multi-scale dilated Inception to enrich and improve the information, enhance the adaptability of the model. Different from the existing deep CNNs, SDINet uses cosine distance learning to calculate the similarity between the leaf image pairs to recognize apple diseases. Experimental results on the apple diseased leaf image dataset validate that the proposed method is effective to recognize apple leaf disease using a small number of training samples.

Exploring Prompts in Few-Shot Cross-Linguistic Topic Classification Scenarios

In recent years, large-scale pretrained language models have become widely used in natural language processing tasks. On this basis, prompt learning has achieved excellent performance in specific few-shot classification scenarios. The core idea of prompt learning is to convert a downstream task into a masked language modelling task. However, different prompt templates can greatly affect the results, and finding an appropriate template is difficult and time-consuming. To this end, this study proposes a novel hybrid prompt approach, which combines discrete prompts and continuous prompts, to motivate the model to learn more semantic knowledge from a small number of training samples. By comparing the performance difference between discrete prompts and continuous prompts, we find that hybrid prompts achieve the best results, reaching a 73.82% F1 value in the test set. In addition, we analyze the effect of different virtual token lengths in continuous prompts and hybrid prompts in a few-shot cross-language topic classification scenario. The results demonstrate that there is a threshold for the length of virtual tokens, and too many virtual tokens decrease the performance of the model. It is better not to exceed the average length of the training set corpus. Finally, this paper designs a method based on vector similarity to explore the real meanings represented by virtual tokens. The experimental results show that the prompt automatically learnt from the virtual token has a certain correlation with the input text.

Two-stage multi-dimensional convolutional stacked autoencoder network model for hyperspectral images classification

Deep learning models have been widely used in hyperspectral images classification. However, the classification results are not satisfactory when the number of training samples is small. Focused on above-mentioned problem, a novel Two-stage Multi-dimensional Convolutional Stacked Autoencoder (TMC-SAE) model is proposed for hyperspectral images classification. The proposed model is composed of two sub-models SAE-1 and SAE-2. The SAE-1 is a 1D autoencoder with asymmetric structre based on full connection layers and 1D convolution layers to reduce spectral dimensionality. The SAE-2 is a hybrid autoencoder composed of 2D and 3D convolution operations to extract spectral-spatial features from the reduced dimensionality data by SAE-1. The SAE-1 is trained with raw data by unsupervised learning and the encoder of SAE-1 is employed to reduce spectral dimensionality of raw data. The data after dimension reduction is used to train the SAE-2 by unsupervised learning. The fine-tuning of SAE-2 encoder and the training of classifier are implemented simultaneously with small number of samples by supervised learning. Comparative experiments are performed on three widely used hyperspectral remote sensing data. The extensive comparative experiments demonstrate that the proposed architecture can effectively extract deep features and maintain high classification accuracy with small number of training samples.

A comparison of data augmentation methods in voice pathology detection

To distinguish pathological voices from healthy voices, automatic voice pathology detection systems can be built using machine learning (ML) and deep learning (DL) techniques. To fully exploit such systems, large quantities of training data are typically required. The amount of training data is, however, small in the area of pathological voice, and therefore data augmentation (DA) becomes a potential technology to artificially increase the quantity of training data. This study presents a systematic comparison between various DA methods in the detection of pathological voice, including three time domain methods (noise addition, pitch shifting and time stretching), one time–frequency domain method (SpecAugment), and two vocoder-based methods (harmonic-to-noise ratio (HNR) modification and glottal pulse length modification). Detection systems were built using four popular spectral feature representations (static mel-frequency cepstral coefficients (MFCCs), dynamic MFCCs, spectrogram and mel-spectrogram). As classifiers, two widely used ML models (support vector machine (SVM) and random forest (RF)) and two DL models (long short-term memory (LSTM) network and convolutional neural network (CNN) with 1-dimensional (1-D) and 2-dimensional (2-D) architectures) were used. These systems were trained using a small number of training samples from two popular databases of pathological voice (HUPA and SVD) to find the best feature/classifier combination for each database. As a result, one ML-based detection system (mel-spectrogram/SVM for HUPA and SVD) and two DL-based detection systems (dynamic MFCCs/2-D CNN for HUPA and mel-spectrogram/2-D CNN for SVD) were selected for the comparison of the DA methods. The results show that by using DA in the system training, detection accuracy increased compared to the baseline systems that were trained without using DA. This improvement in accuracy was, however, clearly larger for the 2D-CNN system than for the SVM system. Furthermore, all six DA methods improved accuracy of the 2-D CNN system compared to the baseline system for both databases. The highest improvements were achieved using the time–frequency domain SpecAugment DA method, which improved accuracy by 1.5% and 3.8% (absolute) for the HUPA and SVD database, respectively.

Deep Multi-View Clustering Based on Reconstructed Self-Expressive Matrix

Deep Multi-view Subspace Clustering is a powerful unsupervised learning technique for clustering multi-view data, which has achieved significant attention during recent decades. However, most current multi-view clustering methods rely on learning self-expressive layers to obtain the ultimate clustering results, where the size of the self-expressive matrix increases quadratically with the number of input data points, making it difficult to handle large-scale datasets. Moreover, since multiple views are rich in information, both consistency and specificity of the input images need to be considered. To solve these problems, we propose a novel deep multi-view clustering approach based on the reconstructed self-expressive matrix (DCRSM). We use a reconstruction module to approximate self-expressive coefficients using only a small number of training samples, while the conventional self-expressive model must train the network with entire datasets. We also use shared layers and specific layers to integrate consistent and specific information of features to fuse information between views. The proposed DCRSM is extensively evaluated on multiple datasets, including Fashion-MNIST, COIL-20, COIL-100, and YTF. The experimental results demonstrate its superiority over several existing multi-view clustering methods, achieving an improvement between 1.94% and 4.2% in accuracy and a maximum improvement of 4.5% in NMI across different datasets. Our DCRSM also yields competitive results even when trained by 50% samples of the whole datasets.

Amortized Bayesian Meta-Learning with Accelerated Gradient Descent Steps

Recent meta-learning models often learn priors from observed tasks using a network optimized via stochastic gradient descent (SGD), which usually takes more training steps to convergence. In this paper, we propose an accelerated Bayesian meta-learning structure with a stochastic inference network (ABML-SIN). The proposed model aims to solve the training procedure of Bayesian meta-learning to improve the training speed and efficiency. Current approaches of meta-learning hardly converge within a few descent steps, owing to the small number of training samples. Therefore, we introduce an accelerated gradient descent learning network based on teacher–student architecture to learn the meta-latent variable θt for task t. With this amortized fast inference network, the meta-learner is able to learn the task-specific latent θt within a few training steps; thus, it improves the learning speed of the meta-learner. To refine the latent variables generated from the transductive amortization network of the meta-learner, SIN—followed by a conventional SGD-optimized network—is introduced as the student–teacher network to online-update the parameters. SIN extracts the local latent variables and accelerates the convergence of the meta-learning network. Our experiments on simulation data demonstrate that the proposed method provides generalization and scalability on unseen samples, and produces competitive/superior uncertainty estimations on few-shot learning tasks on two widely adopted 2D datasets with fewer training epochs compared to the state-of-the-art meta-learning approaches. Furthermore, the parameters generated by SIN act as perturbations on latent weights, enhancing the probability of accelerating the training efficiency of the meta-learner. Extensive qualitative experiments show that our method performs well across different meta-learning tasks in both simulated and real-world circumstances.

A Few-shot learning approach for Monkeypox recognition from a cross-domain perspective

Monkeypox is a zoonotic infectious skin disease initially endemic in Africa only. However, some countries are now beginning to report cases of apparent community transmission. In Computer Aided Diagnosis, deep learning has gained substantial improvement over traditional methods. Commonly, training a supervised deep model requires a large number of labeled samples. However, the collection and annotation of new disease images such as human monkeypox are time-consuming and expensive. Thus, we introduce a few-shot learning based approach for the recognition of human monkeypox in images. It requires merely a small number of training samples. In particular, it is a novel framework built with a normal backbone and auxiliary backbones. They are co-trained with Self-supervised Learning and Cross-domain Adaption techniques. The self-supervision penalty is used to help the auxiliary backbones effectively learn priors from source domain. The combined features across different domains are unified through a power transform layer. Extensive experiments are conducted on a task of recognizing chickenpox, measles, and human monkeypox diseases in a three-way few-shot manner. The results demonstrate that our method outperforms mainstream few-shot learning algorithms such as meta-learning based and fine-tuning based methods.