Meta-learning Methods Research Articles

Meta transfer evidence deep learning for trustworthy few-shot classification

Although the standard meta-learning methods have demonstrated strong performance in few-shot image classification scenarios, the models typically lack the capability to assess the reliability of their predictions, which can lead to risks in certain applications. Aiming at this problem, we first propose a meta-learning-based Evidential Deep Learning (EDL) called Meta Evidence Deep Learning (MetaEDL), which enables reliable prediction in the few-show image classification scenario. Being the same as general meta-learning methods, MetaEDL commonly employed shallow neural networks as feature extractors to avoid overfitting when dealing with few-shot samples, which significantly restricts the model’s ability to extract features. To further address this limitation, we propose a Meta Transfer Evidence Deep Learning (MetaTEDL) to address the few-shot trustworthy classification issue. MetaTEDL adopts a large-scale pre-trained neural network as its feature extractor. In the meta-training process, we only train two lightweight neuron operations Scaling and Shifting to reduce the risk of over-fitting. Then, two evidential head neural networks are trained to integrate evidence from different sources, aiming to improve the quality of the evidence output. We conduct comprehensive experiments on several challenging few-shot classification benchmarks. The results indicate that our proposed method not only outperforms other conventional meta-learning methods in terms of few-shot classification performance, but also has good UQ (uncertainty quantification), Uncertainty-guided active learning, and OOD (Out-of-Distribution) detection capabilities.

Semantic Interaction Meta-Learning Based on Patch Matching Metric.

Metric-based meta-learning methods have demonstrated remarkable success in the domain of few-shot image classification. However, their performance is significantly contingent upon the choice of metric and the feature representation for the support classes. Current approaches, which predominantly rely on holistic image features, may inadvertently disregard critical details necessary for novel tasks, a phenomenon known as "supervision collapse". Moreover, relying solely on visual features to characterize support classes can prove to be insufficient, particularly in scenarios involving limited sample sizes. In this paper, we introduce an innovative framework named Patch Matching Metric-based Semantic Interaction Meta-Learning (PatSiML), designed to overcome these challenges. To counteract supervision collapse, we have developed a patch matching metric strategy based on the Transformer architecture to transform input images into a set of distinct patch embeddings. This approach dynamically creates task-specific embeddings, facilitated by a graph convolutional network, to formulate precise matching metrics between the support classes and the query image patches. To enhance the integration of semantic knowledge, we have also integrated a label-assisted channel semantic interaction strategy. This strategy merges word embeddings with patch-level visual features across the channel dimension, utilizing a sophisticated language model to combine semantic understanding with visual information. Our empirical findings across four diverse datasets reveal that the PatSiML method achieves a classification accuracy improvement of 0.65% to 21.15% over existing methodologies, underscoring its robustness and efficacy.

Using the MSFNet Model to Explore the Temporal and Spatial Evolution of Crop Planting Area and Increase Its Contribution to the Application of UAV Remote Sensing

Remote sensing technology can be used to monitor changes in crop planting areas to guide agricultural production management and help achieve regional carbon neutrality. Agricultural UAV remote sensing technology is efficient, accurate, and flexible, which can quickly collect and transmit high-resolution data in real time to help precision agriculture management. It is widely used in crop monitoring, yield prediction, and irrigation management. However, the application of remote sensing technology faces challenges such as a high imbalance of land cover types, scarcity of labeled samples, and complex and changeable coverage types of long-term remote sensing images, which have brought great limitations to the monitoring of cultivated land cover changes. In order to solve the abovementioned problems, this paper proposed a multi-scale fusion network (MSFNet) model based on multi-scale input and feature fusion based on cultivated land time series images, and further combined MSFNet and Model Diagnostic Meta Learning (MAML) methods, using particle swarm optimization (PSO) to optimize the parameters of the neural network. The proposed method is applied to remote sensing of crops and tomatoes. The experimental results showed that the average accuracy, F1-score, and average IoU of the MSFNet model optimized by PSO + MAML (PSML) were 94.902%, 91.901%, and 90.557%, respectively. Compared with other schemes such as U-Net, PSPNet, and DeepLabv3+, this method has a better effect in solving the problem of complex ground objects and the scarcity of remote sensing image samples and provides technical support for the application of subsequent agricultural UAV remote sensing technology. The study found that the change in different crop planting areas was closely related to different climatic conditions and regional policies, which helps to guide the management of cultivated land use and provides technical support for the realization of regional carbon neutrality.

Contrastive prototype network with prototype augmentation for few-shot classification

In recent years, metric-based meta-learning methods have received widespread attention because of their effectiveness in solving few-shot classification problems. However, the scarcity of data frequently results in suboptimal embeddings, causing a discrepancy between anticipated class prototypes and those derived from the support set. These problems severely limit the generalizability of such methods, necessitating further development of Few-Shot Learning (FSL). In this study, we propose the Contrastive Prototype Network (CPN) consisting of three components: (1) Contrastive learning proposed as an auxiliary path to reduce the distance between homogeneous samples and amplify the differences between heterogeneous samples, thereby enhancing the effectiveness and quality of embeddings; (2) A pseudo-prototype strategy proposed to address the bias in prototypes, whereby the pseudo prototypes generated using query set samples are integrated with the initial prototypes to obtain more representative prototypes; (3) A new data augmentation technique, mixupPatch, introduced to alleviate the issue of insufficient data samples, whereby enhanced images are generated by blending the images and labels from different samples, to increase the number of samples. Extensive experiments and ablation studies conducted on five datasets demonstrated that CPN achieves robust results against recent solutions.

Image classification based on few-shot learning algorithms: a review

Image classification is a critical task in the field of computer vision, and its importance has significantly increased over the past few years. Machine learning and deep learning techniques have demonstrated immense potential in this field. However, traditional image classification models require a vast amount of training data, which can be challenging and expensive to obtain. To overcome this limitation, researchers are turning to few-shot learning, which aims to classify images with limited training samples. This paper presents a detailed analysis of the field of image classification using few-shot learning. First, it investigates the use of data augmentation, transfer learning, and meta-learning methods in this field. Then, it introduces several commonly used datasets and evaluation metrics in few-shot classification, compares several classical few-shot classification methods, and summarizes the experimental results obtained from public datasets. Finally, this paper analyzes the current challenges in few-shot image classification and suggests potential future directions.

Metric-based meta-learning combined with hyperspectral imaging for rapid detection of adulteration in domain-shifted camel milk powder

Camel milk powder possesses high nutritional and economic value. The adulteration of camel milk powder with other varieties seriously compromises consumer rights. In the practical application of hyperspectral analysis for camel milk powder detection, the sample categories used for testing often differ from those used to construct the model. As a learning approach adept at domain-shifted and few shot scenarios, meta-learning is employed to tackle this issue. In this study, we used camel milk powder adulterated with cow milk powder as training samples and adulterated with goat milk powder as test samples. In the detection of eleven adulteration levels, the detection accuracy for pure camel milk powder reached 98.92%. Notably, the detection accuracy for the less conspicuous 70% adulteration level achieved 77.69%. The comprehensive detection accuracy of meta-learning reached 84.4%, showcasing notable improvements compared to SVM, BP, and CNN, which saw increases of 24.67%, 28.16%, and 18.4%, respectively. The detailed analysis of feature vectors and contributions substantiates the reliability and stability of the meta-learning-based qualitative analysis. The introduction of meta-learning methods is poised to make significant contributions to rapid detection by relevant testing agencies and the protection of consumer rights.

Meta-learning Approaches for Few-Shot Learning: A Survey of Recent Advances

Despite its astounding success in learning deeper multi-dimensional data, the performance of deep learning declines on new unseen tasks mainly due to its focus on same-distribution prediction. Moreover, deep learning is notorious for poor generalization from few samples. Meta-learning is a promising approach that addresses these issues by adapting to new tasks with few-shot datasets. This survey first briefly introduces meta-learning and then investigates state-of-the-art meta-learning methods and recent advances in: (i) metric-based, (ii) memory-based, (iii), and learning-based methods. Finally, current challenges and insights for future researches are discussed.

Towards Domain-Aware Stable Meta Learning for Out-of-Distribution Generalization

Deep learning models are often trained on datasets that are limited in size and distribution, which may not fully represent the entire range of data encountered in practice. Thus, making deep learning models generalize to out-of-distribution data has received a significant amount of attention in recent studies due to the critical importance of this ability in real-world applications. Meta learning as an effective knowledge transfer paradigm, which learns a base model with high generalization ability to adapt to new data distributions by minimizing domain shifts across tasks during meta-training. However, most existing meta learning methods assume that the base model can access the labels of different domains, and this assumption is demanding in many real application scenarios. In addition, these methods focus on narrowing data-level domain shifts, while ignoring task-level domain shifts, which may lead to inadequate or even negative transfer. Inspired by human learners who use induction to learn and master new tasks, we propose a novel domain-aware meta learning framework for out-of-distribution generalization, termed SMLG. This framework enables the base model to generalize effectively to unseen domains without relying on domain-specific labels. Specifically, we develop a domain-aware transformation module to obtain meta representation and pseudo domain labels. As a result, the base model can be trained robustly without the need for direct domain label input. Furthermore, to investigate the impact of domain shifts at different levels, we introduce a joint loss function that combines cross-entropy with a domain alignment constraint. Extensive experiments on benchmark datasets demonstrate the efficacy of our framework.

An information fusion-based meta transfer learning method for few-shot fault diagnosis under varying operating conditions

In recent years, meta-learning has gained increasing attention in the field of fault diagnosis due to its advantages of handling small samples and exhibiting fast adaptation across different diagnostic tasks. However, the scenario of sharply varying operating conditions and the heavy computation burden still limit the effective application of meta-learning in the field of transfer fault diagnosis. To address the above challenges, a few-shot meta transfer diagnosis method is proposed based on information fusion-based model agnostic meta-learning (IFMAML). Firstly, the information enhancement method based on sparse principal component analysis is introduced to enhance the domain invariant features and reduce data redundancy. Subsequently, the information fusion strategy of multiple sensor data is proposed to form the red, green, and blue (RGB) channel information of images, which can enrich the diversity of domain invariant features and mine the spatial information of multiple sensors. Then, the IFMAML, with its enhanced potential for diagnostic performance and computational efficiency, is developed to address the challenging few-shot cross-domain transfer diagnosis under varying operating conditions. Finally, two case studies for gearbox fault diagnostics considering sharply varying speed conditions and unknown health conditions have been conducted to demonstrate the effectiveness and superiority of the proposed method. Experimental results have indicated that the proposed IFMAML method can achieve superior diagnostic accuracy and can be quickly adapted to new transfer diagnosis scenarios under varying operating conditions, when compared with several mainstream meta-learning methods.

Task-Similarity is a Crucial Factor for Few-Shot Meta-Learning of Structure-Activity Relationships.

Machine learning models support computer-aided molecular design and compound optimization. However, the initial phases of drug discovery often face a scarcity of training data for these models. Meta-learning has emerged as a potentially promising strategy, harnessing the wealth of structure-activity data available for known targets to facilitate efficient few-shot model training for the specific target of interest. In this study, we assessed the effectiveness of two different meta-learning methods, namely model-agnostic meta-learning (MAML) and adaptive deep kernel fitting (ADKF), specifically in the regression setting. We investigated how factors such as dataset size and the similarity of training tasks impact predictability. The results indicate that ADKF significantly outperformed both MAML and a single-task baseline model on the inhibition data. However, the performance of ADKF varied across different test tasks. Our findings suggest that considerable enhancements in performance can be anticipated primarily when the task of interest is similar to the tasks incorporated in the meta-learning process.

Unsupervised meta-learning with domain adaptation based on a multi-task reconstruction-classification network for few-shot hyperspectral image classification

Although the deep-learning method has achieved great success for hyperspectral image (HSI) classification, the few-shot HSI classification deserves sufficient study because it is difficult and expensive to acquire labeled samples. In fact, the meta-learning methods can improve the performance for few-shot HSI classification effectively. However, most of the existing meta-learning methods for HSI classification are supervised, which still heavily rely on the labeled data for meta-training. Moreover, there are many cross-scene classification tasks in the real world, and domain adaptation of unsupervised meta-learning has been ignored for HSI classification so far. To address the above issues, this paper proposes an unsupervised meta-learning method with domain adaptation based on a multi-task reconstruction-classification network (MRCN) for few-shot HSI Classification. MRCN does not need any labeled data for meta-training, where the pseudo labels are generated by multiple spectral random sampling and data augmentation. The meta-training of MRCN jointly learns a shared encoding representation for two tasks and domains. On the one hand, we design an encoder-classifier to learn the classification task on the source-domain data. On the other hand, we devise an encoder-decoder to learn the reconstruction task on the target-domain data. The experimental results on four HSI datasets demonstrate that MRCN preforms better than several state-of-the-art methods with only two to five labeled samples per class. To the best of our knowledge, the proposed method is the first unsupervised meta-learning method that considers the domain adaptation for few-shot HSI classification.

Successive model-agnostic meta-learning for few-shot fault time series prognosis

Meta learning is a promising technique for solving few-shot fault prediction problems, which have attracted the attention of many researchers in recent years. Existing meta-learning methods for time series prediction, which predominantly rely on random and similarity matching-based task partitioning, facing three major limitations: (1) feature exploitation inefficiency; (2) suboptimal task data allocation; and (3) limited robustness with small samples. To this end, we introduce a novel‘meta-task’ partitioning scheme, underpinned by a differential autoregressive algorithm, that treats a continuous time period of a time series as a meta-task, composed of multiple successive short time periods. This approach allows us to extract more comprehensive features and relationships from the data, resulting in more accurate predictions. Furthermore, our findings indicate that the differential autoregressive approach significantly bolsters robustness across diverse datasets. Extensive experiments on several fault and time series prediction datasets demonstrate that our approach substantially enhances prediction performance and generalization capability under both few-shot and general conditions.

Reweighted Regularized Prototypical Network for Few-Shot Fault Diagnosis.

In this article, we study the challenging few-shot fault diagnosis (FSFD) problem where limited faulty samples are available. Metric-based meta-learning methods have been a prevalent approach toward FSFD; however, most of them rely on learning a generalized distance metric and fall short of leveraging intraclass and interclass distribution information. To this end, we develop a novel reweighted regularized prototypical network to improve the performance of FSFD, where an intraclass reweighting strategy is proposed to reduce the influence of noise and outliers and obtain stable estimations of fault prototypes. In addition, a novel balance-enforcing regularization (BER) is proposed to hedge against the between-class imbalance and improve the discrimination capability. These two remedies help to reduce the intraclass difference and enlarge the interclass difference via episodic training. In this way, an improved metric space and a better diagnostic performance can be attained in a few-shot learning context. Case studies on the Tennessee Eastman benchmark process and a real-world railway turnout dataset demonstrate that the proposed FSFD approach compares favorably against state-of-the-art methodologies with desirable diagnostic performance.

Meta-Learning With Distributional Similarity Preference for Few-Shot Fault Diagnosis Under Varying Working Conditions.

Few-shot fault diagnosis is a challenging problem for complex engineering systems due to the shortage of enough annotated failure samples. This problem is increased by varying working conditions that are commonly encountered in real-world systems. Meta-learning is a promising strategy to solve this point, open issues remain unresolved in practical applications, such as domain adaptation, domain generalization, etc. This article attempts to improve domain adaptation and generalization by focusing on the distribution-shift robustness of meta-learning from the task generation perspective. In fact, few-shot fault diagnosis under varying working conditions allows to address the distribution shift problem in a natural way. An unsupervised across-tasks meta-learning strategy with distributional similarity preference is proposed, where the core is the distribution-distance-weighting mechanism. Differently from the naive random meta-train task generation strategy used in existing meta-learning methods, the source instances that present a more similar distribution with respect to the target instances gain larger weightings in the task generation. This strategy leads to a meta-task training set that is enough diverse, and at the same time can be easily learned due to the distribution similarity features of the source tasks. The proposed method introduces the concept of maximum mean discrepancy that is applied to derive the distribution distance of the measurements. Moreover, a model-agnostic meta-learning is applied to realize few-shot fault diagnosis under varying working conditions. The proposed solutions are verified and compared by considering two public datasets used for bearing fault diagnosis. The results show that the proposed strategy outperforms different related few-shot fault diagnosis methods under varying working conditions. Moreover, it is thus proved that, meta-learning with distribution similarity feature represents an effective approach for domain adaptation and generalization.

Multi-task convex combination interpolation for meta-learning with fewer tasks

Meta-learning methods try to enhance the generalization of the meta-learning model by various tasks. Diverse tasks can provide sufficient knowledge to assist the model in understanding and capturing different types of information. The sufficient number of meta-training tasks is a crucial factor in ensuring the generalization of meta-learning algorithms. However, obtaining a sufficient number of labeled meta-training tasks is often a challenging endeavor in real-world scenarios. Some scholars have addressed the issue of inadequate tasks only by interpolating between two tasks, but they do not take into account the problem of task diversity. To resolve this concern, we propose a Multi-task Convex Combination Interpolation (MCCI) method that simultaneously considers both task quantity and task diversity issues. First, we increase the number of tasks based on interpolation techniques, which provides the possibility to extend it from 2-task interpolation to n-task interpolation. Second, we randomly select multiple tasks and perform convex combination interpolation in the n-task space, in order to increase the task diversity. Third, we prove the positive effect of the convex combination interpolation on the generalization ability and noise resistance of meta-learning algorithms in theory. Finally, we verify the generalization ability and noise resistance of the proposed MCCI on seven datasets. The extensive experimental results show the superior performance of the proposed method compared to the state-of-art methods.

The meta-learning method for the ensemble model based on situational meta-task.

The meta-learning methods have been widely used to solve the problem of few-shot learning. Generally, meta-learners are trained on a variety of tasks and then generalized to novel tasks. However, existing meta-learning methods do not consider the relationship between meta-tasks and novel tasks during the meta-training period, so that initial models of the meta-learner provide less useful meta-knowledge for the novel tasks. This leads to a weak generalization ability on novel tasks. Meanwhile, different initial models contain different meta-knowledge, which leads to certain differences in the learning effect of novel tasks during the meta-testing period. Therefore, this article puts forward a meta-optimization method based on situational meta-task construction and cooperation of multiple initial models. First, during the meta-training period, a method of constructing situational meta-task is proposed, and the selected candidate task sets provide more effective meta-knowledge for novel tasks. Then, during the meta-testing period, an ensemble model method based on meta-optimization is proposed to minimize the loss of inter-model cooperation in prediction, so that multiple models cooperation can realize the learning of novel tasks. The above-mentioned methods are applied to popular few-shot character datasets and image recognition datasets. Furthermore, the experiment results indicate that the proposed method achieves good effects in few-shot classification tasks. In future work, we will extend our methods to provide more generalized and useful meta-knowledge to the model during the meta-training period when the novel few-shot tasks are completely invisible.

Comprehensive analysis of machine learning models for predicting concrete compressive strength

Predicting the compressive strength of concrete provides several benefits including insights into its durability, potential lifespan, and its suitability for specific construction projects. Recent advances in machine learning algorithms have prompted their application to compressive strength modeling and prediction. In this paper, we carry out a comprehensive comparative analysis of the existing machine learning algorithms. Unlike, other current studies in this domain, we consider the entire spectrum of algorithms ranging from basic linear models to advanced meta-learning methods. The results show that there exists a trade-off between speed and accuracy of the models. We find that while ensemble models such as stacking and gradient boost provide superior accuracy, they are significantly slower than linear models.

Few-Shot Classification of Wafer Bin Maps Using Transfer Learning and Ensemble Learning

Abstract The high cost of collecting and annotating wafer bin maps (WBMs) necessitates few-shot WBM classification, i.e., classifying WBM defect patterns using a limited number of WBMs. Existing few-shot WBM classification algorithms mainly utilize meta-learning methods that leverage knowledge learned in several episodes. However, meta-learning methods require a large amount of additional real WBMs, which can be unrealistic. To help train a network with a few real-WBMs while avoiding this challenge, we propose the use of simulated WBMs to pre-train a classification model. Specifically, we employ transfer learning by pre-training a classification network with sufficient amounts of simulated WBMs and then fine-tuning it with a few real-WBMs. We further employ ensemble learning to overcome the overfitting problem in transfer learning by fine-tuning multiple sets of classification layers of the network. A series of experiments on a real-dataset demonstrate that our model outperforms the meta-learning methods that are widely used in few-shot WBM classification. Additionally, we empirically verify that transfer and ensemble learning, the two most important yet simple components of our model, reduce the prediction bias and variance in few-shot scenarios without a significant increase in training time.

Few-shot classification of ultrasound breast cancer images using meta-learning algorithms

Medical datasets often have a skewed class distribution and a lack of high-quality annotated images. However, deep learning methods require a large amount of labeled data for classification. In this study, we present a few-shot learning approach for the classification of ultrasound breast cancer images using meta-learning methods. We used prototypical networks and model agnostic meta-learning (MAML) algorithms as meta-learning methods. The breast ultrasound images (BUSI) dataset, which has three classes and is difficult to use in meta-learning, was used for meta-testing in a cross-domain approach along with other datasets for meta-training. Our proposed approach yielded an accuracy range of 0.882–0.889, achieved by implementing the ResNet50 backbone with ProtoNet in a 10-shot setting. These results represent a significant improvement ranging from 6.27 to 7.10% over the baseline accuracy of 0.831. The results showed that ProtoNet outperformed the MAML method for all k-shot settings. In addition, the use of ResNet models as the backbone network for feature extraction was found to be more successful than the use of a four-layer convolutional model. Our proposed method is the first attempt to apply meta-learning for few-shot classification in the BUSI dataset while providing higher accuracy compared to deep learning methods for medical images with small-scale datasets and few classes. The methodology used in this study can be adapted to other datasets with similar problems.

Improving cross-lingual low-resource speech recognition by Task-based Meta PolyLoss

Multilingual meta learning has emerged as a promising paradigm for transferring knowledge from source languages to facilitate the learning of low-resource target languages. Loss functions are a type of meta-knowledge that is crucial to the effective training of neural networks. However, the misalignment between the loss functions and the learning paradigms of meta learning degrades the network’s performance. To address this challenge, we propose a new method called Task-based Meta PolyLoss (TMPL) for meta learning. By regarding speech recognition tasks as normal samples and applying PolyLoss to the meta loss function, TMPL can be denoted as a linear combination of polynomial functions based on task query loss. Theoretical analysis shows that TMPL improves meta learning by enabling attention adjustment across different tasks, which can be tailored for different datasets. Experiments on three datasets demonstrated that gradient-based meta learning methods achieve superior performance with TMPL. Furthermore, our experiments validate that the task-based loss function effectively mitigates the misalignment issue.