Model-Agnostic Meta-Learning Research Articles

Learn to Adapt for Self-Supervised Monocular Depth Estimation.

Monocular depth estimation is one of the fundamental tasks in environmental perception and has achieved tremendous progress by virtue of deep learning. However, the performance of trained models tends to degrade or deteriorate when employed on other new datasets due to the gap between different datasets. Though some methods utilize domain adaptation technologies to jointly train different domains and narrow the gap between them, the trained models cannot generalize to new domains that are not involved in training. To boost the transferability of self-supervised monocular depth estimation models and mitigate the issue of meta-overfitting, we train the model in the pipeline of meta-learning and propose an adversarial depth estimation task. We adopt model-agnostic meta-learning (MAML) to obtain universal initial parameters for further adaptation and train the network in an adversarial manner to extract domain-invariant representations for easing meta-overfitting. In addition, we propose a constraint to impose upon cross-task depth consistency to compel the depth estimation to be identical in different adversarial tasks, which improves the performance of our method and smoothens the training process. Experiments on four new datasets demonstrate that our method adapts quite fast to new domains. Our method trained after 0.5 epoch achieves comparable results with the state-of-the-art methods trained at least 20 epochs.

The Research of Intra-Pulse Modulated Signal Recognition of Radar Emitter under Few-Shot Learning Condition Based on Multimodal Fusion

Radar radiation source recognition is critical for the reliable operation of radar communication systems. However, in increasingly complex electromagnetic environments, traditional identification methods face significant limitations. These methods often struggle with high noise levels and diverse modulation types, making it difficult to maintain accuracy, especially when the Signal-to-Noise Ratio (SNR) is low or the available training data are limited. These difficulties are further intensified by the necessity to generalize in environments characterized by a substantial quantity of noisy, low-quality signal samples while being constrained by a limited number of desirable high-quality training samples. To more effectively address these issues, this paper proposes a novel approach utilizing Model-Agnostic Meta-Learning (MAML) to enhance model adaptability in few-shot learning scenarios, allowing the model to quickly learn with limited data and optimize parameters effectively. Furthermore, a multimodal fusion neural network, DCFANet, is designed, incorporating residual blocks, squeeze and excitation blocks, and a multi-scale CNN, to fuse I/Q waveform data and time–frequency image data for more comprehensive feature extraction. Our model enables more robust signal recognition, even when the signal quality is severely degraded by noise or when only a few examples of a signal type are available. Testing on 13 intra-pulse modulated signals in an Additive White Gaussian Noise (AWGN) environment across SNRs ranging from −20 to 10 dB demonstrated the approach’s effectiveness. Particularly, under a 5−way5−shot setting, the model achieves high classification accuracy even at −10dB SNR. Our research underscores the model’s ability to address the key challenges of radar emitter signal recognition in low-SNR and data-scarce conditions, demonstrating its strong adaptability and effectiveness in complex, real-world electromagnetic environments.

MetaFollower: Adaptable personalized autonomous car following

Car-following (CF) modeling, a fundamental component in microscopic traffic simulation, has attracted increasing research interest in recent decades. In this study, we propose an adaptable personalized car-following framework —– MetaFollower, by leveraging the power of meta-learning. Specifically, we first utilize Model-Agnostic Meta-Learning (MAML) to extract common driving knowledge from various CF events. Afterward, the pre-trained model can be fine-tuned on new drivers with only a few CF trajectories to achieve personalized CF adaptation. We additionally combine Long Short-Term Memory (LSTM) and Intelligent Driver Model (IDM) to reflect temporal heterogeneity with high interpretability. MetaFollower can accurately capture and simulate the intricate dynamics of car-following behavior while considering the unique driving styles of individual drivers. We demonstrate the versatility and adaptability of MetaFollower by showcasing its ability to adapt to new drivers with limited training data quickly. To evaluate the performance of MetaFollower, we conduct rigorous experiments comparing it with both data-driven and physics-based models. The results reveal that our proposed framework outperforms baseline models in predicting car-following behavior with higher accuracy and safety. To the best of our knowledge, this is the first car-following model aiming to achieve fast adaptation by considering both driver and temporal heterogeneity based on meta-learning.

Federated Model-Agnostic Meta-Learning With Sharpness-Aware Minimization for Internet of Things Optimization

Federated Model-Agnostic Meta-Learning With Sharpness-Aware Minimization for Internet of Things Optimization

Novel meta-learning for few-shot bearing fault diagnosis under varying working conditions

In practical engineering, large amount data and variable working conditions poses a challenge to most existing Deep Learning(DL) methods. To solve this problem, this paper proposes a new meta-learning approach. Under the condition of limited data, the fault diagnosis under variable working conditions is regarded as a problem with fewer lenses, and the fault diagnosis of few samples across working scenes is carried out based on the Model-Agnostic Meta-Learning(MAML). Gradient-by-gradient rules are used for parameter optimization to achieve an efficient representation of these tasks. Then, the attention mechanism is applied to improve the efficiency of the training. Finally, experiments verified the fault diagnosis accuracy under various working conditions.

Predicting heave and pitch motions of an FPSO using meta-learning

Real-time motion prediction is helpful in guaranteeing the operation stability of a Floating Production Storage Offloading (FPSO) unit. Recurrent neural networks (RNNs) are becoming feasible alternatives to numerical simulations for motion prediction as artificial intelligence develops rapidly. In this study, model-agnostic meta-learning (MAML) is combined with RNNs to deterministically predict the heave and pitch motions of a ship-shaped FPSO. This approach is motivated by the fact that MAML improves training efficiency without losing accuracy. The data came from a scaled model test conducted at Shanghai Jiao Tong University’s deepwater wave basin. Before introducing MAML, we verified that long short-term memory (LSTM) and gated recurrent unit (GRU) could accurately predict the heave and pitch of about 7.68 s into the future. With fewer learnable parameters than LSTM, GRU demonstrates slightly better accuracy. Therefore, this study focuses particularly on the combination of GRU and MAML. The parameters of MAML, including order of derivative, step size, number of adaption gradient updates, and batch size of the tasks, are evaluated systemically in terms of accuracy and training efficiency. With the assistance of MAML, GRU’s training efficiency for heave and pitch has significantly improved, increasing by approximately 65% and 55%, respectively. Meanwhile, the prediction error for both has decreased by about 10%. Notably, MAML’s performance is minimally affected by variations in incoming wave direction and sea state, as well as the randomness and temporal variability of the motion. MAML is a powerful tool that enables RNNs to achieve real-time prediction of FPSO motion.

Anomaly detection for multivariate time series in IoT using discrete wavelet decomposition and dual graph attention networks

Effective anomaly detection in multivariate time series data is critical to ensuring the security of Internet of Things (IoT) devices and systems. However, building a high precision and low false positive rate anomaly detection model for the complex and volatile IoT environment is a challenging task. This is often due to issues such as a lack of anomaly labeling, high data volatility, and the complexity of device mechanisms. Traditional machine learning algorithms and sequence models frequently fail to account for feature correlation and temporal dependency in anomaly detection. Although deep learning-based anomaly detection methods have progressed, there is still room for improvement in precision, recall, and generalization ability. In this paper, we propose an anomaly detection model called Meta-MWDG to address these issues. The model is based on a multi-scale discrete wavelet decomposition and a dual graph attention network, which can effectively extract feature correlation and temporal dependency in multivariate time series data. Additionally, model-agnostic meta-learning (MAML) is introduced to improve the model’s generalization performance, enabling it to perform well on new tasks even with a few samples. A gated recurrent unit (GRU) is combined with a multi-head self-attention network to output both prediction and reconstruction results in a joint optimization strategy, improving the precision of anomaly detection. Extensive experimental studies demonstrate that Meta-MWDG outperforms the state-of-the-art methods in anomaly detection.

IMAML-IDCG: Optimization-based meta-learning with ImageNet feature reusing for few-shot invasive ductal carcinoma grading

Model-Agnostic Meta-Learning (MAML) is a widely used few-shot learning (FSL) technique that reduces reliance on large, labeled datasets in deep learning for medical imaging analysis. However, MAML requires backpropagating through all feature layers for task adaptation, leading to suboptimal computational efficiency. We propose IMAML-IDCG (ImageNet Model-Agnostic Meta-Learning in Invasive Ductal Carcinoma Grading), which enhances computational efficiency for few-shot grading of Invasive Ductal Carcinoma (IDC) through three key techniques: (1) ImageNet feature reusing, (2) ImageNet partial freezing strategy, and (3) adaptive inner learning rate. IMAML-IDCG is initialized with ImageNet pre-trained weights. During the inner optimization loop, only the model’s classifier head layer is optimized, leveraging prior ImageNet knowledge (ImageNet feature reusing) and employing an adaptive learning rate for improved task adaptation. In the outer optimization loop, IMAML-IDCG selectively fine-tunes the last few model layers to enhance efficiency and reduce overfitting (ImageNet partial freezing strategy). We evaluated IMAML-IDCG using the BreaKHis dataset (7,909 images) as the base dataset, and the BCHI (282 images) and PathoIDCG (3,744 images) datasets as the novel datasets. Our empirical results demonstrate that IMAML-IDCG outperforms MAML and other FSL methods in few-shot IDC grading tasks across various cross-magnification domain settings. Notably, IMAML-IDCG achieves a 14.64% improvement over MAML on the BCHI dataset and a 6.04% improvement on the PathoIDCG 40X dataset when meta-trained with the BreaKHis 40X dataset in the 3-way 5-shot scenario.

A mutual reconstruction network model for few-shot classification of histological images: addressing interclass similarity and intraclass diversity.

The automated classification of histological images is crucial for the diagnosis of cancer. The limited availability of well-annotated datasets, especially for rare cancers, poses a significant challenge for deep learning methods due to the small number of relevant images. This has led to the development of few-shot learning approaches, which bear considerable clinical importance, as they are designed to overcome the challenges of data scarcity in deep learning for histological image classification. Traditional methods often ignore the challenges of intraclass diversity and interclass similarities in histological images. To address this, we propose a novel mutual reconstruction network model, aimed at meeting these challenges and improving the few-shot classification performance of histological images. The key to our approach is the extraction of subtle and discriminative features. We introduce a feature enhancement module (FEM) and a mutual reconstruction module to increase differences between classes while reducing variance within classes. First, we extract features of support and query images using a feature extractor. These features are then processed by the FEM, which uses a self-attention mechanism for self-reconstruction of features, enhancing the learning of detailed features. These enhanced features are then input into the mutual reconstruction module. This module uses enhanced support features to reconstruct enhanced query features and vice versa. The classification of query samples is based on weighted calculations of the distances between query features and reconstructed query features and between support features and reconstructed support features. We extensively evaluated our model using a specially created few-shot histological image dataset. The results showed that in a 5-way 10-shot setup, our model achieved an impressive accuracy of 92.09%. This is a 23.59% improvement in accuracy compared to the model-agnostic meta-learning (MAML) method, which does not focus on fine-grained attributes. In the more challenging, 5-way 1-shot setting, our model also performed well, demonstrating a 18.52% improvement over the ProtoNet, which does not address this challenge. Additional ablation studies indicated the effectiveness and complementary nature of each module and confirmed our method's ability to parse small differences between classes and large variations within classes in histological images. These findings strongly support the superiority of our proposed method in the few-shot classification of histological images. The mutual reconstruction network provides outstanding performance in the few-shot classification of histological images, successfully overcoming the challenges of similarities between classes and diversity within classes. This marks a significant advancement in the automated classification of histological images.

Bayesian optimization-based Model-Agnostic Meta-Learning: Application to predict maximum cyclic moment resistance of steel bolted T-stub connections

Accurately assessing the maximum moment resistance of steel bolted T-stub connections under cyclic loading is crucial for designing earthquake-resistant structures with such connections. Traditional methods based on design standards like ASCE 41-17 often lack precision. Recently, supervised machine learning techniques, particularly Artificial Neural Network (ANN), have been explored. However, conventional ANNs require substantial data for generalization, which is limited for steel bolted T-stub connections. To address these challenges, this study explores the feasibility of using the Model-Agnostic Meta-Learning (MAML) to predict the maximum cyclic moment resistance of steel bolted T-stub connections. MAML adapts task-specific model parameters rapidly and transfers knowledge across tasks to fine-tune global model parameters, potentially enhancing prediction accuracy with limited data. The MAML model is first optimized using Bayesian optimization with a Gaussian Process model to identify ideal hyperparameters. The optimized MAML model is then compared with two ANN models (one with optimized hyperparameters, another matching MAML's neural network architecture) and ASCE 41-17 method. Results demonstrate the optimized MAML model's superior generalization capabilities, offering a promising approach for steel bolted T-stub connections.

Subgraph-level federated graph neural network for privacy-preserving recommendation with meta-learning

Graph neural networks (GNN) are widely used in recommendation systems, but traditional centralized methods raise privacy concerns. To address this, we introduce a federated framework for privacy-preserving GNN-based recommendations. This framework allows distributed training of GNN models using local user data. Each client trains a GNN using its own user–item graph and uploads gradients to a central server for aggregation. To overcome limited data, we propose expanding local graphs using Software Guard Extension (SGX) and Local Differential Privacy (LDP). SGX computes node intersections for subgraph exchange and expansion, while local differential privacy ensures privacy. Additionally, we introduce a personalized approach with Prototype Networks (PN) and Model-Agnostic Meta-Learning (MAML) to handle data heterogeneity. This enhances the encoding abilities of the federated meta-learner, enabling precise fine-tuning and quick adaptation to diverse client graph data. We leverage SGX and local differential privacy for secure parameter sharing and defense against malicious servers. Comprehensive experiments across six datasets demonstrate our method’s superiority over centralized GNN-based recommendations, while preserving user privacy.

The Research of Multi-Node Collaborative Compound Jamming Recognition Algorithm Based on Model-Agnostic Meta-Learning and Time-Frequency Analysis

Deep learning has presented its spectacular potential in the jamming recognition field. Yet, sufficient samples required by normal deep learning analysis methods are not always available, especially in the 6G communication field. This situation appears to be more challenging in the communication field. In this article, Model-Agnostic Meta-Learning (MAML) is imported into the jamming recognition field in order to accomplish compound jamming recognition in the circumstances of few-shot learning. Further, the existing research on jamming recognition techniques is mostly based on single-node recognition. This technique cannot make full and efficient use of the jamming information collected. Therefore, this article adds a multi-node collaborative technique into the compound jamming recognition algorithm that is based on MAML and time-frequency analysis. Based on the fact that each cognitive node can recognize independently, the recognition results are be sent to the fusion center. The fusion center completes the fusion of the recognition results according to the majority rule. The experiments demonstrate that, with the fusion of the multi-node collaborative technique, the precision of compound jamming recognition in the condition of few-shot learning has been effectively improved.

Zero-shot classification of small target on sea bottom using model-agnostic meta-learning.

A model-agnostic meta-learning (MAML)-based active target classifier to identify small targets (e.g., mines) on the sea bottom in different ocean environments from those present in the training data is proposed. To better classify the targets deviating from those in the training set, MAML is applied to the out-of-distribution samples. Frequency-domain target and clutter scattering signals from various tasks with varying bottom types (silt/clay) and incident angles (low/moderate/high) are utilized as training data samples. MAML significantly outperforms conventional neural networks during the test. The improved generalization of MAML is explained using loss landscape in the form of a smooth convex curve.

MAMLCDA: A Meta-Learning Model for Predicting circRNA-Disease Association Based on MAML Combined With CNN.

Circular RNAs (circRNAs) exist in vivo and are a class of noncoding RNA molecules. They have a single-stranded, closed, annular structure. Many studies have shown that circRNAs and diseases are linked. Therefore, it is critical to build a reliable and accurate predictor to find the circRNA-disease association. In this paper, we presented a meta-learning model named MAMLCDA to identify the circRNA-disease association, which is based on model-agnostic meta-learning (MAML) combined with CNN classification. Specifically, similarities between diseases and circRNAs are extracted and integrated to characterize their relationships, and k-means is used to cluster majority samples and select a certain number of samples from each cluster to obtain the same number of negative samples as the positive samples. To further reduce the dimension of the features and save operation time, we applied probabilistic principal component analysis (PPCA) to compact the integrated circRNA and disease similarity network feature vectors. The feature vectors are converted into images. At this time, the prediction problem is transformed into the 2-way 1-shot problem of the image and input into the model with MAML as the meta-learner and CNN as the base-learner. Comparison results of five-fold cross-validation on two benchmark datasets illustrate that MAMLCDA outperforms several state-of-the-art approaches with the best accuracies of 95.33% and 98%. Therefore, MAMLCDA can help to understand the pathogenesis of complex diseases at the circRNA level.

Meta-LSTM in hydrology: Advancing runoff predictions through model-agnostic meta-learning

In the field of hydrology, deep learning has become a prevalent tool for runoff simulation. However, the limitations stem from their primary focus on normality, which fails to accurately capture rare events. This study introduces a novel Long Short-Term Memory (LSTM) model, termed Meta-LSTM, which is based on the Model-Agnostic Meta-Learning (MAML) framework. The Meta-LSTM model is capable of dynamically fine-tuning its parameters to ensure effective adaptation to various runoff scenarios. From this Meta-LSTM model, the Meta-enhanced paradigm is subsequently proposed, ensuring effective adaptation to diverse models and datasets. We conducted experiments with various models on the CAMELS and CAMELS-AUS datasets. Compared to conventional deep learning models for runoff simulation, Meta-enhanced models exhibit substantial improvements. Specifically, the Meta-LSTM model reduces the PBIAS from −13.86 % to −5.18 %, increasing the Kling-Gupta Efficiency (KGE) by 7 % on the CAMELS dataset. For the Meta-GR4J-CNN model, the PBIAS decreases by 21 %, and the Nash-Sutcliffe Efficiency (NSE) increases by 2 % on the CAMELS-AUS dataset. This demonstrates the ability of the Meta-enhanced model to more accurately represent observed data without significant additional time costs. Our approach overcomes the problem of uniform training in simulations of complex scenarios and promises to significantly improve the accuracy of future hydrologic studies.

Task-Similarity is a Crucial Factor for Few-Shot Meta-Learning of Structure-Activity Relationships.

Machine learning models support computer-aided molecular design and compound optimization. However, the initial phases of drug discovery often face a scarcity of training data for these models. Meta-learning has emerged as a potentially promising strategy, harnessing the wealth of structure-activity data available for known targets to facilitate efficient few-shot model training for the specific target of interest. In this study, we assessed the effectiveness of two different meta-learning methods, namely model-agnostic meta-learning (MAML) and adaptive deep kernel fitting (ADKF), specifically in the regression setting. We investigated how factors such as dataset size and the similarity of training tasks impact predictability. The results indicate that ADKF significantly outperformed both MAML and a single-task baseline model on the inhibition data. However, the performance of ADKF varied across different test tasks. Our findings suggest that considerable enhancements in performance can be anticipated primarily when the task of interest is similar to the tasks incorporated in the meta-learning process.

Few-Shot Learning for Medical Image Segmentation Using 3D U-Net and Model-Agnostic Meta-Learning (MAML).

Deep learning has attained state-of-the-art results in general image segmentation problems; however, it requires a substantial number of annotated images to achieve the desired outcomes. In the medical field, the availability of annotated images is often limited. To address this challenge, few-shot learning techniques have been successfully adapted to rapidly generalize to new tasks with only a few samples, leveraging prior knowledge. In this paper, we employ a gradient-based method known as Model-Agnostic Meta-Learning (MAML) for medical image segmentation. MAML is a meta-learning algorithm that quickly adapts to new tasks by updating a model's parameters based on a limited set of training samples. Additionally, we use an enhanced 3D U-Net as the foundational network for our models. The enhanced 3D U-Net is a convolutional neural network specifically designed for medical image segmentation. We evaluate our approach on the TotalSegmentator dataset, considering a few annotated images for four tasks: liver, spleen, right kidney, and left kidney. The results demonstrate that our approach facilitates rapid adaptation to new tasks using only a few annotated images. In 10-shot settings, our approach achieved mean dice coefficients of 93.70%, 85.98%, 81.20%, and 89.58% for liver, spleen, right kidney, and left kidney segmentation, respectively. In five-shot sittings, the approach attained mean Dice coefficients of 90.27%, 83.89%, 77.53%, and 87.01% for liver, spleen, right kidney, and left kidney segmentation, respectively. Finally, we assess the effectiveness of our proposed approach on a dataset collected from a local hospital. Employing five-shot sittings, we achieve mean Dice coefficients of 90.62%, 79.86%, 79.87%, and 78.21% for liver, spleen, right kidney, and left kidney segmentation, respectively.

Research on reactor power prediction of nuclear power plant based on multivariate optimization GRU model

In the operation of nuclear power plants, the accurate prediction of power change trends is crucial for ensuring safety and stability. In this work, a ML-GRU-RS method, based on model-agnostic meta-learning (MAML), gate recurrent unit (GRU), and random search optimization, is proposed for the long-term prediction of key parameters of nuclear power plants. This method combines the fast adaptability of MAML, the time series data processing capability of GRU, and the optimization efficiency of random search to achieve high-precision predictions under varying power conditions. The results demonstrate that this method can effectively predict the future trends of key parameters in nuclear power plants. The ability of operators to anticipate these trends has been significantly enhanced, contributing to the overall safety of the nuclear power plants.

An empirical evaluation of meta residual network for classifying sensor drift samples

With the advancements in materials and pattern recognition algorithms, electronic nose (E-nose) has been continuously developing and finding increasingly widespread applications. However, E-nose is prone to drift during practical usage. Currently, the primary methods for addressing the issue of sensor drift in sample classification are semi-supervised learning and unsupervised learning. The implementation of supervised learning has been difficult in the past due to the requirement for a large number of up-to-date samples to retrain the models. But now, this challenge can be overcome with the help of meta-learning. We applied the model-agnostic meta-learning (MAML) architecture from meta-learning and designed a residual classifier with learnable per-parameter learning rates. Leveraging the ability of ‘learning to learn’, our method do not require a large amount of target domain data and there is no need to repeat extensive computations when transferring to any other target domain compared to other algorithms, resulting it highly deployable. Through the evaluation of multiple experiments, we demonstrate the feasibility of our approach and achieve very high classification accuracy on a publicly available long-term drift dataset compared with other methods.

Learning to Diagnose: Meta-Learning for Efficient Adaptation in Few-Shot AIOps Scenarios

With the advancement of technologies like 5G, cloud computing, and microservices, the complexity of network management systems and the variety of technical components have greatly increased. This rise in complexity has rendered traditional operations and maintenance methods inadequate for current monitoring and maintenance demands. Consequently, artificial intelligence for IT operations (AIOps), which harnesses AI and big data technologies, has emerged as a solution. AIOps plays a crucial role in enhancing service quality and customer satisfaction, boosting engineering productivity, and reducing operational costs. This article delves into the primary tasks involved in AIOps, such as anomaly detection, and log fault analysis and classification. A significant challenge identified in many AIOps tasks is the scarcity of fault sample data, indicating a natural alignment of these tasks with few-shot learning. Inspired by model-agnostic meta-learning (MAML), we propose a new anomaly detector, MAML-KAD, for application in various AIOps tasks. Observations confirm that meta-learning algorithms effectively enhance AIOps tasks, showcasing the wide-ranging application prospects of meta-learning algorithms in the field of AIOps. Moreover, we introduced an AIOps platform that embeds meta-learning within its diagnostic core and features streamlined log collection, caching, and alerting to automate the AIOps workflow.