Meta-learning Approach Research Articles

Meta Learning to Rank for Sparsely Supervised Queries

Supervisory signals are a critical resource for training learning to rank models. In many real-world search and retrieval scenarios, these signals may not be readily available or could be costly to obtain for some queries. The examples include domains where labeling requires professional expertise, applications with strong privacy constraints, and user engagement information that are too scarce. We refer to these scenarios as sparsely supervised queries which pose significant challenges to traditional learning to rank models. In this work, we address sparsely supervised queries by proposing a novel meta learning to rank framework which leverages fast learning and adaption capability of meta-learning. The proposed approach accounts for the fact that different queries have different optimal parameters for their rankers, in contrast to traditional learning to rank models which only learn a global ranking model applied to all the queries. In consequence, the proposed method would yield significant advantages especially when new queries are of different characteristics with the training queries. Moreover, the proposed meta learning to rank framework is generic and flexible. We conduct a set of comprehensive experiments on both public datasets and a real-world e-commerce dataset. The results demonstrate that the proposed meta-learning approach can significantly enhance the performance of learning to rank models with sparsely labeled queries.

A Novel Federated Meta-Learning Approach for Discriminating Sedentary Behavior From Wearable Data

A Novel Federated Meta-Learning Approach for Discriminating Sedentary Behavior From Wearable Data

Автономное следование безэкипажного судна по траектории и предотвращение столкновений

In this paper we propose a method to solve the problem of autonomous vessel navigation in highly uncertain conditions. The primary goal was to make the Reinforcement Learning agent to learn the policy allowing the AV to follow a certain path while avoid collisions with any other objects. Maneuvering of AV in the proposed conditions is the principal subject of this paper. Several scenarios were explored during the research with static and dynamic objects. The agent was trained with model free off-policy algorithms. The training process was divided in several parts where we were experimenting with meta-learning approach elements to achieve the robustness of the agent behavior.

Combining graph neural networks and transformers for few-shot nuclear receptor binding activity prediction

Nuclear receptors (NRs) play a crucial role as biological targets in drug discovery. However, determining which compounds can act as endocrine disruptors and modulate the function of NRs with a reduced amount of candidate drugs is a challenging task. Moreover, the computational methods for NR-binding activity prediction mostly focus on a single receptor at a time, which may limit their effectiveness. Hence, the transfer of learned knowledge among multiple NRs can improve the performance of molecular predictors and lead to the development of more effective drugs. In this research, we integrate graph neural networks (GNNs) and Transformers to introduce a few-shot GNN-Transformer, Meta-GTNRP to predict the binding activity of compounds using the combined information of different NRs and identify potential NR-modulators with limited data. The Meta-GTNRP model captures the local information in graph-structured data and preserves the global-semantic structure of molecular graph embeddings for NR-binding activity prediction. Furthermore, a few-shot meta-learning approach is proposed to optimize model parameters for different NR-binding tasks and leverage the complementarity among multiple NR-specific tasks to predict binding activity of compounds for each NR with just a few labeled molecules. Experiments with a compound database containing annotations on the binding activity for 11 NRs shows that Meta-GTNRP outperforms other graph-based approaches. The data and code are available at: https://github.com/ltorres97/Meta-GTNRP.Scientific contributionThe proposed few-shot GNN-Transformer model, Meta-GTNRP captures the local structure of molecular graphs and preserves the global-semantic information of graph embeddings to predict the NR-binding activity of compounds with limited available data; A few-shot meta-learning framework adapts model parameters across NR-specific tasks for different NRs in a joint learning procedure to predict the binding activity of compounds for each NR with just a few labeled molecules in highly imbalanced data scenarios; Meta-GTNRP is a data-efficient approach that combines the strengths of GNNs and Transformers to predict the NR-binding properties of compounds through an optimized meta-learning procedure and deliver robust results valuable to identify potential NR-based drug candidates.

Meta-Hybrid: Integrate Meta-Learning to Enhance Class Imbalance Graph Learning

The class imbalance problem is a significant challenge in node classification tasks. Since majority class samples dominate imbalanced data, the model tends to favor the majority class, resulting in insufficient ability to identify minority classes. Evaluation indicators such as accuracy may not fully reflect the model’s performance. To solve these undesirable effects, we propose a framework for synthesizing minority class samples, GraphSHX, to balance the number of samples of different classes, and integrate the XGBoost model for node classification prediction during the training process. Conventional graph neural networks (GNNs) yielded unsatisfactory results, possibly due to the limited number of newly generated nodes. Therefore, we introduce a meta-mechanism to deal with small-sample problems, and employ the meta-learning approach to enhance performance on small-sample tasks by learning from a large number of tasks. An empirical evaluation of node classification on six publicly available datasets demonstrated that our balanced data set method outperforms existing optimal loss repair methods and synthetic node methods. The addition of the XGBoost model and meta-learning improves the accuracy by more than 5% to 10%, with the overall accuracy of the improved model being 15% higher than that of the baseline method.

A Meta-Learning Approach for Classifying Multimodal Retinal Images of Retinal Vein Occlusion With Limited Data.

To propose and validate a meta-learning approach for detecting retinal vein occlusion (RVO) from multimodal images with only a few samples. In this cross-sectional study, we formulate the problem as meta-learning. The meta-training dataset consists of 1254 color fundus (CF) images from 39 different fundus diseases. Two meta-testing datasets include a public domain dataset and an independent dataset from Kandze Prefecture People's Hospital. The proposed meta-learning models comprise two modules: the feature extraction networks and the prototypical networks (PNs). We use two deep learning models (the ResNet and the Contrastive Language-Image Pre-Training networks [CLIP]) for feature extraction. We evaluate the performance of the algorithms using accuracy, area under the receiver operating characteristic curve (AUCROC), F1-score, and recall. CLIP-based PNs outperform across all meta-testing datasets. For the public APTOS dataset, meta-learning algorithms achieve good results with an accuracy of 86.06% and an AUCROC of 0.87 with only 16 training images. In the hospital datasets, meta-learning algorithms show excellent diagnostic capability for detecting RVO with a very low number of shots (AUCROC above 0.99 for n = 4, 8, and 16, respectively). Notably, even though the meta-training dataset does not include fluorescein angiography (FA) images, meta-learning algorithms also have excellent diagnostic capability for detecting RVO from images with a different modality (AUCROC above 0.93 for n = 4, 8, and 16, respectively). The proposed meta-learning models excel in detecting RVO, not only on CF images but also on FA images from a different imaging modality. The proposed meta-learning models could be useful in automatically detecting RVO on CF and FA images.

Few-Shot Learning in Wi-Fi-Based Indoor Positioning.

This paper explores the use of few-shot learning in Wi-Fi-based indoor positioning, utilizing convolutional neural networks (CNNs) combined with meta-learning techniques to enhance the accuracy and efficiency of positioning systems. The focus is on addressing the challenge of limited labeled data, a prevalent issue in extensive indoor environments. The study explores various scenarios, comparing the performance of the base CNN and meta-learning models. The meta-learning approach involves few-shot learning tasks, such as three-way N-shot, five-way N-shot, etc., to enhance the model's ability to generalize from limited data. The experiments were conducted across various scenarios, evaluating the performance of the models with different numbers of samples per class (K) after filtering by cosine similarity (FCS) during both the stages of data preprocessing and meta-learning. The scenarios included both base classes and novel classes, with and without meta-learning. The results indicated that the base CNN model achieved varying accuracy levels depending on the scenario and the number of samples per class retained after FCS. Meta-learning performed acceptably in scenarios with fewer samples, which are the distinct datasets pertaining to novel classes. With 20 samples per class, the base CNN achieved an accuracy of 0.80 during the pre-training stage, while meta-learning (three-way one-shot) achieved an accuracy of 0.78 on a new small dataset with novel classes.

Novel meta-learning for few-shot bearing fault diagnosis under varying working conditions

In practical engineering, large amount data and variable working conditions poses a challenge to most existing Deep Learning(DL) methods. To solve this problem, this paper proposes a new meta-learning approach. Under the condition of limited data, the fault diagnosis under variable working conditions is regarded as a problem with fewer lenses, and the fault diagnosis of few samples across working scenes is carried out based on the Model-Agnostic Meta-Learning(MAML). Gradient-by-gradient rules are used for parameter optimization to achieve an efficient representation of these tasks. Then, the attention mechanism is applied to improve the efficiency of the training. Finally, experiments verified the fault diagnosis accuracy under various working conditions.

A Meta-learning Approach to Mitigating the Estimation Bias of Q-learning

It is a longstanding problem that Q-learning suffers from the overestimation bias. This issue originates from the fact that Q-learning uses the expectation of maximum Q-value to approximate the maximum expected Q-value. A number of algorithms, such as Double Q-learning, were proposed to address this problem by reducing the estimation of maximum Q-value, but this may lead to an underestimation bias. Note that this underestimation bias may have a larger performance penalty than the overestimation bias. Different from previous algorithms, this paper studies this issue from a fresh perspective, i.e., meta-learning view, which leads to our Meta-Debias Q-learning. The main idea is to extract the maximum expected Q-value with meta-learning over multiple tasks to remove the estimation bias of maximum Q-value and help the agent choose the optimal action more accurately. However, there are two challenges: (1) How to automatically select suitable training tasks? (2) How to positively transfer the meta-knowledge from selected tasks to remove the estimation bias of maximum Q-value? To address the two challenges mentioned above, we quantify the similarity between the training tasks and the test task. This similarity enables us to select appropriate “partial” training tasks and helps the agent extract the maximum expected Q-value to remove the estimation bias. Extensive experiment results show that our Meta-Debias Q-learning outperforms SOTA baselines drastically in three evaluation indicators, i.e., maximum Q-value, policy, and reward. More specifically, our Meta-Debias Q-learning only underestimates \(1.2*10^{-3}\) than the maximum expected Q-value in the multi-armed bandit environment; and only differs \(5.04\%-5\%=0.04\%\) than the optimal policy in the two states MDP environment. In addition, we compare the uniform weight and our similarity weight. Experiment results reveal fundamental insights into why our proposed algorithm outperforms in the maximum Q-value, policy, and reward.

Meta-learning Approaches for Few-Shot Learning: A Survey of Recent Advances

Despite its astounding success in learning deeper multi-dimensional data, the performance of deep learning declines on new unseen tasks mainly due to its focus on same-distribution prediction. Moreover, deep learning is notorious for poor generalization from few samples. Meta-learning is a promising approach that addresses these issues by adapting to new tasks with few-shot datasets. This survey first briefly introduces meta-learning and then investigates state-of-the-art meta-learning methods and recent advances in: (i) metric-based, (ii) memory-based, (iii), and learning-based methods. Finally, current challenges and insights for future researches are discussed.

Enhancing Plant Disease Detection through Transfer Learning by Incorporating MemoryAugmented Networks and Meta-Learning Approaches

Transfer learning has revolutionized automated plant disease detection by leveraging pre-trained convolutional neural networks (CNNs) on large-scale datasets like ImageNet. This paper explores advanced methodologies in transfer learning, focusing on the integration of memory-augmented networks and meta-learning approaches. These enhancements aim to improve model adaptation to new disease types and environmental conditions, thereby enhancing accuracy and robustness in agricultural applications. The paper reviews existing literature, discusses methodologies, and suggests future research directions to advance the field of AI-driven plant pathology.

Returns to solar panels in the housing market: A meta learner approach

This paper aims to estimate the returns to solar panels in the UK residential housing market. Our analysis applies a causal machine learning approach to Zoopla property data containing about 5 million observations. Drawing on meta-learner algorithms, we provide strong evidence documenting that solar panels are directly capitalized into sale prices. Our results point to a selling price premium above 6% (range between 6.1% to 7.1% depending on the meta-learner) associated with solar panels. Considering that the average selling price is £230,536 in our sample, this corresponds to an additional £14,062 to £16,368 selling price premium for houses with solar panels. Our results are robust to traditional hedonic pricing models and matching techniques, with the lowest estimates at 3.5% using the latter. Despite the declining trend, the additional analyses demonstrate that the positive premium associated with solar panels persists over the years.

A Meta-learner approach to multistep-ahead time series prediction

AbstractThe utilization of machine learning has become ubiquitous in addressing contemporary challenges in data science. Moreover, there has been significant interest in democratizing the decision-making process for selecting machine learning algorithms, achieved through the incorporation of meta-features and automated machine learning techniques for both classification and regression tasks. However, this paradigm has not been readily applied to multistep-ahead time series prediction problems. Unlike regression and classification problems, which utilize independent variables not derived from the target variable, time series models typically rely on past values of the series to forecast future outcomes. The structure of a time series is often characterized by features such as trend, seasonality, cyclicality and irregularity. In our study, we illustrate how time series metrics representing these features, in conjunction with an ensemble-based regression Meta-Learner, were employed to predict the standardized mean square error of candidate time series prediction models. Our experiments utilized datasets covering a broad feature space, facilitating the selection of the most effective model by researchers. A rigorous evaluation was conducted to assess the performance of the Meta-Learner on both synthetic and real time series data.

Online meta-learning approach for sensor fault diagnosis using limited data

The accurate and timely diagnosis of sensor faults plays a critical role in ensuring the reliability and performance of structural health monitoring (SHM) systems. However, the challenge is detecting, locating, and estimating sensor faults in an online manner using limited training data. To resolve this problem, a novel approach for online sensor fault diagnosis is proposed for SHM. The proposed approach is based on meta-learning, which enables superior model generalization capabilities using limited data. The detection, localization, and estimation of typical sensor faults in an online manner can be achieved efficiently by the proposed approach. First, a one-dimensional convolutional neural network (1D CNN) is designed to detect and locate faulty sensors. The initial model parameters of the 1D CNN are optimized using a model-agnostic meta-learning training strategy. This strategy allows the acquisition of transferable prior knowledge, which can speed up the learning process on new sensor fault detection and localization tasks. The meta-learning strategy also enables efficient and accurate detection and localization of potential faulty sensors with limited data. After detecting and locating the faulty sensors, an online updating algorithm based on a dual Kalman filter is used to estimate the severity of sensor faults and structural states simultaneously. The proposed approach is demonstrated with simulated sensor faults that cover a numerical example and field measurements from the Canton Tower. The results show that the proposed approach is applicable for online sensor fault diagnosis in SHM.

A Lightweight Task-Agreement Meta Learning for Low-Resource Speech Recognition

Meta-learning has proven to be a powerful paradigm for transferring knowledge from prior tasks to facilitate the quick learning of new tasks in automatic speech recognition. However, the differences between languages (tasks) lead to variations in task learning directions, causing the harmful competition for model’s limited resources. To address this challenge, we introduce the task-agreement multilingual meta-learning (TAMML), which adopts the gradient agreement algorithm to guide the model parameters towards a direction where tasks exhibit greater consistency. However, the computation and storage cost of TAMML grows dramatically with model’s depth increases. To address this, we further propose a simplification called TAMML-Light which only uses the output layer for gradient calculation. Experiments on three datasets demonstrate that TAMML and TAMML-Light achieve outperform meta-learning approaches, yielding superior results.Furthermore, TAMML-Light can reduce at least 80 %\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document} of the relative increased computation expenses compared to TAMML.

Advances and Challenges in Meta-Learning: A Technical Review.

Meta-learning empowers learning systems with the ability to acquire knowledge from multiple tasks, enabling faster adaptation and generalization to new tasks. This review provides a comprehensive technical overview of meta-learning, emphasizing its importance in real-world applications where data may be scarce or expensive to obtain. The article covers the state-of-the-art meta-learning approaches and explores the relationship between meta-learning and multi-task learning, transfer learning, domain adaptation and generalization, self-supervised learning, personalized federated learning, and continual learning. By highlighting the synergies between these topics and the field of meta-learning, the article demonstrates how advancements in one area can benefit the field as a whole, while avoiding unnecessary duplication of efforts. Additionally, the article delves into advanced meta-learning topics such as learning from complex multi-modal task distributions, unsupervised meta-learning, learning to efficiently adapt to data distribution shifts, and continual meta-learning. Lastly, the article highlights open problems and challenges for future research in the field. By synthesizing the latest research developments, this article provides a thorough understanding of meta-learning and its potential impact on various machine learning applications. We believe that this technical overview will contribute to the advancement of meta-learning and its practical implications in addressing real-world problems.

Joint Sensing and Communication for mmWave VR in Metaverse: A Meta-Learning Approach

Joint Sensing and Communication for mmWave VR in Metaverse: A Meta-Learning Approach

Meta-2OM: A multi-classifier meta-model for the accurate prediction of RNA 2'-O-methylation sites in human RNA.

2'-O-methylation (2-OM or Nm) is a widespread RNA modification observed in various RNA types like tRNA, mRNA, rRNA, miRNA, piRNA, and snRNA, which plays a crucial role in several biological functional mechanisms and innate immunity. To comprehend its modification mechanisms and potential epigenetic regulation, it is necessary to accurately identify 2-OM sites. However, biological experiments can be tedious, time-consuming, and expensive. Furthermore, currently available computational methods face challenges due to inadequate datasets and limited classification capabilities. To address these challenges, we proposed Meta-2OM, a cutting-edge predictor that can accurately identify 2-OM sites in human RNA. In brief, we applied a meta-learning approach that considered eight conventional machine learning algorithms, including tree-based classifiers and decision boundary-based classifiers, and eighteen different feature encoding algorithms that cover physicochemical, compositional, position-specific and natural language processing information. The predicted probabilities of 2-OM sites from the baseline models are then combined and trained using logistic regression to generate the final prediction. Consequently, Meta-2OM achieved excellent performance in both 5-fold cross-validation training and independent testing, outperforming all existing state-of-the-art methods. Specifically, on the independent test set, Meta-2OM achieved an overall accuracy of 0.870, sensitivity of 0.836, specificity of 0.904, and Matthew's correlation coefficient of 0.743. To facilitate its use, a user-friendly web server and standalone program have been developed and freely available at http://kurata35.bio.kyutech.ac.jp/Meta-2OM and https://github.com/kuratahiroyuki/Meta-2OM.

A Meta-Learning Approach for Diabetic Retinopathy Severity Grading

A Meta-Learning Approach for Diabetic Retinopathy Severity Grading

A lifelong meta-learning approach for learning deep grey-box representative thermal dynamics models for residential buildings

Thermal dynamics models of residential buildings are crucial for managing energy use and maintaining desirable indoor environment quality, in the context of decarbonizing the electric grid. Despite recent developments in data-driven models, the implementation with a massive number of buildings in the real world remains infeasible because the computation is expensive and the thermostat data of buildings can be online and sparse. A representative model for aggregated study and acceleration of personalizing models is needed for many downstream tasks. In addition, most existing data-driven models lack physical interpretability questioning the representativeness of any aggregated model. In this paper, we present an online framework for learning meta-models of thermal dynamics across various residential buildings. The meta-models are physically explainable according to the building types and can accelerate fine-tuning personalized models for any building saving data and computation. Experiments suggest that our method performs well with a real-world online dataset that produces representative models of different climate zones and effective initialization for training individual models.