Meta-learning Approach Research Articles

Addressing the Algorithm Selection Problem through an Attention-Based Meta-Learner Approach

In the algorithm selection problem, where the task is to identify the most suitable solving technique for a particular situation, most methods used as performance mapping mechanisms have been relatively simple models such as logistic regression or neural networks. In the latter case, most implementations tend to have a shallow and straightforward architecture and, thus, exhibit a limited ability to extract relevant patterns. This research explores the use of attention-based neural networks as meta-learners to improve the performance mapping mechanism in the algorithm selection problem and fully take advantage of the model’s capabilities for pattern extraction. We compare the proposed use of an attention-based meta-learner method as a performance mapping mechanism against five models from the literature: multi-layer perceptron, k-nearest neighbors, softmax regression, support vector machines, and decision trees. We used a meta-data dataset obtained by solving the vehicle routing problem with time window (VRPTW) instances contained in the Solomon benchmark with three different configurations of the simulated annealing meta-heuristic for testing purposes. Overall, the attention-based meta-learner model yields better results when compared to the other benchmark methods in consistently selecting the algorithm that best solves a given VRPTW instance. Moreover, by significantly outperforming the multi-layer perceptron, our findings suggest promising potential in exploring more recent and novel advancements in neural network architectures.

Meta-learning biologically plausible plasticity rules with random feedback pathways

Backpropagation is widely used to train artificial neural networks, but its relationship to synaptic plasticity in the brain is unknown. Some biological models of backpropagation rely on feedback projections that are symmetric with feedforward connections, but experiments do not corroborate the existence of such symmetric backward connectivity. Random feedback alignment offers an alternative model in which errors are propagated backward through fixed, random backward connections. This approach successfully trains shallow models, but learns slowly and does not perform well with deeper models or online learning. In this study, we develop a meta-learning approach to discover interpretable, biologically plausible plasticity rules that improve online learning performance with fixed random feedback connections. The resulting plasticity rules show improved online training of deep models in the low data regime. Our results highlight the potential of meta-learning to discover effective, interpretable learning rules satisfying biological constraints.

A Meta-Learning Approach for Few-Shot Face Forgery Segmentation and Classification.

The technology for detecting forged images is good at detecting known forgery methods. It trains neural networks using many original and corresponding forged images created with known methods. However, when encountering unseen forgery methods, the technology performs poorly. Recently, one suggested approach to tackle this problem is to use a hand-crafted generator of forged images to create a range of fake images, which can then be used to train the neural network. However, the aforementioned method has limited detection performance when encountering unseen forging techniques that the hand-craft generator has not accounted for. To overcome the limitations of existing methods, in this paper, we adopt a meta-learning approach to develop a highly adaptive detector for identifying new forging techniques. The proposed method trains a forged image detector using meta-learning techniques, making it possible to fine-tune the detector with only a few new forged samples. The proposed method inputs a small number of the forged images to the detector and enables the detector to adjust its weights based on the statistical features of the input forged images, allowing the detection of forged images with similar characteristics. The proposed method achieves significant improvement in detecting forgery methods, with IoU improvements ranging from 35.4% to 127.2% and AUC improvements ranging from 2.0% to 48.9%, depending on the forgery method. These results show that the proposed method significantly improves detection performance with only a small number of samples and demonstrates better performance compared to current state-of-the-art methods in most scenarios.

A meta-learning approach to personalized blood glucose prediction in type 1 diabetes

Accurate blood glucose prediction is a critical element in modern artificial pancreas systems. Recently, many deep learning-based models have been proposed for glucose prediction, showing encouraging results in population modeling. However, due to the large amount of data required for training deep learning-based models, few studies have successfully addressed personalized modeling, which is critical to ensure safe policies in a closed-loop scheme given the high inter-patient variability. To address this concern, we propose a meta-learning-based technique for accurate personalized modeling that requires minimal data volume to personalize from its population version, needs few training iterations, and has a low risk of over-fitting. Results using the UVA/Padova simulator show that the proposed technique generalizes better and outperforms other approaches in standard and task-specific metrics, particularly for longer prediction horizons and higher degrees of distributional shifts.

Distilled Meta-learning for Multi-Class Incremental Learning

Meta-learning approaches have recently achieved promising performance in multi-class incremental learning. However, meta-learners still suffer from catastrophic forgetting, i.e., they tend to forget the learned knowledge from the old tasks when they focus on rapidly adapting to the new classes of the current task. To solve this problem, we propose a novel distilled meta-learning (DML) framework for multi-class incremental learning that integrates seamlessly meta-learning with knowledge distillation in each incremental stage. Specifically, during inner-loop training, knowledge distillation is incorporated into the DML to overcome catastrophic forgetting. During outer-loop training, a meta-update rule is designed for the meta-learner to learn across tasks and quickly adapt to new tasks. By virtue of the bilevel optimization, our model is encouraged to reach a balance between the retention of old knowledge and the learning of new knowledge. Experimental results on four benchmark datasets demonstrate the effectiveness of our proposal and show that our method significantly outperforms other state-of-the-art incremental learning methods.

Ensemble machine learning approach for examining critical process parameters and scale-up opportunities of microbial electrochemical systems for hydrogen peroxide production

Hydrogen peroxide (H2O2) production in microbial electrochemical systems (MESs) is an attractive option for enabling a circular economy in the water/wastewater sector. Here, a machine learning algorithm was developed, using a meta-learning approach, to predict the H2O2 production rates in MES based on the seven input variables, including various design and operating parameters. The developed models were trained and cross-validated using the experimental data collected from 25 published reports. The final ensemble meta-learner model (combining 60 models) demonstrated a high prediction accuracy with very high R2 (0.983) and low root-mean-square error (RMSE) (0.647 kg H2O2 m−3 d−1) values. The model identified the carbon felt anode, GDE cathode, and cathode-to-anode volume ratio as the top three most important input features. Further scale-up analysis for small-scale wastewater treatment plants indicated that proper design and operating conditions could increase the H2O2 production rate to as high as 9 kg m−3 d−1.

Face Recognition System for Real Life Applications using Augmentation Techniques

A facial recognition system may recognise a person by comparing his face to the ones in a database of faces taken from a digital image or video frame. It is primarily used to verify the identity of users by analysing facial features from an image. According to several studies, facial recognition technology has made enormous strides with the growth of science and technology, but there is still opportunity for its improvement in practical use. Some issues like rotation, occlusion, and meta learning approach can be handled for improved model accuracy by using 3D technologies like image augmentation to supplement 2D images. Today's facial recognition technology can be used for more than only identifying people; it can also be used for security purposes in residential, business, and commercial applications. Due to its convenience, face recognition technology is widely employed in the financial sector as well. This research proposes face recognition for real life applications using Siamese network and Deep Neural Network, Facenet.

Potential and limitations of machine meta-learning (ensemble) methods for predicting COVID-19 mortality in a large inhospital Brazilian dataset

The majority of early prediction scores and methods to predict COVID-19 mortality are bound by methodological flaws and technological limitations (e.g., the use of a single prediction model). Our aim is to provide a thorough comparative study that tackles those methodological issues, considering multiple techniques to build mortality prediction models, including modern machine learning (neural) algorithms and traditional statistical techniques, as well as meta-learning (ensemble) approaches. This study used a dataset from a multicenter cohort of 10,897 adult Brazilian COVID-19 patients, admitted from March/2020 to November/2021, including patients [median age 60 (interquartile range 48–71), 46% women]. We also proposed new original population-based meta-features that have not been devised in the literature. Stacking has shown to achieve the best results reported in the literature for the death prediction task, improving over previous state-of-the-art by more than 46% in Recall for predicting death, with AUROC 0.826 and MacroF1 of 65.4%. The newly proposed meta-features were highly discriminative of death, but fell short in producing large improvements in final prediction performance, demonstrating that we are possibly on the limits of the prediction capabilities that can be achieved with the current set of ML techniques and (meta-)features. Finally, we investigated how the trained models perform on different hospitals, showing that there are indeed large differences in classifier performance between different hospitals, further making the case that errors are produced by factors that cannot be modeled with the current predictors.

A Meta-Learning Approach for Device-Free Indoor Localization

Although fingerprint-based methods could achieve high location accuracy for device-free indoor localization, it requires cost-expensive massive labeling. In order to fully exploit the previously collected CSI fingerprints for various indoor localization tasks, this letter proposes a meta-learning approach for indoor localization, with which an excellent initialization solution for a new localization task is resorted and further training with a few labeled samples results in sound performance improvement over the state-of-the-art deep learning (DL) approach. By recognizing the unequal contribution of an individual training task to the target task, a novel weighted loss is introduced under the framework of meta-learning. Experiments show that the proposed meta-learning approach can gain improvement of 20% Root-Mean-Square-Error (RMSE) enhancement over the DL approach.

Meta-ED: Cross-lingual Event Detection Using Meta-learning for Indian Languages

Lack of annotated data is a major concern in Event Detection (ED) tasks for low-resource languages. Cross-lingual ED seeks to address this issue by transferring information across various languages to improve overall performance. In this article, we propose a method for cross-lingual ED with a few training instances. We present a model agnostic meta-learning approach for few-shot cross-lingual ED that is able to find good parameter initialization and enables fast adaptation to new low-resource languages. We evaluate our model on four Indian languages. The results show that our approach significantly outperforms the base model.

Meta-learning for dynamic tuning of active learning on stream classification

Supervised data stream learning depends on the incoming sample’s true label to update a classifier’s model. In real life, obtaining the ground truth for each instance is a challenging process; it is highly costly and time consuming. Active Learning has already bridged this gap by finding a reduced set of instances to support the creation of a reliable stream classifier. However, identifying a reduced number of informative instances to support a suitable classifier update and drift adaptation is very tricky. To better adapt to concept drifts using a reduced number of samples, we propose an online tuning of the Uncertainty Sampling threshold using a meta-learning approach. Our approach exploits statistical meta-features from adaptive windows to meta-recommend a suitable threshold to address the trade-off between the number of labelling queries and high accuracy. Experiments exposed that the proposed approach provides the best trade-off between accuracy and query reduction by dynamic tuning the uncertainty threshold using lightweight meta-features.

Spatially transferable dwelling extraction from Multi-Sensor imagery in IDP/Refugee Settlements: A meta-Learning approach

Dwelling information is very important for various applications in humanitarian emergency response. For this, Earth observation is crucial to have spatially explicit and temporally frequent observations. Coupled with advances in computer vision, especially with the proliferation of state-of-the-art deep learning models, are providing a new opportunity for automatic information retrieval from remotely sensed imagery. Despite their proven performance, they have two known limitations, viz, the requirement of intensive data for training and lack of universal generalization under changing scene characteristics and respective data distributions. To tackle this problem, the current study has investigated the relevance of a meta-learning approach for the creation of a spatially transferable optimal model for dwelling extraction in IDP/refugee settlements. The study followed a Model Agnostic meta-Learning (MAML) strategy with newly designed and tested variates with weighted loss gradient update plus self-supervision in the adaptation phase to the target locations using a few samples. The approach is tested using multi-sensor, multi-temporal satellite imagery from eight IDP settlements. Furthermore, a thorough investigation is undertaken on task-specific transfers and their association with deep-embedded feature space and image structural similarity. Results indicate that for some target sites, task-specific transfers perform better than MAML approaches. When MAML is trained with a weighted loss gradient update, it yielded better performance. The best performance (MIoU 0.623 and an F-1 score of 76.7 %) was achieved when MAML is aided with self-supervision using pseudo-labels from unlabelled target data. In all experimental setups, though increasing adaptation samples contribute to positive transfer, the marginal contribution from additional samples is decreasing and stagnates when the adaptation sample size reaches ∼ 35 % of the target dataset.

A metalearning approach for Physics-Informed Neural Networks (PINNs): Application to parameterized PDEs

Physics-informed neural networks (PINNs) as a means of discretizing partial differential equations (PDEs) are garnering much attention in the Computational Science and Engineering (CS&E) world. At least two challenges exist for PINNs at present: an understanding of accuracy and convergence characteristics with respect to tunable parameters and identification of optimization strategies that make PINNs as efficient as other computational science tools. The cost of PINNs training remains a major challenge of Physics-informed Machine Learning (PiML) – and, in fact, machine learning (ML) in general. This paper is meant to move towards addressing the latter through the study of PINNs on new tasks, for which parameterized PDEs provides a good testbed application as tasks can be easily defined in this context. Following the ML world, we introduce metalearning of PINNs with application to parameterized PDEs. By introducing metalearning and transfer learning concepts, we can greatly accelerate the PINNs optimization process. We present a survey of model-agnostic metalearning, and then discuss our model-aware metalearning applied to PINNs as well as implementation considerations and algorithmic complexity. We then test our approach on various canonical forward parameterized PDEs that have been presented in the emerging PINNs literature.

Power transformers fault diagnosis based on a meta-learning approach to kernel extreme learning machine with opposition-based learning sparrow search algorithm

Considering the power transformers fault diagnosis model has unstable performance and prone to over-fitting, we propose a transformers fault diagnosis model based on a meta-learning approach to kernel extreme learning machine with opposition-based learning sparrow search algorithm optimization (Meta-OSSA-KELM) in this paper. Its learning proceeds in two steps. Firstly, the base-learner KELMs is trained on the disjoint training subset. Then, meta-learner KELM is trained with the hidden codes of training set in base-learner KELMs that have been trained. In this paper, chaotic mapping and opposition-based learning are integrated into Sparrow search algorithm(SSA) and used it to optimize each learner. We simulate this model with measured dissolved gas analysis(DGA) data, the results show that compared with PSO and SSA, opposition-based learning sparrow search algorithm(OSSA) has better global search-ability on the optimization for the proposed model. In addition, compared with Adaboost.M1, BPNN, SVM and KELM, Meta-OSSA-KELM has a higher average accuracy (90.9% vs 78.5%, 74.0%, 76.9%, 76.9%) and a lower standard deviation (1.56×10–2 vs 4.21×10–2, 10.5×10–2, 3.7×10–2, 2.18×10–2) in simulation tests for 30 times. It is shown that the proposed model is a stable and better performance transformers fault diagnosis method.

Meta Learning Based Deception Detection from Speech

It is difficult to overestimate the importance of detecting human deception, specifically by using speech cues. Indeed, several works attempt to detect deception from speech. Unfortunately, most works use the same people and environments in training and in testing. That is, they do not separate training samples from test samples according to the people who said each statement or by the environments in which each sample was recorded. This may result in less reliable detection results. In this paper, we take a meta-learning approach in which a model is trained on a variety of learning tasks to enable it to solve new learning tasks using only a few samples. In our approach, we split the data according to the persons (and recording environment), i.e., some people are used for training, and others are used for testing only, but we do assume a few labeled samples for each person in the data set. We introduce CHAML, a novel deep learning architecture that receives as input the sample in question along with two more truthful samples and non-truthful samples from the same person. We show that our method outperforms other state-of-the-art methods of deception detection based on speech and other approaches for meta-learning on our data-set. Namely, CHAML reaches an accuracy of 61.34% and an F1-Score of 0.3857, compared to an accuracy of only 55.82% and an F1-score of only 0.3444, achieved by a previous, most recent approach.

Constructing a Meta-Learner for Unsupervised Anomaly Detection

Unsupervised anomaly detection (AD) is critical for a wide range of practical applications, from network security to health and medical tools. Due to the diversity of problems, no single algorithm has been found to be superior for all AD tasks. Choosing an algorithm, otherwise known as the Algorithm Selection Problem (ASP), has been extensively examined in supervised classification problems, through the use of meta-learning and AutoML, however, it has received little attention in unsupervised AD tasks. This research proposes a new meta-learning approach that identifies an appropriate unsupervised AD algorithm given a set of meta-features generated from the unlabelled input dataset. The performance of the proposed meta-learner is superior to the current state of the art solution. In addition, a mixed model statistical analysis has been conducted to examine the impact of the meta-learner components: the meta-model, meta-features, and the base set of AD algorithms, on the overall performance of the meta-learner. The analysis was conducted using more than 10,000 datasets, which is significantly larger than previous studies. Results indicate that a relatively small number of meta-features can be used to identify an appropriate AD algorithm, but the choice of a meta-model in the meta-learner has a considerable impact.

Layered Meta-Learning Algorithm for Predicting Adverse Events in Type 1 Diabetes

Type 1 diabetes mellitus (T1D) is a chronic disease that, if not treated properly, can lead to serious complications. We propose a layered meta-learning approach based on multi-expert systems to predict adverse events in T1D. The base learner is composed of three deep neural networks and exploits only continuous glucose monitoring data as an input feature. Each network specializes in predicting whether the patient is about to experience hypoglycemia, hyperglycemia, or euglycemia. The output of the experts is passed to a meta-learner to provide the final model classification. In addition, we formally introduce a novel parameter, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\alpha $ </tex-math></inline-formula> , to evaluate the advance by which a prediction is performed. We evaluate the proposed approach on both a public and a private dataset and implement it on an edge device to test its feasibility in real life. On average, on the Ohio T1DM dataset, our system was able to predict hypoglycemia events with a time gain of 22.8 minutes, hyperglycemia ones with an advance of 24.0 minutes. Our model not only outperforms presented models in the literature in terms of events predicted with sufficient advance, but also with regard to the number of false positives, achieving on average 0.45 and 0.46 hypo- and hyperglycemic false alarms per day, respectively. Furthermore, the meta-learning approach effectively improves performance in a new cohort of patients by training only the meta-learner with a limited amount of data. We believe our approach would be an essential ally for the patients to control the glycemic fluctuations and adjust their insulin therapy and dietary intakes, enabling them to speed up decision-making and improve personal self-management, resulting in a reduced risk of acute and chronic complications. As our last contribution, we assessed the validity of the approach by exploiting only blood glucose variations as well as in combination with the information of the insulin boluses, the skin temperature, and the galvanic skin response. In general, we have observed that providing other information but CGM leads to slightly lower performances with respect to considering CGM alone.

Learning Adaptive Probabilistic Models for Uncertainty-Aware Air Pollution Prediction

The air pollution problem has been a serious issue for public health and city development in recent years, which rises an urgent demand for accurate air pollution prediction models. Traditional time series prediction scheme has been challenged in such case because the air pollution signals can be highly dynamic and uncertain. Distinct latent physical processes and complex environment changes over time make it hard for one fixed model to give consistently good performance. Also probabilistic prediction and uncertainty-aware estimation are critical when the model is used for public decisions in practice. However, few previous works can simultaneously provide adaptive and probabilistic air pollution prediction abilities, which are important practical issues for the real-world implementation of data-driven air pollution prediction models. In this paper we propose an adversarial meta learning approach to address these concerns. After reformulating the target signals as a collection of independent but similar mini-tasks along time, we adversarially learn a meta model that includes an implicit generator to adaptively give more informative probabilistic predictions across tasks. We further provide a bayesian meta learning interpretation that recasts the proposed model as an approximate minimizer for the Wasserstein distance between a generative latent model and the true data distribution. Our model provides an algorithm that explicitly models how the prediction distribution is conditioned on underlying data patterns and simultaneously gives adaptive uncertainty estimation. Experiments on both synthetic and real-world air pollution datasets show that the proposed model can simultaneously provide better probabilistic and adaptive prediction while keeping stratifying point prediction error.

Meta-Learning Multi-Scale Radiology Medical Image Super-Resolution

High-resolution medical images have important medical value, but are difficult to obtain directly. Limited by hardware equipment and patient’s physical condition, the resolution of directly acquired medical images is often not high. Therefore, many researchers have thought of using super-resolution algorithms for secondary processing to obtain high-resolution medical images. However, current super-resolution algorithms only work on a single scale, and multiple networks need to be trained when super-resolution images of different scales are needed. This definitely raises the cost of acquiring high-resolution medical images. Thus, we propose a multi-scale super-resolution algorithm using meta-learning. The algorithm combines a meta-learning approach with an enhanced depth of residual super-resolution network to design a meta-upscale module. The meta-upscale module utilizes the weight prediction property of meta-learning and is able to perform the super-resolution task of medical images at any scale. Meanwhile, we design a non-integer mapping relation for super-resolution, which allows the network to be trained under non-integer magnification requirements. Compared to the state-of-the-art single-image super-resolution algorithm on computed tomography images of the pelvic region. The meta-learning multiscale super-resolution algorithm obtained a surpassing of about 2% at a smaller model volume. Testing on different parts proves the high generalizability of our algorithm. Multi-scale super-resolution algorithms using meta-learning can compensate for hardware device defects and reduce secondary harm to patients while obtaining high-resolution medical images. It can be of great use in imaging related fields.

A Meta-Learning Approach for Aircraft Trajectory Prediction

The aviation industry has seen significant advancements in safety procedures over the past few decades, resulting in a steady decline in aviation deaths worldwide. However, the safety standards in General Aviation (GA) are still lower compared to those in commercial aviation. With the anticipated growth in air travel, there is an imminent need to improve operational safety in GA. One way to improve aircraft and operational safety is through trajectory prediction. Trajectory prediction plays a key role in optimizing air traffic control and improving overall flight safety. This paper proposes a meta-learning approach to predict short- to mid-term trajectories of aircraft using historical real flight data collected from multiple GA aircraft. The proposed solution brings together multiple models to improve prediction accuracy. In this paper, we are combining two models, Random Forest Regression (RFR) and Long Short-term Memory (LSTM), using k-Nearest Neighbors (k-NN), to output the final prediction based on the combined output of the individual models. This approach gives our model an edge over single-model predictions. We present the results of our meta-learner and evaluate its performance against individual models using the Mean Absolute Error (MAE), Absolute Altitude Error (AAE), and Root Mean Squared Error (RMSE) evaluation metrics. The proposed methodology for aircraft trajectory forecasting is discussed in detail, accompanied by a literature review and an overview of the data preprocessing techniques used. The results demonstrate that the proposed meta-learner outperforms individual models in terms of accuracy, providing a more robust and proactive approach to improve operational safety in GA.