Combines Transfer Learning Research Articles

Combining transfer learning and hyperspectral reflectance analysis to assess leaf nitrogen concentration across different plant species datasets

Accurate estimation of leaf nitrogen concentration (LNC) is critical to characterize ecosystem and plant physiological processes for example in carbon fixation. Remote sensing can capture LNC, while interrelated traits and spectral diversity across plant species prevent development of transferable LNC assessment models based on leaf reflectance. Here, we developed a new transfer learning method by coupling transfer component analysis with the support vector regression, namely TCA-SVR, to transfer LNC assessment models across different plant species. We benchmarked the performance of TCA-SVR against a well-established partial least squares regression (PLSR) model with five remote sensing datasets on 60 plant species measured from three spectroradiometers with varied spectral resolutions and illumination and viewing angles. The result showed that leaf reflectance presented the high spectral diversity in different spectral regions, plant species, and growth stages. The combination of visible (VIS), near infrared (NIR), and shortwave infrared (SWIR) reflectance (e.g. 550–2300 nm) achieved the optimal LNC assessment across all datasets. Results on the testing datasets showed that the transferability of the PLSR models highly depended on the LNC distribution and spectral features, which were associated with the differences in plant species, spectral measurements, and growth conditions between datasets. These differences led to the large variations in LNC and leaf reflectance, which thus produced the overestimations and underestimations of LNC. Compared to the PLSR model, TCA-SVR greatly improved the transferability of the LNC assessment model by reducing the average root mean square error by 36.76%. Further, the implementation of modeling updating can help TCA-SVR learn the features related to the difference in plant species and LNC ranges by transferring samples from the target dataset to the source dataset. Our model updating approach improved the performance of TCA-SVR and only needed 5% of the off-site samples to supplement the source dataset to achieve an effective assessment of LNC. Refining the proposed method with new remote sensing datasets will aid rapid monitoring of plant nitrogen status and may improve carbon‑nitrogen interactions in existing ecosystem models.

Dealing with a data-limited regime: Combining transfer learning and transformer attention mechanism to increase aqueous solubility prediction performance

Aqueous solubility is a key chemical property that drives various processes in chemistry and biology. Its computational prediction is challenging, as evidenced by the fact that it has been a subject of considerable interest for several decades. Recent work has explored fingerprint-based, feature-based and graph-based representations with different machine learning and deep learning methodologies. In general, many traditional methods have been proposed, but they rely heavily on the quality of the rule-based, hand-crafted features. On the other hand, limitations in the quality of aqueous solubility data become a handicap when training deep models. In this study, we have developed a novel structure-aware method for the prediction of aqueous solubility by introducing a new deep network architecture and then employing a transfer learning approach. The model was proven to be competitive, obtaining an RMSE of 0.587 during both cross-validation and a test on an independent dataset. To be more precise, the method is evaluated on molecules downloaded from the Online Chemical Database and Modeling Environment (OCHEM). Beyond aqueous solubility prediction, the strategy presented in this work may be useful for modeling any kind of (chemical or biological) properties for which there is a limited amount of data available for model training.

A Novel COVID-19 Diagnosis Support System Using the Stacking Approach and Transfer Learning Technique on Chest X-Ray Images.

COVID-19 is an infectious disease-causing flu-like respiratory problem with various symptoms such as cough or fever, which in severe cases can cause pneumonia. The aim of this paper is to develop a rapid and accurate medical diagnosis support system to detect COVID-19 in chest X-ray images using a stacking approach combining transfer learning techniques and KNN algorithm for selection of the best model. In deep learning, we have multiple approaches for building a classification system for analyzing radiographic images. In this work, we used the transfer learning technique. This approach makes it possible to store and use the knowledge acquired from a pretrained convolutional neural network to solve a new problem. To ensure the robustness of the proposed system for diagnosing patients with COVID-19 using X-ray images, we used a machine learning method called the stacking approach to combine the performances of the many transfer learning-based models. The generated model was trained on a dataset containing four classes, namely, COVID-19, tuberculosis, viral pneumonia, and normal cases. The dataset used was collected from a six-source dataset of X-ray images. To evaluate the performance of the proposed system, we used different common evaluation measures. Our proposed system achieves an extremely good accuracy of 99.23% exceeding many previous related studies.

Domain adaptation for segmentation of critical structures for prostate cancer therapy

Preoperative assessment of the proximity of critical structures to the tumors is crucial in avoiding unnecessary damage during prostate cancer treatment. A patient-specific 3D anatomical model of those structures, namely the neurovascular bundles (NVB) and the external urethral sphincters (EUS), can enable physicians to perform such assessments intuitively. As a crucial step to generate a patient-specific anatomical model from preoperative MRI in a clinical routine, we propose a multi-class automatic segmentation based on an anisotropic convolutional network. Our specific challenge is to train the network model on a unique source dataset only available at a single clinical site and deploy it to another target site without sharing the original images or labels. As network models trained on data from a single source suffer from quality loss due to the domain shift, we propose a semi-supervised domain adaptation (DA) method to refine the model’s performance in the target domain. Our DA method combines transfer learning and uncertainty guided self-learning based on deep ensembles. Experiments on the segmentation of the prostate, NVB, and EUS, show significant performance gain with the combination of those techniques compared to pure TL and the combination of TL with simple self-learning ({p}<0.005 for all structures using a Wilcoxon’s signed-rank test). Results on a different task and data (Pancreas CT segmentation) demonstrate our method’s generic application capabilities. Our method has the advantage that it does not require any further data from the source domain, unlike the majority of recent domain adaptation strategies. This makes our method suitable for clinical applications, where the sharing of patient data is restricted.

A pre-training and self-training approach for biomedical named entity recognition.

Named entity recognition (NER) is a key component of many scientific literature mining tasks, such as information retrieval, information extraction, and question answering; however, many modern approaches require large amounts of labeled training data in order to be effective. This severely limits the effectiveness of NER models in applications where expert annotations are difficult and expensive to obtain. In this work, we explore the effectiveness of transfer learning and semi-supervised self-training to improve the performance of NER models in biomedical settings with very limited labeled data (250-2000 labeled samples). We first pre-train a BiLSTM-CRF and a BERT model on a very large general biomedical NER corpus such as MedMentions or Semantic Medline, and then we fine-tune the model on a more specific target NER task that has very limited training data; finally, we apply semi-supervised self-training using unlabeled data to further boost model performance. We show that in NER tasks that focus on common biomedical entity types such as those in the Unified Medical Language System (UMLS), combining transfer learning with self-training enables a NER model such as a BiLSTM-CRF or BERT to obtain similar performance with the same model trained on 3x-8x the amount of labeled data. We further show that our approach can also boost performance in a low-resource application where entities types are more rare and not specifically covered in UMLS.

Side Scan Sonar Shipwreck Image Recognition Algorithm based on Transfer Learning and Deep Learning

Current underwater shipwreck side scan sonar samples are few and difficult to label. With small sample sizes, their image recognition accuracy with a convolutional neural network model is low. In this study, we proposed an image recognition method for shipwreck side scan sonar that combines transfer learning with deep learning. In the non-transfer learning, shipwreck sonar sample data were used to train the network, and the results were saved as the control group. The weakly correlated data were applied to train the network, then the network parameters were transferred to the new network, and then the shipwreck sonar data was used for training. These steps were repeated using strongly correlated data. Experiments were carried out on Lenet-5, AlexNet, GoogLeNet, ResNet and VGG networks. Without transfer learning, the highest accuracy was obtained on the ResNet network (86.27%). Using weakly correlated data for transfer training, the highest accuracy was on the VGG network (92.16%). Using strongly correlated data for transfer training, the highest accuracy was also on the VGG network (98.04%). In all network architectures, transfer learning improved the correct recognition rate of convolutional neural network models. Experiments show that transfer learning combined with deep learning improves the accuracy and generalization of the convolutional neural network in the case of small sample sizes.

Feature-Selection-Based Transfer Learning for Intracortical Brain–Machine Interface Decoding

The time spent in collecting current samples for decoder calibration and the computational burden brought by high-dimensional neural recordings remain two challenging problems in intracortical brain-machine interfaces (iBMIs). Decoder calibration optimization approaches have been proposed, and neuron selection methods have been used to reduce computational burden. However, few methods can solve both problems simultaneously. In this article, we present a symmetrical-uncertainty-based transfer learning (SUTL) method that combines transfer learning with feature selection. The proposed method uses symmetrical uncertainty to quantitatively measure three indices for feature selection: stationarity, importance and redundancy of the feature. By selecting the stationary features, the disparities between the historical data and current data can be diminished, and the historical data can be effectively used for decoder calibration, thereby reducing the demand for current data. After selecting the important and non-redundant features, only the channels corresponding to them need to work; thus, the computational burden is reduced. The proposed method was tested on neural data recorded from two rhesus macaques to decode the reaching position or grasping gesture. The results showed that the SUTL method diminished the disparities between the historical data and current data, while achieving superior decoding performance with the needs of only ten current samples each category, less than 10% the number of features and 30% the number of neural recording channels. Additionally, unlike most studies on iBMIs, feature selection was implemented instead of neuron selection, and the average decoding accuracy achieved by the former was 6.6% higher.

Bearing Fault Diagnosis under Variable Working Conditions Based on Deep Residual Shrinkage Networks and Transfer Learning

Nowadays, deep learning has made great achievements in the field of rotating machinery fault diagnosis. But in the practical engineering scenarios, when facing a large number of unlabeled data and variable operating conditions, only using a deep learning algorithm may reduce the performance. In order to solve the above problem, this paper uses a method of combining transfer learning with deep learning. First, the deep shrinkage residual network is constructed by adding soft thresholds to extract the characteristics of bearing vibration data under noise redundancy. Then, the joint maximum mean deviation (JMMD) criterion and conditional domain adversarial (CDA) learning domain adapting network are used to align the source and target domains. At the same time, adding transferable semantic augmentation (TSA) regular items improves alignment performance between classes. Finally, the proposed model is verified by three experiments: variable load, variable speed, and variable noise, which overcomes the shortcomings of traditional deep learning and shallow transfer learning algorithms.

Multisignal VGG19 Network with Transposed Convolution for Rotating Machinery Fault Diagnosis Based on Deep Transfer Learning

To realize high-precision and high-efficiency machine fault diagnosis, a novel deep learning framework that combines transfer learning and transposed convolution is proposed. Compared with existing methods, this method has faster training speed, fewer training samples per time, and higher accuracy. First, the raw data collected by multiple sensors are combined into a graph and normalized to facilitate model training. Next, the transposed convolution is utilized to expand the image resolution, and then the images are treated as the input of the transfer learning model for training and fine-tuning. The proposed method adopts 512 time series to conduct experiments on two main mechanical datasets of bearings and gears in the variable-speed gearbox, which verifies the effectiveness and versatility of the method. We have obtained advanced results on both datasets of the gearbox dataset. The dataset shows that the test accuracy is 99.99%, achieving a significant improvement from 98.07% to 99.99%.

Automatic extraction of wheat lodging area based on transfer learning method and deeplabv3+ network

To provide technical support for lodging-resistant wheat breeding, post-disaster assessment, and analysis of factors affecting lodging, the dynamic and accurate extraction of lodging area is particularly important. Most existing methods of monitoring wheat lodging aim to extract the lodging area at a single growth stage, rendering the dynamic monitoring of wheat lodging highly difficult. Thus, this study was aimed at developing a method of estimating wheat lodging at multiple growth stages. For this purpose, nitrogen fertilizers were utilized at different levels to induce different lodging conditions in wheat fields. Unmanned aerial vehicles (UAVs) were used to obtain Red, Green and Blue (RGB) and multispectral images of the field at different wheat-growth stages. Based on these two types of images, a new method combining transfer learning and the DeepLabv3+ network is proposed herein to extract lodging areas at various wheat-growth stages. The proposed method was compared with the commonly used UNet for the extraction of the lodging area. The results show that the proposed method and UNet achieved dice coefficients of 0.82 and 0.75 (early flowering), 0.88 and 0.80 (late flowering), 0.89 and 0.86 (filling stage), 0.90 and 0.87 (early maturity), and 0.90 and 0.88 (late maturity), respectively, using RGB images; further, the proposed method and UNet achieved dice coefficients of 0.91 and 0.51 (early flowering), 0.89 and 0.28 (late flowering), 0.91 and 0.82 (filling stage), 0.93 and 0.76 (early maturity), and 0.92 and 0.56 (late maturity), respectively, at different wheat-growth stages using multispectral image data. Thus, the proposed method can be used to predict lodging at multiple wheat-growth stages, and it outperforms UNet. An effective tool for dynamic monitoring of wheat lodging has been proposed herein.

The fault diagnosis method of rolling bearing under variable working conditions based on deep transfer learning

The vibration signals of rolling bearing obtained under variable working conditions do not obey the same independent distribution so that the traditional method of bearing fault diagnosis has low accuracy, a fault diagnosis method about rolling bearing based on sparse denoising autoencoder (SDAE) for deep feature extraction combining transfer learning is proposed. First, the bearing vibration signal in the time domain is transformed for frequency domain signal via Fourier transform, which is input into the SDAE for adaptive deep feature extraction. Then, the joint geometrical and statistical alignment is introduced to deal with the deep feature samples for reducing the domain discrepancy both statistically and geometrically. Finally, the k-nearest neighbor classification algorithm is used for completing the fault diagnosis of rolling bearing under variable working conditions. The experimental results show that the method presented in the paper improves the accuracy rate of fault diagnosis about rolling bearing under variable working conditions, verifies its feasibility and effectiveness.

What Makes Politicians’ Instagram Posts Popular? Analyzing Social Media Strategies of Candidates and Office Holders with Computer Vision

Previous research on the success of politicians’ messages on social media has so far focused on a limited number of platforms, especially Facebook and Twitter, and predominately studied the effects of textual content. This research reported here applies computer vision analysis to a total of 59,020 image posts published by 172 Instagram accounts of U.S. politicians, both candidates and office holders, and examines how visual attributes influence audience engagement such as likes and comments. In particular, this study introduces an unsupervised approach that combines transfer learning and clustering techniques to discover hidden categories from large-scale visual data. The results reveal that different self-personalization strategies in visual media, for example, images featuring politicians in private, nonpolitical settings, showing faces, and displaying emotions, generally increase audience engagement. Yet, a significant portion of politician’s Instagram posts still fell into the traditional, “politics-as-usual” type of political communication, showing professional settings and activities. The analysis explains how self-personalization is embodied in specific visual portrayals and how different self-presentation strategies affect audience engagement on a popular but less studied social media platform.

Biomedical image classification made easier thanks to transfer and semi-supervised learning

Background and objectivesDeep learning techniques are the state-of-the-art approach to solve image classification problems in biomedicine; however, they require the acquisition and annotation of a considerable volume of images. In addition, using deep learning libraries and tuning the hyperparameters of the networks trained with them might be challenging for several users. These drawbacks prevent the adoption of these techniques outside the machine-learning community. In this work, we present an Automated Machine Learning (AutoML) method to deal with these problems. MethodsOur AutoML method combines transfer learning with a new semi-supervised learning procedure to train models when few annotated images are available. In order to facilitate the dissemination of our method, we have implemented it as an open-source tool called ATLASS. Finally, we have evaluated our method with two benchmarks of biomedical image classification datasets. ResultsOur method has been thoroughly tested both with small datasets and partially annotated biomedical datasets; and, it outperforms, both in terms of speed and accuracy, the existing AutoML tools when working with small datasets; and, might improve the accuracy of models up to a 10% when working with partially annotated datasets. ConclusionsThe work presented in this paper allows the use of deep learning techniques to solve an image classification problem with few resources. Namely, it is possible to train deep models with small, and partially annotated datasets of images. In addition, we have proven that our AutoML method outperforms other AutoML tools both in terms of accuracy and speed when working with small datasets.

Multispecies bioacoustics classification using transfer learning of deep convolutional neural networks with pseudo-labeling

In this study, we evaluated deep convolutional neural networks for classifying the calls of 24 birds and amphibian species detected in ambient field recordings from the tropical mountains of Puerto Rico. Training data were collected using a template-based detection algorithm and manually validated with a graphical interface. To reduce the labor intensive and time-consuming process of manual validation, as well as to increase the accuracy of species classification with acoustic recordings, we propose a novel approach that combines transfer learning of a pre-trained deep convolutional neural network (CNN) model, a semi-supervised pseudo-labeling method, and a custom training loss function. While generating sufficient training data is a major challenge for many deep learning applications, our proposed methodology enables the network to be trained in a supervised fashion with labeled and unlabeled data simultaneously, which effectively increases the size of training set and thus boosts the model performance. As a result, the model achieves 97.7% sensitivity, 96.4% specificity and 96.6% accuracy in classifying a test set of manually validated true and false positive template-based detections. This multi-label multi-species classification methodology and its framework can be easily expanded to other acoustic classification problems.

Reinforcement Learning Based Fast Self-Recalibrating Decoder for Intracortical Brain-Machine Interface.

Background: For the nonstationarity of neural recordings in intracortical brain–machine interfaces, daily retraining in a supervised manner is always required to maintain the performance of the decoder. This problem can be improved by using a reinforcement learning (RL) based self-recalibrating decoder. However, quickly exploring new knowledge while maintaining a good performance remains a challenge in RL-based decoders. Methods: To solve this problem, we proposed an attention-gated RL-based algorithm combining transfer learning, mini-batch, and weight updating schemes to accelerate the weight updating and avoid over-fitting. The proposed algorithm was tested on intracortical neural data recorded from two monkeys to decode their reaching positions and grasping gestures. Results: The decoding results showed that our proposed algorithm achieved an approximate 20% increase in classification accuracy compared to that obtained by the non-retrained classifier and even achieved better classification accuracy than the daily retraining classifier. Moreover, compared with a conventional RL method, our algorithm improved the accuracy by approximately 10% and the online weight updating speed by approximately 70 times. Conclusions: This paper proposed a self-recalibrating decoder which achieved a good and robust decoding performance with fast weight updating and might facilitate its application in wearable device and clinical practice.

Device-free passive wireless localization system with weighted transferable discriminative dimensionality reduction method

Device-free passive wireless indoor localization has attracted great interest due to the widespread deployment of Wi-Fi devices and the rapid growth in demand for location-based services. In this paper, we propose a novel device-free passive wireless localization system with a weighted transferable discriminative dimensionality reduction method (termed TLLOC). It utilizes the channel state information (CSI) extracted from a single link to estimate the location of the target, neither requiring the target to wear any electronic equipment nor deploying a large number of APs and Monitor Devices. To cope with the problem of reduced localization accuracy caused by the unpredictable nature of CSI over time that ignored by most previous CSI-based localization works, a novel weighted transferable discriminative dimensionality reduction (termed WTR) method combining transfer learning and dimensionality reduction is proposed. The WTR method constructs a low-dimensional latent space, which can simultaneously improve the discrimination of training samples and narrow the distribution divergence between the training samples and the test samples, further enhancing the performance of our system. Experimental results are presented to confirm that TLLOC can effectively improve localization accuracy while saving a great amount of the calibration cost, compared with the other existing methods in a representative indoor environment.

Radio Frequency Signal Identification Using Transfer Learning Based on LSTM

Radio frequency distinct native attribute (RF-DNA) technology is very important in distinguishing RF devices. This paper presents a new method for distinguishing RF devices by combining transfer learning and long short-term memory (TL-LSTM). The main purpose of this paper is to identify which RF device sent the unknown RF signals. The data were collected from almost the same eight RF devices produced in 2011, 2014 or 2016. These RF devices emitted unintended signals at 2.4G bandwidth with frequency shift keying. The proposed method first used late production RF devices in 2011 or 2014 as source domain and transferred the trained model to target domain produced in 2016 and then used neural network LSTM model to identify the RF signals. The proposed method is advantageous because it does not require a huge amount of sampling data, and this technique is better than traditional strategies to select optimal features in the multi-domain feature space. The results reveal that the proposed method TL-LSTM can solve the problem of small sample training very well.

Multispecies bioacoustic classification using transfer learning of deep convolutional neural networks with pseudo-labeling

In this study, we evaluated deep convolutional neural networks for classifying the calls of 24 birds and amphibian species detected in ambient field recordings from the tropical mountains of Puerto Rico. Training data were collected using a template-based detection algorithm followed by a manual validation process. As preparing sufficient training data is a major challenge for many deep learning applications, we propose a novel approach that combines transfer learning of a pre-trained deep convolutional neural network (CNN) model and a semi-supervised pseudo-labeling method with a custom loss function to meet this challenge. Our proposed methodology enables the network to be trained in a supervised fashion with labeled and unlabeled data simultaneously, which effectively increases the size of training set and thus boosts the model performance. In classifying a test set of manually validated positive and negative template-based detections, our proposed model achieves 97.7% sensitivity (true positive rate), 96.4% specificity (true negative rate) and 99.5% Area Under a Curve (AUC). This multi-label multi-species classification methodology and its framework can be easily adopted by other acoustic classification problems.

Stacked autoencoder-based community detection method via an ensemble clustering framework

Community detection is a challenging issue because most existing methods are not well suited for complex social networks with ambiguous structures. In this paper, we propose a novel community detection method named Stacked Autoencoder-Based Community Detection Method via Ensemble Clustering (CDMEC). This is the first time that we have attempted to apply four different complex network similarity representations to the community detection problem. This work makes up for the insufficiency of the single similarity matrix to describe the similarity relationship between nodes. These similarity representations can fully describe and consider the sufficient local information between nodes in a network topology. Our CDMEC framework combines transfer learning and a stacked autoencoder to obtain an efficient low-dimensional feature representation of complex networks and aggregates multiple inputs through a novel ensemble clustering framework. This novel framework first uses the basic clustering results to construct a consistent matrix, and then it employs the nonnegative matrix factorization (NMF)-based clustering method to detect reliable clustering results from the consistent matrix. The results of various extensive experiments on artificial benchmark networks and real-world networks showed that the proposed CDMEC framework is superior to the existing state-of-the-art community detection methods and has great potential in solving the community detection problems.

Lightweight and Robust Representation of Economic Scales from Satellite Imagery

Satellite imagery has long been an attractive data source providing a wealth of information regarding human-inhabited areas. While high-resolution satellite images are rapidly becoming available, limited studies have focused on how to extract meaningful information regarding human habitation patterns and economic scales from such data. We present READ, a new approach for obtaining essential spatial representation for any given district from high-resolution satellite imagery based on deep neural networks. Our method combines transfer learning and embedded statistics to efficiently learn the critical spatial characteristics of arbitrary size areas and represent such characteristics in a fixed-length vector with minimal information loss. Even with a small set of labels, READ can distinguish subtle differences between rural and urban areas and infer the degree of urbanization. An extensive evaluation demonstrates that the model outperforms state-of-the-art models in predicting economic scales, such as the population density in South Korea (R2=0.9617), and shows a high use potential in developing countries where district-level economic scales are unknown.