Deep Transfer Learning Approach Research Articles

A novel transfer learning approach for wind power prediction based on a serio-parallel deep learning architecture

Although machine learning methods have been widely applied in the wind power prediction field, they are not suitable for building the prediction model of a new-built wind farm because of no sufficient historical data. In this study, a novel deep transfer learning approach is proposed for addressing the few-shot learning problem in multi-step ahead wind power prediction. In the pre-training stage, several convolutional neural networks (CNNs) in parallel are separately connected to the long short-term memory network (LSTM), thus forming a unique serio-parallel CNNs-LSTM (CL) feature extractor. The CL utilizes the CNNs and LSTM to extract both the meteorological and temporal feature information of the neighboring wind farms for facilitating the prediction modeling of the source wind farm. In the transfer-training stage, a transfer strategy is designed to transfer partial network parameters of a well-trained CL feature extractor to construct the prediction model of the target wind farm. In addition, a personalized-training strategy is implemented by using crisscross optimization (CSO) to retrain the parameters of fully-connected layer. The proposed method is validated on a swarm of wind farms located in China and the experimental results show its obvious superiority over the non-transfer models involved in this study.

Resilient power network structure for stable operation of energy systems: A transfer learning approach

With increasing dynamic loads, short-term voltage stability (STVS) problems are emerging in sub-transmission expansion planning (SEP), which threats the stable operation of energy systems. However, it is computationally intensive to evaluate all possible network structures in SEP, since STVS is traditionally analyzed for a fixed network structure at a certain operating condition using time-domain simulations. Taking advantage of big data analytics, a deep transfer learning approach based on bi-directional long short-term memory (BiLSTM) is proposed to identify resilient network structures with better STVS performance efficiently. First, an improved voltage recovery index (IVRI) is introduced to quantify the STVS of different network structures with a higher degree of distinguishment. Then, a BiLSTM-based STVS evaluation machine is devised to identify resilient network structures with better STVS performances with high efficiency, which predicts the STVS of various network structures without resorting to time-consuming time-domain simulations. Finally, the STVS evaluation machine is transferred to adapt to new systems with different numbers of buses in the context of SEP. Numerical tests on the IEEE benchmarks and the real Guangdong Power Grid have verified the effectiveness of the proposed approach. An illustrative application example indicates the potential of the proposed approach in tackling STVS-based SEP for the stable operation of energy systems.

A deep transfer learning approach for the detection and diagnosis of maxillary sinusitis on panoramic radiographs.

To investigate the use of transfer learning when applying a deep learning source model from one institution (institution A) to another institution (institution B) for creating effective models (target models) for the detection of maxillary sinuses and diagnosis of maxillary sinusitis on panoramic radiographs. In addition, to determine appropriate numbers of training data for the transfer learning. Source model was created using 350 panoramic radiographs from institution A as training data. Transfer learning was performed by adding 25, 50, 100, 150, or 225 panoramic radiographs as training data from institution B to the source model; this yielded the target models T25, T50, T100, T150 and T225. Each model was then evaluated using test data that comprised 40 images from institution A, 30 images from institution B. The performance indices (recall, precision and F1 score) for detecting the maxillary sinuses by the source model exceeded 0.98 when using test data A from institution A, but they deteriorated when using test data B from institution B. In the evaluation of target models using test data B, model T25 showed improved detection performance (recall of 0.967). The diagnostic performance of model T50 for maxillary sinusitis exceeded 0.9 in sensitivity. Transfer learning, which involves applying a small amount of data to the source model, yielded high performances in detecting the maxillary sinuses and diagnosing the maxillary sinusitis on panoramic radiographs. This study serves as a reference when adapting source models to other institutions.

Deep Transfer Learning Approach for Automatic Recognition of Drug Toxicity and Inhibition of SARS-CoV-2.

Severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2) causes COVID-19 and is responsible for the ongoing pandemic. Screening of potential antiviral drugs against SARS-CoV-2 depend on in vitro experiments, which are based on the quantification of the virus titer. Here, we used virus-induced cytopathic effects (CPE) in brightfield microscopy of SARS-CoV-2-infected monolayers to quantify the virus titer. Images were classified using deep transfer learning (DTL) that fine-tune the last layers of a pre-trained Resnet18 (ImageNet). To exclude toxic concentrations of potential drugs, the network was expanded to include a toxic score (TOX) that detected cell death (CPETOXnet). With this analytic tool, the inhibitory effects of chloroquine, hydroxychloroquine, remdesivir, and emetine were validated. Taken together we developed a simple method and provided open access implementation to quantify SARS-CoV-2 titers and drug toxicity in experimental settings, which may be adaptable to assays with other viruses. The quantification of virus titers from brightfield images could accelerate the experimental approach for antiviral testing.

Adaptive Technique for Brightness Enhancement of Automated Knife Detection in Surveillance Video with Deep Learning

Detecting knives in surveillance videos are very urgent for public safety. In general, the research in identifying dangerous weapons is relatively new. Knife detection is a very challenging task because knives vary in size and shape. Besides, it easily reflects lights that reduce the visibility of knives in a video sequence. The reflection of light on the surface of the knife and the brightness on its surface makes the detection process extremely difficult, even impossible. This paper presents an adaptive technique for brightness enhancement of knife detection in surveillance systems. This technique overcomes the brightness problem that faces the steel weapons and improves the knife detection process. It suggests an automatic threshold to assess the level of frame brightness. Depending on this threshold, the proposed technique determines if the frame needs to enhance its brightness or not. Experimental results verify the efficiency of the proposed technique in detecting knives using the deep transfer learning approach. Moreover, the most four famous models of deep convolutional neural networks are tested to select the best in detecting knives. Finally, a comparison is made with the-state-of-the-art techniques, and the proposed technique proved its superiority.

Human-Centered Reflective Writing Analytics with a Deep Transfer Learning Approach

Human-Centered Reflective Writing Analytics with a Deep Transfer Learning Approach

Classification of Planetary Nebulae through Deep Transfer Learning

This study investigate the effectiveness of using Deep Learning (DL) for the classification of planetary nebulae (PNe). It focusses on distinguishing PNe from other types of objects, as well as their morphological classification. We adopted the deep transfer learning approach using three ImageNet pre-trained algorithms. This study was conducted using images from the Hong Kong/Australian Astronomical Observatory/Strasbourg Observatory H-alpha Planetary Nebula research platform database (HASH DB) and the Panoramic Survey Telescope and Rapid Response System (Pan-STARRS). We found that the algorithm has high success in distinguishing True PNe from other types of objects even without any parameter tuning. The Matthews correlation coefficient is 0.9. Our analysis shows that DenseNet201 is the most effective DL algorithm. For the morphological classification, we found for three classes, Bipolar, Elliptical and Round, half of objects are correctly classified. Further improvement may require more data and/or training. We discuss the trade-offs and potential avenues for future work and conclude that deep transfer learning can be utilized to classify wide-field astronomical images.

Effect of deep transfer and multi-task learning on sperm abnormality detection

Analyzing the abnormality of morphological characteristics of male human sperm has been studied for a long time mainly because it has many implications on the male infertility problem, which accounts for approximately half of the infertility problems in the world. Yet, detecting such abnormalities by embryologists has several downsides. To clarify, analyzing sperms through visual inspection of an expert embryologist is a highly subjective and biased process. Furthermore, it takes much time for a specialist to make a diagnosis. Hence, in this paper, we proposed two deep learning algorithms that are able to automate this process. The first algorithm uses a network-based deep transfer learning approach, while the second technique, named Deep Multi-task Transfer Learning (DMTL), employs a novel combination of network-based deep transfer learning and multi-task learning to classify sperm's head, vacuole, and acrosome as either normal or abnormal. This DMTL technique is capable of classifying all the aforementioned parts of the sperm in a single prediction. Moreover, this is the first time that the concept of multi-task learning has been introduced to the field of Sperm Morphology Analysis (SMA). To benchmark our algorithms, we employed a freely-available SMA dataset named MHSMA. During our experiments, our algorithms reached the state-of-the-art results on the accuracy, precision, and f0.5, as well as other important metrics, such as the Matthews Correlation Coefficient on one, two, or all three labels. Notably, our algorithms increased the accuracy of the head, acrosome, and vacuole by 6.66%, 3.00%, and 1.33%, and reached the accuracy of 84.00%, 80.66%, and 94.00% on these labels, respectively. Consequently, our algorithms can be used in health institutions, such as fertility clinics, with further recommendations to practically improve the performance of our algorithms.

Brain MRI‐based Wilson disease tissue classification: an optimised deep transfer learning approach

Wilson's disease (WD) is caused by the excessive accumulation of copper in the brain and liver, leading to death if not diagnosed early. WD shows its prevalence as white matter hyperintensity (WMH) in MRI scans. It is challenging and tedious to classify WD against controls when comparing visually, primarily due to subtle differences in WMH. This Letter presents a computer-aided design-based automated classification strategy that uses optimised transfer learning (TL) utilising two novel paradigms known as (i) MobileNet and (ii) the Visual Geometric Group-19 (VGG-19). Further, the authors benchmark TL systems against a machine learning (ML) paradigm. Using four-fold augmentation, VGG-19 is superior to MobileNet demonstrating accuracy and area under the curve (AUC) pairs as 95.46 ± 7.70 %, 0.932 (p < 0.0001 ) and 86.87 ± 2.23 %, 0.871 (p < 0.0001 ), respectively. Further, MobileNet and VGG-19 showed an improvement of 3.4 and 13.5%, respectively, when benchmarked against the ML-based soft classifier – Random Forest.

Deep transfer with minority data augmentation for imbalanced breast cancer dataset

Clinical diagnosis of breast cancer is a challenging problem in the biomedical domain. The BreakHis breast cancer histopathological image dataset consists of two classes: Benign (Minority class) and Malignant (Majority class). The imbalanced class distribution results in the degradation of performance of the classifier model due to biased classification towards the majority class. To tackle this problem, a novel learning strategy that involves a deep transfer network has been proposed in this paper, in collaboration with Deep Convolution Generative Adversarial network (DCGAN). DCGAN is used in the initial phase for data augmentation of the minority class only. The dataset, with the class distribution now balanced, is applied as input to the deep transfer network. The proposed deep transfer architecture has at its core, the initial pre-trained layers (until block 4 pool layer) of the VGG16 deep network architecture pre-trained on the ImageNet object classification dataset. The higher end of our transfer network comprises of Batch Normalization, 2D Convolutional (CONV2D) layer, Global Average Pooling 2D, Dropout and Dense layers that are added to enhance the network’s performance. Experiments on the benchmark BreakHis dataset for different magnification factors: 40X, 100X, 200X and 400X validate the efficiency of the proposed deep transfer learning approach due to the high scores achieved as compared to the state-of-the-art deep networks.

Noise reduction with cross-tracer and cross-protocol deep transfer learning for low-dose PET

Previous studies have demonstrated the feasibility of reducing noise with deep learning-based methods for low-dose fluorodeoxyglucose (FDG) positron emission tomography (PET). This work aimed to investigate the feasibility of noise reduction for tracers without sufficient training datasets using a deep transfer learning approach, which can utilize existing networks trained by the widely available FDG datasets. In this study, the deep transfer learning strategy based on a fully 3D patch-based U-Net was investigated on a 18F-fluoromisonidazole (18F-FMISO) dataset using single-bed scanning and a 68Ga-DOTATATE dataset using whole-body scanning. The datasets of 18F-FDG by single-bed scanning and whole-body scanning were used to obtain pre-trained U-Nets separately for subsequent cross-tracer and cross-protocol transfer learning. The full-dose PET images were used as the labels while low-dose PET images from 10% counts were used as the inputs. Three types of U-Nets were obtained: a U-Net trained by FDG dataset, a pre-trained FDG U-Net fine-tuned by another less-available tracer (FMISO/DOATATE), and a U-Net completely trained by a large number of less-available tracer datasets (FMISO/DOATATE), used as the reference U-Net. The denoising performance of the three types of U-Nets was evaluated on single-bed 18F-FMISO and whole-body 68Ga-DOTATATE separately and compared using normalized root-mean-square error (NRMSE), signal-to-noise ratio (SNR), and relative bias of region of interest (ROI). For cross-tracer transfer learning, all the U-Nets provided denoised images with similar quality for both tracers. There was no significant difference in terms of NRMSE and SNR when comparing the former two U-Nets with the reference U-Net. The ROI biases for these U-Nets were similar. For cross-tracer and cross-protocol transfer learning, the pre-trained single-bed FDG U-Net fine-tuned by whole-body DOTATATE data provided the most consistent images with the reference U-Net. Fine-tuning significantly reduced the NRMSE and the ROI bias and improved the SNR when comparing the fine-tuned U-Net with the U-Net trained by single-bed FDG only (NRMSE: 96.3% ± 21.1% versus 120.6% ± 18.5%, ROI bias: −10.5% ± 13.0% versus −14.7% ± 6.4%, SNR: 4.2 ± 1.4 versus 3.9 ± 1.6, for fine-tuned U-Net and the U-Net trained by single-bed FDG, respectively, with p < 0.01 in all cases). This work demonstrated that it is feasible to utilize existing networks well-trained by FDG datasets to reduce the noise for other less-available tracers and other scanning protocols by using the fine-tuning strategy.

Deep Transfer Learning for Signal Detection in Ambient Backscatter Communications

Tag signal detection is one of the key tasks in ambient backscatter communication (AmBC) systems. However, obtaining perfect channel state information (CSI) is challenging and costly, which makes AmBC systems suffer from a high bit error rate (BER). To eliminate the requirement of channel estimation and to improve the system performance, in this paper, we adopt a deep transfer learning (DTL) approach to implicitly extract the features of channel and directly recover tag symbols. To this end, we develop a DTL detection framework which consists of offline learning, transfer learning, and online detection. Specifically, a DTL-based likelihood ratio test (DTL-LRT) is derived based on the minimum error probability (MEP) criterion. As a realization of the developed framework, we then apply convolutional neural networks (CNN) to intelligently explore the features of the sample covariance matrix, which facilitates the design of a CNN-based algorithm for tag signal detection. Exploiting the powerful capability of CNN in extracting features of data in the matrix formation, the proposed method is able to further improve the system performance. In addition, an asymptotic explicit expression is also derived to characterize the properties of the proposed CNN-based method when the number of samples is sufficiently large. Finally, extensive simulation results demonstrate that the BER performance of the proposed method is comparable to that of the optimal detection method with perfect CSI.

Towards More Accurate Automatic Sleep Staging via Deep Transfer Learning

Despite recent significant progress in the development of automatic sleep staging methods, building a good model still remains a big challenge for sleep studies with a small cohort due to the data-variability and data-inefficiency issues. This work presents a deep transfer learning approach to overcome these issues and enable transferring knowledge from a large dataset to a small cohort for automatic sleep staging. We start from a generic end-to-end deep learning framework for sequence-to-sequence sleep staging and derive two networks as the means for transfer learning. The networks are first trained in the source domain (i.e. the large database). The pretrained networks are then finetuned in the target domain (i.e. the small cohort) to complete knowledge transfer. We employ the Montreal Archive of Sleep Studies (MASS) database consisting of 200 subjects as the source domain and study deep transfer learning on three different target domains: the Sleep Cassette subset and the Sleep Telemetry subset of the Sleep-EDF Expanded database, and the Surrey-cEEGrid database. The target domains are purposely adopted to cover different degrees of data mismatch to the source domains. Our experimental results show significant performance improvement on automatic sleep staging on the target domains achieved with the proposed deep transfer learning approach. These results suggest the efficacy of the proposed approach in addressing the above-mentioned data-variability and data-inefficiency issues. As a consequence, it would enable one to improve the quality of automatic sleep staging models when the amount of data is relatively small.11The source code and the pretrained models are published at https://github.com/pquochuy/sleep_transfer_learning.

Deep Transfer Learning Approach for Detection of Covid-19 from Chest X-Ray Images

Deep Transfer Learning Approach for Detection of Covid-19 from Chest X-Ray Images

Sparse Array Selection Across Arbitrary Sensor Geometries With Deep Transfer Learning

Sparse sensor array selection arises in many engineering applications, where it is imperative to obtain maximum spatial resolution from a limited number of array elements. Recent research shows that computational complexity of array selection is reduced by replacing the conventional optimization and greedy search methods with a deep learning network. However, in practice, sufficient and well-calibrated labeled training data are unavailable and, more so, for arbitrary array configurations. To address this, we adopt a deep transfer learning (TL) approach, wherein we train a deep convolutional neural network (CNN) with data of a source sensor array for which calibrated data are readily available and reuse this pre-trained CNN for a different, data-insufficient target array geometry to perform sparse array selection. Numerical experiments with uniform rectangular and circular arrays demonstrate enhanced performance of TL-CNN on the target model than the CNN trained with insufficient data from the same model. In particular, our TL framework provides approximately 20% higher sensor selection accuracy and 10% improvement in the direction-of-arrival estimation error.

Human Identification for Activities of Daily Living: A Deep Transfer Learning Approach

ABSTRACT Sensor-based home Activities of Daily Living (ADLs) monitoring systems have emerged to monitor elderly people’s self-care ability remotely. However, the unobtrusive, privacy-friendly object motion sensor-based systems face challenges such as scarce labeled data and ADL performer confusion in a multi-resident setting. This study adopts the design science paradigm to develop an innovative deep transfer learning framework for human identification (DTL-HID) to address both challenges. A novel convolutional neural network (CNN) is proposed to automatically extract comprehensive temporal and cross-axial motion patterns for the DTL-HID framework. We rigorously evaluate the DTL-HID framework against state-of-the-art benchmarks (e.g., k Nearest Neighbors, Support Vector Machines, and alternative CNN designs). Results demonstrate our proposed DTL-HID framework can identify the ADL performer accurately even on a small amount of labeled data. We demonstrate a case study and discuss how stakeholders can further apply this approach to unobtrusive smart home monitoring for senior citizens. Beyond demonstrating the framework’s practical utility, we discuss two implications of our design principles to mobile analytics and design science research: (1) extracting temporal and axial local dependencies can capture richer information from multi-axial time-series data and (2) transferring knowledge learned on a relevant source domain with sufficient data can improve the performance of the desired task on the target domain with scarce data.

A Deep Transfer Learning Approach to Reducing the Effect of Electrode Shift in EMG Pattern Recognition-Based Control.

An important barrier to commercialization of pattern recognition myoelectric control of prostheses is the lack of robustness to confounding factors such as electrode shift, skin impedance variations, and learning effects. To overcome this challenge, a novel supervised adaptation approach based on transfer learning (TL) with convolutional neural networks (CNNs) is proposed which requires only a short training session (a few seconds for each class) to recalibrate the system. TL is proposed as a solution to the problem of insufficient calibration data due to short training times for both classification and regression-based control schemes. This approach was validated for electrode shift of roughly 2.5cm with 13 able-bodied subjects to estimate individual and combined wrist motions. With this method, the original CNN (trained before the shift) was fine-tuned with the calibration data from after shifting. The results show that the proposed technique outperforms training a CNN from scratch (random initialization of weights) or a support vector machine (SVM) using the minimal calibration data. Moreover, it demonstrates superior performance than previous LDA and QDA-based adaptation approaches. As the outcomes confirm, the proposed CNN TL method provides a practical solution for adaptation to external factors, improving the robustness of electromyogram (EMG) pattern recognition systems.

Deep Transfer Bug Localization

Many projects often receive more bug reports than what they can handle. To help debug and close bug reports, a number of bug localization techniques have been proposed. These techniques analyze a bug report and return a ranked list of potentially buggy source code files. Recent development on bug localization has resulted in the construction of effective supervised approaches that use historical data of manually localized bugs to boost performance. Unfortunately, as highlighted by Zimmermann et al., sufficient bug data is often unavailable for many projects and companies. This raises the need for cross-project bug localization - the use of data from a project to help locate bugs in another project. To fill this need, we propose a deep transfer learning approach for cross-project bug localization. Our proposed approach named TRANP-CNN extracts transferable semantic features from source project and fully exploits labeled data from target project for effective cross-project bug localization. We have evaluated TRANP-CNN on curated high-quality bug datasets and our experimental results show that TRANP-CNN can locate buggy files correctly at top 1, top 5, and top 10 positions for 29.9, 51.7, 61.3 percent of the bugs respectively, which significantly outperform state-of-the-art bug localization solution based on deep learning and several other advanced alternative solutions considering various standard evaluation metrics.

An Effective MR-Guided CT Network Training for Segmenting Prostate in CT Images.

Segmentation of prostate in medical imaging data (e.g., CT, MRI, TRUS) is often considered as a critical yet challenging task for radiotherapy treatment. It is relatively easier to segment prostate from MR images than from CT images, due to better soft tissue contrast of the MR images. For segmenting prostate from CT images, most previous methods mainly used CT alone, and thus their performances are often limited by low tissue contrast in the CT images. In this article, we explore the possibility of using indirect guidance from MR images for improving prostate segmentation in the CT images. In particular, we propose a novel deep transfer learning approach, i.e., MR-guided CT network training (namely MICS-NET), which can employ MR images to help better learning of features in CT images for prostate segmentation. In MICS-NET, the guidance from MRI consists of two steps: (1) learning informative and transferable features from MRI and then transferring them to CT images in a cascade manner, and (2) adaptively transferring the prostate likelihood of MRI model (i.e., well-trained convnet by purely using MR images) with a view consistency constraint. To illustrate the effectiveness of our approach, we evaluate MICS-NET on a real CT prostate image set, with the manual delineations available as the ground truth for evaluation. Our methods generate promising segmentation results which achieve (1) six percentages higher Dice Ratio than the CT model purely using CT images and (2) comparable performance with the MRI model purely using MR images.

Automatic Detection of Diabetic Retinopathy in Retinal Fundus Photographs Based on Deep Learning Algorithm.

PurposeTo achieve automatic diabetic retinopathy (DR) detection in retinal fundus photographs through the use of a deep transfer learning approach using the Inception-v3 network.MethodsA total of 19,233 eye fundus color numerical images were retrospectively obtained from 5278 adult patients presenting for DR screening. The 8816 images passed image-quality review and were graded as no apparent DR (1374 images), mild nonproliferative DR (NPDR) (2152 images), moderate NPDR (2370 images), severe NPDR (1984 images), and proliferative DR (PDR) (936 images) by eight retinal experts according to the International Clinical Diabetic Retinopathy severity scale. After image preprocessing, 7935 DR images were selected from the above categories as a training dataset, while the rest of the images were used as validation dataset. We introduced a 10-fold cross-validation strategy to assess and optimize our model. We also selected the publicly independent Messidor-2 dataset to test the performance of our model. For discrimination between no referral (no apparent DR and mild NPDR) and referral (moderate NPDR, severe NPDR, and PDR), we also computed prediction accuracy, sensitivity, specificity, area under the receiver operating characteristic curve (AUC), and κ value.ResultsThe proposed approach achieved a high classification accuracy of 93.49% (95% confidence interval [CI], 93.13%–93.85%), with a 96.93% sensitivity (95% CI, 96.35%–97.51%) and a 93.45% specificity (95% CI, 93.12%–93.79%), while the AUC was up to 0.9905 (95% CI, 0.9887–0.9923) on the independent test dataset. The κ value of our best model was 0.919, while the three experts had κ values of 0.906, 0.931, and 0.914, independently.ConclusionsThis approach could automatically detect DR with excellent sensitivity, accuracy, and specificity and could aid in making a referral recommendation for further evaluation and treatment with high reliability.Translational RelevanceThis approach has great value in early DR screening using retinal fundus photographs.