Deep Transfer Learning Techniques Research Articles

Deep transfer learning techniques with hybrid optimization in early prediction and diagnosis of different types of oral cancer

Deep transfer learning techniques with hybrid optimization in early prediction and diagnosis of different types of oral cancer

Multi-Scale Hybrid Network for Polyp Detection in Wireless Capsule Endoscopy and Colonoscopy Images.

The trade-off between speed and precision is a key step in the detection of small polyps in wireless capsule endoscopy (WCE) images. In this paper, we propose a hybrid network of an inception v4 architecture-based single-shot multibox detector (Hyb-SSDNet) to detect small polyp regions in both WCE and colonoscopy frames. Medical privacy concerns are considered the main barriers to WCE image acquisition. To satisfy the object detection requirements, we enlarged the training datasets and investigated deep transfer learning techniques. The Hyb-SSDNet framework adopts inception blocks to alleviate the inherent limitations of the convolution operation to incorporate contextual features and semantic information into deep networks. It consists of four main components: (a) multi-scale encoding of small polyp regions, (b) using the inception v4 backbone to enhance more contextual features in shallow and middle layers, and (c) concatenating weighted features of mid-level feature maps, giving them more importance to highly extract semantic information. Then, the feature map fusion is delivered to the next layer, followed by some downsampling blocks to generate new pyramidal layers. Finally, the feature maps are fed to multibox detectors, consistent with the SSD process-based VGG16 network. The Hyb-SSDNet achieved a 93.29% mean average precision (mAP) and a testing speed of 44.5 FPS on the WCE dataset. This work proves that deep learning has the potential to develop future research in polyp detection and classification tasks.

Computer vision classification of dry beans (Phaseolus vulgaris L.) based on deep transfer learning techniques

Computer vision classification of dry beans (Phaseolus vulgaris L.) based on deep transfer learning techniques

A comparative study of shallow learning and deep transfer learning techniques for accurate fingerprints vitality detection

• Deep Transfer Learning architectures are compared against shallow learning image descriptors under the same conditions. • Deep transfer learning for fingerprint liveness detection is a viable way. • XAI shows that deep transfer learning extracts complex nonlinear patterns able to discern minutiae for liveness detection. This work inspects deep learning architectures and shallow learning techniques to determine whether the image of a fingerprint is real (Live) or not (Fake). It is known that Deep Learning techniques deliver, in general, good accuracies being able to automatically extract relevant patterns, at the same time, it is also known that these algorithms require large amounts of data. For this reason, transfer learning aims to transfer the knowledge learnt over a huge dataset to a new, smaller dataset. In this work, because of the limited size of the LivDet2019 dataset, three well known deep learning architectures such as Inception V3, ResNet50 and NASNet Large have been modified to perform transfer learning from the huge imagenet dataset to the smaller LivDet2019. The hypothesis at the very basis of this work is that the deep learning architectures trained on the huge imagenet dataset would learn to extract relevant patterns like lines, shapes, curves, jump between curves, etc… Later, the extracted knowledge, is fine-tuned on the LivDet2019 dataset to recognize fingerprint minuities as a non-linear combination of the previously learned patterns. For sake of completeness, state of art shallow learning image descriptors, finetuned for fingerprint recognition, such as Binarized Statistical Image Features (BSIF), Local Phase Quantization (LPQ) and Weber Local Descriptor (WLD) are used for extracting features from the LivDet2019 dataset. The classification on each of these extracted features is performed both with a linear and non-linear (gaussian) support vector machine. Accuracies suggest that both shallow learning and deep learning techniques are on par with the accuracies of reviewed works and thus transfer learning in fingerprint liveness detection is a feasible strategy that deserve attention and future research with the aim of increasing fingerprint detection accuracies.

Development and Comparison of Skin Cancer Diagnosis Models

Skin cancer is the uncontrolled growth of abnormal cells in the epidermis, the outermost layer of skin. The rapid growth and proliferation of abnormal cells creates malignant tumors of the skin. With the computer analysis of skin images, researchers are made to distinguish whether the skin spot is benign or malign It is automatically possible to classify whether a skin spot is benign or malignant by computer analysis of skin images. In this study, it was aimed to diagnose malignant skin images by computer analysis. The stained appearance on the skin is classified as benign or malignant using deep transfer learning techniques. Benign or malignant skin spot image data were used in network training. In image classification, darkNet-19, darkNet-53, squeezeNet, shufleNet architectures available in the Matlab deep learning toolbox were compared. High accuracy results have been obtained. The highest performance was achieved with the rate of 89.89% with darkNet-19 architecture. The performances of the networks darkNet-53, shuffleNet, squeezeNet architectures are 87.36%, 86.15%, 84.23% respectively.

A perspective survey on deep transfer learning for fault diagnosis in industrial scenarios: Theories, applications and challenges

Deep Transfer Learning (DTL) is a new paradigm of machine learning, which can not only leverage the advantages of Deep Learning (DL) in feature representation, but also benefit from the superiority of Transfer Learning (TL) in knowledge transfer. As a result, DTL techniques can make DL-based fault diagnosis methods more reliable, robust and applicable, and they have been widely developed and investigated in the field of Intelligent Fault Diagnosis (IFD). Although several systematic and valuable review articles have been published on the topic of IFD, they summarized relevant research only from an algorithm perspective and overlooked practical applications in industry scenarios. Furthermore, a comprehensive review on DTL-based IFD methods is still lacking. From this insight, it is particularly important and more necessary to comprehensively survey the relevant publications of DTL-based IFD, which will help readers to conveniently understand the current state-of-the-art techniques and to quickly design an effective solution for solving IFD problems in practice. First, theoretical backgrounds of DTL are briefly introduced to present how the transfer learning techniques can be integrated with deep learning models. Then, major applications of DTL and their recent developments in the field of IFD are detailed and discussed. More importantly, suggestions on how to select DTL algorithms in practical applications, and some future challenges are shared. Finally, conclusions of this survey are given. At last, we have reason to believe that the works done in this article can provide convenience and inspiration for the researchers who want to devote their efforts in the progress and advance of IFD.

Star cluster classification in the PHANGS–HST survey: Comparison between human and machine learning approaches

ABSTRACT When completed, the PHANGS–HST project will provide a census of roughly 50 000 compact star clusters and associations, as well as human morphological classifications for roughly 20 000 of those objects. These large numbers motivated the development of a more objective and repeatable method to help perform source classifications. In this paper, we consider the results for five PHANGS–HST galaxies (NGC 628, NGC 1433, NGC 1566, NGC 3351, NGC 3627) using classifications from two convolutional neural network architectures (RESNET and VGG) trained using deep transfer learning techniques. The results are compared to classifications performed by humans. The primary result is that the neural network classifications are comparable in quality to the human classifications with typical agreement around 70 to 80 per cent for Class 1 clusters (symmetric, centrally concentrated) and 40 to 70 per cent for Class 2 clusters (asymmetric, centrally concentrated). If Class 1 and 2 are considered together the agreement is 82 ± 3 per cent. Dependencies on magnitudes, crowding, and background surface brightness are examined. A detailed description of the criteria and methodology used for the human classifications is included along with an examination of systematic differences between PHANGS–HST and LEGUS. The distribution of data points in a colour–colour diagram is used as a ‘figure of merit’ to further test the relative performances of the different methods. The effects on science results (e.g. determinations of mass and age functions) of using different cluster classification methods are examined and found to be minimal.

Exploring Deep Transfer Learning Techniques for Alzheimer's Dementia Detection.

Examination of speech datasets for detecting dementia, collected via various speech tasks, has revealed links between speech and cognitive abilities. However, the speech dataset available for this research is extremely limited because the collection process of speech and baseline data from patients with dementia in clinical settings is expensive. In this paper, we study the spontaneous speech dataset from a recent ADReSS challenge, a Cookie Theft Picture (CTP) dataset with balanced groups of participants in age, gender, and cognitive status. We explore state-of-the-art deep transfer learning techniques from image, audio, speech, and language domains. We envision that one advantage of transfer learning is to eliminate the design of handcrafted features based on the tasks and datasets. Transfer learning further mitigates the limited dementia-relevant speech data problem by inheriting knowledge from similar but much larger datasets. Specifically, we built a variety of transfer learning models using commonly employed MobileNet (image), YAMNet (audio), Mockingjay (speech), and BERT (text) models. Results indicated that the transfer learning models of text data showed significantly better performance than those of audio data. Performance gains of the text models may be due to the high similarity between the pre-training text dataset and the CTP text dataset. Our multi-modal transfer learning introduced a slight improvement in accuracy, demonstrating that audio and text data provide limited complementary information. Multi-task transfer learning resulted in limited improvements in classification and a negative impact in regression. By analyzing the meaning behind the AD/non-AD labels and Mini-Mental State Examination (MMSE) scores, we observed that the inconsistency between labels and scores could limit the performance of the multi-task learning, especially when the outputs of the single-task models are highly consistent with the corresponding labels/scores. In sum, we conducted a large comparative analysis of varying transfer learning models focusing less on model customization but more on pre-trained models and pre-training datasets. We revealed insightful relations among models, data types, and data labels in this research area.

Classifying maturity of cherry tomatoes using Deep Transfer Learning techniques

This research studies a method to classify the tomatoes’ maturity by using deep transfer learning techniques. We carry out sorting systems adopting three pre-trained convolutional neural networks of VGG16, VGG19, and ResNet101. The experimental results show that the VGG19 model obtains a high precision on both the train set and the test set.

Brain tumour classification using siamese neural network and neighbourhood analysis in embedded feature space

AbstractThe application of deep transfer learning techniques has been successful in developing accurate systems for brain tumour classification on large‐scale medical image databases. For small databases, feature learning by deep neural networks is not robust. The systems based on domain‐specific hand‐crafted features have limited accuracy. In this paper, the authors focus on developing accurate models that could be trained effectively using a smaller number of data samples. A siamese neural network (SNN) is designed to extract features from brain magnetic resonance imaging (MRI) images. The SNN is realised using a 3‐layer, fully connected neural network. The designed SNN has lesser complexity and fewer parameters than deep transfer‐learned convolutional neural networks (CNN). A nearest neighbourhood analysis, using Euclidean and Mahalanobis distances, is conducted on the SNN encoded feature space. The encoded feature space is two dimensional, such that the neighbourhood analysis is computationally less intensive. For the neighbourhood analysis, a k‐nearest neighbour (k‐NN) model is utilised. The proposed method is evaluated using three publicly available datasets, namely, Radiopaedia, Harvard and Figshare repositories. The respective classification accuracy on cross‐validation is 92.6%, 98.5% and 92.6%. Other metrics used for the performance evaluation include F‐score, Specificity and balanced accuracy. The underlying network architecture and the design choice of network layers allow the implementation of the SNN in environments with low computational resources. The SNN features are found to be more effective than the hand‐designed features, and the deep transfer learned features for the stated problem.

On migratable traffic risk estimation in urban sensing: A social sensing based deep transfer network approach

This paper focuses on the migratable traffic risk estimation problem in intelligent transportation systems using the social sensing. The goal is to accurately estimate the traffic risk of a target area where the ground truth traffic accident reports are not available by leveraging an estimation model from a source area where such data is available. Two important challenges exist. The first challenge lies in the discrepancy between source and target areas and such discrepancy would prevent a direct application of a model from the source area to the target area. The second challenge lies in the difficulty of identifying all potential features in the migratable traffic risk estimation problem and decide the importance of identified features due to the lack of ground truth labels in the target area. To address these challenges, we develop DeepRisk, a social sensing based migratable traffic risk estimation scheme using deep transfer learning techniques. The evaluation results on a real world dataset in New York City show the DeepRisk significantly outperforms the state-of-the-art baselines in accurately estimating the traffic risk of locations in a city.

Facial Expression Recognition Based on Weighted-Cluster Loss and Deep Transfer Learning Using a Highly Imbalanced Dataset.

Facial expression recognition (FER) is a challenging problem in the fields of pattern recognition and computer vision. The recent success of convolutional neural networks (CNNs) in object detection and object segmentation tasks has shown promise in building an automatic deep CNN-based FER model. However, in real-world scenarios, performance degrades dramatically owing to the great diversity of factors unrelated to facial expressions, and due to a lack of training data and an intrinsic imbalance in the existing facial emotion datasets. To tackle these problems, this paper not only applies deep transfer learning techniques, but also proposes a novel loss function called weighted-cluster loss, which is used during the fine-tuning phase. Specifically, the weighted-cluster loss function simultaneously improves the intra-class compactness and the inter-class separability by learning a class center for each emotion class. It also takes the imbalance in a facial expression dataset into account by giving each emotion class a weight based on its proportion of the total number of images. In addition, a recent, successful deep CNN architecture, pre-trained in the task of face identification with the VGGFace2 database from the Visual Geometry Group at Oxford University, is employed and fine-tuned using the proposed loss function to recognize eight basic facial emotions from the AffectNet database of facial expression, valence, and arousal computing in the wild. Experiments on an AffectNet real-world facial dataset demonstrate that our method outperforms the baseline CNN models that use either weighted-softmax loss or center loss.

Comparison of Deep Transfer Learning Techniques in Human Skin Burns Discrimination

While visual assessment is the standard technique for burn evaluation, computer-aided diagnosis is increasingly sought due to high number of incidences globally. Patients are increasingly facing challenges which are not limited to shortage of experienced clinicians, lack of accessibility to healthcare facilities and high diagnostic cost. Certain number of studies were proposed in discriminating burn and healthy skin using machine learning leaving a huge and important gap unaddressed; whether burns and related skin injuries can be effectively discriminated using machine learning techniques. Therefore, we specifically use transfer learning by leveraging pre-trained deep learning models due to deficient dataset in this paper, to discriminate two classes of skin injuries—burnt skin and injured skin. Experiments were extensively conducted using three state-of-the-art pre-trained deep learning models that includes ResNet50, ResNet101 and ResNet152 for image patterns extraction via two transfer learning strategies—fine-tuning approach where dense and classification layers were modified and trained with features extracted by base layers and in the second approach support vector machine (SVM) was used to replace top-layers of the pre-trained models, trained using off-the-shelf features from the base layers. Our proposed approach records near perfect classification accuracy in categorizing burnt skin ad injured skin of approximately 99.9%.

Deep transfer learning for gesture recognition with WiFi signals

Different gestures have different action speeds, directions, and trajectories that can cause distinctive effects on the propagation of WiFi signals. In this paper, we present a new approach that uses deep transfer learning techniques to recognize gestures based on the channel state information (CSI) extracted from WiFi signals. Firstly, the CSI streams of gestures are captured and the gesture segments are extracted based on the CSI amplitude changes, and then the WiFi-based gesture recognition problem is innovatively converted to an image classification problem by expressing CSI streams as an image matrix. After that, two deep transfer learning methods are applied to recognize gestures using high-level features extracted by deep convolutional neural network (CNN) and fine-tuned CNN models. We evaluated our method using a collected dataset with 12 gestures in two environments, and the experimental results demonstrated that the proposed method outperformed other state-of-the-art WiFi-based gesture recognition methods.

Computer-aided diagnosis of congenital abnormalities of the kidney and urinary tract in children based on ultrasound imaging data by integrating texture image features and deep transfer learning image features

Anatomic characteristics of kidneys derived from ultrasound images are potential biomarkers of children with congenital abnormalities of the kidney and urinary tract (CAKUT), but current methods are limited by the lack of automated processes that accurately classify diseased and normal kidneys. The objective of the study was to evaluate the diagnostic performance of deep transfer learning techniques to classify kidneys of normal children and those with CAKUT. A transfer learning method was developed to extract features of kidneys from ultrasound images obtained during routine clinical care of 50 children with CAKUT and 50 controls. To classify diseased and normal kidneys, support vector machine classifiers were built on the extracted features using (1) transfer learning imaging features from a pretrained deep learning model, (2) conventional imaging features, and (3) their combination. These classifiers were compared, and their diagnosis performance was measured using area under the receiver operating characteristic curve (AUC), accuracy, specificity, and sensitivity. The AUC for classifiers built on the combination features were 0.92, 0.88, and 0.92 for discriminating the left, right, and bilateral abnormal kidney scans from controls with classification rates of 84%, 81%, and 87%; specificity of 84%, 74%, and 88%; and sensitivity of 85%, 88%, and 86%, respectively. These classifiers performed better than classifiers built on either the transfer learning features or the conventional features alone (p<0.001). The present study validated transfer learning techniques for imaging feature extraction of ultrasound images to build classifiers for distinguishing children with CAKUT from controls. The experiments have demonstrated that the classifiers built on the transfer learning features and conventional image features could distinguish abnormal kidney images from controls with AUCs greater than 0.88, indicating that classification of ultrasound kidney scans has a great potential to aid kidney disease diagnosis. A limitation of the present study is the moderate number of patients that contributed data to the transfer learning approach. The combination of transfer learning and conventional imaging features yielded the best classification performance for distinguishing children with CAKUT from controls based on ultrasound images of kidneys.