CNN-based Transfer Learning Research Articles

CNN-based transfer learning for forest aboveground biomass prediction from ALS point cloud tomography

ABSTRACT This study presents a new approach for predicting forest aboveground biomass (AGB) from airborne laser scanning (ALS) data: AGB is predicted from sequences of images depicting vertical cross-sections through the ALS point clouds. A 3D version of the VGG16 convolutional neural network (CNN) with initial weights transferred from pre-training on the ImageNet dataset was used. The approach was tested on datasets from Canada, Poland, and the Czech Republic. To analyse the effect of training sample size on model performance, different-sized samples ranging from 10 to 375 ground plots were used. The CNNs were compared with random forest models (RFs) trained on point cloud metrics. At the maximum number of training samples, the difference in RMSE between observed and predicted AGB of CNNs and RFs ranged from −2 t/ha to 5 t/ha, and the difference in squared Pearson correlation coefficient ranged from −0.05 to 0.06. Additional pre-training on synthetic data derived from virtual laser scanning of simulated forest stands could only improve the prediction performance of the CNNs when only a few real training samples (10–40) were available. While 3D CNNs trained on cross-section images derived from real data showed promising results, RFs remain a competitive alternative.

Optimizing mammography interpretation education: leveraging deep learning for cohort-specific error detection to enhance radiologist training.

Accurate interpretation of mammograms presents challenges. Tailoring mammography training to reader profiles holds the promise of an effective strategy to reduce these errors. This proof-of-concept study investigated the feasibility of employing convolutional neural networks (CNNs) with transfer learning to categorize regions associated with false-positive (FP) errors within screening mammograms into categories of "low" or "high" likelihood of being a false-positive detection for radiologists sharing similar geographic characteristics. Mammography test sets assessed by two geographically distant cohorts of radiologists (cohorts A and B) were collected. FP patches within these mammograms were segmented and categorized as "difficult" or "easy" based on the number of readers committing FP errors. Patches outside 1.5 times the interquartile range above the upper quartile were labeled as difficult, whereas the remaining patches were labeled as easy. Using transfer learning, a patch-wise CNN model for binary patch classification was developed utilizing ResNet as the feature extractor, with modified fully connected layers for the target task. Model performance was assessed using 10-fold cross-validation. Compared with other architectures, the transferred ResNet-50 achieved the highest performance, obtaining receiver operating characteristics area under the curve values of 0.933 ( ) and 0.975 ( ) on the validation sets for cohorts A and B, respectively. The findings highlight the feasibility of employing CNN-based transfer learning to predict the difficulty levels of local FP patches in screening mammograms for specific radiologist cohort with similar geographic characteristics.

Intelligent waste sorting for sustainable environment: A hybrid deep learning and transfer learning model

The significance of waste disposal, classification, and monitoring has dramatically increased due to the increase in industrial development and the progress of intelligent urbanization. Since the last few decades, the utilization of Deep learning techniques has grown increasingly in waste management research. The efficiency of a waste reuse and recycling process relies on its capacity to restore resources to their original state, thereby minimizing pollution and promoting an ecologically sustainable framework. Selecting the optimal deep-learning method for classifying and predicting waste is challenging and time-consuming. This paper proposed intelligent garbage categorization using Bidirectional Long Short-Term Memory (Bi-LSTM) and CNN-based transfer learning to improve environmental sustainability. Organic and recyclable garbage are separated. To simplify trash categorization, a hybrid model combines TL-based CNN and Bi-LSTM. This study extensively examined the suggested technique with numerous CNN computational methods, including VGG-19, ResNet-34, AlexNet, ResNet-50, and VGG-16, using the ’TrashNet Waste’ database. Key findings show that our hybrid model outperforms existing models. Our classification accuracy is 96.78 %, 5.27 % higher than the best model. Our model also reduces misclassification by 7.25 %, proving its reliability. This comprehensive examination examined the computer models’ trash classification performance and provided specific viewpoints. The results explain each technique’s pros and cons and show how useful they are in real-world circumstances. Waste classification is practical and sophisticated with a hybrid model. The effectiveness and cleverness of this model improve sustainable environmental practices. The proposed method’s excellent performance suggests its seamless integration into practical waste management solutions.

Evaluating Deep Learning Techniques for Detecting Aneurysmal Subarachnoid Hemorrhage: A Comparative Analysis of Convolutional Neural Network and Transfer Learning Models

Machine learning and deep learning techniques offer a promising multidisciplinary solution for subarachnoid hemorrhage (SAH) detection. The novel transfer learning approach mitigates the time constraints associated with the traditional techniques and demonstrates a superior performance. This study aims to evaluate the effectiveness of convolutional neural networks (CNNs) and CNN-based transfer learning models in differentiating between aneurysmal SAH and nonaneurysmal SAH. Data from Istanbul Ümraniye Training and Research Hospital, which included 15,600 digital imaging and communications in medicine images from 123 patients with aneurysmal SAH and 7793 images from 80 patients with nonaneurysmal SAH, were used. The study employed 4 models: Inception-V3, EfficientNetB4, single-layer CNN, and three-layer CNN. Transfer learning models were customized by modifying the last 3 layers and using the Adam optimizer. The models were trained on Google Collaboratory and evaluated based on metrics such as F-score, precision, recall, and accuracy. EfficientNetB4 demonstrated the highest accuracy (99.92%), with a better F-score (99.82%), recall (99.92%), and precision (99.90%) than the other methods. The single- and three-layer CNNs and the transfer learning models produced comparable results. No overfitting was observed, and robust models were developed. CNN-based transfer learning models can accurately diagnose the etiology of SAH from computed tomography images and is a valuable tool for clinicians. This approach could reduce the need for invasive procedures such as digital subtraction angiography, leading to more efficient medical resource utilization and improved patient outcomes.

Brain tumor detection from images and comparison with transfer learning methods and 3-layer CNN

Health is very important for human life. In particular, the health of the brain, which is the executive of the vital resource, is very important. Diagnosis for human health is provided by magnetic resonance imaging (MRI) devices, which help health decision makers in critical organs such as brain health. Images from these devices are a source of big data for artificial intelligence. This big data enables high performance in image processing classification problems, which is a subfield of artificial intelligence. In this study, we aim to classify brain tumors such as glioma, meningioma, and pituitary tumor from brain MR images. Convolutional Neural Network (CNN) and CNN-based inception-V3, EfficientNetB4, VGG19, transfer learning methods were used for classification. F-score, recall, imprinting and accuracy were used to evaluate these models. The best accuracy result was obtained with VGG16 with 98%, while the F-score value of the same transfer learning model was 97%, the Area Under the Curve (AUC) value was 99%, the recall value was 98%, and the precision value was 98%. CNN architecture and CNN-based transfer learning models are very important for human health in early diagnosis and rapid treatment of such diseases.

Negative Stances Detection from Multilingual Data Streams in Low-Resource Languages on Social Media Using BERT and CNN-Based Transfer Learning Model

Online social media allows users to connect with a large number of people across the globe and facilitate the exchange of information efficiently. These platforms cater to many of our day-to-day needs. However, at the same time, social media have been increasingly used to transmit negative stances such as derogatory language, hate speech, and cyberbullying. The task of identifying the negative stances from social media posts or comments or tweets is termed negative stance detection . One of the major challenges associated with negative stance detection is that most of the content published on social media is often in a multilingual format. This work aims to identify negative stances from multilingual data streams in low-resource languages on social media using a hybrid transfer learning and deep convolutional neural network approach. The proposed work starts by preprocessing the multilingual datasets by removing irrelevant information such as special characters and hyperlinks. The processed dataset is then passed through a pretrained BERT (bidirectional encoder representations from Transformers) model to generate embeddings by fine-tuning the model as per the dataset under consideration. The generated word embeddings are then passed to a deep convolutional neural network for extracting the latent features from the texts and removing the unessential information. This helps our model to achieve robustness and effectiveness for efficient learning on the given dataset and make appropriate predictions on zero-shot data. The article utilizes several optimization strategies for examining the impact of fine-tuning different BERT layers on the model’s performance. Intensive experiments on a variety of languages — namely, English, French, Italian, Danish, Arabic, Spanish, Indonesian, German, and Portuguese — are performed. The experimental results demonstrate the effectiveness and efficiency of the proposed framework.

Exploring CNN-based transfer learning approaches for Arabic alphabets sign language recognition using the ArSL2018 dataset

Exploring CNN-based transfer learning approaches for Arabic alphabets sign language recognition using the ArSL2018 dataset

Chem2Side: A Deep Learning Model with Ensemble Augmentation (Conventional + Pix2Pix) for COVID-19 Drug Side-Effects Prediction from Chemical Images

Drug side effects (DSEs) or adverse drug reactions (ADRs) are a major concern in the healthcare industry, accounting for a significant number of annual deaths in Europe alone. Identifying and predicting DSEs early in the drug development process is crucial to mitigate their impact on public health and reduce the time and costs associated with drug development. Objective: In this study, our primary objective is to predict multiple drug side effects using 2D chemical structures, especially for COVID-19, departing from the conventional approach of relying on 1D chemical structures. We aim to develop a novel model for DSE prediction that leverages the CNN-based transfer learning architecture of ResNet152V2. Motivation: The motivation behind this research stems from the need to enhance the efficiency and accuracy of DSE prediction, enabling the pharmaceutical industry to identify potential drug candidates with fewer adverse effects. By utilizing 2D chemical structures and employing data augmentation techniques, we seek to revolutionize the field of drug side-effect prediction. Novelty: This study introduces several novel aspects. The proposed study is the first of its kind to use 2D chemical structures for predicting drug side effects, departing from the conventional 1D approaches. Secondly, we employ data augmentation with both conventional and diffusion-based models (Pix2Pix), a unique strategy in the field. These innovations set the stage for a more advanced and accurate approach to DSE prediction. Results: Our proposed model, named CHEM2SIDE, achieved an impressive average training accuracy of 0.78. Moreover, the average validation and test accuracy, precision, and recall were all at 0.73. When evaluated for COVID-19 drugs, our model exhibited an accuracy of 0.72, a precision of 0.79, a recall of 0.72, and an F1 score of 0.73. Comparative assessments against established transfer learning and machine learning models (VGG16, MobileNetV2, DenseNet121, and KNN) showcased the exceptional performance of CHEM2SIDE, marking a significant advancement in drug side-effect prediction. Conclusions: Our study introduces a groundbreaking approach to predicting drug side effects by using 2D chemical structures and incorporating data augmentation. The CHEM2SIDE model demonstrates remarkable accuracy and outperforms existing models, offering a promising solution to the challenges posed by DSEs in drug development. This research holds great potential for improving drug safety and reducing the associated time and costs.

Microseismic event waveform classification using CNN-based transfer learning models

The efficient processing of large amounts of data collected by the microseismic monitoring system (MMS), especially the rapid identification of microseismic events in explosions and noise, is essential for mine disaster prevention. Currently, this work is primarily performed by skilled technicians, which results in severe workloads and inefficiency. In this paper, CNN-based transfer learning combined with computer vision technology was used to achieve automatic recognition and classification of multi-channel microseismic signal waveforms. First, data collected by MMS was generated into 6-channel original waveforms based on events. After that, sample data sets of microseismic events, blasts, drillings, and noises were established through manual identification. These datasets were split into training sets and test sets according to a certain proportion, and transfer learning was performed on AlexNet, GoogLeNet, and ResNet50 pre-training network models, respectively. After training and tuning, optimal models were retained and compared with support vector machine classification. Results show that transfer learning models perform well on different test sets. Overall, GoogLeNet performed best, with a recognition accuracy of 99.8%. Finally, the possible effects of the number of training sets and the imbalance of different types of sample data on the accuracy and effectiveness of classification models were discussed.

CNN-based Transfer Learning in Intelligent Recognition of Scrap Bundles

CNN-based Transfer Learning in Intelligent Recognition of Scrap Bundles

Robust detection of seam carving with low ratio via pixel adjacency subtraction and CNN-based transfer learning

Seam carving is an outstanding content-aware image resizing technique. When the resizing ratio is relatively lower, seam carving can achieve image resizing without leaving any visual obvious artifacts. However, it may also be used for malicious purposes, such as changing image semantic content by object removal. Seam caving changes the relationship between neighboring pixels, thus seam carved images should be different from natural ones. Inspired by this, an end-to-end seam carving detection network, namely SCDNet, is proposed to detect seam carved images. To highlight seam carving artifacts, a pre-processing layer, namely the subtractive pixel adjacency module, is designed as the first layer of SCDNet. Then, 16 residual blocks with shortcut learn features from the input, and a spatial pyramid pooling module is embedded at the end of the residual block to generate fixed-length features, which can eliminate the requirement of fixed-resolution input images for existing deep models. Finally, the features are fed into the fully-connected layer for final classification. In addition, to expose seam carved images with a small carving ratio, a transfer learning strategy is exploited to gradually clarify the artifacts. Extensive experiments demonstrate that SCDNet obtains better detection accuracy than state-of-the-art methods, especially for seam carving with small scaling ratios.

Skin Lesion Classification Using CNN-based Transfer Learning Model

The computer-aided diagnosis (CAD) and the analysis of skin lesions using deep learning models have become common in the last decade. The proposed CAD systems have considered various datasets and deep learning models. The transfer of knowledge from particular pre-trained models to others has also gained importance due to the efficient convergence and superior results. This study presents the design and implementation of a transfer learning model using Convolutional Neural Networks (CNN) with variable training epoch numbers to classify skin lesion images obtained by smartphones. The model is divided into the inner and external CNN models to train and transfer the knowledge, and the preprocessing and data augmentation are not applied. Several experiments are performed to classify cancerous and non-cancerous skin lesions and all skin lesion types provided in the dataset separately. The designed model increased the classification rates by 20% compared to the conventional CNN. The transfer learning model achieved 0.81, 0.88, and 0.86 mean recall, mean specificity, and mean accuracy in detecting cancerous lesions, and 0.83, 0.90, and 0.86 macro recall, macro precision, and macro F1 score in classifying six skin lesions. The obtained results show the efficacy of transfer learning in skin lesion diagnosis.

IRIS-Based Human Identity Recognition with Deep Learning Methods

Abstract: One of the most important computer system modules is the one in charge of user security. It has been demonstrated that simple logins and passwords cannot provide high efficiency and are simple for hackers to crack. The popular substitute is identity identification using bio-metrics. Iris as a biometric characteristic has attracted increased attention in recent years. It was brought on by the high efficiency and precision this quantifiable aspect ensured. Throughout the literature, the effects of this curiosity can be seen. Several authors have put forth a variety of various approaches. This paper describe methods for an irisbased algorithm for recognizing human identity. Artificial neural networks (ANN) and a CNN-based transfer learning model (Mobile-net) were employed in the classification process. Once the output has been categorized, the iris component is segmented using a process called segmentation. Tests that have been run have demonstrated that the suggested procedure can produce results that are adequate.

Transfer Learning for Image-Based Malware Detection for IoT

The tremendous growth in online activity and the Internet of Things (IoT) led to an increase in cyberattacks. Malware infiltrated at least one device in almost every household. Various malware detection methods that use shallow or deep IoT techniques were discovered in recent years. Deep learning models with a visualization method are the most commonly and popularly used strategy in most works. This method has the benefit of automatically extracting features, requiring less technical expertise, and using fewer resources during data processing. Training deep learning models that generalize effectively without overfitting is not feasible or appropriate with large datasets and complex architectures. In this paper, a novel ensemble model, Stacked Ensemble-autoencoder, GRU, and MLP or SE-AGM, composed of three light-weight neural network models-autoencoder, GRU, and MLP-that is trained on the 25 essential and encoded extracted features of the benchmark MalImg dataset for classification was proposed. The GRU model was tested for its suitability in malware detection due to its lesser usage in this domain. The proposed model used a concise set of malware features for training and classifying the malware classes, which reduced the time and resource consumption in comparison to other existing models. The novelty lies in the stacked ensemble method where the output of one intermediate model works as input for the next model, thereby refining the features as compared to the general notion of an ensemble approach. Inspiration was drawn from earlier image-based malware detection works and transfer learning ideas. To extract features from the MalImg dataset, a CNN-based transfer learning model that was trained from scratch on domain data was used. Data augmentation was an important step in the image processing stage to investigate its effect on classifying grayscale malware images in the MalImg dataset. SE-AGM outperformed existing approaches on the benchmark MalImg dataset with an average accuracy of 99.43%, demonstrating that our method was on par with or even surpassed them.

Modulation format recognition using CNN-based transfer learning models

Transfer learning (TL) appears to be a potential method for transferring information from general to specialized activities. Unfortunately, experimenting using various TL models does not yield good results. In this paper, we propose a model built from scratch with the Hough transform (HT) of constellation diagrams to improve modulation format recognition. The HT is utilized to project points on the constellation diagrams on the Hough space. The HT translates constellation diagram points into lines. Features can then be extracted from the HT domain. Constellation diagrams for eight different modulation formats (2/4/8/16—PSK and 8/16/32/64—QAM) are obtained at optical signal-to-noise ratios (OSNRs) ranging from 5 to 30 dB. The proposed system is based on classification and TL. The obtained results indicate that even at low OSNR values, the proposed system can blindly recognize the wireless optical modulation format with a classification accuracy of up to 99%.

Speech Emotion Recognition Based on Two-Stream Deep Learning Model Using Korean Audio Information

Identifying a person’s emotions is an important element in communication. In particular, voice is a means of communication for easily and naturally expressing emotions. Speech emotion recognition technology is a crucial component of human–computer interaction (HCI), in which accurately identifying emotions is key. Therefore, this study presents a two-stream-based emotion recognition model based on bidirectional long short-term memory (Bi-LSTM) and convolutional neural networks (CNNs) using a Korean speech emotion database, and the performance is comparatively analyzed. The data used in the experiment were obtained from the Korean speech emotion recognition database built by Chosun University. Two deep learning models, Bi-LSTM and YAMNet, which is a CNN-based transfer learning model, were connected in a two-stream architecture to design an emotion recognition model. Various speech feature extraction methods and deep learning models were compared in terms of performance. Consequently, the speech emotion recognition performance of Bi-LSTM and YAMNet was 90.38% and 94.91%, respectively. However, the performance of the two-stream model was 96%, which was a minimum of 1.09% and up to 5.62% improved compared with a single model.

CNN-Based Transfer Learning for 3D Knuckle Recognition

A trustworthy and secure identity verification system is in great demand nowadays. The automatic recognition of the 3D middle finger knuckle is a new biometric identifier that could offer a precise, practical, and efficient alternative for personal identification. According to earlier studies, deep learning algorithms could be used for biometric identification. However, the accuracy of the current 3D middle finger knuckle recognition model is relatively low. Motivated by this fact, in this study, seven deep learning neural networks have been modified and trained to identify 3D middle finger knuckle patterns using transfer learning. Using the Hong Kong Polytechnic University’s 3D knuckle image dataset, an extensive experiment was performed. Two sessions of data from different camera lenses were used to assess the performance of the suggested deep learning model. The results show that the InceptionV3 method significantly enhanced the recognition of 3D middle finger knuckle patterns with 99.07% accuracy, followed by Xception, NasNetMobile, and DenseNet201 (97.35%, 92.92%, and 92.59%, respectively), which is superior to the current middle finger knuckle recognition model. This accurate, fast, and automatic middle finger knuckle identification will help to be implemented in real-time and small-scale settings like offices, schools, or personal devices like laptops and smartphones, where training is simple.

Using Transfer Learning of Convolutional Neural Network on Neck Radiographs to Identify Acute Epiglottitis.

Acute epiglottitis (AE) is a life-threatening condition and needs to be recognized timely. Diagnosis of AE with a lateral neck radiograph yields poor reliability and sensitivity. Convolutional neural networks (CNN) are powerful tools to assist the analysis of medical images. This study aimed to develop an artificial intelligence model using CNN-based transfer learning to identify AE in lateral neck radiographs. All cases in this study are from two hospitals, a medical center, and a local teaching hospital in Taiwan. In this retrospective study, we collected 251 lateral neck radiographs of patients with AE and 936 individuals without AE. Neck radiographs obtained from patients without and with AE were used as the input for model transfer learning in a pre-trained CNN including Inception V3, Densenet201, Resnet101, VGG19, and Inception V2 to select the optimal model. We used five-fold cross-validation to estimate the performance of the selected model. The confusion matrix of the final model was analyzed. We found that Inception V3 yielded the best results as the optimal model among all pre-train models. Based on the average value of the fivefold cross-validation, the confusion metrics were obtained: accuracy = 0.92, precision = 0.94, recall = 0.90, and area under the curve (AUC) = 0.96. Using the Inception V3-based model can provide an excellent performance to identify AE based on radiographic images. We suggest using the CNN-based model which can offer a non-invasive, accurate, and fast diagnostic method for AE in the future.

Rotor Fault Diagnosis Method Using CNN-Based Transfer Learning with 2D Sound Spectrogram Analysis

This study discusses a failure detection algorithm that uses frequency analysis and artificial intelligence to determine whether a rotor used in an industrial setting has failed. A rotor is a standard component widely used in industrial sites, and continuous friction and corrosion frequently result in motor and bearing failures. As workers inspecting failure directly are at risk of serious accidents, an automated environment that can operate unmanned and a system for accurate failure determination are required. This study proposes an algorithm to detect faults by introducing convolutional neural networks (CNNs) after converting the fault sound from the rotor into a spectrogram through STFT analysis and visually processing it. A binary classifier for distinguishing between normal and failure states was added to the output part of the neural network structure used, which was based on the transfer learning methodology. We mounted the proposed structure on a designed embedded system to conduct performance discrimination experiments and analyze various outcome indicators using real-world fault data from various situations. The analysis revealed that failure could be detected in response to various normal and fault sounds of the field system and that both training and validation accuracy were greater than 99%. We further intend to investigate artificial intelligence algorithms that train and learn by classifying fault types into early, middle, and late stages to identify more specific faults.

A CNN-based transfer learning method for leakage detection of pipeline under multiple working conditions with AE signals

Pipeline leakage detection is a crucial part of pipeline integrity management. Acoustic emission (AE) based leakage detection is widely used in this field. The latest detection methods are combined AE with convolutional neural networks. However, these methods are often confined to the complex signal processing and computing power and only target specific working conditions. To address these issues, this study proposes a convolutional neural network-based transfer learning (CNN-TL) method for pipeline leakage detection under multiple working conditions. Seven AE datasets are collected from pipeline leakage experiments under different work environments, transporting medium, and fluid pressure. The proposed method converted raw AE signals into three-channel images by a novel conversion method, which avoids reliance on expert knowledge and complex signal processing. CNN-TL is investigated by two different approaches, feature-based CNN-TL and parameter-based CNN-TL. The following nine pre-trained CNN models are used to select the optimal CNN-TL model: Alexnet, Squeezenet, Vgg19, Googlenet, Inceptionv3, Mobilenetv2, Xception, Resnet101, and Densenet201. Results show that the proposed feature-based CNN-TL method significantly outperformed parameter-based CNN-TL and traditional CNN methods, especially on two-phase flow datasets. The highest accuracy of seven AE datasets obtained by the feature-based CNN-TL methods are 100.00%, 100.00%, 100.00%, 99.33%, 85.67%, 87.67%, 74.33%, 83.33% respectively. Moreover, the computation time of proposed method is 16.78 s on average by using the best layers in feature-based CNN-TL. It can be concluded that the proposed method does not rely on signal processing, requires less computational power, and can accomplish accurate detection of pipeline leaks under multiple working conditions.