Transfer Learning System Research Articles

Jasmine Flower Classification with CNN Architectures: A Comparative Study of NasNetMobile, VGG16, and Xception in Agricultural Technology

Jasmine flowers have many benefits and uses such as for traditional medicine, tea, perfume, cosmetics, decoration, and others. in the selection of fresh jasmine flowers for making tea is very important, currently the classification of jasmine flowers for making tea is mostly still using manual methods. Often influenced by individual preferences, opinions, or biases. this causes a lack of objectivity and uncertainty in the classification of jasmine flowers. The manual method is very weak due to human visual limitations and fatigue levels which can result in less than the optimal jasmine flower classification. Therefore, in the research that has been done, a transfer learning system was applied that can classify fresh jasmine flowers with rotten jasmine flowers. This study aims to compare three different Convolutional Neural Network architectures: NasNetMobile, VGG16, and Xception. The results on the three architectures can show maximum results, namely 99.21% for NasNetMobile, 98.69% for VGG16 and 97.91% for Xception. This study provides insight into the classification of good and bad jasmine flowers to encourage further exploration in the field of agriculture.

Retinal vascular morphological characteristics in diabetic retinopathy: an artificial intelligence study using a transfer learning system to analyze ultra-wide field images.

To investigate the morphological characteristics of retinal vessels in patients with different severity of diabetic retinopathy (DR) and in patients with or without diabetic macular edema (DME). The 239 eyes of DR patients and 100 eyes of healthy individuals were recruited for the study. The severity of DR patients was graded as mild, moderate and severe non-proliferative diabetic retinopathy (NPDR) according to the international clinical diabetic retinopathy (ICDR) disease severity scale classification, and retinal vascular morphology was quantitatively analyzed in ultra-wide field images using RU-net and transfer learning methods. The presence of DME was determined by optical coherence tomography (OCT), and differences in vascular morphological characteristics were compared between patients with and without DME. Retinal vessel segmentation using RU-net and transfer learning system had an accuracy of 99% and a Dice metric of 0.76. Compared with the healthy group, the DR group had smaller vessel angles (33.68±3.01 vs 37.78±1.60), smaller fractal dimension (Df) values (1.33±0.05 vs 1.41±0.03), less vessel density (1.12±0.44 vs 2.09±0.36) and fewer vascular branches (206.1±88.8 vs 396.5±91.3), all P<0.001. As the severity of DR increased, Df values decreased, P=0.031. No significant difference between the DME and non-DME groups were observed in vascular morphological characteristics. In this study, an artificial intelligence retinal vessel segmentation system is used with 99% accuracy, thus providing with relatively satisfactory performance in the evaluation of quantitative vascular morphology. DR patients have a tendency of vascular occlusion and dropout. The presence of DME does not compromise the integral retinal vascular pattern.

Efficient breast cancer diagnosis using multi‐level progressive feature aggregation based deep transfer learning system

AbstractBreast cancer is a worldwide fatal disease that exists mostly among women. The deep learning technique has proven its effectiveness, but the performance of the existing deep learning systems is quite compromising. In this work, a deep transfer learning system is suggested for efficient breast cancer classification from histopathology images. This system is based on a novel multi‐level progressive feature aggregation (MPFA) and a spatial domain learning approach. The combination of a pretrained Resnet101 backbone network with MPFA is implemented to extract more significant features. In addition, a mixed‐dilated spatial domain learning network (MSLN) is further incorporated to enhance the receptive field and increase discrimination between features. The proposed method achieved superior performance as compared to the existing state‐of‐the‐art methods, offering 99.24% accuracy, a 98.79% F‐1 score, 98.59% precision, and 98.99% recall values over BreaKHis dataset. An ablation study is carried out over the ICIAR2018 dataset to verify the generalizability and effectiveness of the system.

Design Analysis of Stroke Risk Prediction Model Employing the Hybrid Structure Implementation of Deep Transfer Learning System

Design Analysis of Stroke Risk Prediction Model Employing the Hybrid Structure Implementation of Deep Transfer Learning System

Genetic Programming for Document Classification: A Transductive Transfer Learning System.

Document classification is a challenging task to the data being high-dimensional and sparse. Many transfer learning methods have been investigated for improving the classification performance by effectively transferring knowledge from a source domain to a target domain, which is similar to but different from the source domain. However, most of the existing methods cannot handle the case that the training data of the target domain does not have labels. In this study, we propose a transductive transfer learning system, utilizing solutions evolved by genetic programming (GP) on a source domain to automatically pseudolabel the training data in the target domain in order to train classifiers. Different from many other transfer learning techniques, the proposed system pseudolabels target-domain training data to retrains classifiers using all target-domain features. The proposed method is examined on nine transfer learning tasks, and the results show that the proposed transductive GP system has better prediction accuracy on the test data in the target domain than existing transfer learning approaches including subspace alignment-domain adaptation methods, feature-level-domain adaptation methods, and one latest pseudolabeling strategy-based method.

Optimizing Brain Tumor Classification Accuracy Through Transfer Learning and Internet of Things Integration

Brain tumor classification using medical images is crucial for identification and therapy. However, brain tumors are complex and vary, making grouping them difficult. This work demonstrates a novel transfer learning method for brain tumor classification. We employ trained Convolutional Neural Networks (CNNs) models and data enrichment approaches to extract meaningful information from medical images. We want to fine-tune the models built on our dataset to uncover hierarchical patterns that distinguish tumor types. Through data enrichment, the training sample becomes more diverse and richer, making the model more generic and robust. Our team's extensive testing and research have shown that the suggested procedure can identify brain tumors. Our machine-learning approach performs better than others in terms of accuracy, sensitivity, specificity, and precision. Our technique improves brain tumor categorization and assures accurate clinical diagnosis. Automated testing systems are one way for physicians to assist patients in selecting the best course of treatment. Researchers may improve classification performance by incorporating modern imaging technology or topic-specific data. The Internet of Things, or IoT, is helping to drive the development of complex real-time data collection, processing, and sharing systems. These technological advancements have transformed medical imaging. This graphic depicts a cutting-edge transfer learning system that may be able to identify brain cancer from medical photos. This technology has the potential to enhance data collection and processing via the Internet of Things. Data augmentation and pre-trained convolutional neural networks may help to extract interpretable medical images. The Internet of Things improved the model's flexibility, resilience, and utility. We achieved this by expanding the training data set. Rapid categorization advancements have made clinical diagnosis more efficient. Classification, deep learning, medical imaging, machine learning, transfer learning, tumor detection, and image analysis all relate to this topic.

Targeted Display Advertising using Machine Learning

This paper delves into the intricate challenges of problem formulation and data representation in the context of a large-scale machine learning system for targeted display advertising. Unlike traditional models, this system is not just conceptual but has been operational for years across thousands of advertising campaigns. Since obtaining ideal training data is cost-prohibitive, data is sourced from related domains and tasks and then adapted for the target task. The paper outlines the architecture of this multi-stage transfer learning system, emphasizing the problem formulation aspects. Extensive experiments demonstrate the value of each transfer stage. Real-world results with diverse advertising clients from various industries showcase the system's performance. The paper concludes with valuable insights gained from over half a decade of work on this complex, widely deployed machine learning system.

Automatic type Identification of Sub - Axial Vertebral Column Fractures using Deep learning Technique

Traumas like falls, sports injuries, or vehicle accidents are the main causes of Vertebral Column Fracture(VCFs) or spine fracture. The symptoms of VCF include back pain, swelling, numbness, and height decrease. VCF causes extreme pain, paralysis, difficulty in movement, etc. CT scans are inexpensive and effective in providing precise and quick VCF type detection. The determination of type VCF, however subject to inter-observer variability. To address this limitation this work introduces an automatic system for the detection of VCF type based on an ensemble of deep fine tuned models. This can assist the orthopaedicians in type identification of VCF which decreases the image interpretation time and increases the patient care time. For type identification, a total of two fine-tuned CNN architectures are used. Then, to further enhance the type identification performance, we developed an ensemble model of fine tuned VGG16 and ResNet50 deep learning models, which average the outputs of the models during final prediction. We used a dataset of 2820 CT images from eight different VCF types that were collected from Kasturba Medical college, Manipal Academy of Higher education, Manipal. The results shows that the ensemble model performs well compare to individual fine-tuned models with accuracy of 82%, precision of 0.84, recall of 0.82 and F1-score of 0.81 for VCF type identification from CT scans. This study shows that type identification of VCF in CT scans may be successfully accomplished using an ensemble deep transfer learning system with various fine tuned CNN architectures.

Cross-Domain Transfer Learning for Natural Scene Classification of Remote-Sensing Imagery

Natural scene classification, which has potential applications in precision agriculture, environmental monitoring, and disaster management, poses significant challenges due to variations in the spatial resolution, spectral resolution, texture, and size of remotely sensed images of natural scenes on Earth. For such challenging problems, deep-learning-based algorithms have demonstrated amazing performances in recent years. Among these methodologies, transfer learning is a useful technique which employs the learned features already extracted from the pre-trained models from large-scale datasets for the problem at hand, resulting in quicker and more accurate models. In this study, we deployed cross-domain transfer learning for the land-cover classification of remotely sensed images of natural scenes. We conducted extensive experiments to measure the performance of the proposed method and explored the factors that affect the performance of the models. Our findings suggest that fine-tuning the ResNet-50 model outperforms various other models in terms of the classification accuracy. The experimental results showed that the deployed cross-domain transfer-learning system achieved outstanding (99.5% and 99.1%) accurate performances compared to previous benchmarks on the NaSC-TG2 dataset with the final tuning of the whole structure and only the last three layers, respectively.

Novel Task-Based Unification and Adaptation (TUA) Transfer Learning Approach for Bilingual Emotional Speech Data

Modern developments in machine learning methodology have produced effective approaches to speech emotion recognition. The field of data mining is widely employed in numerous situations where it is possible to predict future outcomes by using the input sequence from previous training data. Since the input feature space and data distribution are the same for both training and testing data in conventional machine learning approaches, they are drawn from the same pool. However, because so many applications require a difference in the distribution of training and testing data, the gathering of training data is becoming more and more expensive. High performance learners that have been trained using similar, already-existing data are needed in these situations. To increase a model’s capacity for learning, transfer learning involves transferring knowledge from one domain to another related domain. To address this scenario, we have extracted ten multi-dimensional features from speech signals using OpenSmile and a transfer learning method to classify the features of various datasets. In this paper, we emphasize the importance of a novel transfer learning system called Task-based Unification and Adaptation (TUA), which bridges the disparity between extensive upstream training and downstream customization. We take advantage of the two components of the TUA, task-challenging unification and task-specific adaptation. Our algorithm is studied using the following speech datasets: the Arabic Emirati-accented speech dataset (ESD), the English Speech Under Simulated and Actual Stress (SUSAS) dataset and the Ryerson Audio-Visual Database of Emotional Speech and Song dataset (RAVDESS). Using the multidimensional features and transfer learning method on the given datasets, we were able to achieve an average speech emotion recognition rate of 91.2% on the ESD, 84.7% on the RAVDESS and 88.5% on the SUSAS datasets, respectively.

Morphological characteristics of retinal vessels in eyes with high myopia: Ultra-wide field images analyzed by artificial intelligence using a transfer learning system.

The purpose of this study is to investigate the retinal vascular morphological characteristics in high myopia patients of different severity. 317 eyes of high myopia patients and 104 eyes of healthy control subjects were included in this study. The severity of high myopia patients is classified into C0-C4 according to the Meta Analysis of the Pathologic Myopia (META-PM) classification and their vascular morphological characteristics in ultra-wide field imaging were analyzed using transfer learning methods and RU-net. Correlation with axial length (AL), best corrected visual acuity (BCVA) and age was analyzed. In addition, the vascular morphological characteristics of myopic choroidal neovascularization (mCNV) patients and their matched high myopia patients were compared. The RU-net and transfer learning system of blood vessel segmentation had an accuracy of 98.24%, a sensitivity of 71.42%, a specificity of 99.37%, a precision of 73.68% and a F1 score of 72.29. Compared with healthy control group, high myopia group had smaller vessel angle (31.12 ± 2.27 vs. 32.33 ± 2.14), smaller fractal dimension (Df) (1.383 ± 0.060 vs. 1.424 ± 0.038), smaller vessel density (2.57 ± 0.96 vs. 3.92 ± 0.93) and fewer vascular branches (201.87 ± 75.92 vs. 271.31 ± 67.37), all P < 0.001. With the increase of myopia maculopathy severity, vessel angle, Df, vessel density and vascular branches significantly decreased (all P < 0.001). There were significant correlations of these characteristics with AL, BCVA and age. Patients with mCNV tended to have larger vessel density (P < 0.001) and more vascular branches (P = 0.045). The RU-net and transfer learning technology used in this study has an accuracy of 98.24%, thus has good performance in quantitative analysis of vascular morphological characteristics in Ultra-wide field images. Along with the increase of myopic maculopathy severity and the elongation of eyeball, vessel angle, Df, vessel density and vascular branches decreased. Myopic CNV patients have larger vessel density and more vascular branches.

Skin lesion segmentation using two-phase cross-domain transfer learning framework

Background and ObjectiveDeep learning (DL) models have been used for medical imaging for a long time but they did not achieve their full potential in the past because of insufficient computing power and scarcity of training data. In recent years, we have seen substantial growth in DL networks because of improved technology and an abundance of data. However, previous studies indicate that even a well-trained DL algorithm may struggle to generalize data from multiple sources because of domain shifts. Additionally, ineffectiveness of basic data fusion methods, complexity of segmentation target and low interpretability of current DL models limit their use in clinical decisions. To meet these challenges, we present a new two-phase cross-domain transfer learning system for effective skin lesion segmentation from dermoscopic images. MethodsOur system is based on two significant technical inventions. We examine a two- phase cross-domain transfer learning approach, including model-level and data-level transfer learning, by fine-tuning the system on two datasets, MoleMap and ImageNet. We then present nSknRSUNet, a high-performing DL network, for skin lesion segmentation using broad receptive fields and spatial edge attention feature fusion. We examine the trained model's generalization capabilities on skin lesion segmentation to quantify these two inventions. We cross-examine the model using two skin lesion image datasets, MoleMap and HAM10000, obtained from varied clinical contexts. ResultsAt data-level transfer learning for the HAM10000 dataset, the proposed model obtained 94.63% of DSC and 99.12% accuracy. In cross-examination at data-level transfer learning for the Molemap dataset, the proposed model obtained 93.63% of DSC and 97.01% of accuracy. ConclusionNumerous experiments reveal that our system produces excellent performance and improves upon state-of-the-art methods on both qualitative and quantitative measures.

Transfer Learning Based Neural Machine Translation of English-Khasi on Low-Resource Settings

Machine translation for low-resource language can be improved using various techniques. One such technique is the application of knowledge learned by training a model with high-resource language pair to another model with a low-resource language pair. The paper discusses the experiments and improvement of the results of neural machine translation using transfer learning for the English-Khasi language pair. Long short-term memory is used as the backbone architecture for the transfer learning model. The essential technique is the shared vocabulary, constructed utilizing the subword unit of byte pair encoding of the two pairs of languages and the subword unit of byte pair encoded datasets. Analysis and evaluation of the experimental output using human subjective evaluation, statistical evaluation, and automatic evaluation show positive results for the transfer learning system. A thorough analysis of word order agreement and comparisons of the outputs between the baseline and the transfer learning system is made. The analysis and evaluation methods portray that neural machine translation using transfer learning improves the translation accuracy for Khasi, a low-resource language.

Facial beauty prediction fusing transfer learning and broad learning system

AbstractFacial beauty prediction (FBP) is an important and challenging problem in the fields of computer vision and machine learning. Not only it is easily prone to overfitting due to the lack of large-scale and effective data, but also difficult to quickly build robust and effective facial beauty evaluation models because of the variability of facial appearance and the complexity of human perception. Transfer Learning can be able to reduce the dependence on large amounts of data as well as avoid overfitting problems. Broad learning system (BLS) can be capable of quickly completing models building and training. For this purpose, Transfer Learning was fused with BLS for FBP in this paper. Firstly, a feature extractor is constructed by way of CNNs models based on transfer learning for facial feature extraction, in which EfficientNets are used in this paper, and the fused features of facial beauty extracted are transferred to BLS for FBP, called E-BLS. Secondly, on the basis of E-BLS, a connection layer is designed to connect the feature extractor and BLS, called ER-BLS. Finally, experimental results show that, compared with the previous BLS and CNNs methods existed, the accuracy of FBP was improved by E-BLS and ER-BLS, demonstrating the effectiveness and superiority of the method presented, which can also be widely used in pattern recognition, object detection and image classification.

Breast Cancer Detection in Mammography Image using Convolutional Neural Network

Breast cancer is one of the non-contagious diseases that tends to increase every year. This disease occurs almost entirely in women, but can also occur in men. One way to detect this disease is by observing mammography images. However, mammography images often tend to be blurry with low quality so that it is possible to detect them incorrectly. Therefore, in this study, automatic classification of breast cancer on mammographic images was carried out using the Convolutional Neural Network (CNN). This proposed system uses the VGG16 architecture with a transfer learning system. The proposed system is then optimized using Adam optimizers and RMSprop optimizers. The results of system testing for normal, benign, and malignant classifications obtained an accuracy value of 80% - 90% with the highest accuracy achieved using Adam's optimizers. With this proposed system, it is hoped that it can help in the clinical diagnosis of breast cancer.

Voting in Transfer Learning System for Ground-Based Cloud Classification

Cloud classification is a great challenge in meteorological research. The different types of clouds, currently known and present in our skies, can produce radioactive effects that impact the variation of atmospheric conditions, with consequent strong dominance over the earth’s climate and weather. Therefore, identifying their main visual features becomes a crucial aspect. In this paper, the goal is to adopt pretrained deep neural networks-based architecture for clouds image description, and subsequently, classification. The approach is pyramidal. Proceeding from the bottom up, it partially extracts previous knowledge of deep neural networks related to original task and transfers it to the new task. The updated knowledge is integrated in a voting context to provide a classification prediction. The framework trains the neural models on unbalanced sets, a condition that makes the task even more complex, and combines the provided predictions through statistical measures. An experimental phase on different cloud image datasets is performed, and the results achieved show the effectiveness of the proposed approach with respect to state-of-the-art competitors.

A Low-Overhead Indoor Positioning System Using CSI Fingerprint Based on Transfer Learning

With the rapidly growing demand for Location-Based Services in indoor environments, fingerprint-based indoor positioning has caused great interest due to its high positioning accuracy and low equipment cost. However, the standard signal radio map cannot provide consistent high positioning accuracy under environmental changes and new scenarios. To address this problem, we present a novel indoor positioning Transfer Learning(TL) system based on improved TrAdaBoost. First perform phase correction on the raw CSI(Channel State Information) phase, then use One-vs-Rest algorithm and One-Hot coding, which can realize the multi-classification ability of the TrAdaBoost algorithm. Meanwhile, optimize the iterative process of the algorithm by using a correction factor. In addition, Confidence regression is used to obtain the final estimated position. Experimental results show that the positioning accuracy can be improved by 35% in dynamic environment conditions. The proposed method can improve the positioning accuracy by an average of 30% in new scenes and the Site Survey Overhead(SSO) is reduced by 40%. Compared to other position b algorithm, our proposed method has more robustness in time and space.

A machine learning perspective on the emotional content of Parkinsonian speech

Patients with Parkinson's disease (PD) have distinctive voice patterns, often perceived as expressing sad emotion. While this characteristic of Parkinsonian speech has been supported through the perspective of listeners, where both PD and healthy control (HC) subjects repeat the same speaking tasks, it has never been explored through a machine learning modelling approach. Our work provides an objective evaluation of this characteristic of the PD speech, by building a transfer learning system to assess how the PD pathology affects the sadness perception. To do so we introduce a Mixture-of-Experts (MoE) architecture for speech emotion recognition designed to be transferable across datasets. Firstly, by relying on publicly available emotional speech corpora, we train the MoE model and then we use it to quantify perceived sadness in never seen before PD and matched HC speech recordings. To build our models (experts), we extracted spectral features of the voicing parts of speech and we trained a gradient boosting decision trees model in each corpus to predict happiness vs. sadness. MoE predictions are created by weighting each expert's prediction according to the distance between the new sample and the expert-specific training samples. The MoE approach systematically infers more negative emotional characteristics in PD speech than in HC. Crucially, these judgments are related to the disease severity and the severity of speech impairment in the PD patients: the more impairment, the more likely the speech is to be judged as sad. Our findings pave the way towards a better understanding of the characteristics of PD speech and show how publicly available datasets can be used to train models that provide interesting insights on clinical data.

A Multicenter Study on Carotid Ultrasound Plaque Tissue Characterization and Classification Using Six Deep Artificial Intelligence Models: A Stroke Application

Atherosclerotic plaque in carotid arteries can ultimately lead to cerebrovascular events if not monitored. The objectives of this study are (a) to design a set of artificial intelligence (AI)-based tissue characterization and classification (TCC) systems (Atheromatic 2.0, AtheroPoint, CA, USA) using ultrasound-based carotid artery plaque scans collected from multiple centers and (b) to evaluate the AI performance. We hypothesize that symptomatic plaque is more scattered than asymptomatic plaque. Therefore, the AI system can learn, characterize, and classify them automatically. We developed six kinds of AI systems: four machine learning (ML) systems, one transfer learning (TL) system, and one deep learning (DL) architecture with different layers. Atheromatic 2.0 uses two types of plaque characterization: (a) an AI-based mean feature strength (MFS) and (b) bispectrum analysis. Three kinds of data were collected: London, Lisbon, and Combined (London + Lisbon). We balanced and then augmented five folds to conduct 3-D optimization for optimal number of AI layers versus folds. Using K10 (90% training, 10% testing), the mean accuracies for DL, TL, and ML over the mean of the three data sets were 93.55%, 94.55%, and 89%, respectively. The corresponding mean AUCs were 0.938, 0.946, and 0.889 ( $p ), respectively. AI paradigms showed an improvement by 10.41% and 3.32% for London and Lisbon in comparison to Atheromatic 1.0, respectively. On characterization, for all three data sets, MFS (symptomatic) > MFS (asymptomatic) by 46.56%, 19.40%, and 53.84%, respectively, thus validating our hypothesis. Atheromatic 2.0 showed consistent and stable results and is useful for carotid plaque tissue classification and characterization for vascular surgery applications.

Hybrid Transfer Learning and Broad Learning System for Wearing Mask Detection in the COVID-19 Era.

In the era of Corona Virus Disease 2019 (COVID-19), wearing a mask can effectively protect people from infection risk and largely decrease the spread in public places, such as hospitals and airports. This brings a demand for the monitoring instruments that are required to detect people who are wearing masks. However, this is not the objective of existing face detection algorithms. In this article, we propose a two-stage approach to detect wearing masks using hybrid machine learning techniques. The first stage is designed to detect candidate wearing mask regions as many as possible, which is based on the transfer model of Faster_RCNN and InceptionV2 structure, while the second stage is designed to verify the real facial masks using a broad learning system. It is implemented by training a two-class model. Moreover, this article proposes a data set for wearing mask detection (WMD) that includes 7804 realistic images. The data set has 26403 wearing masks and covers multiple scenes, which is available at "https://github.com/BingshuCV/WMD." Experiments conducted on the data set demonstrate that the proposed approach achieves an overall accuracy of 97.32% for simple scene and an overall accuracy of 91.13% for the complex scene, outperforming the compared methods.