Hybrid Transfer Learning Research Articles

Identification of medicinal plant using hybrid transfer learning technique

Ayurveda is one of the oldest holistic treatment systems in the world. Finding an accurate and correct plant is the key to the working of Ayurvedic treatment. Identification of medicinal plants is a tedious job due to the look-alike feature and availability issue of medicinal plants with other plants. This paper emphases on the ideal identification and classification of plants of medicinal use using deep learning approaches. Previously researchers have used traditional machine learning techniques to identify medicinal plants, which lead to mixed results. Such results are good but not enough as the identification of medicinal plants may lead to a worsening situation for patients. This research is conducted to get results closer to an ayurvedic expert. The dataset used for this research has been taken from Mendeley Data. The dataset comprises 30 different species of medicinal plants. Hybrid transfer learning has been applied to this dataset. The model has generated a test accuracy of 95.25% which is better than the other popular transfer learning techniques.

Prediction of Treatment Outcome in Major Depressive Disorder Using Ensemble of Hybrid Transfer Learning and Long Short-Term Memory Based on EEG Signal

Major Depressive Disorder (MDD) is a widespread global mental disease. The Effectiveness of Selective Serotonin Reuptake Inhibitors (SSRIs) antidepressants prescribed for MDD patients is limited and pre-treatment assessment of treatment outcome is a vital task. In this study, a hybrid model based on Transfer Learning (TL) of powerful pre-trained deep Convolutional Neural Networks (CNNs) empowered with bidirectional Long Short-term Memory (BLSTM) cells and attention mechanism were developed to classify Responders (R) and Non-Responders (NR) to SSRI antidepressants, using raw data images of pre-treatment EEG signal obtained from 30 MDD patients. TL-LSTM-Attention models based on VGG16, Xception, and Densenet121 models were created for the classification of R/NR. Results show that the highest performance of hybrid TL-LSTM-Attention models was achieved by VGG16-LSTM-Attention with an accuracy of 98.21%, sensitivity of 96.22%, and specificity of 99.67%. An Ensemble model based on weighted majority voting among hybrid models is constructed to surpass the performance of each single hybrid model. Our proposed ensemble model gained accuracy, sensitivity, and specificity of 98.84%, 97.80%, and 99.60%, respectively. Hence, the proposed model which is an ensemble of TL-LSTM-Attention models fed by raw data images of EEG signal leads to high confidence in the prediction of antidepressants treatment outcome.

Preparing Well for Esophageal Endoscopic Detection Using a Hybrid Model and Transfer Learning.

Early detection of esophageal cancer through endoscopic imaging is pivotal for effective treatment. However, the intricacies of endoscopic diagnosis, contingent on the physician's expertise, pose challenges. Esophageal cancer features often manifest ambiguously, leading to potential confusions with other inflammatory esophageal conditions, thereby complicating diagnostic accuracy. In recent times, computer-aided diagnosis has emerged as a promising solution in medical imaging, particularly within the domain of endoscopy. Nonetheless, contemporary AI-based diagnostic models heavily rely on voluminous data sources, limiting their applicability, especially in scenarios with scarce datasets. To address this limitation, our study introduces novel data training strategies based on transfer learning, tailored to optimize performance with limited data. Additionally, we propose a hybrid model integrating EfficientNet and Vision Transformer networks to enhance prediction accuracy. Conducting rigorous evaluations on a carefully curated dataset comprising 1002 endoscopic images (comprising 650 white-light images and 352 narrow-band images), our model achieved exceptional outcomes. Our combined model achieved an accuracy of 96.32%, precision of 96.44%, recall of 95.70%, and f1-score of 96.04%, surpassing state-of-the-art models and individual components, substantiating its potential for precise medical image classification. The AI-based medical image prediction platform presents several advantageous characteristics, encompassing superior prediction accuracy, a compact model size, and adaptability to low-data scenarios. This research heralds a significant stride in the advancement of computer-aided endoscopic imaging for improved esophageal cancer diagnosis.

Enhancing P300-Based Brain-Computer Interfaces with Hybrid Transfer Learning: A Data Alignment and Fine-Tuning Approach

A new approach is introduced to address the subject dependency problem in P300-based brain-computer interfaces (BCI) by using transfer learning. The occurrence of P300, an event-related potential, is primarily associated with changes in natural neuron activity and elicited in response to infrequent stimuli, which can be monitored non-invasively through an electroencephalogram. However, implementing P300-based BCI in real-time requires many training samples and time-consuming calibration, making it challenging to use in practical applications. To tackle these challenges, the proposed approach harnesses the high-level feature extraction capability of a deep neural network, achieved through fine-tuning. To ensure similar distributions of feature extraction data, the approach of aligning data in Euclidean space is employed, which is then applied to a discriminatively restricted Boltzmann machine with a single layer for P300 detection. The performance of the proposed method on the BCI Competition III dataset II and the BCI competition II dataset II, the state-of-the-art dataset, was evaluated and compared with previous studies. The results showed that robust performance could be achieved using a small number of training samples, demonstrating the effectiveness of the transfer learning approach in P300-based BCI applications.

Visual Diagnostics of Dental Caries through Deep Learning of Non-Standardised Photographs Using a Hybrid YOLO Ensemble and Transfer Learning Model

Access to oral healthcare is not uniform globally, particularly in rural areas with limited resources, which limits the potential of automated diagnostics and advanced tele-dentistry applications. The use of digital caries detection and progression monitoring through photographic communication, is influenced by multiple variables that are difficult to standardize in such settings. The objective of this study was to develop a novel and cost-effective virtual computer vision AI system to predict dental cavitations from non-standardised photographs with reasonable clinical accuracy. A set of 1703 augmented images was obtained from 233 de-identified teeth specimens. Images were acquired using a consumer smartphone, without any standardised apparatus applied. The study utilised state-of-the-art ensemble modeling, test-time augmentation, and transfer learning processes. The "you only look once" algorithm (YOLO) derivatives, v5s, v5m, v5l, and v5x, were independently evaluated, and an ensemble of the best results was augmented, and transfer learned with ResNet50, ResNet101, VGG16, AlexNet, and DenseNet. The outcomes were evaluated using precision, recall, and mean average precision (mAP). The YOLO model ensemble achieved a mean average precision (mAP) of 0.732, an accuracy of 0.789, and a recall of 0.701. When transferred to VGG16, the final model demonstrated a diagnostic accuracy of 86.96%, precision of 0.89, and recall of 0.88. This surpassed all other base methods of object detection from free-hand non-standardised smartphone photographs. A virtual computer vision AI system, blending a model ensemble, test-time augmentation, and transferred deep learning processes, was developed to predict dental cavitations from non-standardised photographs with reasonable clinical accuracy. This model can improve access to oral healthcare in rural areas with limited resources, and has the potential to aid in automated diagnostics and advanced tele-dentistry applications.

Quantum transfer learning for image classification

Quantum machine learning, an important element of quantum computing, recently has gained research attention around the world. In this paper, we have proposed a quantum machine learning model to classify images using a quantum classifier. We exhibit the results of a comprehensive quantum classifier with transfer learning applied to image datasets in particular. The work uses hybrid transfer learning technique along with the classical pre-trained network and variational quantum circuits as their final layers on a small scale of dataset. The implementation is carried out in a quantum processor of a chosen set of highly informative functions using PennyLane a cross-platform software package for using quantum computers to evaluate the high-resolution image classifier. The performance of the model proved to be more accurate than its counterpart and outperforms all other existing classical models in terms of time and competence.

A hybrid transfer learning method for transient stability prediction considering sample imbalance

Data-driven transient stability prediction (TSP) exists with issues of model robustness and sample imbalance. An instance-based and parameter-based of hybrid transfer learning (HTL) method for TSP considering sample imbalance is proposed to address these two issues. The instance-based transfer learning is firstly utilized to select those applicable samples from the source domain system when the boundary conditions such as the network topology and the operational mode of power system change, which may significantly shorten the time for time-domain simulation (TDS) of the target domain system. Subsequently, the conditional generative adversarial network (CGAN) is employed to augment the unstable samples for obtaining a more balanced training data set. Finally, the parameter-based transfer learning is adopted to quickly update the model for TSP. Case studies conducted on a New England 10-machine 39-bus system, an IEEE 50-machine 145-bus system and a Western Electricity Coordinating Council (WECC) 29-machine 179-bus system demonstrate superior quality and diversity of the generated samples obtained by the CGAN-based data augmentation algorithm than the ones of traditional methods involving SMOTE, ADASYN, GAN and DCGAN. Furthermore, the results from numerous numerical experiments also indicate that the proposed HTL method considering sample imbalance improves the model robustness of TSP.

A Novel Hybrid Transfer Learning Approach for Small-Sample High-Voltage Circuit Breaker Fault Diagnosis On-site

Although the data-driven fault diagnosis method has achieved perfect diagnosis of high-voltage circuit breakers (HVCBs) mechanical fault under the massive data built in the laboratory, it is still a challenge to train a high-precision and robust diagnosis model under the condition of small samples on-site at this stage. To solve the above issues, this paper proposes a novel hybrid transfer learning to realize small-sample HVCB fault diagnosis on-site. To fully learn domain discriminative features and domain matching, this paper simultaneously introduces domain adaptation transfer learning and domain adversarial training into small-sample HVCB diagnosis on-site. At the same time, the two kinds of feature transfer learning are combined through ensemble learning to get the final diagnosis result. To extract discriminative features that characterize HVCB faults, this paper constructs a one-dimensional attention residual convolutional neural network, which can ensure that the network pays attention to key features while fully extracting temporal fine-grained information. The experimental results show that the hybrid transfer learning proposed in this paper achieves 94.69% accuracy of small-sample HVCB fault diagnosis on-site, which is significantly higher than other methods. It has laid a solid foundation for small-sample HVCB fault diagnosis on-site.

Accuracy improvement of laser-induced breakdown spectroscopy coal analysis by hybrid transfer learning.

Laser-induced breakdown spectroscopy (LIBS) has been applied in coal analysis for advantages such as real-time online analysis. Fine-tuning is a transfer learning method that has been utilized in LIBS to improve accuracy in the target domain with a limited training set by introducing a model trained on a different but related source domain. This research proposed a hybrid transfer learning method (HTr-LIBS) to further enhance the performance of LIBS coal analysis by combining fine-tuning with sample reweighting. A neural network was pre-trained on the source domain and target domain training set. The sample weights of the source domain were iteratively adjusted according to the prediction errors. The pre-trained neural network with optimal sample weights was then fine-tuned using the target domain training set. The proposed method significantly improved the analytical accuracy compared to direct modeling using small training sets. When the training set size increased to 19, the R2P of direct modeling for ash content and volatile matter content were 0.8105 and 0.9440, respectively. HTr-LIBS increased the R2P for ash content and volatile matter content to 0.9029 and 0.9627, respectively. The improvements were more significant and stable than fine-tuning of the source domain model without sample reweighting. The introduction of target domain data during pre-training and the iterative adjustment of sample weights both contributed to the improvements.

A Calibration-Based Hybrid Transfer Learning Framework for RUL Prediction of Rolling Bearing Across Different Machines

The effective remaining useful life (RUL) prediction of rolling bearings could guarantee mechanical equipment reliability and stability. The hybrid physical and data-driven prognosis model (HPDM) is recently receiving increasing attention. However, HPDM approaches suffer from two significant challenges that limit their applicability to complex prognosis scenarios: (1) the reality gap between the simulation and measurement data and (2) the limited model generality to accommodate different working conditions and machines. From the perspective of leveraging physical model inference as ’teachers’ for the data-driven model, this article proposes a calibrated-based hybrid transfer learning framework to improve the data fidelity and model generality. Firstly, a 5-DOF dynamic model of rolling bearing is constructed. Comprehensively considering the crack and spall behaviors of degradation evolution, the physical model could provide various failure trajectories. Secondly, the particle filter-based calibration is proposed to retain the high fidelity of the physical simulation. Finally, a Physics-informed Bayes Deep Dual Network (PI-BDDN) is designed. The designed network fuses the physical calibrated simulation as augmented input space to learn representative prognosis features and makes the transfer learning process interpretable by combining the physical model parameters into adversarial learning to selectively identify the most informative knowledge for RUL prediction. The effectiveness of the proposed method is verified on two representative bearing datasets, and comparative results show the superiority of the proposed method on prediction accuracy and uncertainty quantification.

Cross-domain feature selection and diagnosis of oil and gas pipeline defects based on transfer learning

Developing intelligent pipeline defect diagnosis technology is of great significance to ensure the efficiency and safety of pipeline transportation. However, a challenge in real-world tasks is the inadaptability and instability of these technologies for processing data from different distributions. Therefore, this paper proposes an intelligent transfer defect diagnosis framework that combines a hybrid feature selection method and transfer learning (TL) technology. The hybrid feature selection method consists of two cross-domain filtering feature selection methods and feature subset evaluation. A Monte Carlo simulation wrapper is used to obtain the best transfer feature subset. The obtained optimal feature subset is fed to the feature adaptation-based classification method for defect diagnosis. We used two different datasets as the target domain and source domain and simulated two transfer scenarios to verify the framework. Experimental results demonstrate that the proposed diagnosis framework achieves high accuracy and effectiveness in pipeline defect diagnosis cross-domain transfer tasks. This will facilitate pipeline maintenance engineers in solving the problem of pipeline defect diagnosis and reasonably arranging pipeline maintenance.

Classification Framework for Medical Diagnosis of Brain Tumor with an Effective Hybrid Transfer Learning Model.

Brain tumors (BTs) are deadly diseases that can strike people of every age, all over the world. Every year, thousands of people die of brain tumors. Brain-related diagnoses require caution, and even the smallest error in diagnosis can have negative repercussions. Medical errors in brain tumor diagnosis are common and frequently result in higher patient mortality rates. Magnetic resonance imaging (MRI) is widely used for tumor evaluation and detection. However, MRI generates large amounts of data, making manual segmentation difficult and laborious work, limiting the use of accurate measurements in clinical practice. As a result, automated and dependable segmentation methods are required. Automatic segmentation and early detection of brain tumors are difficult tasks in computer vision due to their high spatial and structural variability. Therefore, early diagnosis or detection and treatment are critical. Various traditional Machine learning (ML) techniques have been used to detect various types of brain tumors. The main issue with these models is that the features were manually extracted. To address the aforementioned insightful issues, this paper presents a hybrid deep transfer learning (GN-AlexNet) model of BT tri-classification (pituitary, meningioma, and glioma). The proposed model combines GoogleNet architecture with the AlexNet model by removing the five layers of GoogleNet and adding ten layers of the AlexNet model, which extracts features and classifies them automatically. On the same CE-MRI dataset, the proposed model was compared to transfer learning techniques (VGG-16, AlexNet, SqeezNet, ResNet, and MobileNet-V2) and ML/DL. The proposed model outperformed the current methods in terms of accuracy and sensitivity (accuracy of 99.51% and sensitivity of 98.90%).

Underwater crack pixel-wise identification and quantification for dams via lightweight semantic segmentation and transfer learning

Remotely operated vehicles (ROVs) with cameras provide a solution for dam underwater information acquisition, but problems like massive high-dimensional data processing and effective damage-related information extraction also occur. This paper thereby proposes a real-time pixel-level dam underwater crack automatic segmentation and quantification framework using lightweight semantic segmentation network LinkNet and two-stage hybrid transfer learning(TL). With the combination of in-domain and cross-domain TL, the modeling cost and computational burden can be significantly reduced by transferring knowledge learned in relevant domains to the target domain. The proposed method shows strong identification capability in complicated underwater scenarios(motion blur, uneven illumination, and obstacle blocking), achieving performance with 0.8924 mIOU, 0.9444 precision, 0.9151 recall, and 0.9295 F1-score in the test set. Combined with infrared laser-assisted ranging systems, the geometric features and physical sizes of cracks are quantified using the proposed method. Finally, a visual GUI software with both offline and online detection patterns is developed to perform real-time detection in practice.

Attention-optimized DeepLab V3 + for automatic estimation of cucumber disease severity

BackgroundAutomatic and accurate estimation of disease severity is critical for disease management and yield loss prediction. Conventional disease severity estimation is performed using images with simple backgrounds, which is limited in practical applications. Thus, there is an urgent need to develop a method for estimating the disease severity of plants based on leaf images captured in field conditions, which is very challenging since the intensity of sunlight is constantly changing, and the image background is complicated.ResultsThis study developed a simple and accurate image-based disease severity estimation method using an optimized neural network. A hybrid attention and transfer learning optimized semantic segmentation model was proposed to obtain the disease segmentation map. The severity was calculated by the ratio of lesion pixels to leaf pixels. The proposed method was validated using cucumber downy mildew, and powdery mildew leaves collected under natural conditions. The results showed that hybrid attention with the interaction of spatial attention and channel attention can extract fine lesion and leaf features, and transfer learning can further improve the segmentation accuracy of the model. The proposed method can accurately segment healthy leaves and lesions (MIoU = 81.23%, FWIoU = 91.89%). In addition, the severity of cucumber leaf disease was accurately estimated (R2 = 0.9578, RMSE = 1.1385). Moreover, the proposed model was compared with six different backbones and four semantic segmentation models. The results show that the proposed model outperforms the compared models under complex conditions, and can refine lesion segmentation and accurately estimate the disease severity.ConclusionsThe proposed method was an efficient tool for disease severity estimation in field conditions. This study can facilitate the implementation of artificial intelligence for rapid disease severity estimation and control in agriculture.

Profile Decomposition Based Hybrid Transfer Learning for Cold-Start Data Anomaly Detection

Anomaly detection is an essential task for quality management in smart manufacturing. An accurate data-driven detection method usually needs enough data and labels. However, in practice, there commonly exist newly set-up processes in manufacturing, and they only have quite limited data available for analysis. Borrowing the name from the recommender system, we call this process a cold-start process. The sparsity of anomaly, the deviation of the profile, and noise aggravate the detection difficulty. Transfer learning could help to detect anomalies for cold-start processes by transferring the knowledge from more experienced processes to the new processes. However, the existing transfer learning and multi-task learning frameworks are established on task- or domain-level relatedness. We observe instead, within a domain, some components (background and anomaly) share more commonality, others (profile deviation and noise) not. To this end, we propose a more delicate component-level transfer learning scheme, i.e., decomposition-based hybrid transfer learning ( DHTL ): It first decomposes a domain (e.g., a data source containing profiles) into different components (smooth background, profile deviation, anomaly, and noise); then, each component’s transferability is analyzed by expert knowledge; Lastly, different transfer learning techniques could be tailored accordingly. We adopted the Bayesian probabilistic hierarchical model to formulate parameter transfer for the background, and “ L 2,1 + L 1 ”-norm to formulate low dimension feature-representation transfer for the anomaly. An efficient algorithm based on Block Coordinate Descend is proposed to learn the parameters. A case study based on glass coating pressure profiles demonstrates the improved accuracy and completeness of detected anomaly, and a simulation demonstrates the fidelity of the decomposition results.

Deep Neural Network Models for Colon Cancer Screening.

Simple SummaryDeep learning models have been shown to achieve high performance in diagnosing colon cancer compared to conventional image processing and hand-crafted machine learning methods. Hence, several studies have focused on developing hybrid learning, end-to-end, and transfer learning techniques to reduce manual interaction and for labelling the regions of interest. However, these weak learning techniques do not always provide a clear diagnosis. Therefore, it is necessary to develop a clear explainable learning method that can highlight factors and form the basis of clinical decisions. However, there has been little research carried out employing such transparent approaches. This study discussed the aforementioned models for colon cancer diagnosis.Early detection of colorectal cancer can significantly facilitate clinicians’ decision-making and reduce their workload. This can be achieved using automatic systems with endoscopic and histological images. Recently, the success of deep learning has motivated the development of image- and video-based polyp identification and segmentation. Currently, most diagnostic colonoscopy rooms utilize artificial intelligence methods that are considered to perform well in predicting invasive cancer. Convolutional neural network-based architectures, together with image patches and preprocesses are often widely used. Furthermore, learning transfer and end-to-end learning techniques have been adopted for detection and localization tasks, which improve accuracy and reduce user dependence with limited datasets. However, explainable deep networks that provide transparency, interpretability, reliability, and fairness in clinical diagnostics are preferred. In this review, we summarize the latest advances in such models, with or without transparency, for the prediction of colorectal cancer and also address the knowledge gap in the upcoming technology.

Short-Term Canyon Wind Speed Prediction Based on CNN—GRU Transfer Learning

Due to the particularity of the site selection of hydropower stations, the canyon wind with large fluctuations often occurs during the construction of the hydropower station, which will seriously affect the safety of construction personnel. Especially in the early stage of the construction of the hydropower station, the historical data and information on the canyon wind are scarce. Short-term forecasting of canyon wind speed has become extremely important. The main innovation of this paper is to propose a time series prediction method based on transfer learning. This method can achieve short-term prediction when there are few wind speed sample data, and the model is relatively simple while ensuring the accuracy of prediction. Considering the temporal and nonlinear characteristics of canyon wind speed data, a hybrid transfer learning model based on a convolutional neural network (CNN) and gated recurrent neural network (GRU) is proposed to predict short-term canyon wind speed with fewer observation data. In this method, the time sliding window is used to extract time series from historical wind speed data and temperature data of adjacent cities as the input of the neural network. Next, CNN is used to extract the feature vector from the input, and the feature vector can form time series. Then, the GRU network is used for short-term wind speed prediction by the time series. Experimental results show that the proposed method improves MAE and RMSE by nearly 20%, which will provide new ideas for the application of wind speed forecasting in canyons under complex terrain. The research contents of this paper contribute to the actual construction of hydropower stations.

A Novel Hybrid Transfer Learning Framework for Dynamic Cutterhead Torque Prediction of the Tunnel Boring Machine

A tunnel boring machine (TBM) is an important large-scale engineering machine, which is widely applied in tunnel construction. Precise cutterhead torque prediction plays an essential role in the cost estimation of energy consumption and safety operation in the tunneling process, since it directly influences the adaptable adjustment of excavation parameters. Complicated and variable geological conditions, leading to operational and status parameters of the TBM, usually exhibit some spatio-temporally varying characteristic, which poses a serious challenge to conventional data-based methods for dynamic cutterhead torque prediction. In this study, a novel hybrid transfer learning framework, namely TRLS-SVR, is proposed to transfer knowledge from a historical dataset that may contain multiple working patterns and alleviate fresh data noise interference when addressing dynamic cutterhead torque prediction issues. Compared with conventional data-driven algorithms, TRLS-SVR considers long-ago historical data, and can effectively extract and leverage the public latent knowledge that is implied in historical datasets for current prediction. A collection of in situ TBM operation data from a tunnel project located in China is utilized to evaluate the performance of the proposed framework.

An Improved Hybrid Transfer Learning-Based Deep Learning Model for PM2.5 Concentration Prediction

With the improvement of the living standards of the residents, it is a very important and challenging task to continuously improve the accuracy of PM2.5 (particulate matter less than 2.5 μm in diameter) prediction. Deep learning-based networks, such as LSTM and CNN, have achieved good performance in recent years. However, these methods require sufficient data to train the model. The performance of these methods is limited for the sites where the data is lacking, such as the newly constructed monitoring sites. To deal with this problem, an improved deep learning model based on the hybrid transfer learning strategy is proposed for predicting PM2.5 concentration in this paper. In the proposed model, the maximum mean discrepancy (MMD) is used to select which station in the source domain is most suitable for migration to the target domain. An improved dual-stage two-phase (DSTP) model is used to extract the spatial–temporal features of the source domain and the target domain. Then the domain adversarial neural network (DANN) is used to find the domain invariant features between the source and target domains by domain adaptation. Thus, the model trained by source domain site data can be used to assist the prediction of the target site without degradation of the prediction performance due to domain drift. At last, some experiments are conducted. The experimental results show that the proposed model can effectively improve the accuracy of the PM2.5 prediction at the sites lacking data, and the proposed model outperforms most of the latest models.

Understanding dance semantics using spatio-temporal features coupled GRU networks

The efforts are taken for cultural heritage preservation lead to many digitization initiatives. Indian Classical Dance and its tradition has proven historical importance. Computer-aided archiving and preservation of Indian classical dance resources open enormous opportunities for computational analysis. With the help of recent computational advancements in the field of Computer Vision, these archives can be transformed into intelligent information retrieval systems. In this work, we propose a novel method for understanding the dance semantics by making use of the Spatio-temporal variations of dance features. A video archive is created as a part of this work, from live recordings of different trained dancers and clippings from the internet. The Deep Pose Estimator coupled GRU Model deals with the spatial aspects with a deep learning pose estimator and handles the temporal perspective with GRU Network. The efficiency of the proposed method was compared with benchmark methods such as A 3D-Convolutional Neural Network-based Model, Time Distributed CNN-LSTM Model, and Hybrid Transfer Learning - LSTM Model and the results show the proposed method outperformed others even with different video resolutions.