Combines Transfer Learning Research Articles

Wi-TCG: A WiFi gesture recognition method based on transfer learning and conditional generative adversarial networks

Abstract With the rapid progress of WiFi technology, WiFi-based wireless sensing technology has opened up new ways for contactless human-computer interaction. However, hand gesture recognition technology faces low-quality data sets, insufficient model robustness, poor universality when the application scenario changes, high training costs, and weak generalization ability. To this end, this study innovatively proposes the Wi-TCG method, which combines transfer learning and conditional generative adversarial network (CGAN) to optimize WiFi gesture recognition. This method uses commercial Wi-Fi devices to collect channel state information (CSI) of gesture actions. It innovatively extracts Doppler shift image data as the input of CGAN to generate virtual data with similar characteristics to expand the training sample set. The network is fine-tuned using transfer learning techniques to recognize multiple gesture action categories in different scenarios accurately. In tests of two new natural scenes and six new gesture categories, the Wi-TCG method achieved a high recognition accuracy of 93.1%, providing strong support for applying WiFi-based wireless sensing technology in contactless human-computer interaction.

Deep Learning-Based Bone Cancer Detection and Classification from X-Ray Images Using the Owl Search Algorithm

Abstract—Bone cancer is a serious health condition that often leads to high mortality rates. Early detection plays a crucial role in limiting the spread of cancerous cells and improving patient outcomes. However, manual detection is a time-consuming and labor-intensive process, necessitating th6+e development of an automated system to efficiently classify and identify cancerous and healthy bone tissues. This paper introduces the Owl Search Algorithm with Deep Learning-Driven Bone Cancer Detection and Classification (OSADL-BCDC) model, which combines transfer learning and hyperparameter optimization for effective bone cancer detection. The OSADL-BCDC approach utilizes the Inception v3 model as a pretrained feature extractor, eliminating the need for manual image segmentation. The Owl Search Algorithm (OSA) is employed as a hyperparameter optimizer to enhance the performance of Inception v3. Additionally, Long Short-Term Memory (LSTM) networks are leveraged to detect the presence of bone cancer. The proposed OSADL-BCDC method reduces diagnosis time and achieves faster convergence, demonstrating superior performance in experimental evaluations compared to existing algorithms. The results of the comparison emphasize the effectiveness and improved accuracy of the OSADL-BCDC model for bone cancer detection. Keywords—Bone cancer, Owl search, hyperparameter tuning, Inception v3

Dynamic liquid level prediction for multiple oil wells based on transfer learning and multidimensional feature fusion network

Abstract Accurate prediction of the dynamic liquid level (DLL) in oil wells is crucial for the intelligent optimization of pumping systems. It provides real-time insights into the operational conditions of the pumping system but also supports the optimization of operational parameters with data. However, due to the long-term operation of oil wells and their complex internal environments, direct measurement of the DLL is challenging, leading to low reliability of the obtained data. Therefore, this paper conducts an in-depth analysis of the parameters involved in the pumping process, identifies the model’s input features, and develops a DLL prediction model for multiple wells based on multidimensional feature fusion (MFF). This model captures the characteristics of DLL changes and the diversity of input features. To address the issues of slow model training and low prediction accuracy caused by insufficient datasets in practical applications, this paper integrates transfer learning (TL) techniques. It proposes a new model, the DLL model for multiple wells based on TL and multidimensional feature fusion (TMFF). Initially, the Euclidean distance and maximum mean discrepancy methods are employed to verify the feature similarity between the source and target domains, using highly similar DLL data as experimental data. By combining TL techniques with the MFF model, the TMFF model is established. The model’s capabilities are validated using field-collected data with broad representativeness. Experimental results demonstrate that the proposed MFF model possesses high accuracy and generalization capability. Additionally, the TMFF model effectively resolves the issue of insufficient data during model training. In summary, the methods proposed in this paper can provide accurate DLL data for practical applications in intelligent oilfields.

Combining transfer learning and statistical measures to predict performance of composite materials with limited data

AbstractPredicting the performance of composite materials is crucial for their application in civil infrastructure, yet limited experimental data often hinder the development of accurate and generalizable models. This study introduces a deep neural network (DNN) approach that combines summarizing statistics (SS) and transfer learning (TL)—termed the SSTL‐DNN approach—to address data scarcity in modeling composite materials. The computational novelty lies in the SS method's ability to extract comprehensive information from limited datasets by converting complex constitutive laws into concise statistical representations, thereby enabling efficient and effective model training. Simultaneously, the TL method enhances computational efficiency by leveraging knowledge from related tasks with abundant data to improve learning in the target task with scarce data. This combination not only reduces dependency on large datasets but also significantly improves model generalization. The proposed SSTL‐DNN approach is validated through two case studies: fiber‐reinforced polymer confined concrete and engineered cementitious composites. In both case studies, the SSTL‐DNN model reduces the required dataset size by up to 75% and decreases the validation error by 39%, compared to traditional deep learning models. These results demonstrate that the SSTL‐DNN approach not only overcomes data scarcity but also provides accurate predictions and generalization to unseen data, offering a practical solution for modeling composite materials with limited data.

A fault diagnosis method for bearings and gears in rotating machinery based on data fusion and transfer learning

Abstract Rotating machinery is a crucial component of industrial equipment, and the fault diagnosis of bearings and gears, as vital elements of rotating machinery, is essential since they often fail under harsh working conditions, leading to significant property losses and serious personal safety problems. However, fault data for gears and bearings are often sparse in actual condition, and it is a challenge to ensure the reliability and stability of fault diagnosis results by extracting the features of a single data. To solve the above problems, this paper proposes a fault diagnosis method that combines Transfer Learning and data fusion techniques. Firstly, in this method, two kinds of fault signals are transformed into Gramian Angular Difference Fields and Recurrence Plot. Next, a U-shaped feature fusion dual discriminator generative adversarial network is used to fuse two-dimensional images from multiple sensor data. Its feature fusion module deeply integrates the features of the two images, thereby solving the impact of single data on the reliability and stability of fault diagnosis. Moreover, open-source datasets are used for Transfer Learning training to tackle the small sample problem. Finally, a decision-level information fusion classifier, the Dual-Branch Dempster-Shafer Classifier (DB-DSC), classifies the fused images. This classifier incorporates an improved soft threshold function and D-S evidence theory to achieve adaptive gradient changes and improve the robustness and accuracy of classification results. The experimental results show the effectiveness and stability of the proposed method, and the generated images get high score in several metrics. The average classification accuracy of the classification network reaches 93% and 92.5% on the two datasets, Therefore, the proposed method exhibits strong fault diagnosis capabilities under the small sample conditions of bearings and gears.

Improved transfer learning two-branch convolutional neural network image dehazing

In view of the existing image dehazing algorithms, there are problems of incomplete dehazing and image color distortion. A dehazing network combining Transfer learning sub-net and Residual attention sub-net is proposed. First, the pretrained model of the transfer learning subnet is adopted to enhance the feature attributes of the samples. Second, the struct ure of the dual-branch network is constructed, and the residual attention sub-network is used to assist the transfer learning subnetwork to train the parameters of the network model. Finally, the method of tail ensemble learning is used to fuse the features of the dual network to obtain the model parameters of the dehazed image, so as to complete the image restoration task. The experimental results show that the algorithm proposed in the paper improves the PSNR index by 1.87 dB and 4.22 dB on the RESIDE data set and the O-HAZE data set respectively, and the SSIM index on the O-HAZE data set by 6.7%.

Few-shot learning based histopathological image classification of colorectal cancer

Background Colorectal cancer is a prevalent and deadly disease worldwide, posing significant diagnostic challenges. Traditional histopathologic images classification is often inefficient and subjective. Although some histopathologists use computer-aided diagnosis (CAD) to improve efficiency, these methods depend heavily on extensive data and specific annotations, limiting their development. To address these challenges, this paper proposes a method based on few-shot learning. Methods This study introduces a few-shot learning approach that combines transfer learning and contrastive learning to classify colorectal cancer histopathology images into benign and malignant categories. The model comprises modules for feature extraction, dimensionality reduction, and classification, trained using a combination of contrast loss and cross-entropy loss. In this paper, we detail the setup of hyperparameters:n-way, k-shot, β and the creation of support, query, and test datasets. Results Our method achieved over 98% accuracy on a query dataset with 35 samples per category using only 10 training samples per category. We documented the model’s loss, accuracy, and the confusion matrix of the results. Additionally, we employed the t-SNE algorithm to analyze and assess the model’s classification performance. Conclusion The proposed model demonstrates significant advantages in accuracy and minimal data dependency, performing robustly across all tested n-way, k-shot scenarios. It consistently achieved over 93% accuracy on comprehensive test datasets, including 1916 samples, confirming its high classification accuracy and strong generalization capability. Our research advances the use of few-shot learning in medical diagnostics, and also lays the groundwork for extending it to deal with rare, difficult-to-diagnose cases.

A Multifunctional Tactile Sensory System for Robotic Intelligent Identification and Manipulation Perception.

Humans recognize and manipulate objects relying on the multidimensional force features captured by the tactile sense of skin during the manipulation. Since the current sensors integrated in robots cannot support the robots to sense the multiple interaction states between manipulator and objects, achieving human-like perception and analytical capabilities remains a major challenge for service robots. Prompted by the tactile perception involved in robots performing complex tasks, a multimodal tactile sensory system is presented to provide in situ simultaneous sensing for robots when approaching, touching, and manipulating objects. The system comprises a capacitive sensor owning the high sensitivity of 1.11E-2pFmm-1, a triboelectricity nanogenerator with the fast response speed of 30ms, and a pressure sensor array capable of 3D force detection. By Combining transfer learning models, which fuses multimodal tactile information to achieve high-precision (up to 95%) recognition of the multi-featured targets such as random hardness and texture information under random sampling conditions, including random grasp force and velocity. This sensory system is expected to enhance the intelligent recognition and behavior-planning capabilities of autonomous robots when performing complex tasks in undefined surrounding environments.

Combining Transfer Learning and Ensemble Algorithms for Improved Citrus Leaf Disease Classification

Accurate categorization and timely control of leaf diseases are crucial for citrus growth. We proposed the Multi-Models Fusion Network (MMFN) for citrus leaf diseases detection based on model fusion and transfer learning. Compared to traditional methods, the algorithm (integrating transfer learning Alexnet, VGG, and Resnet) we proposed can address the issues of limited categories, slow processing speed, and low recognition accuracy. By constructing efficient deep learning models and training and optimizing them with a large dataset of citrus leaf images, we ensured the broad applicability and accuracy of citrus leaf disease detection, achieving high-precision classification. Herein, various deep learning algorithms, including original Alexnet, VGG, Resnet, and transfer learning versions Resnet34 (Pre_Resnet34) and Resnet50 (Pre_Resnet50) were also discussed and compared. The results demonstrated that the MMFN model achieved an average accuracy of 99.72% in distinguishing between diseased and healthy leaves. Additionally, the model attained an average accuracy of 98.68% in the classification of multiple diseases (citrus huanglongbing (HLB), greasy spot disease and citrus canker), insect pests (citrus leaf miner), and deficiency disease (zinc deficiency). These findings conclusively illustrate that deep learning model fusion networks combining transfer learning and integration algorithms can automatically extract image features, enhance the automation and accuracy of disease recognition, demonstrate the significant potential and application value in citrus leaf disease classification, and potentially drive the development of smart agriculture.

Combining transfer learning and numerical modelling to deal with the lack of training data in data-based SHM

Structural health monitoring (SHM) involves continuously surveilling the performance of structures to identify progressive damage or deterioration that might evolve over time. Recently, machine learning (ML) algorithms have been successfully employed in various SHM applications, including damage detection. However, supervised ML algorithms often require labelled data for multiple possible damage states of the structure for successful damage identification. Although it may be feasible to gather such data for low-value structures, obtaining damage data for expensive structures such as aircraft could be highly challenging. Herein, this data insufficiency is addressed by combining Finite Element (FE) models with domain adaptation, specifically transfer component analysis (TCA) and joint domain adaptation (JDA). The proposed methodology is showcased in two case studies, a Brake–Reuß beam, where damage scenarios correspond to different torque settings on a lap joint and a wingbox laboratory structure where damage is introduced as saw-cuts. Supervised learning algorithms in the form of Artificial Neural Networks (ANNs) and K-Nearest Neighbours (KNNs) are trained based on FE data after domain adaptation is applied and are then tested with the experimental data. It is shown that even though the performance of classifiers in distinct scenarios of dual, three, four and five-class cases is sensitive to choices in the training stage, the use of TCA or JDA allows for the use of FE data for training and significantly reduces the need for expensive experimental damage data to be used for training. These results can pave the way for a broader use of ML algorithms in SHM of critical and/or expensive structures.

Research on tool wear condition monitoring based on deep transfer learning and residual network

To address the issues of sample scarcity and insufficient recognition accuracy in existing deep learning models for tool wear monitoring, this study developed a milling tool wear monitoring model that combines transfer learning (TL) and deep residual networks (ResNet). The model uses continuous wavelet transform to convert vibration signals into time-frequency maps, which are then fed into the network model for analysis. ResNet50 was selected as the base feature extraction model, and transfer learning techniques were employed to update the classification layer’s weights, enabling tool wear detection. The model based on ResNet-TL achieved a detection accuracy of 94%, significantly exceeding the threshold for intelligent tool wear recognition. This accomplishment markedly improves the precision and stability of tool wear state monitoring, providing more reliable technical support for tool management in manufacturing processes. Additionally, the method demonstrated superiority in addressing the small sample problem, paving the way for future research in tool wear monitoring. By integrating advanced deep learning techniques with transfer learning, the model not only enhances detection capabilities but also improves adaptability and robustness in practical industrial applications.

MnasNet-SimAM: An Improved Deep Learning Model for the Identification of Common Wheat Diseases in Complex Real-Field Environments.

Deep learning approaches have been widely applied for agricultural disease detection. However, considerable challenges still exist, such as low recognition accuracy in complex backgrounds and high misjudgment rates for similar diseases. This study aimed to address these challenges through the detection of six prevalent wheat diseases and healthy wheat in images captured in a complex natural context, evaluating the recognition performance of five lightweight convolutional networks. A novel model, named MnasNet-SimAM, was developed by combining transfer learning and an attention mechanism. The results reveal that the five lightweight convolutional neural networks can recognize the six different wheat diseases with an accuracy of more than 90%. The MnasNet-SimAM model attained an accuracy of 95.14%, which is 1.7% better than that of the original model, while only increasing the model's parameter size by 0.01 MB. Additionally, the MnasNet-SimAM model reached an accuracy of 91.20% on the public Wheat Fungi Diseases data set, proving its excellent generalization capacity. These findings reveal that the proposed model can satisfy the requirements for rapid and accurate wheat disease detection.

Meta-model structural monitoring with cutting-edge AAE-VMD fusion alongside optimized machine learning methods

Transfer learning significantly enhances machine learning by leveraging knowledge from one dataset to boost performance on another, which is particularly beneficial when labelled data ares limited or costly. Sensor fusion, an essential technique in structural health monitoring, improves the effectiveness of transfer learning by combining data from multiple sensors to extract more informative features. This article proposes a novel meta-model structural monitoring using a new data fusion method named the adversarial autoencoder (AAE)–variational mode decomposition (VMD) algorithm, which integrates optimization algorithms, transfer learning, and machine learning techniques for damage detection tasks. The proposed meta-model combines transfer learning with an optimized machine learning framework for training and employs pretrained models for testing across diverse datasets. First, the tensors are fused using the AAE approach into 300 data points, reducing noise and outliers, performing dimensionality reduction, and enriching the training and test datasets. Then, a preprocessing step involves decomposing and denoising tensors using the VMD algorithm, followed by selecting the most informative intrinsic mode functions (IMFs) based on their variance. These selected IMFs are concatenated, and 13 statistical features are extracted from them and used as inputs for machine learning models. Various optimization algorithms are employed to fine-tune the hyperparameters of the machine learning models for optimal classification results. Validation of this meta-model utilizes the dataset collected from a steel grandstand structure, available in benchmark dataset format. For decomposed fused tensors, both particle swarm optimization and covariance matrix adaptation evolution strategy emerge as equally effective optimization techniques for K-nearest neighbours (KNN), consistently achieving the highest mean test accuracy and F1 score of 99.5%. Conversely, KNN stands out as a robust and reliable classification algorithm for denoised fused tensors in transfer learning classification problems, consistently demonstrating perfect accuracy and an F1 score of 1.0 ± 0.0 across all optimization techniques.

Harnessing Deep Learning for Video Based Weapon Detection

This research addresses the escalating global issue of handgun-related crimes by proposing an innovative Intelligent Video Surveillance System (IVSS) that leverages advanced deep learning (DL) techniques for remote firearm detection and timely threat response. The system employs Convolutional Neural Networks (CNN) and the YOLO v3 model, uniquely integrating Transfer Learning (TL) to enhance adaptability and efficacy. Experimental validation using the Internet Movie Firearms Database (IMFDB) demonstrates the system's versatility in detecting various pistols and guns, achieving promising results that surpass existing systems in accuracy and efficiency. Challenges in real-time weapon recognition, such as the absence of a standardized weapon dataset, occlusion, and small object sizes, are acknowledged. Emphasis is placed on the critical need for reliable data acquisition, precise labeling, and preprocessing tailored to different detection algorithms. The implementation encompasses video collection, preprocessing, model loading, algorithm application, segmentation, and classification, alongside a user-friendly webcam interface for real-time detection. Additionally, the system integrates the pyttsx3 library for voice alerts and the Twilio API for voice call alerts to enhance responsiveness. In summary, this study presents a novel CNN-based model combining Transfer Learning with YOLO v3, achieving superior weapon identification and distinguishing between real and fake firearms, representing a significant advancement in intelligent video surveillance and contributing to the reduction of weapon violence.

Transfer learning with convolutional neural networks for hydrological streamline delineation

Hydrological streamline delineation is critical for effective environmental management, influencing agriculture sustainability, river dynamics, watershed planning, and more. This study develops a novel approach to combining transfer learning with convolutional neural networks that capitalize on image-based pre-trained models to improve the accuracy and transferability of streamline delineation. We evaluate the performance of eleven image-based pre-trained models and a baseline model using datasets from Rowan County, North Carolina, and Covington River, Virginia in the USA. Our results demonstrate that when models are adapted to a new area, the fine-tuned ImageNet pre-trained model exhibits superior predictive accuracy, markedly higher than the models trained from scratch or those only fine-tuned on the same area. Moreover, the pre-trained model achieves better smoothness and connectivity between classified streamline channels. These findings underline the effectiveness of transfer learning in enhancing the delineation of hydrological streamlines across varied geographies, offering a scalable solution for accurate and efficient environmental modelling.

Leveraging transfer learning and active learning for data annotation in passive acoustic monitoring of wildlife

Passive Acoustic Monitoring (PAM) has emerged as a pivotal technology for wildlife monitoring, generating vast amounts of acoustic data. However, the successful application of machine learning methods for sound event detection in PAM datasets heavily relies on the availability of annotated data, which can be laborious to acquire. In this study, we investigate the effectiveness of transfer learning and active learning techniques to address the data annotation challenge in PAM. Transfer learning allows us to use pre-trained models from related tasks or datasets to bootstrap the learning process for sound event detection. Furthermore, active learning promises strategic selection of the most informative samples for annotation, effectively reducing the annotation cost and improving model performance. We evaluate an approach that combines transfer learning and active learning to efficiently exploit existing annotated data and optimize the annotation process for PAM datasets. Our transfer learning observations show that embeddings produced by BirdNet, a model trained on high signal-to-noise recordings of bird vocalisations, can be effectively used for predicting anurans in PAM data: a linear classifier constructed using these embeddings outperforms the benchmark by 21.7%. Our results indicate that active learning is superior to random sampling, although no clear winner emerges among the strategies employed. The proposed method holds promise for facilitating broader adoption of machine learning techniques in PAM and advancing our understanding of biodiversity dynamics through acoustic data analysis.

EEG-based TSK fuzzy graph neural network for driver drowsiness estimation

With the development of brain-computer interface (BCI), electroencephalogram (EEG) is considered to be one of the best physiological signals to detect the fatigue state of drivers due to its advantages of extremely high time resolution and low use cost. However, since EEG is sensitive to noise/artifacts, and there is non-stationarity and low signal-to-noise ratio in inter-subject/intra-subject, the accurate estimation of driver drowsiness state remains challenging. Therefore, this paper proposes the EEG-based TSK fuzzy graph neural network (TSKG) to solve the regression problem of driver drowsiness estimation by combining transfer learning (TL), graph convolutional neural network (GCN), information theory, and TSK fuzzy neural network. TSKG extracts the relevant features in EEG through mutual information minimization and regards the fuzzy rules as graph structure data. The extracted relevant features and the original features are used to optimize the fuzzy rules through GCN. TSKG is tested on the fatigue driving dataset, and Root Mean Square Error (RMSE) and Correlation Coefficient (CC) are 0.1681 and 0.7118, respectively. Especially when dealing with difficult samples, TSKG has significantly better generalization performance.

Scalability and diversity of StarGANv2-VC in Arabic emotional voice conversion: Overcoming data limitations and enhancing performance

Emotional Voice Conversion (EVC) for under-resourced languages like Arabic faces challenges due to limited emotional speech data. This study explored strategies to mitigate dataset scarcity and improve Arabic EVC performance. Fundamental experiments (Speaker-Dependent, Gender-Dependent, Gender-Independent) were conducted using the KSUEmotions dataset to analyze speaker, gender, and model impacts. Data augmentation techniques like time stretching and phase shuffling artificially increased data diversity. Attention mechanisms integrated into StarGANv2-VC aimed to better capture emotional cues. Transfer learning leveraged the larger English Emotional Speech Database (ESD) to enhance the Arabic system. A novel “Reordering Speaker-Emotion Data” approach treated each emotion as a separate speaker to expand the emotional variability. Our comprehensive approach, combining transfer learning, data augmentation, and architectural modifications, demonstrates the potential to overcome dataset limitations and enhance the performance of Arabic EVC systems.

Vision Transformer for Flood Detection Using Satellite Images from Sentinel-1 and Sentinel-2

Floods are devastating phenomena that occur almost all around the world and are responsible for significant losses, in terms of both human lives and economic damages. When floods occur, one of the challenges that emergency response agencies face is the identification of the flooded area so that access points and safe routes can be determined quickly. This study presents a flood detection methodology that combines transfer learning with vision transformers and satellite images from open datasets. Transformers are powerful models that have been successfully applied in Natural Language Processing (NLP). A variation of this model is the vision transformer (ViT), which can be applied to image classification tasks. The methodology is applied and evaluated for two types of satellite images: Synthetic Aperture Radar (SAR) images from Sentinel-1 and Multispectral Instrument (MSI) images from Sentinel-2. By using a pre-trained vision transformer and transfer learning, the model is fine-tuned on these two datasets to train the models to determine whether the images contain floods. It is found that the proposed methodology achieves an accuracy of 84.84% on the Sentinel-1 dataset and 83.14% on the Sentinel-2 dataset, revealing its insensitivity to the image type and applicability to a wide range of available visual data for flood detection. Moreover, this study shows that the proposed approach outperforms state-of-the-art CNN models by up to 15% on the SAR images and 9% on the MSI images. Overall, it is shown that the combination of transfer learning, vision transformers, and satellite images is a promising tool for flood risk management experts and emergency response agencies.

Exploring the effective implementation of population-based SHM in existing buildings. Part II: Damage-feature classification for decision-making

In the context of structural condition assessment, transfer learning methods overcome some of the difficulties associated with the paucity of information on the actual structural condition of a target structure. This study aims to associate the building’s response with a population of nominally identical buildings for which a form is derived from existing empirical models relating certain basic characteristics (e.g., structure height) with the fundamental resonance period. This paper presents a structural condition classification based on the measured resonance period of the target structure, presented as the Build’Health™ solution. First, damage thresholds are defined by the shift of the fundamental period, which is considered to be a damage-sensitive characteristic for a given building population form, derived from almost thirty published references. The implicit period variation due to certain weather conditions is also included. Multinomial logistic regression and Gaussian mixture models are then used to classify damage according to the performance levels used in earthquake engineering (i.e., Operational Condition, Immediate Occupancy, Life Safety and Collapse Prevention). A performance-based probabilistic framework using a traffic-light system (green-orange-red classification) is finally used to classify structural condition. The method is tested and validated on several buildings surveyed after weak to strong earthquakes with different structural conditions. We show the complementarity of combining transfer learning, which gives the actual state of the target specimen with respect to a nominally identical population form of buildings, with multinomial logistic regressions and Gaussian mixture models for operational condition-based decision-making defined by the measured resonance period. This manuscript is the second in a series aimed at developing the Build’Health™ operational method for assessing the condition of real buildings (Part I on damage detection using transfer learning and Part II on classification using Gaussian mixture models and multinomial logistic regressions) based on basic building information.