Transfer Learning Method Research Articles

Universal Remaining Useful Life Prediction for OECTs under Different Aging Conditions

Abstract It is significantly important to predict remaining useful life (RUL) of organic electrochemical transistors (OECTs) for next-generation offshore electronics with stable and reliable performance. Most existing RUL prediction models are not suitable for OECT RUL prediction tasks as they are based on the premise that components have the same aging conditions. In fact, aging conditions of different OECTs often exist discrepancy, leading to performance degradation of RUL prediction models. Although a few methods addressed this issue via transfer learning methods, they still suffer from the challenge in terms of an obvious discrepancy of aging data distribution caused by different aging conditions. To address this issue, we developed a novel universal RUL prediction model for OECTs, called adaptive transformer-based network (ATFN), to reduce the obvious discrepancy among different aging data. First, a transformer-based feature extractor is used to capture the temporal and spatial aging features from some aging precursors. Then, a multi-scale feature alignment metric is adopted to align the aging features of OECTs by reducing discrepancy at different feature scales. Last, an adversarial manner is developed to obtain aging-condition-invariant features for further feature alignment. Extensive experiments are conducted on a real-world OECTs cycling stability aging tests dataset. The average MSE of our method is reduced by two orders of magnitude compared to the one of baseline, which indicates that our method achieves great progress for universal RUL prediction of OECTs under different aging conditions.

Combining transfer learning and statistical measures to predict performance of composite materials with limited data

AbstractPredicting the performance of composite materials is crucial for their application in civil infrastructure, yet limited experimental data often hinder the development of accurate and generalizable models. This study introduces a deep neural network (DNN) approach that combines summarizing statistics (SS) and transfer learning (TL)—termed the SSTL‐DNN approach—to address data scarcity in modeling composite materials. The computational novelty lies in the SS method's ability to extract comprehensive information from limited datasets by converting complex constitutive laws into concise statistical representations, thereby enabling efficient and effective model training. Simultaneously, the TL method enhances computational efficiency by leveraging knowledge from related tasks with abundant data to improve learning in the target task with scarce data. This combination not only reduces dependency on large datasets but also significantly improves model generalization. The proposed SSTL‐DNN approach is validated through two case studies: fiber‐reinforced polymer confined concrete and engineered cementitious composites. In both case studies, the SSTL‐DNN model reduces the required dataset size by up to 75% and decreases the validation error by 39%, compared to traditional deep learning models. These results demonstrate that the SSTL‐DNN approach not only overcomes data scarcity but also provides accurate predictions and generalization to unseen data, offering a practical solution for modeling composite materials with limited data.

A seed-epidermis-feature-recognition-based lightweight peanut seed selection method for embedded systems

High-quality seeds improve the germination rate. Therefore, seed selection before sowing peanuts is crucial. Current peanut seed selection methods include color sorters and sieving machines, which are efficient but lack accuracy due to their reliance on a single indicator. Manual selection is inefficient and subject to subjective influences. Although machine vision and deep learning have performed well in crops such as corn and pepper, most existing research is based on PC or MATLAB platforms, which are not portable and are prone to interference, making them unsuitable for field applications. This study developed a lightweight model for recognizing peanut seed epidermal features. The model was based on deep learning and model quantization techniques. The transfer learning method was used to use four pre-trained models, EfficientNet_b0, EfficientNetv2-b0, MobileNet_v2_35_224, and NasNet_Mobile, as feature extraction layers, the input layer was added before the feature extraction layer, and the dropout and dense layers were added after the feature extraction layer to construct a classifier. The peanut seed selection network(PSSNet) models were constructed and named PSSNet-E, PSSNet-E2, PSSNet-M, and PSSNet-N, respectively, and trained on the Huayu 22 peanut seed dataset constructed in this study. The constructed models were compressed using model quantization technology, and four quantized models were obtained, namely PSSNet-Ef, PSSNet-E2f, PSSNet-Mf, and PSSNet-Nf. Finally, PSSNet-Mf, which had the best model evaluation, was selected as the peanut selection model for this study and deployed on a prototype for testing. Compared with the unquantized model, the size of the quantized model was reduced by two-thirds and the running speed was increased by 37%. A total of 400 Huayu 22 peanut seeds were selected as test samples, and 5 selection tests were conducted.The results showed that the average accuracy was 95.3%, which met the requirements of peanut selection at the production site.

MatSwarm: trusted swarm transfer learning driven materials computation for secure big data sharing

The rapid advancement of Industry 4.0 necessitates close collaboration among material research institutions to accelerate the development of novel materials. However, multi-institutional cooperation faces significant challenges in protecting sensitive data, leading to data silos. Additionally, the heterogeneous and non-independent and identically distributed (non-i.i.d.) nature of material data hinders model accuracy and generalization in collaborative computing. In this paper, we introduce the MatSwarm framework, built on swarm learning, which integrates federated learning with blockchain technology. MatSwarm features two key innovations: a swarm transfer learning method with a regularization term to enhance the alignment of local model parameters, and the use of Trusted Execution Environments (TEE) with Intel SGX for heightened security. These advancements significantly enhance accuracy, generalization, and ensure data confidentiality throughout the model training and aggregation processes. Implemented within the National Material Data Management and Services (NMDMS) platform, MatSwarm has successfully aggregated over 14 million material data entries from more than thirty research institutions across China. The framework has demonstrated superior accuracy and generalization compared to models trained independently by individual institutions.

Chest x-ray images: transfer learning model in COVID-19 detection.

This research aims to develop an effective algorithm for diagnosing COVID-19 in chest X-rays using the transfer learning method and support vector machines. In total, data was collected from 10 clinics, including both large city hospitals and smaller medical institutions. This ensured a diverse range of geographical and demographic information in the sample. An extensive data set was collected, including 10,000 chest X-ray images. 5000 images represent normal cases, 3993 images represent pneumonia cases, and 1007 images represent COVID-19 cases. Machine learning methods were applied to develop a classification model, and the results were compared with seven state-of-the-art models and a lightweight CNN architecture. The results showed that the proposed method achieves high accuracy values (Accuracy): 0.95 for COVID-19, 0.89 for pneumonia, and 0.92 for normal images (p < 0.05). Comparison with other models demonstrates statistically significant superiority of our method in accuracy across all three classes. The EfficientNet-B0 model surpasses our method only in accuracy for normal images with p < 0.01, confirming the advantages of our method. Our method demonstrates high sensitivity values (Sensitivity): 0.96 for COVID-19, 0.88 for pneumonia, and 0.93 for normal images (p < 0.05), outperforming most of the compared models. Correlation analysis showed Pearson coefficients of 0.92, 0.89, and 0.94 for COVID-19, pneumonia, and normal images, respectively, confirming a high degree of consistency between predicted and true class labels. In addition, the model was validated on external datasets to assess its generalizability. This validation confirmed its high level of effectiveness in a variety of clinical settings. This study confirms the importance of applying machine learning methods in medical applications and opens new perspectives for early diagnosis of infectious diseases. The practical application of the obtained results can enhance the efficiency of diagnosis and control the spread of COVID-19, as well as contribute to the development of innovative methods in medical practice.

Cephalometric landmark annotation using transfer learning: Detectron2 and YOLOv8 baselines on a diverse cephalometric image dataset

BackgroundCephalometric landmark annotation is a key challenge in radiographic analysis, requiring automation due to its time-consuming process and inherent subjectivity. This study investigates the application of advanced transfer learning techniques to enhance the accuracy of anatomical landmarks in cephalometric images, which is a vital aspect of orthodontic diagnosis and treatment planning. MethodsWe assess the suitability of transfer learning methods by employing state-of-the-art pose estimation models. The first framework is Detectron2, with two baselines featuring different ResNet backbone architectures: rcnn_R_50_FPN_3x and rcnn_R_101_FPN_3x. The second framework is YOLOv8, with three variants reflecting different network sizes: YOLOv8s-pose, YOLOv8m-pose, and YOLOv8l-pose. These pose estimation models are adopted for the landmark annotation task. The models are trained and evaluated on the DiverseCEPH19 dataset, comprising 1692 radiographic images with 19 landmarks, and their performance is analyzed across various images categories within the dataset. Additionally, the study is extended to a benchmark dataset of 400 images to investigate how dataset size impacts the performance of these frameworks. ResultsDespite variations in objectives and evaluation metrics between pose estimation and landmark localization tasks, the results are promising. Detectron2's variant outperforms others with an accuracy of 85.89%, compared to 72.92% achieved by YOLOv8's variant on the DiverseCEPH19 dataset. This superior performance is also observed in the smaller benchmark dataset, where Detectron2 consistently maintains higher accuracy than YOLOv8. ConclusionThe noted enhancements in annotation precision suggest the suitability of Detectron2 for deployment in applications that require high precision while taking into account factors such as model size, inference time, and resource utilization, the evidence favors YOLOv8 baselines.

Theories and Methods for Indoor Positioning Systems: A Comparative Analysis, Challenges, and Prospective Measures

In the era of the Internet of Things (IoT), the demand for accurate positioning services has become increasingly critical, as location-based services (LBSs) depend on users’ location data to deliver contextual functionalities. While the Global Positioning System (GPS) is widely regarded as the standard for outdoor localization due to its reliability and comprehensive coverage, its effectiveness in indoor positioning systems (IPSs) is limited by the inherent complexity of indoor environments. This paper examines the various measurement techniques and technological solutions that address the unique challenges posed by indoor environments. We specifically focus on three key aspects: (i) a comparative analysis of the different wireless technologies proposed for IPSs based on various methodologies, (ii) the challenges of IPSs, and (iii) forward-looking strategies for future research. In particular, we provide an in-depth evaluation of current IPSs, assessing them through multidimensional matrices that capture diverse architectural and design considerations, as well as evaluation metrics established in the literature. We further examine the challenges that impede the widespread deployment of IPSs and highlight the potential risk that these systems may not be recognized with a single, universally accepted standard method, unlike GPS for outdoor localization, which serves as the golden standard for positioning. Moreover, we outline several promising approaches that could address the existing challenges of IPSs. These include the application of transfer learning, feature engineering, data fusion, multisensory technologies, hybrid techniques, and ensemble learning methods, all of which hold the potential to significantly enhance the accuracy and reliability of IPSs. By leveraging these advanced methodologies, we aim to improve the overall performance of IPSs, thus paving the way for more robust and dependable LBSs in indoor environments.

SGTL-SUIE: semantic attention-guided transfer learning method for stylization underwater image enhancement

SGTL-SUIE: semantic attention-guided transfer learning method for stylization underwater image enhancement

Performance Analysis of Transfer Learning Models for Identifying AI-Generated and Real Images

This study aims to analyze and compare the performance of three transfer learning methods, namely InceptionV3, VGG16, and DenseNet121, in detecting AI-generated and real images. The background of this research is the unknown performance of transfer learning methods for detecting AI-generated and real images. This study introduces innovation by conducting 54 experiments involving three types of transfer learning, three dataset split ratios (60:40, 70:30, and 80:20), three optimizers (Adam, SGD, and RMSprop), two numbers of epochs (20 and 50), and the addition of dense and flatten layers during fine tuning. Performance evaluation was conducted using binary cross entropy loss and confusion matrix. This research provides significant benefits in determining the most effective transfer learning model for detecting AI-generated and real images and offers practical guidance for further development. The results show that the InceptionV3 model with the Adam optimizer, an 80:20 split ratio, and 20 epochs achieved the highest accuracy of 84.26%, with a loss of 39.54%, precision of 81.33%, recall of 82.43%, and an F1-Score of 81.88%.

Ensemble Learning Development Based on Transfer Learning for Indonesian Traditional Food Detection

Development of traditional food competes with other traditional foods now. They must compete with fast food and food from abroad. In 2013, the food and beverage sector were the second highest contributor to tourist expenditure after accommodation. This shows its very important role in the economy. That caused, we need a model that can predict traditional Indonesian foods and snacks. We used ensemble learning. It had 2 transfer learning methods, namely VGG-19 and Xception. They will be combined to improve the performance of the existing model. The research result shown output. It has found that the ensemble learning model achieved accuracy of up to 97% on training data and 91% on testing data. It is hoped that this prediction model can help people recognize typical Indonesian food and increase interest in and preserve the food around them.

Fault diagnosis of bearings under variable operating conditions based on domain adaptation

Aiming at the problem of low fault recognition rate caused by different distribution of training samples and test samples and imbalance of various fault data in bearing fault diagnosis, a domain adaptive fault diagnosis method based on improved residual network ( ResNet ) is designed. In the first layer of the diagnosis network, a multi-dimensional convolution structure is used for feature extraction to obtain fault feature information of different dimensions; the local maximum mean difference (LMMD) is used in the domain adaptive layer to align the distribution of the source domain and the target domain to obtain more fine-grained information; the class balance loss function ( CBLoss ) is used to solve the training problem of unbalanced data, and the Adam optimization network is used to realize fault diagnosis. Experimental results show that the proposed improved method can achieve higher diagnosis results under the imbalance of fault sample categories. Experimental verification is carried out on two bearing data sets and collected wind turbine data. The results show that the proposed improved method has certain advantages. The diagnostic performance of the proposed improved method in the case of unbalanced data samples is better than other deep transfer learning methods, and it can be used as an effective cross-condition fault analysis method.

Brief Study on Convolutional Neural Networks

Convolutional Neural Networks (CNNs) have revolutionized the field of computer vision and pattern recognition, offering state-of-the-art performance in tasks such as image classification, object detection, and segmentation. This study explores the architecture, training strategies, and applications of CNNs, focusing on their ability to automatically learn spatial hierarchies of features through layers of convolutional filters. The research evaluates various CNN architectures, including Lent, Alex Net, and ResNet, highlighting their improvements in accuracy and efficiency. Additionally, the study investigates advanced techniques such as data augmentation, transfer learning, and regularization methods (e. g, dropout and batch normalization), which enhance the model's generalization capabilities. Through empirical experiments, we demonstrate the effectiveness of CNNs in real-world scenarios, including medical image analysis, autonomous driving, and facial recognition. The study concludes with a discussion of the future trends of CNNs, particularly in the context of deep learning optimization, hardware acceleration, and integration with emerging technologies like edge computing and quantum machine learning.

An overview of current developments and methods for identifying diabetic foot ulcers: A survey

AbstractDiabetic foot ulcers (DFUs) present a substantial health risk across diverse age groups, creating challenges for healthcare professionals in the accurate classification and grading. DFU plays a crucial role in automated health monitoring and diagnosis systems, where the integration of medical imaging, computer vision, statistical analysis, and gait information is essential for comprehensive understanding and effective management. Diagnosing DFU is imperative, as it plays a major role in the processes of diagnosis, treatment planning, and neuropathy research within automated health monitoring and diagnosis systems. To address this, various machine learning and deep learning‐based methodologies have emerged in the literature to support healthcare practitioners in achieving improved diagnostic analyses for DFU. This survey paper investigates various diagnostic methodologies for DFU, spanning traditional statistical approaches to cutting‐edge deep learning techniques. It systematically reviews key stages involved in diabetic foot ulcer classification (DFUC) methods, including preprocessing, feature extraction, and classification, explaining their benefits and drawbacks. The investigation extends to exploring state‐of‐the‐art convolutional neural network models tailored for DFUC, involving extensive experiments with data augmentation and transfer learning methods. The overview also outlines datasets commonly employed for evaluating DFUC methodologies. Recognizing that neuropathy and reduced blood flow in the lower limbs might be caused by atherosclerotic blood vessels, this paper provides recommendations to researchers and practitioners involved in routine medical therapy to prevent substantial complications. Apart from reviewing prior literature, this survey aims to influence the future of DFU diagnostics by outlining prospective research directions, particularly in the domains of personalized and intelligent healthcare. Finally, this overview is to contribute to the continual evolution of DFU diagnosis in order to provide more effective and customized medical care.This article is categorized under: Application Areas &gt; Health Care Technologies &gt; Machine Learning Technologies &gt; Artificial Intelligence

Remaining Useful Life of the Rolling Bearings Prediction Method Based on Transfer Learning Integrated with CNN-GRU-MHA

To accurately predict the remaining useful life (RUL) of rolling bearings under limited data and fluctuating load conditions, we propose a new method for constructing health indicators (HI) and a transfer learning prediction framework, which integrates Convolutional Neural Networks (CNN), Gated Recurrent Units (GRU), and Multi-head attention (MHA). Firstly, we combined Convolutional Neural Networks (CNN) with Gated Recurrent Units (GRU) to fully extract temporal and spatial features from vibration signals. Then, the Multi-head attention mechanism (MHA) was added for weighted processing to improve the expression ability of the model. Finally, a new method for constructing Health indicators (HIs) was proposed in which the noise reduction and normalized vibration signals were taken as a HI, the L1 regularization method was added to avoid overfitting, and the model-based transfer learning method was used to realize the RUL prediction of bearings under small samples and variable load conditions. Experiments were conducted using the PHM2012 dataset from the FEMTO-ST research institute and XJTU-SY dataset. Three sets of 12 migration experiments were conducted under three different operating conditions on the PHM2012 dataset. The results show that the average RMSE of the proposed method was 0.0443, indicating high prediction accuracy under variable loads and small sample conditions. Three different operating conditions and two sets of four migration experiments were conducted on the XJTU-SY dataset, and the results show that the average RMSE of the proposed method was 0.0693, verifying the good generalization of the model under variable load conditions. In summary, the proposed HI construction method and prediction framework can effectively reduce the differences between features, with high stability and good generalizability.

Damage mechanics coupled with a transfer learning approach for the fatigue life prediction of bronze/steel diffusion welded bimetallic material

The bronze/steel diffusion welded (BSDW) bimetallic material is often applied in the rotors of piston pumps to withstand complex alternating loads under high-speed operating conditions. Although diffusion welding is a type of solid-phase welding method to achieve high-quality material connections, the fatigue problems still deserve our attention, especially the very high cycle fatigue (VHCF) and high cycle fatigue (HCF) problems. However, due to the high cost of obtaining data, it is necessary to find an efficient and high-precision fatigue life prediction method for diffusion welded materials with a small sample size. In this study, a novel method continuum damage mechanics − transfer learning method (CDM-TLM) for fatigue life prediction of BSDW material is proposed based on the transfer learning (TL) and continuum damage mechanics − finite element method (CDM-FEM). In comparison with the test results, the predicted values of BSDW material fatigue life all fall within the twice error band of the median values of the test life. The influence of frozen layers during TL and training samples in source and target models on the prediction performance is further discussed. CDM-TLM is an effective life prediction method for high-precision life prediction of BSDW material with a small sample size.

Rail State Recognition Method Based on a Convolutional Block Attention Module

Abstract In order to solve the problems of high subjectivity and serious lag in traditional rail surface state recognition based on manual experience, a rail surface state recognition model based on convolution attention is proposed. First, the transfer learning method is used to pre-train the ImageNet data set to obtain the model parameters. Secondly, the 3×3 convolution in the ResNet-50 residual block is replaced with a convolutional block attention module (CBAM) to obtain a new CBAM-ResNet. Finally, the model identification results of the rail surface status are obtained through the Softmax classifier. The results show that this method can effectively identify the rail surface status, and the identification accuracy can reach 99.68%, which is better than other deep neural network models.

Ultrasonic Rough Crack Characterization Using Time-of-Flight Diffraction With Self-Attention Neural Network.

Time-of-flight diffraction (ToFD) is a widely used ultrasonic nondestructive evaluation (NDE) method for locating and characterizing rough defects, with high accuracy in sizing smooth cracks. However, naturally grown defects often have irregular surfaces, complicating the received tip diffraction waves and affecting the accuracy of defect characterization. This article proposes a self-attention (SA) deep learning method to interpret the ToFD A-scan signals for sizing rough defects. A high-fidelity finite-element (FE) simulation software Pogo is used to generate the synthetic datasets for training and testing the deep learning model. Besides, the transfer learning (TL) method is used to fine-tune the deep learning model trained by the Gaussian rough defects to boost the performance of characterizing realistic thermal fatigue rough defects. An ultrasonic experiment using 2-D rough crack samples made by additive manufacturing is conducted to validate the performance of the developed deep learning model. To demonstrate the accuracy of the proposed method, the crack characterization results are compared with those obtained using the conventional Hilbert peak-to-peak sizing method. The results indicate that the deep learning method achieves significantly reduced uncertainty and error in rough defect characterization, in comparison with traditional sizing approaches used in ToFD measurements.

Surface quality prediction in-situ monitoring system: A deep transfer learning-based regression approach with audible signal

Surface roughness plays an indispensable and fundamental role as a leading indicator of the surface quality of machined parts in the manufacturing process. The precise and effective monitoring and prediction of surface roughness is crucial for surface quality control. In this regard, the development of an in-process surface quality monitoring system is necessary, which has the promising potential to achieve this goal. Such a system typically comprises data-driven models for decision-making and sensing techniques for detecting associated process information. However, some challenges still exist in building such systems. Firstly, the architecture design and deployment of data-driven models, specifically deep learning (DL)-based models, demand adequate domain knowledge. Secondly, most models trained on specific tasks with limited datasets are prone to suppressing their versatility and generalization across different machining conditions. Additionally, in most cases, reliance on handcrafted features to represent dynamic information on various signals during model training necessitates extensive expertise in selecting appropriate feature types. Furthermore, due to the nature of their low dimensionality, handcrafted features have difficulty in capturing of overall process-related underlying patterns from dynamics signatures, which is time-varying and often occurs in transient events. To address these challenges, this paper proposes the regression-based pre-trained convolutional neural network (pre-trained CNN) combined with Mel-spectrogram images based on the transfer learning method for surface roughness prediction. Within the context, the architecture of the transfer model is slightly adapted from already well-trained CNNs. Initial weights in each layer of the CNN model are directly inherited and then fine-tuned through the Bayesian optimization tuning method. Besides, the audible sound signals are captured and subsequently converted into 2D Mel-spectrogram images with variant time lengths, which are separately engaged to retrain and validate four existing pre-trained CNN models (VGG16, VGG19, ResNet50V2 and InceptionResNetV2). Eventually, the effectiveness of proposed models and comparison of their predictive capabilities are further validated through a case study in the turning process. The results demonstrate that each applied pre-trained CNN model is capable of effectively predicting surface quality with satisfactory prediction results. Therefore, the proposed method can facilitate the establishment of a machining monitoring system concerning its accuracy, reliability, and robustness.

Adaptive soft-sensor update by Latest Sample Targeting Frustratingly Easy Domain Adaptation

Soft-sensors are widely used in manufacturing processes to estimate key process variables; however, their performance may deteriorate when process characteristics change. Although Just-In-Time (JIT) modeling techniques have been proposed for adaptive soft-sensor design, they do not always adapt to abrupt changes. Transfer learning (TL) has been suggested as a means to address this issue, with Frustratingly Easy Domain Adaptation (FEDA) being used for soft-sensor design. This study proposes a new TL method called Latest Sample Targeting-FEDA (LST-FEDA) for JIT-based soft-sensor, which can handle both sudden and gradual changes in process characteristics. LST-FEDA updates soft-sensors using a fixed number of latest samples whenever a new sample is obtained. The effectiveness of the proposed method was demonstrated using simulation data from a vinyl acetate monomer (VAM) process and actual operation data from a fluorine-based monomer (FM) process. LST-FEDA accurately estimated objective variables during sudden malfunctions and scheduled maintenance, contributing to efficient and safe process operation.

A Semantic Alignment Guided Transfer-Learning Method for Electro-Mechanical Actuator Fault Diagnosis Under Variable Working Conditions

A Semantic Alignment Guided Transfer-Learning Method for Electro-Mechanical Actuator Fault Diagnosis Under Variable Working Conditions