Traditional Machine Learning Methods Research Articles

PT-KGNN: A framework for pre-training biomedical knowledge graphs with graph neural networks

Biomedical knowledge graphs (KGs) serve as comprehensive data repositories that contain rich information about nodes and edges, providing modeling capabilities for complex relationships among biological entities. Many approaches either learn node features through traditional machine learning methods, or leverage graph neural networks (GNNs) to directly learn features of target nodes in the biomedical KGs and utilize them for downstream tasks. Motivated by the pre-training technique in natural language processing (NLP), we propose a framework named PT-KGNN (Pre-Training the biomedical KG with GNNs) to learn embeddings of nodes in a broader context by applying GNNs on the biomedical KG.We design several experiments to evaluate the effectivity of our proposed framework and the impact of the scale of KGs. The results of tasks consistently improve as the scale of the biomedical KG used for pre-training increases. Pre-training on large-scale biomedical KGs significantly enhances the drug-drug interaction (DDI) and drug-disease association (DDA) prediction performance on the independent dataset. The embeddings derived from a larger biomedical KG have demonstrated superior performance compared to those obtained from a smaller KG.By applying pre-training techniques on biomedical KGs, rich semantic and structural information can be learned, leading to enhanced performance on downstream tasks. it is evident that pre-training techniques hold tremendous potential and wide-ranging applications in bioinformatics.

Landslide Hazard Analysis Combining BGA-Net-Based Landslide Susceptibility Perception and Small Baseline Subset Interferometric Synthetic Aperture Radar in the Baige Section in the Upper Reaches of Jinsha River

The geological and topographic conditions in the upper reaches of the Jinsha River are intricate, with frequent occurrences of landslides. Landslide Susceptibility Prediction (LSP) in this area is a crucial aspect of geological disaster risk management. This study constructs an LSP model using a Convolutional Neural Network (CNN) based on a Bilateral Aggregation Guidance (BAG) strategy, termed BGA-Net. A comprehensive landslide hazard analysis, integrating static landslide susceptibility zonation with dynamic surface deformation monitoring, was therefore conducted. The study area selected was the upper reaches of the Jinsha River, particularly the site of the Baige landslide. The BGA-Net model was first proposed for LSP generation, achieving an accuracy exceeding 85%, while the Small Baseline Subset Interferometric Synthetic Aperture Radar (SBAS-InSAR) technology was jointly applied to comprehensively analyze the dynamic geological hazard risk at a regional scale. The final results were presented in a lookup table format and mapped to delineate and grade each risk level. The results show the method is practical, with high feasibility. Compared with traditional machine learning methods, the BGA-strategy-oriented CNN model more effectively differentiated the extremely low- and extremely high-susceptibility areas, enhancing decision-making processes.

Enhanced analysis of tabular data through Multi-representation DeepInsight

Tabular data analysis is a critical task in various domains, enabling us to uncover valuable insights from structured datasets. While traditional machine learning methods can be used for feature engineering and dimensionality reduction, they often struggle to capture the intricate relationships and dependencies within real-world datasets. In this paper, we present Multi-representation DeepInsight (MRep-DeepInsight), a novel extension of the DeepInsight method designed to enhance the analysis of tabular data. By generating multiple representations of samples using diverse feature extraction techniques, our approach is able to capture a broader range of features and reveal deeper insights. We demonstrate the effectiveness of MRep-DeepInsight on single-cell datasets, Alzheimer's data, and artificial data, showcasing an improved accuracy over the original DeepInsight approach and machine learning methods like random forest, XGBoost, LightGBM, FT-Transformer and L2-regularized logistic regression. Our results highlight the value of incorporating multiple representations for robust and accurate tabular data analysis. By leveraging the power of diverse representations, MRep-DeepInsight offers a promising new avenue for advancing decision-making and scientific discovery across a wide range of fields.

Dual Stream Long Short-Term Memory Feature Fusion Classifier for Surface Electromyography Gesture Recognition.

Gesture recognition using electromyography (EMG) signals has prevailed recently in the field of human-computer interactions for controlling intelligent prosthetics. Currently, machine learning and deep learning are the two most commonly employed methods for classifying hand gestures. Despite traditional machine learning methods already achieving impressive performance, it is still a huge amount of work to carry out feature extraction manually. The existing deep learning methods utilize complex neural network architectures to achieve higher accuracy, which will suffer from overfitting, insufficient adaptability, and low recognition accuracy. To improve the existing phenomenon, a novel lightweight model named dual stream LSTM feature fusion classifier is proposed based on the concatenation of five time-domain features of EMG signals and raw data, which are both processed with one-dimensional convolutional neural networks and LSTM layers to carry out the classification. The proposed method can effectively capture global features of EMG signals using a simple architecture, which means less computational cost. An experiment is conducted on a public DB1 dataset with 52 gestures, and each of the 27 subjects repeats every gesture 10 times. The accuracy rate achieved by the model is 89.66%, which is comparable to that achieved by more complex deep learning neural networks, and the inference time for each gesture is 87.6 ms, which can also be implied in a real-time control system. The proposed model is validated using a subject-wise experiment on 10 out of the 40 subjects in the DB2 dataset, achieving a mean accuracy of 91.74%. This is illustrated by its ability to fuse time-domain features and raw data to extract more effective information from the sEMG signal and select an appropriate, efficient, lightweight network to enhance the recognition results.

A Survey of Machine Learning in Edge Computing: Techniques, Frameworks, Applications, Issues, and Research Directions

Internet of Things (IoT) devices often operate with limited resources while interacting with users and their environment, generating a wealth of data. Machine learning models interpret such sensor data, enabling accurate predictions and informed decisions. However, the sheer volume of data from billions of devices can overwhelm networks, making traditional cloud data processing inefficient for IoT applications. This paper presents a comprehensive survey of recent advances in models, architectures, hardware, and design requirements for deploying machine learning on low-resource devices at the edge and in cloud networks. Prominent IoT devices tailored to integrate edge intelligence include Raspberry Pi, NVIDIA’s Jetson, Arduino Nano 33 BLE Sense, STM32 Microcontrollers, SparkFun Edge, Google Coral Dev Board, and Beaglebone AI. These devices are boosted with custom AI frameworks, such as TensorFlow Lite, OpenEI, Core ML, Caffe2, and MXNet, to empower ML and DL tasks (e.g., object detection and gesture recognition). Both traditional machine learning (e.g., random forest, logistic regression) and deep learning methods (e.g., ResNet-50, YOLOv4, LSTM) are deployed on devices, distributed edge, and distributed cloud computing. Moreover, we analyzed 1000 recent publications on “ML in IoT” from IEEE Xplore using support vector machine, random forest, and decision tree classifiers to identify emerging topics and application domains. Hot topics included big data, cloud, edge, multimedia, security, privacy, QoS, and activity recognition, while critical domains included industry, healthcare, agriculture, transportation, smart homes and cities, and assisted living. The major challenges hindering the implementation of edge machine learning include encrypting sensitive user data for security and privacy on edge devices, efficiently managing resources of edge nodes through distributed learning architectures, and balancing the energy limitations of edge devices and the energy demands of machine learning.

Analyzing electroencephalograph signals for early Alzheimer’s disease detection: deep learning vs. traditional machine learning approaches

Alzheimer’s disease (AD) stands as a progressive neurodegenerative disorder with a significant global public health impact. It is imperative to establish early and accurate diagnoses of AD to facilitate effective interventions and treatments. Recent years have witnessed the emergence of machine learning (ML) and deep learning (DL) techniques, displaying promise in various medical domains, including AD diagnosis. This study undertakes a comprehensive contrast between conventional machine learning methods and advanced deep learning strategies for early AD diagnosis. Conventional ML algorithms like support vector machines, decision trees, and K nearest neighbor have been extensively employed for AD diagnosis through relevant feature extraction from heterogeneous data sources. Conversely, deep learning techniques such as multilayer perceptron (MLP) and convolutional neural networks (CNNs) have demonstrated exceptional aptitude in autonomously uncovering intricate patterns and representations from unprocessed data like EEG data. The findings reveal that while traditional ML methods may perform adequately with limited data, deep learning techniques excel when ample data is available, showcasing their potential for early and precise AD diagnosis. In conclusion, this research paper contributes to the ongoing discourse surrounding the choice of appropriate methodologies for early Alzheimer’s disease diagnosis.

Deep Neural Networks and Tabular Data: A Survey.

Heterogeneous tabular data are the most commonly used form of data and are essential for numerous critical and computationally demanding applications. On homogeneous datasets, deep neural networks have repeatedly shown excellent performance and have therefore been widely adopted. However, their adaptation to tabular data for inference or data generation tasks remains highly challenging. To facilitate further progress in the field, this work provides an overview of state-of-the-art deep learning methods for tabular data. We categorize these methods into three groups: data transformations, specialized architectures, and regularization models. For each of these groups, our work offers a comprehensive overview of the main approaches. Moreover, we discuss deep learning approaches for generating tabular data and also provide an overview over strategies for explaining deep models on tabular data. Thus, our first contribution is to address the main research streams and existing methodologies in the mentioned areas while highlighting relevant challenges and open research questions. Our second contribution is to provide an empirical comparison of traditional machine learning methods with 11 deep learning approaches across five popular real-world tabular datasets of different sizes and with different learning objectives. Our results, which we have made publicly available as competitive benchmarks, indicate that algorithms based on gradient-boosted tree ensembles still mostly outperform deep learning models on supervised learning tasks, suggesting that the research progress on competitive deep learning models for tabular data is stagnating. To the best of our knowledge, this is the first in-depth overview of deep learning approaches for tabular data; as such, this work can serve as a valuable starting point to guide researchers and practitioners interested in deep learning with tabular data.

CNN-LSTM for Heartbeat Sound Classification

Cardiovascular disorders are among the primary causes of death. Regularly monitoring the heart is of paramount importance in preventing fatalities arising from heart diseases. Heart disease monitoring encompasses various approaches, including the analysis of heartbeat sounds. The auditory patterns of a heartbeat can serve as indicators of heart health. This study aims to build a new model for categorizing heartbeat sounds based on associated ailments. The Phonocardiogram (PCG) method digitizes and records heartbeat sounds. By converting heartbeat sounds into digital data, researchers are empowered to develop a deep learning model capable of discerning heart defects based on distinct cardiac rhythms. This study proposes the utilization of Mel-frequency cepstral coefficients for feature extraction, leveraging their application in voice data analysis. These extracted features are subsequently employed in a multi-step classification process. The classification process merges a convolutional neural network (CNN) with a long short-term memory network (LSTM), forming a comprehensive deep learning architecture. This architecture is further enhanced through optimization utilizing the Adagrad optimizer. To examine the effectiveness of the proposed method, its classification performance is evaluated using the "Heartbeat Sounds" dataset sourced from Kaggle. Experimental results underscore the effectiveness of the proposed method by comparing it with simple CNN, CNN with vanilla LSTM, and traditional machine learning methods (MLP, SVM, Random Forest, and k-NN).

Classification of Dermoscopic Images Using CNN-SVM

Traditional machine learning methods like GLCM and ABCD rules have long been employed for image classification tasks. However, they come with inherent limitations, primarily the need for manual feature extraction. This manual feature extraction process is time-consuming and relies on expert domain knowledge, making it challenging for non-experts to use effectively. Deep learning methods, specifically Convolutional Neural Networks (CNN), have revolutionized image classification by automating the feature extraction. CNNs can learn hierarchical features directly from the raw pixel values, eliminating the need for manual feature engineering. Despite their powerful capabilities, CNNs have limitations, mainly when working with small image datasets. They may overfit the data or struggle to generalize effectively. In light of these considerations, this study adopts a hybrid approach that leverages the strengths of both deep learning and traditional machine learning. CNNs are automatic feature extractors, allowing the model to capture meaningful image patterns. These extracted features are then fed into a Support Vector Machine (SVM) classifier, known for its efficiency and effectiveness in handling small datasets. The results of this study are encouraging, with an accuracy of 0.94 and an AUC score of 0.94. Notably, these metrics outperform Abbas' previous research by a significant margin, underscoring the effectiveness of the hybrid CNN-SVM approach. This research reinforces that SVM classifiers are well-suited for tasks involving limited image data, yielding improved classification accuracy and highlighting the potential for broader applications in image analysis.

A deep learning approach for acute liver failure prediction with combined fully connected and convolutional neural networks.

Acute Liver Failure (ALF) is a critical medical condition with rapid development, often caused by viral infections, hepatotoxic drug abuse, or other severe liver diseases. Timely and accurate prediction of ALF occurrence is clinically crucial. However, predicting ALF poses challenges due to the diverse physiological differences among patients and the dynamic nature of the disease. This study introduces a deep learning approach that combines fully connected and convolutional neural networks for effective ALF prediction. The goal is to overcome limitations of traditional machine learning methods and enhance predictive model performance and generalization. The proposed model integrates a fully connected neural network for handling basic patient features and a convolutional neural network dedicated to capturing temporal patterns in patient data. The combination allows automatic learning of complex patterns and abstract features present in highly dynamic medical data associated with ALF. The model's effectiveness is demonstrated through comprehensive experiments and performance evaluations. It outperforms traditional machine learning methods, achieving 94.8% accuracy and superior generalization capabilities. The study highlights the potential of deep learning in ALF prediction, emphasizing the importance of considering individualized medical factors. Future research should focus on improving model robustness, addressing imbalanced data, and further exploring personalized features for enhanced predictive accuracy in real-world clinical scenarios.

Applications of deep learning models in precision prediction of survival rates for heart failure patients.

Heart failure poses a significant challenge in the global health domain, and accurate prediction of mortality is crucial for devising effective treatment plans. In this study, we employed a Seq2Seq model from deep learning, integrating 12 patient features. By finely modeling continuous medical records, we successfully enhanced the accuracy of mortality prediction. The objective of this research was to leverage the Seq2Seq model in conjunction with patient features for precise mortality prediction in heart failure cases, surpassing the performance of traditional machine learning methods. The study utilized a Seq2Seq model in deep learning, incorporating 12 patient features, to intricately model continuous medical records. The experimental design aimed to compare the performance of Seq2Seq with traditional machine learning methods in predicting mortality rates. The experimental results demonstrated that the Seq2Seq model outperformed conventional machine learning methods in terms of predictive accuracy. Feature importance analysis provided critical patient risk factors, offering robust support for formulating personalized treatment plans. This research sheds light on the significant applications of deep learning, specifically the Seq2Seq model, in enhancing the precision of mortality prediction in heart failure cases. The findings present a valuable direction for the application of deep learning in the medical field and provide crucial insights for future research and clinical practices.

A Review on Skin Melanoma Classification using different ML and DL Models

This review explores the classification of skin melanoma utilizing various machine learning (ML) and deep learning (DL) models, highlighting the advancements and comparative performance of these methodologies. Skin melanoma, a serious type of skin cancer, demands early and accurate diagnosis for effective treatment. The review covers a range of ML techniques such as support vector machines, decision trees, and ensemble methods, alongside \DL approaches including convolutional neural networks and recurrent neural networks. Emphasis is placed on the models' accuracy, computational efficiency, and the datasets used for training and validation. The review underscores the potential of DL models to outperform traditional ML methods due to their ability to automatically extract and learn intricate features from large datasets, thus offering promising prospects for enhanced diagnostic precision in melanoma classification.

Connectome-based schizophrenia prediction using structural connectivity - Deep Graph Neural Network(sc-DGNN).

Connectome is understanding the complex organization of the human brain's structural and functional connectivity is essential for gaining insights into cognitive processes and disorders. To improve the prediction accuracy of brain disorder issues, the current study investigates dysconnected subnetworks and graph structures associated with schizophrenia. By using the proposed structural connectivity-deep graph neural network (sc-DGNN) model and compared with machine learning (ML) and deep learning (DL) models.This work attempts to focus on eighty-eight subjects of diffusion magnetic resonance imaging (dMRI), three classical ML, and five DL models. The structural connectivity-deep graph neural network (sc-DGNN) model is proposed to effectively predict dysconnectedness associated with schizophrenia and exhibits superior performance compared to traditional ML and DL (GNNs) methods in terms of accuracy, sensitivity, specificity, precision, F1-score, and Area under receiver operating characteristic (AUC). The classification task on schizophrenia using structural connectivity matrices and experimental results showed that linear discriminant analysis (LDA) performed 72% accuracy rate in ML models and sc-DGNN performed at a 93% accuracy rate in DL models to distinguish between schizophrenia and healthy patients.

Deep learning algorithm-enabled sediment characterization techniques to determination of water saturation for tight gas carbonate reservoirs in Bohai Bay Basin, China

Understanding water saturation levels in tight gas carbonate reservoirs is vital for optimizing hydrocarbon production and mitigating challenges such as reduced permeability due to water saturation (Sw) and pore throat blockages, given its critical role in managing capillary pressure in water drive mechanisms reservoirs. Traditional sediment characterization methods such as core analysis, are often costly, invasive, and lack comprehensive spatial information. In recent years, several classical machine learning models have been developed to address these shortcomings. Traditional machine learning methods utilized in reservoir characterization encounter various challenges, including the ability to capture intricate relationships, potential overfitting, and handling extensive, multi-dimensional datasets. Moreover, these methods often face difficulties in dealing with temporal dependencies and subtle patterns within geological formations, particularly evident in heterogeneous carbonate reservoirs. Consequently, despite technological advancements, enhancing the reliability, interpretability, and applicability of predictive models remains imperative for effectively characterizing tight gas carbonate reservoirs. This study employs a novel data-driven strategy to prediction of water saturation in tight gas reservoir powered by three recurrent neural network type deep/shallow learning algorithms—Gated Recurrent Unit (GRU), Recurrent Neural Networks (RNN), Long Short-Term Memory (LSTM), Support Vector Machine (SVM), K-nearest neighbor (KNN) and Decision tree (DT)—customized to accurately forecast sequential sedimentary structure data. These models, optimized using Adam's optimizer algorithm, demonstrated impressive performance in predicting water saturation levels using conventional petrophysical data. Particularly, the GRU model stood out, achieving remarkable accuracy (an R-squared value of 0.9973) with minimal errors (RMSE of 0.0198) compared to LSTM, RNN, SVM, KNN and, DT algorithms, thus showcasing its proficiency in processing extensive datasets and effectively identifying patterns. By achieving unprecedented accuracy levels, this study not only enhances the understanding of sediment properties and fluid saturation dynamics but also offers practical implications for reservoir management and hydrocarbon exploration in complex geological settings. These insights pave the way for more reliable and efficient decision-making processes, thereby advancing the forefront of reservoir engineering and petroleum geoscience.

A robust deep learning system for screening of obstructive sleep apnea using T-F spectrum of ECG signals

Obstructive sleep apnea (OSA) is a non-communicable sleep-related medical condition marked by repeated disruptions in breathing during sleep. It may induce various cardiovascular and neurocognitive complications. Electrocardiography (ECG) is a useful method for detecting numerous health-related disorders. ECG signals provide a less complex and non-invasive solution for the screening of OSA. Automated and accurate detection of OSA may enhance diagnostic performance and reduce the clinician’s workload. Traditional machine learning methods typically involve several labor-intensive manual procedures, including signal decomposition, feature evaluation, selection, and categorization. This article presents the time-frequency (T-F) spectrum classification of de-noised ECG data for the automatic screening of OSA patients using deep convolutional neural networks (DCNNs). At first, a filter-fusion algorithm is used to eliminate the artifacts from the raw ECG data. Stock-well transform (S-T) is employed to change filtered time-domain ECG into T-F spectrums. To discriminate between apnea and normal ECG signals, the obtained T-F spectrums are categorized using benchmark Alex-Net and Squeeze-Net, along with a less complex DCNN. The superiority of the presented system is measured by computing the sensitivity, specificity, accuracy, negative predicted value, precision, F1-score, and Fowlkes-Mallows index. The results of comparing all three utilized DCNNs reveal that the proposed DCNN requires fewer learning parameters and provides higher accuracy. An average accuracy of 95.31% is yielded using the proposed system. The presented deep learning system is lightweight and faster than Alex-Net and Squeeze-Net as it utilizes fewer learnable parameters, making it simple and reliable.

Advancing Sea Surface Height Retrieval through Global Navigation Satellite System Reflectometry: A Model Interaction Approach with Cyclone Global Navigation Satellite System and FengYun-3E Measurements

The measurement of sea surface height (SSH), which is of great importance in the field of oceanography, can be obtained through the innovative technique of GNSS-R for remote sensing. This research utilizes the dataset from spaceborne GNSS-R platforms, the Cyclone Global Navigation Satellite System (CYGNSS) and FengYun-3E (FY-3E), as the primary source of data for retrieving sea surface height (SSH). The utilization of artificial neural networks (ANNs) allows for the accurate estimation of ocean surface height with a precision of meter-level accuracy throughout the period of 1–17 August 2022. As a traditional machine learning method, an ANN is employed to extract pertinent data features, facilitating the acquisition of precise sea surface height estimations. Additionally, separate models are devised for both GNSS-R platforms, one based on constant velocity (CV) and the other on constant acceleration (CA). The Interactive Multiple Model (IMM) is utilized as the main method to combine the four models and convert the likelihood of each model. The transition between the models allows the filters to effectively adapt to dynamic changes and complex environments. This approach relies on the fundamental notion of the Kalman filter (KF), which showcases robust noise handling capabilities in predicting the SSH, separately. The results demonstrate that the model interaction technology is capable of efficiently filtering and integrating SSH data, yielding a Root Mean Square Error (RMSE) of 1.03 m. This corresponds to a 9.84% enhancement compared to the retrieved height from CYGNSS and a 37.19% enhancement compared to the retrieved height from FY-3E. The model proposed in this paper provides a potential scheme for the GNSS-R data fusion of multiple platforms and multiple models. In the future, more data sources and more models can be added to achieve more accurate adaptive fusion.

Explainable AI for myocardial infarction using the vectorcardiogram

Abstract Background and purpose Myocardial infarction (MI), a widespread cardiovascular ailment, often lacks immediate patient awareness being early and accurate MI diagnosis crucial for lifesaving interventions. In artificial intelligence studies, ECG signals are extensively explored, while fewer examine the potential of vectorcardiogram (VCG) which offers a three-dimensional cardiac dipole representation. Traditional Machine Learning methods manually extract features; some studies advocate VCG features. Deep Learning, especially Convolutional Neural Networks (CNN), automates feature extraction. Although ECG contains redundant information from the VCG's 12 projections, no studies directly apply CNNs to VCG, nor assess the impact of redundancy on predictive ability. This study aims to bridge this gap by investigating CNNs' diagnostic capabilities with VCG inputs, specifically focusing on improving MI classification. Methods The PTB-XL database was used for this study, containing 10-seconds ECG data from various pathologies, including MI. The database encompasses 9,481 records from healthy patients and 4,129 records from patients with MI. Data was divided into 80% for training and 20% for testing. Consequently, the dataset was expanded to have 52% representing MI and 48% representing healthy cases. We designed a 1D-CNN classifier (Figure 1.A) with three convolutional layers. The network was trained using both ECG and VCG reconstruction through the Inverse Dower Transform. Results The trained model achieved accuracy values of 0.895 and 0.914 for the test set when utilizing ECG and VCG, respectively. These results underscore how the information condensation in the VCG contributes to an enhanced predictive capacity of the model. In a qualitative analysis, activation maps extracted from the CNN illustrate heightened activation in the QRS complex when the disease is present, contrasting with diminished activation in its absence. These maps depict the weight assigned to each signal segment, enabling the classification of signals into healthy and diseased categories. The presented figure exemplifies a notable activation in the QRS loop in the case of myocardial infarction (MI), aligning with the observed alteration in the cardiac signal associated with this pathology (Figure 1.B). Conclusion In this study, an interpretable CNN has been designed for MI detection, with the VCG yielding better results than the ECG for this task due to information compression. This provides a new insight for enhancing classification capability in Deep Learning models, allowing for advancements in the MI detection.

Transfer learning for acoustic cement bond evaluation: An image classification approach using acoustic variable Density log

Ensuring the integrity of wellbore cement is critical for the environmental protection, safety, and economic viability of oil, gas, geothermal, and Carbon Capture and Storage (CCS) operations. Acoustic logging is the most commonly used evaluation method, and interpreting its data requires significant expertise, often informs high-stakes decisions. Machine Learning (ML) has been used to help the data interpretation, but they are heavily reliant on meticulous feature engineering—a labor-intensive and expertise-driven task. Moreover, ML methods cannot tell which portions of input acoustic data are most important in the cement evaluation task. This level of analysis is vital, as it can significantly inform and improve future strategies for acoustic logging data collection and signal processing, ultimately improving the cement evaluation results. To address these challenges, we present a generalized workflow. First, we convert Variable Density Log (VDL) data into images using the Continuous Wavelet Transform (CWT). Then, we apply transfer learning for image classification to enhance the classification of wellbore cement isolation. In transfer learning stage, the images are processed using several pre-trained image classifiers—Xception, VGG16, MobileNetV2, and ResNet50—originally trained on the extensive ImageNet database containing over 14 million images. To adapt these models for our specific task, we added a global average pooling layer to reduce feature map dimensionality, followed by a fully connected layer with a Rectified Linear Unit (ReLU) activation function to introduce non-linearity and enhance learning capability. We replaced the original final layer of each classifier with a new three-neuron layer using softmax activation, tailored for multi-class classification. We preserved the general characteristics learned from the ImageNet dataset by freezing all original layers except for the newly added ones and compiled the models using the Adam optimizer. The performance of these adapted models was evaluated based on accuracy, loss, and F1 score. We applied this workflow to two distinct datasets: the first from a Norwegian wellbore and the second from a hydrocarbon well in China. In the Norwegian case, the Xception model achieved a classification accuracy of 97.4%, which was further refined to 99.0% through the implementation of Gradient-weighted Class Activation Mapping (Grad-CAM), revealing critical frequency ranges that traditional ML methods could not identify. In the Chinese case, employing the same models and workflow, the MobileNetV2 algorithm demonstrated exceptional performance, achieving a 99.6% accuracy in predicting cement isolation. In addition to achieving high accuracy, our results reveal that the VDL frequency content between 20 and 25 kHz is more critical for cement evaluation in the investigated wells than other frequency ranges, a level of analysis unattainable with traditional ML methods. Furthermore, our approach eliminates the need for deep knowledge in feature engineering, typically requisite in traditional ML, and operates effectively without any feature engineering. These results underscore the efficacy of our approach across diverse geological settings and highlight the potential of transfer learning to revolutionize cement evaluation automation, inspiring its broader application in the industry.

Comparative analysis of heat transfer prediction for falling film evaporation on the horizontal tube based on machine learning methods

Precise prediction of heat transfer for falling film evaporation is necessary for the horizontal tube falling film evaporator design. In this study, models to predict the heat transfer are compared, which are developed by traditional Least Square method (LSM) and Machine Learning (ML) methods, including Support Vector Regression (SVR), Artificial Neural Network (ANN) and Random Forest Regression (RFR). The training data is obtained by a series of experiments, containing 510 experimental data of fresh water and seawater of salinities from 30 to 60 g/kg. The experiments were conducted with the test tube diameter of 19, 25.4 and 38 mm and with the heat flux of 7.7–12 kW/m2. All ML models behave better performances than the LSM model and the SVR model is suggested as the best model to predict the heat transfer, considering the prediction accuracy and generalization ability comprehensively. SHapley Additive exPlanations (SHAP) is used to explain the relationships between Nusselt number and input variables, which also helps to resolve controversial issues about the effects of spray density and tube diameter on the heat transfer. The importance of variables is ranked, in which the tube pitch and seawater salinity are regarded as the most influential variables. Besides, the limitations and performance of models are analyzed in terms of both dataset and algorithms.

A survey on wind power forecasting with machine learning approaches

AbstractWind power forecasting techniques have been well developed over the last half-century. There has been a large number of research literature as well as review analyses. Over the past 5 decades, considerable advancements have been achieved in wind power forecasting. A large body of research literature has been produced, including review articles that have addressed various aspects of the subject. However, these reviews have predominantly utilized horizontal comparisons and have not conducted a comprehensive analysis of the research that has been undertaken. This survey aims to provide a systematic and analytical review of the technical progress made in wind power forecasting. To accomplish this goal, we conducted a knowledge map analysis of the wind power forecasting literature published in the Web of Science database over the last 2 decades. We examined the collaboration network and development context, analyzed publication volume, citation frequency, journal of publication, author, and institutional influence, and studied co-occurring and bursting keywords to reveal changing research hotspots. These hotspots aim to indicate the progress and challenges of current forecasting technologies, which is of great significance for promoting the development of forecasting technology. Based on our findings, we analyzed commonly used traditional machine learning and advanced deep learning methods in this field, such as classical neural networks, and recent Transformers, and discussed emerging technologies like large language models. We also provide quantitative analysis of the advantages, disadvantages, forecasting accuracy, and computational costs of these methods. Finally, some open research questions and trends related to this topic were discussed, which can help improve the understanding of various power forecasting methods. This survey paper provides valuable insights for wind power engineers.