Learning Model Research Articles

Comparative Evaluation of Sentiment Analysis Methods: From Traditional Techniques to Advanced Deep Learning Models

Abstract. Sentiment evaluation plays a crucial role in deciphering public perception and consumer responses in today's digital landscape. This investigation offers a thorough assessment of diverse sentiment evaluation techniques, contrasting conventional machine learning methodologies with cutting-edge deep learning frameworks. In particular, the research scrutinizes the efficacy of Bidirectional Encoder Representations from Transformers (BERT)-derived architectures (BERT-Base and Robustly Optimized BERT Pretraining Approach (RoBERTa)), Convolutional Neural Networks (CNN), Long Short-Term Memory Networks (LSTM), Support Vector Machines (SVM), and Naive Bayes classifiers. The study gauges these approaches based on their precision, recall, F1-metric, overall accuracy, and computational efficiency using an extensive sentiment evaluation dataset. The results reveal that BERT-based models, particularly RoBERTa, achieve the highest accuracy (87.44%) and F1-score (0.8746), though they also require the longest training time (approximately 3 hours). CNN and LSTM models strike a balance between performance and efficiency, while traditional methods like SVM and Naive Bayes offer faster training and deployment with moderate accuracy. The insights gained from this study are valuable for both researchers and practitioners, highlighting the trade-offs between model performance, computational demands, and practical deployment considerations in sentiment analysis applications.

Advancing EGFR mutation subtypes prediction in NSCLC by combining 3D pretrained ConvNeXt, radiomics, and clinical features

PurposeThe aim of this study was to develop a novel approach for predicting the expression status of Epidermal Growth Factor Receptor (EGFR) and its subtypes in patients with Non-Small Cell Lung Cancer (NSCLC) using a Three-Dimensional Convolutional Neural Network (3D-CNN) ConvNeXt, radiomics features and clinical features.Materials and methodsA total of 732 NSCLC patients with available CT imaging and EGFR expression data were included in this retrospective study. The region of interest (ROI) was manually segmented, and clinicopathological features were collected. Radiomic and deep learning features were extracted. The instances were randomly divided into training, validation, and test sets. Feature selection was performed, and XGBoost was used to create solo models and combined models to predict the presence of EGFR and subtypes mutations. The effectiveness of the models was assessed using ROC and PRC curves.ResultsWe established the following models: ModelCNN, Modelradiomic, Modelclinical, ModelCNN+radiomic, ModelCNN+clinical, Modelradiomic+clinical, and ModelCNN+radiomic+clinical, which were based on deep learning features, radiomic features, clinical data and combinations of these, respectively. In predicting EGFR mutations, ModelCNN+radiomic+clinical demonstrated superior performance compared to other prediction models, achieving an AUC of 0.801. For distinguishing between EGFR subtypes ex19del and L858R, ModelCNN+radiomic reached the highest AUC value of 0.775.ConclusionsBoth deep learning models and radiomic signature-based models offer reasonably accurate non-invasive predictions of EGFR status and its subtypes. Fusion models hold the potential to enhance noninvasive methods for predicting EGFR mutations and subtypes, presenting a more reliable prediction approach.

Conv‐ELSTM: An ensemble deep learning approach for predicting short‐term wind power

AbstractAccurate and reliable forecasting of wind power is essential for the stable integration of wind energy into the electrical grid. However, the chaotic nature of wind power presents a significant challenge in utilizing data for effective short‐term forecasting, such as 60‐min predictions. This article introduces a hybrid data‐driven framework that employs an ensemble deep learning model to provide highly precise short‐term wind power predictions. The framework leverages a data‐driven approach to identify the intrinsic components of wind power data, including high‐frequency and low‐frequency components. A convolutional layer‐based feature fusion network is then established to properly extract important information from irrelevant wind energy features. Subsequently, an ensemble of long short‐term memory (LSTM) networks is developed to forecast wind power using the fused features, thereby mitigating the disadvantage of a single prediction model. The numerical experiment is carried out based on two different real‐life datasets. The results demonstrate the effectiveness of the proposed method in forecasting short‐term wind power compared to five benchmarks.

Predicting neurodevelopmental disorders using machine learning models and electronic health records - status of the field.

Machine learning (ML) is increasingly used to identify patterns that could predict neurodevelopmental disorders (NDDs), such as autism spectrum disorder (ASD) and attention-deficit hyperactivity disorder (ADHD). One key source of multilevel data for ML prediction models includes population-based registers and electronic health records. These can contain rich information on individual and familial medical histories and socio-demographics. This review summarizes studies published between 2010-2022 that used ML algorithms to develop predictive models for NDDs using population-based registers and electronic health records. A literature search identified 1191 articles, of which 32 were retained. Of these, 47% developed ASD prediction models and 25% ADHD models. Classical ML methods were used in 82% of studies and in particular tree-based prediction models performed well. The sensitivity of the models was lower than 75% for most studies, while the area under the curve (AUC) was greater than 75%. The most important predictors were patient and familial medical history and sociodemographic factors. Using private in-house datasets makes comparing and validating model generalizability across studies difficult. The ML model development and reporting guidelines were adopted only in a few recently reported studies. More work is needed to harness the power of data for detecting NDDs early.

Impact of Feature Selection Techniques on the Performance of Machine Learning Models for Depression Detection Using EEG Data

Major depressive disorder (MDD) poses a significant challenge in mental healthcare due to difficulties in accurate diagnosis and timely identification. This study explores the potential of machine learning models trained on EEG-based features for depression detection. Six models and six feature selection techniques were compared, highlighting the crucial role of feature selection in enhancing classifier performance. This study investigates the six feature selection methods: Elastic Net, Mutual Information (MI), Chi-Square, Forward Feature Selection with Stochastic Gradient Descent (FFS-SGD), Support Vector Machine-based Recursive Feature Elimination (SVM-RFE), and Minimal-Redundancy-Maximal-Relevance (mRMR). These methods were combined with six diverse classifiers: Logistic Regression, Support Vector Machine (SVM), Random Forest, Extreme Gradient Boosting (XGBoost), Categorical Boosting (CatBoost), and Light Gradient Boosting Machine (LightGBM). The results demonstrate the substantial impact of feature selection on model performance. SVM-RFE with SVM achieved the highest accuracy (93.54%) and F1 score (95.29%), followed by Logistic Regression with an accuracy of 92.86% and F1 score of 94.84%. Elastic Net also delivered strong results, with SVM and Logistic Regression both achieving 90.47% accuracy. Other feature selection methods yielded lower performance, emphasizing the importance of selecting appropriate feature selection and machine learning algorithms. These findings suggest that careful selection and application of feature selection techniques can significantly enhance the accuracy of EEG-based depression detection.

Comparative Analysis of Machine Learning Models for Predicting Rice Yield: Insights from Agricultural Inputs and Practices in Rwanda

Food security is a global challenge, especially in developing countries like Rwanda. With a growing population and limited agricultural land, Rwanda struggles to meet increasing food demands. Rice is a staple food crop in Rwanda, playing a crucial role in the country’s food security. However, factors like climate variability, soil nutrient management, and limited access to high-quality inputs hinder rice yield optimization. This paper investigates the most effective machine learning model for predicting rice crop yield in Rwanda, using agricultural inputs and practices. The study used secondary datasets from the National Institute of Statistics of Rwanda (NISR) for rice yield prediction. Eight supervised machine learning algorithms were used, including Linear Regression, Random Forest, Gradient Boosting, Support Vector Machine, Artificial Neural Network, eXtreme Gradient Boosting Tree, and AdaBoost. The models were evaluated based on their accuracy in predicting rice yields, with RMSE, MAE, and Relative Error as primary metrics. The feature importance analysis was also conducted to identify significant factors influencing yield predictions. The study’s findings revealed that the Adaptive Boosting Tree model outperformed the other machine learning models in predicting rice yield. This model achieved RMSE, MAE and Relative Error of 0.69, 0.46 and 12.4%, respectively, indicating a high level of predictive accuracy. The feature importance analysis further highlighted the key factors that contributed to rice yield predictions, with the quantity of inorganic fertilizer, degree of erosion, season, and seed type emerging as the most influential variables. The study demonstrates the effectiveness of machine learning models, particularly the Adaptive Boosting Tree, in improving rice yield predictions and highlighting the crucial role of agricultural inputs like fertilizer and seed type, in influencing crop yields. The output from this study will help the farmers and stakeholders to make data-driven decisions about resource use and crop management.

Uncertainty quantification for a deep learning models for image-based crack segmentation

Uncertainty quantification for a deep learning models for image-based crack segmentation

Evaluating the Impact of Technological Innovations on Operational Risk Management in Financial Institutions

This study evaluates the impact of technological innovations on operational risk management in financial institutions, focusing on how emerging technologies influence risk identification, mitigation, and management processes. Employing a qualitative approach through literature review and library research, this study synthesizes recent findings on the integration of technologies such as artificial intelligence (AI), blockchain, big data analytics, and cloud computing within financial risk frameworks. Findings indicate that these technologies offer substantial improvements in risk management by enhancing accuracy in risk detection, increasing response speed, and reducing human error. AI and machine learning models, for instance, are shown to enable real-time monitoring and predictive analytics, allowing institutions to identify potential risks before they materialize. Blockchain technology improves transparency and security in transaction processing, thereby mitigating fraud-related risks. Big data analytics provides insights into customer behavior, which can help detect anomalous activities and improve decision-making. However, the study also highlights challenges, including the need for robust cybersecurity measures, potential regulatory gaps, and skill shortages in handling complex technological systems. This study provides insights for financial institutions aiming to balance technological integration with effective risk management, underscoring the need for comprehensive frameworks that align technological advancements with regulatory compliance and organizational capability. Future research should focus on empirical studies assessing the long-term implications of these technologies on risk management efficacy in diverse financial settings.

ResNet-Transformer deep learning model-aided detection of dens evaginatus.

Dens evaginatus is a dental morphological developmental anomaly. Failing to detect it may lead to tubercles fracture and pulpal/periapical disease. Consequently, early detection and intervention of dens evaginatus are significant to preserve vital pulp. This study aimed to develop a deep learning model to assist dentists in early diagnosing dens evaginatus, thereby supporting early intervention and mitigating the risk of severe consequences. In this study, a deep learning model was developed utilizing panoramic radiograph images sourced from 1410 patients aged 3-16 years, with high-quality annotations to enable the automatic detection of dens evaginatus. Model performance and model's efficacy in aiding dentists were evaluated. The findings indicated that the current deep learning model demonstrated commendable sensitivity (0.8600) and specificity (0.9200), outperforming dentists in detecting dens evaginatus with an F1-score of 0.8866 compared to their average F1-score of 0.8780, indicating that the model could detect dens evaginatus with greater precision. Furthermore, with its support, young dentists heightened their focus on dens evaginatus in tooth germs and achieved improved diagnostic accuracy. Based on these results, the integration of deep learning for dens evaginatus detection holds significance and can augment dentists' proficiency in identifying such anomaly.

DB-Net and DVR-Net: Optimized New Deep Learning Models for Efficient Cardiovascular Disease Prediction

Cardiovascular Disease (CVD) is one of the main causes of death in recent years. To overcome the challenges faced during diagnosing CVD at an early stage, deep learning has been used. With advancements in technology, the clinical practice in the health care industry is likely to transform significantly. To predict CVD, we constructed two models: Dense Belief Network (DB-Net) and Deep Vanilla Recurrent Network (DVR-Net). Proximity Weighted Random Affine Shadow sampling balancing technique is used for balancing the highly imbalanced Heart Disease Health Indicator dataset. SHapley Additive exPlanations exhibits each feature’s contribution. It is used to visualize features contribution to the output of DB-Net and DVR-Net in CVD prediction. Furthermore, 10-Fold Cross Validation is performed for evaluating the proposed models performance. Cross-dataset evaluation is also conducted on proposed models to see how well our proposed models generalize on unseen data. Various evaluation measures are used for assessment of models. The proposed DB-Net outperforms all the base models by achieving an accuracy of 91%, F1-score of 91%, precision of 93%, recall of 89%, and execution time of 1883 s on 30 epochs with batch size 32. The DVR-Net beats the state-of-art models with an accuracy of 90%, F1-score of 90%, precision of 90%, recall of 90%, and execution time of 2853 s on 30 epochs with batch size 32.

Deep active learning for multi label text classification.

Given a set of labels, multi-label text classification (MLTC) aims to assign multiple relevant labels for a text. Recently, deep learning models get inspiring results in MLTC. Training a high-quality deep MLTC model typically demands large-scale labeled data. And comparing with annotations for single-label data samples, annotations for multi-label samples are typically more time-consuming and expensive. Active learning can enable a classification model to achieve optimal prediction performance using fewer labeled samples. Although active learning has been considered for deep learning models, there are few studies on active learning for deep multi-label classification models. In this work, for the deep MLTC model, we propose a deep Active Learning method based on Bayesian deep learning and Expected confidence (BEAL). It adopts Bayesian deep learning to derive the deep model's posterior predictive distribution and defines a new expected confidence-based acquisition function to select uncertain samples for deep MLTC model training. Moreover, we perform experiments with a BERT-based MLTC model, where BERT can achieve satisfactory performance by fine-tuning in various classification tasks. The results on benchmark datasets demonstrate that BEAL enables more efficient model training, allowing the deep model to achieve training convergence with fewer labeled samples.

Harvest Harmony: Integrating Linear and Nonlinear Machine Learning Models for Precision Crop Recommendation

This research presents a comprehensive crop recommendation system that combines linear and nonlinear machine learning models to forecast suitable crops based on soil and environmental factors. The study evaluates the performance of Logistic Regression (linear) alongside Random Forest, Gradient Boosting, and XGBoost (nonlinear models) in delivering accurate crop suggestions. Precision, recall, and F1-score metrics are employed to assess the models' effectiveness. The dataset incorporates vital soil characteristics, such as nitrogen, phosphorus, and potassium levels, and environmental parameters like temperature, humidity, pH, and rainfall. The results demonstrate that while linear models like Logistic Regression offer simplicity, nonlinear models such as XGBoost excel in capturing complex data patterns, providing more precise recommendations. This dual-model strategy empowers farmers to make informed decisions tailored to specific farming conditions, boosting agricultural productivity. The study underscores the importance of integrating both linear and nonlinear machine learning techniques to achieve optimal crop recommendations, encouraging resource efficiency and sustainable agricultural practices.

The Influence of the STAD and NHT Learning Models on Students' Cognitive Achievement and Learning Motivation on Colloid Material in Banyumas

This study aims to assess the impact of the Student Team Achievement Division (STAD) and Numbered Head Together (NHT) learning models on students’ cognitive achievement and learning motivation in colloidal material at MA Negeri 1 Banyumas. The study uses a quantitative approach with a quasi-experimental design, involving students from six XI science classes of MA Negeri 1 Banyumas. The first experimental group was taught using the STAD model, while the second group was taught using the NHT model. The sampling technique used was random sampling. Instruments included multiple-choice tests to measure cognitive achievement and learning motivation. Data analysis was carried out using an independent t-test. The results showed that both the STAD and NHT models had a significant impact on students’ cognitive achievement and learning motivation in colloid material. The p-values ​​for the NHT learning model were 0.47 and 0.39, while the p-values ​​for the STAD learning model were 0.17 and 0.16.

Robust double machine learning model with application to omics data.

Recently, there has been a growing interest in combining causal inference with machine learning algorithms. Double machine learning model (DML), as an implementation of this combination, has received widespread attention for their expertise in estimating causal effects within high-dimensional complex data. However, the DML model is sensitive to the presence of outliers and heavy-tailed noise in the outcome variable. In this paper, we propose the robust double machine learning (RDML) model to achieve a robust estimation of causal effects when the distribution of the outcome is contaminated by outliers or exhibits symmetrically heavy-tailed characteristics. In the modelling of RDML model, we employed median machine learning algorithms to achieve robust predictions for the treatment and outcome variables. Subsequently, we established a median regression model for the prediction residuals. These two steps ensure robust causal effect estimation. Simulation study show that the RDML model is comparable to the existing DML model when the data follow normal distribution, while the RDML model has obvious superiority when the data follow mixed normal distribution and t-distribution, which is manifested by having a smaller RMSE. Meanwhile, we also apply the RDML model to the deoxyribonucleic acid methylation dataset from the Alzheimer's disease (AD) neuroimaging initiative database with the aim of investigating the impact of Cerebrospinal Fluid Amyloid 42 (CSF A 42) on AD severity. These findings illustrate that the RDML model is capable of robustly estimating causal effect, even when the outcome distribution is affected by outliers or displays symmetrically heavy-tailed properties.

The Effect of STAD-Cooperative Learning Model with Concept Maps in Promoting Students’ Self-Efficacy and Conceptual Understanding of Thermochemistry

The research sought to explore how employing the Student Team Achievement Division (STAD) cooperative learning model, along with concept maps, impacts the enhancement of students' self-efficacy and grasp of thermochemistry concepts. This study employed a quasi-experimental approach using a pretest-posttest control group design. It was carried out among 127 11th-grade science students from MAN 1 Bengkulu City, chosen through cluster random sampling. The information was gathered by utilizing self-efficacy questionnaires and conceptual understanding tests and was analyzed using Multivariate Analysis of Variance (MANOVA). The findings indicated that the obtained p-value was less than 0.05 (p<0.05), revealing a statistically significant difference in self-efficacy and conceptual understanding between students in the experimental and control groups. This suggests that students in the experimental group showed greater enhancements in self-efficacy and conceptual understanding when compared to those in the controlled group. Thus, These findings revealed that STAD-cooperative learning with concept maps had a better influence in enhancing self-efficacy and student conceptual understanding of thermochemistry.

Multi-condition hawser tension prediction of offshore offloading system based on long and short-term memory network and transfer learning

Real-time prediction of hawser tension of the side-by-side oil offloading system of Floating Production Storage and Offloading (FPSO) can provide early warnings for hawser breakages and ship collision risk, thus improving structural, property, and environmental security. Although offline numerical models and Long Short-Term Memory (LSTM) could offer substantial precision, they confront intensive time costs in recalibrating or retraining models. This paper proposes an integration method of LSTM networks and transfer learning for real-time tension prediction considering inputs of actual remote wave elevations. We use short-term environmental data and numerical simulation data of correlated hawser tensions during offloading operations to train a pre-trained benchmark model for transfer learning. Then, a highly generalized and efficient transferred model is constructed by using a small sample to realize short-term tension predictions in time-varying environments. The results show the occurrence time and value of extreme tensions predicted by transfer learning nearly match the reference data, and their maximum errors are 3 s and 0.11, respectively, superior to LSTM direct training. Therefore, it provides sufficient demand for real-time prediction and early collision risk prevention in dynamically changing ocean environment. The research results could provide an alternative framework for intelligent monitoring of large-scale marine structures.

A Deep Learning-Based Approach to Characterize Skull Physical Properties: A Phantom Study.

Transcranial ultrasound imaging is a popular method to study cerebral functionality and diagnose brain injuries. However, the detected ultrasound signal is greatly distorted due to the aberration caused by the skull bone. The aberration mechanism mainly depends on thickness and porosity, two important skull physical characteristics. Although skull bone thickness andporosity can be estimated from CT or MRI scans, there is significant value in developing methods for obtaining thickness and porosity information from ultrasound itself. Here, we extracted various features from ultrasound signals using physical skull-mimicking phantoms of a range of thicknesses with embedded porosity-mimicking acoustic mismatches and analyzed them using machine learning (ML) and deep learning (DL) models. The performance evaluation demonstrated that both ML- and DL-trained models could predict the physical characteristics of a variety of skull phantoms with reasonable accuracy. The proposed approach could be expanded upon and utilized for the development of effective skull aberration correction methods.

UNIQUE: A Framework for Uncertainty Quantification Benchmarking.

Machine learning (ML) models have become key in decision-making for many disciplines, including drug discovery and medicinal chemistry. ML models are generally evaluated prior to their usage in high-stakes decisions, such as compound synthesis or experimental testing. However, no ML model is robust or predictive in all real-world scenarios. Therefore, uncertainty quantification (UQ) in ML predictions has gained importance in recent years. Many investigations have focused on developing methodologies that provide accurate uncertainty estimates for ML-based predictions. Unfortunately, there is no UQ strategy that consistently provides robust estimates about model's applicability on new samples. Depending on the dataset, prediction task, and algorithm, accurate uncertainty estimations might be unfeasible to obtain. Moreover, the optimum UQ metric also varies across applications, and previous investigations have shown a lack of consistency across benchmarks. Herein, the UNIQUE (UNcertaInty QUantification bEnchmarking) framework is introduced to facilitate a comparison of UQ strategies in ML-based predictions. This Python library unifies the benchmarking of multiple UQ metrics, including the calculation of nonstandard UQ metrics (combining information from the dataset and model), and provides a comprehensive evaluation. In this framework, UQ metrics are evaluated for different application scenarios, e.g., eliminating the predictions with the lowest confidence or obtaining a reliable uncertainty estimate for an acquisition function. Taken together, this library will help to standardize UQ investigations and evaluate new methodologies.

Concrete Creep Prediction Based on Improved Machine Learning and Game Theory: Modeling and Analysis Methods

Understanding the impact of creep on the long-term mechanical features of concrete is crucial, and constructing an accurate prediction model is the key to exploring the development of concrete creep under long-term loads. Therefore, in this study, three machine learning (ML) models, a Support Vector Machine (SVM), Random Forest (RF), and Extreme Gradient Boosting Machine (XGBoost), are constructed, and the Hybrid Snake Optimization Algorithm (HSOA) is proposed, which can reduce the risk of the ML model falling into the local optimum while improving its prediction performance. Simultaneously, the contributions of the input features are ranked, and the optimal model’s prediction outcomes are explained through SHapley Additive exPlanations (SHAP). The research results show that the optimized SVM, RF, and XGBoost models increase their accuracies on the test set by 9.927%, 9.58%, and 14.1%, respectively, and the XGBoost has the highest precision in forecasting the concrete creep. The verification results of four scenarios confirm that the optimized model can precisely capture the compliance changes in long-term creep, meeting the requirements for forecasting the nature of concrete creep.

Detection of Elementary White Mucosal Lesions by an AI System: A Pilot Study

Aim: Accurately identifying primary lesions in oral medicine, particularly elementary white lesions, is a significant challenge, especially for trainee dentists. This study aimed to develop and evaluate a deep learning (DL) model for the detection and classification of elementary white mucosal lesions (EWMLs) using clinical images. Materials and Methods: A dataset was created by collecting photographs of various oral lesions, including oral leukoplakia, OLP plaque-like and reticular forms, OLL, oral candidiasis, and hyperkeratotic lesions from the Unit of Oral Medicine. The SentiSight.AI (Neurotechnology Co.®, Vilnius, Lithuania) AI platform was used for image labeling and model training. The dataset comprised 221 photos, divided into training (n = 179) and validation (n = 42) sets. Results: The model achieved an overall precision of 77.2%, sensitivity of 76.0%, F1 score of 74.4%, and mAP of 82.3%. Specific classes, such as condyloma and papilloma, demonstrated high performance, while others like leucoplakia showed room for improvement. Conclusions: The DL model showed promising results in detecting and classifying EWMLs, with significant potential for educational tools and clinical applications. Expanding the dataset and incorporating diverse image sources are essential for improving model accuracy and generalizability.