Advanced Deep Learning Models Research Articles

Comparative Evaluation of Sentiment Analysis Methods: From Traditional Techniques to Advanced Deep Learning Models

Abstract. Sentiment evaluation plays a crucial role in deciphering public perception and consumer responses in today's digital landscape. This investigation offers a thorough assessment of diverse sentiment evaluation techniques, contrasting conventional machine learning methodologies with cutting-edge deep learning frameworks. In particular, the research scrutinizes the efficacy of Bidirectional Encoder Representations from Transformers (BERT)-derived architectures (BERT-Base and Robustly Optimized BERT Pretraining Approach (RoBERTa)), Convolutional Neural Networks (CNN), Long Short-Term Memory Networks (LSTM), Support Vector Machines (SVM), and Naive Bayes classifiers. The study gauges these approaches based on their precision, recall, F1-metric, overall accuracy, and computational efficiency using an extensive sentiment evaluation dataset. The results reveal that BERT-based models, particularly RoBERTa, achieve the highest accuracy (87.44%) and F1-score (0.8746), though they also require the longest training time (approximately 3 hours). CNN and LSTM models strike a balance between performance and efficiency, while traditional methods like SVM and Naive Bayes offer faster training and deployment with moderate accuracy. The insights gained from this study are valuable for both researchers and practitioners, highlighting the trade-offs between model performance, computational demands, and practical deployment considerations in sentiment analysis applications.

Dynamic inferences of crash risks in freeway merging zones: a spatio-temporal deep learning model

Freeway merging zones are critical for freeway operations and management due to potential crashes arising from complex vehicle merging behaviours. This paper investigates the application of a spatio-temporal deep learning model to infer crash risks in the zones. We first introduce a crash risk index based on Time-To-Collision and vehicle merging patterns. An innovation is that the developed spatio-temporal transformer model can analyze the evolving risk index. This model effectively captures dynamic risk features through a multi-head attention mechanism within its spatio-temporal learning components. Numerical experiments on nine inference tasks with varying spatial resolutions show improved performance of the model with lower resolution. Moreover, the ST-Transformer model is benchmarked against three advanced deep learning models, which consistently demonstrates its superiority in capturing spatio-temporal dependence in risk sequences. This investigation significantly contributes to a richer understanding of proactive traffic safety, providing valuable insights for advanced freeway management and driver assistance systems.

Accurate Acupoint Localization in 2D Hand Images: Evaluating HRNet and ResNet Architectures for Enhanced Detection Performance

Introduction: This research assesses HRNet and ResNet architectures for their precision in localizing hand acupoints on 2D images, which is integral to automated acupuncture therapy. Objectives: The primary objective was to advance the accuracy of acupoint detection in traditional Korean medicine through the application of these advanced deep-learning models, aiming to improve treatment efficacy. Background: Acupoint localization in traditional Korean medicine is crucial for effective treatment, and the study aims to enhance this process using advanced deep-learning models. Methods: The study employs YOLOv3, YOLOF, and YOLOX-s for object detection within a top-down framework, comparing HRNet and ResNet architectures. These models were trained and tested using datasets annotated by technicians and their mean values, with performance evaluated based on Average Precision at two IoU thresholds. Results: HRNet consistently demonstrated lower mean distance errors across various acupoints compared to ResNet, particularly at a 256x256 pixel resolution. Notably, the HRNet-w48 model surpassed human annotators, including medical experts, in localization accuracy. Conclusion: HRNet's superior performance in acupoint localization suggests its potential to improve the precision and efficacy of acupuncture treatments. The study highlights the promising role of machine learning in enhancing traditional medical practices and underscores the importance of accurate acupoint localization in clinical acupuncture.

Advanced Deep Learning Models for Accurate Citrus Disease Classification: Performance Analysis and Insights

Advanced Deep Learning Models for Accurate Citrus Disease Classification: Performance Analysis and Insights

IDENTIFICATION AND LOCALIZATION OF VULNERABILITIES IN SMART CONTRACTS USING ATTENTION VECTORS ANALYSIS IN A BERT-BASED MODEL

Context. With the development of blockchain technology and the increasing use of smart contracts, which are automatically executed in blockchain networks, the significance of securing these contracts has become extremely relevant. Traditional code auditing methods often prove ineffective in identifying complex vulnerabilities, which can lead to significant financial losses. For example, the reentrancy vulnerability that led to the DAO attack in 2016 resulted in the loss of 3.6 million ethers and the split of the Ethereum blockchain network. This underscores the necessity for early detection of vulnerabilities. Objective. The objective of this work is to develop and test an innovative approach for identifying and localizing vulnerabilities in smart contracts based on the analysis of attention vectors in a model using BERT architecture. Method. The methodology described includes data preparation and training a transformer-based model for analyzing smart contract code. The proposed attention vector analysis method allows for the precise identification of vulnerable code segments. The use of the CodeBERT model significantly improves the accuracy of vulnerability identification compared to traditional methods. Specifically, three types of vulnerabilities are considered: reentrancy, timestamp dependence, and tx.origin vulnerability. The data is preprocessed, which includes the standardization of variables and the simplification of functions. Results. The developed model demonstrated a high F-score of 95.51%, which significantly exceeds the results of contemporary approaches, such as the BGRU-ATT model with an F-score of 91.41%. The accuracy of the method in the task of localizing reentrancy vulnerabilities was 82%. Conclusions. The experiments conducted confirmed the effectiveness of the proposed solution. Prospects for further research include the integration of more advanced deep learning models, such as GPT-4 or T5, to improve the accuracy and reliability of vulnerability detection, as well as expanding the dataset to cover other smart contract languages, such as Vyper or LLL, to enhance the applicability and efficiency of the model across various blockchain platforms. Thus, the developed CodeBERT-based model demonstrates high results in detecting and localizing vulnerabilities in smart contracts, which opens new opportunities for research in the field of blockchain platform security.

Speech Emotion Recognition Using Transfer Learning: Integration of Advanced Speaker Embeddings and Image Recognition Models

Automatic Speech Emotion Recognition (SER) plays a vital role in making human–computer interactions more natural and effective. A significant challenge in SER development is the limited availability of diverse emotional speech datasets, which hinders the application of advanced deep learning models. Transfer learning is a machine learning technique that helps address this issue by utilizing knowledge from pre-trained models to improve performance on a new task in a target domain, even with limited data. This study investigates the use of transfer learning from various pre-trained networks, including speaker embedding models such as d-vector, x-vector, and r-vector, and image classification models like AlexNet, GoogLeNet, SqueezeNet, ResNet-18, and ResNet-50. We also propose enhanced versions of the x-vector and r-vector models incorporating Multi-Head Attention Pooling and Angular Margin Softmax, alongside other architectural improvements. Additionally, reverberation from the Room Impulse Response datasets was added to the speech utterances to diversify and augment the available data. Notably, the enhanced r-vector model achieved classification accuracies of 74.05% Unweighted Accuracy (UA) and 73.68% Weighted Accuracy (WA) on the IEMOCAP dataset, and 80.25% UA and 79.81% WA on the CREMA-D dataset, outperforming the existing state-of-the-art methods. This study shows that using cross-domain transfer learning is beneficial for low-resource emotion recognition. The enhanced models developed in other domains (for non-emotional tasks) can further improve the accuracy of SER.

Comprehensive segmentation of gray matter structures on T1-weighted brain MRI: A Comparative Study of CNN, CNN hybrid-transformer or -Mamba architectures.

Recent advances in deep learning have shown promising results in medical image analysis and segmentation. However, most brain MRI segmentation models are limited by the size of their datasets and/or the number of structures they can identify. This study evaluates the performance of six advanced deep learning models in segmenting 122 brain structures from T1-weighted MRI scans, aiming to identify the most effective model for clinical and research applications. 1,510 T1-weighted MRIs were used to compare six deep-learning models for the segmentation of 122 distinct gray matter structures: nnU-Net, SegResNet, SwinUNETR, UNETR, U-Mamba_BOT and U-Mamba_ Enc. Each model was rigorously tested for accuracy using the Dice Similarity Coefficient (DSC) and the 95th percentile Hausdorff Distance (HD95). Additionally, the volume of each structure was calculated and compared between normal control (NC) and Alzheimer's Disease (AD) patients. U-Mamba_Bot achieved the highest performance with a median DSC of 0.9112 [IQR:0.8957, 0.9250]. nnU-Net achieved a median DSC of 0.9027 [IQR: 0.8847, 0.9205] and had the highest HD95 of 1.392[IQR: 1.174, 2.029]. The value of each HD95 (<3mm) indicates its superior capability in capturing detailed brain structures accurately. Following segmentation, volume calculations were performed, and the resultant volumes of normal controls and AD patients were compared. The volume changes observed in thirteen brain substructures were all consistent with those reported in existing literature, reinforcing the reliability of the segmentation outputs. This study underscores the efficacy of U-Mamba_Bot as a robust tool for detailed brain structure segmentation in T1-weighted MRI scans. The congruence of our volumetric analysis with the literature further validates the potential of advanced deep-learning models to enhance the understanding of neurodegenerative diseases such as AD. Future research should consider larger datasets to validate these findings further and explore the applicability of these models in other neurological conditions. AD = Alzheimer's Disease; ADNI = Alzheimer's Disease Neuroimaging Initiative; DSC = Dice Similarity Coefficient; HD95 = the 95th Percentile Hausdorff Distance; IQR = Interquartile Range; NC = Normal Control; SSMs = State-space Sequence Models.

Hydro-informer: a deep learning model for accurate water level and flood predictions

AbstractThis study aims to develop an advanced deep learning model, Hydro-Informer, for accurate water level and flood predictions, emphasizing extreme event forecasting. Utilizing a comprehensive dataset from the Slovak Hydrometeorological Institute SHMI (2008–2020), which includes precipitation, water level, and discharge data, the model was trained using a ladder technique with a custom loss function to enhance focus on extreme values. The architecture integrates Recurrent and Convolutional Neural Networks (RNN, CNN), and Multi-Head Attention layers. Hydro-Informer achieved significant performance, with a Coefficient of Determination (R2) of 0.88, effectively predicting extreme water levels 12 h in advance in a river environment free from human regulation and structures. The model’s strong performance in identifying extreme events highlights its potential for enhancing flood management and disaster preparedness. By integrating with diverse data sources, the model can be used to develop a well-functioning warning system to mitigate flood impacts. This work proposes a novel architecture suitable for locations without water regulation structures.

Sibling Discrimination Using Linear Fusion on Deep Learning Face Recognition Models

Facial recognition technology has revolutionised human identification, providing a non-invasive alternative to traditional biometric methods like signatures and voice recognition. The integration of deep learning has significantly enhanced the accuracy and adaptability of these systems, now widely used in criminal identification, access control, and security. Initial research focused on recognising full-frontal facial features, but recent advancements have tackled the challenge of identifying partially visible faces, a scenario that often reduces recognition accuracy. This study aims to identify siblings based on facial features, particularly in cases where only partial features like eyes, nose, or mouth are visible. Utilising advanced deep learning models such as VGG19, VGG16, VGGFace, and FaceNet, the research introduces a framework to differentiate between sibling images effectively. To boost discrimination accuracy, the framework employs a linear fusion technique that merges insights from all the models used. The methodology involves preprocessing image pairs, extracting embeddings with pre-trained models, and integrating information through linear fusion. Evaluation metrics, including confusion matrix analysis, assess the framework's robustness and precision. Custom datasets of cropped sibling facial areas form the experimental basis, testing the models under various conditions like different facial poses and cropped regions. Model selection emphasises accuracy and extensive training on large datasets to ensure reliable performance in distinguishing subtle facial differences. Experimental results show that combining multiple models' outputs using linear fusion improves the accuracy and realism of sibling discrimination based on facial features. Findings indicate a minimum accuracy of 96% across different facial regions. Although this is slightly lower than the accuracy achieved by a single model like VGG16 with full-frontal poses, the fusion approach provides a more realistic outcome by incorporating insights from all four models. This underscores the potential of advanced deep learning techniques in enhancing facial recognition systems for practical applications.

Improving thyroid cancer diagnosis with advanced deep learning techniques

The accurate diagnosis of thyroid carcinoma is a complex and critical challenge in medical practice due to the clinical and pathological diversity of thyroid tumors. This study explores the application of advanced deep learning models, specifically EfficientNet B4 and MobileNetV3, to enhance the classification of thyroid histopathological images. By leveraging transfer learning, these models were fine-tuned on a dataset of 7,272 images encompassing three types of carcinoma: Medullary, Papillary, and Vesicular. The performance of these models was evaluated using metrics such as accuracy, precision, recall, and F1-score. The results indicate that both models are effective, with EfficientNet B4 demonstrating slightly superior accuracy, particularly in more challenging diagnostic scenarios. Despite these promising results, the study acknowledges limitations related to dataset size and variability, which could affect the generalizability of the models. To address these issues, future research should focus on expanding the dataset and incorporating additional diagnostic data, such as genetic and molecular markers, to improve model performance and clinical applicability. This study highlights the potential of deep learning models to significantly enhance the diagnostic accuracy of thyroid carcinomas, paving the way for more reliable and precise clinical decision-making.

Advanced deep learning models for comprehensive analysis and optimization of nucleate boiling heat transfer systems

Advanced deep learning models for comprehensive analysis and optimization of nucleate boiling heat transfer systems

RNA-Seq analysis for breast cancer detection: a study on paired tissue samples using hybrid optimization and deep learning techniques

ProblemBreast cancer is a leading global health issue, contributing to high mortality rates among women. The challenge of early detection is exacerbated by the high dimensionality and complexity of gene expression data, which complicates the classification process.AimThis study aims to develop an advanced deep learning model that can accurately detect breast cancer using RNA-Seq gene expression data, while effectively addressing the challenges posed by the data’s high dimensionality and complexity.MethodsWe introduce a novel hybrid gene selection approach that combines the Harris Hawk Optimization (HHO) and Whale Optimization (WO) algorithms with deep learning to improve feature selection and classification accuracy. The model’s performance was compared to five conventional optimization algorithms integrated with deep learning: Genetic Algorithm (GA), Artificial Bee Colony (ABC), Cuckoo Search (CS), and Particle Swarm Optimization (PSO). RNA-Seq data was collected from 66 paired samples of normal and cancerous tissues from breast cancer patients at the Jawaharlal Nehru Cancer Hospital & Research Centre, Bhopal, India. Sequencing was performed by Biokart Genomics Lab, Bengaluru, India.ResultsThe proposed model achieved a mean classification accuracy of 99.0%, consistently outperforming the GA, ABC, CS, and PSO methods. The dataset comprised 55 female breast cancer patients, including both early and advanced stages, along with age-matched healthy controls.ConclusionOur findings demonstrate that the hybrid gene selection approach using HHO and WO, combined with deep learning, is a powerful and accurate tool for breast cancer detection. This approach shows promise for early detection and could facilitate personalized treatment strategies, ultimately improving patient outcomes.

Facia-fix: mobile application for bell’s palsy diagnosis and assessment using computer vision and deep learning

Facial paralysis (FP) is a condition characterized by the inability to move some or all of the muscles on one or both sides of the face. Diagnosing FP presents challenges due to the limitations of traditional methods, which are time-consuming, uncomfortable for patients, and require specialized clinicians. Additionally, more advanced tools are often uncommonly available to all healthcare providers. Early and accurate detection of FP is crucial, as timely intervention can prevent long-term complications and improve patient outcomes. To address these challenges, our research introduces Facia-Fix, a mobile application for Bell's palsy diagnosis, integrating computer vision and deep learning techniques to provide real-time analysis of facial landmarks. The classification algorithms are trained on the publicly available YouTube FP (YFP) dataset, which is labeled using the House-Brackmann (HB) method, a standardized system for assessing the severity of FP. Different deep learning models were employed to classify the FP severity, such as MobileNet, CNN, MLP, VGG16, and Vision Transformer. The MobileNet model which uses transfer learning, achieved the highest performance (Accuracy: 0.9812, Precision: 0.9753, Recall: 0.9727, F1 Score: 0.974), establishing it as the optimal choice among the evaluated models. The innovation of this approach lies in its use of advanced deep learning models to provide accurate, objective, non-invasive and real-time comprehensive quantitative assessment of FP severity. Preliminary results highlight the potential of Facia-Fix to significantly improve the diagnostic and follow-up experiences for both clinicians and patients.

A Comprehensive Review on Text Detection and Recognition in Scene Images

Detecting and recognizing text in natural scene images and videos is vital for several real-world applications, such as in the analysis of Crime scene CCTV footage, sports videos, and autonomous driving, to name a few. Therefore, one can expect several challenges, namely, arbitrarily oriented and shaped Text detection and identification in movies and natural environments. Many methods have been developed in the past to address these challenges, including advanced deep-learning models and transformers. Due to several methods available in the literature, it is not so easy to understand the open challenges, applications, directions, scope, limitations, and weaknesses of the methods. Therefore, there is a need to write a survey/review to highlight and discuss the strengths and weaknesses of the developed methods. This survey/review presents different categories of work and discusses their importance, limitations, new challenges, applications, and, finally, directions such that readers can choose appropriate methods and directions to carry out research work in the field of text detection/recognition in the natural scene and videos.

Comparative analysis of advanced deep learning models for predicting evapotranspiration based on meteorological data in bangladesh.

Evapotranspiration is one of the crucial elements in water balance equations and plays a pivotal role in the water and energy cycle of an area. An accurate and precise estimation and prediction of reference evapotranspiration (ETo) is necessary for regional management of water resources and irrigation scheduling. The challenge of predicting daily evapotranspiration with limited meteorological data in Bangladesh. This study aims to predict daily evapotranspiration using limited meteorological data of Bangladesh by three deep learning (CNN, GRU, LSTM) and one hybrid (CNN-GRU) model. The novel method of hybrid CNN-GRU models, which have not been commonly used for this purpose. The performance of models was evaluated by five accuracy matrices R2, RMSE, MAE, MAPE, and CE and comparison is visualized by radar graphs. The study's novelty lies in the use of hybrid CNN-GRU models to estimate reference evapotranspiration, as this algorithm has not been commonly used for this purpose. In the case of the Rangpur station, the hybrid CNN-GRU algorithm outperformed other models, achieving the best values across various statistical metrics during both the training and testing phases. The highest correlation coefficient values of approximately 0.994 and 0.995. Moreover, during training and testing stages, the hybrid model had the lowest MAE (0.076, 0.068) and RMSE (0.138, 0.106) at the Rangpur station. Additionally, in the Sreemangal station, it can be notable that the statistical parameter RSME found superior results in the hybrid model around 0.225 and 0.174, respectively. In addition, the highest R2 and CE values were noted as 0.986, 0.987 and 0.985, 0.986 during the training and testing phases, respectively. The comparison suggests that the hybrid model will be best suited for prediction with the limited meteorological data. The outcome of the present research signifies the ability of deep learning methods in the prediction of evapotranspiration and the dominant variables affecting the changes the in context of Bangladesh.

Dynamic Spatio-Temporal Modeling for Enhanced Air Quality Prediction: Implications for Information Management and Public Health Decision Support Systems

Air quality forecasting has significant implications for environmental monitoring, public health, and information management systems. This study proposes three advanced deep learning models: Graph Neural Networks with Dynamic Spatio-Temporal Attention (GNN-DSTA), Multi-Resolution Convolutional Recurrent Neural Networks (MRC-RNN), and Variational Autoencoders with Spatio-Temporal Latent Embeddings (VAE-STLE). These models address the challenges of capturing complex spatio-temporal dependencies in environments with sparse data samples. The GNN-DSTA model introduces a temporal attention mechanism that dynamically captures evolving spatial-temporal dependencies. MRC-RNN combines CNN's spatial pattern recognition with RNN's temporal modeling across multiple spatial resolutions. VAE-STLE provides a probabilistic framework for robust and interpretable forecasting. Experimental results demonstrate significant improvements in prediction accuracy: GNN-DSTA reduces RMSE by 15-20%, MRC-RNN improves accuracy by 12-15%, and VAE-STLE shows a 10-12% improvement with enhanced uncertainty estimation. These models advance AQI predictions through dynamic attention mechanisms, multi-resolution analysis, and probabilistic forecasting. Furthermore, this study explores the implications of these advanced predictions for information management and public health decision support systems. We discuss the integration of real-time data, scalability considerations for large-scale deployments, user interface design for effective communication of predictions, and ethical considerations in using AI-driven models for public health decision-making. The proposed approach not only enhances the accuracy and reliability of AQI predictions but also provides a framework for developing more effective and responsive public health interventions and environmental policies.

SENTIMENT ANALYSIS OF CUSTOMER FEEDBACK IN THE BANKING SECTOR: A COMPARATIVE STUDY OF MACHINE LEARNING MODELS

This study investigates the application of sentiment analysis to customer feedback in the banking sector, utilizing natural language processing (NLP) techniques and machine learning models to classify customer sentiments into positive, neutral, and negative categories. Feedback was sourced from online platforms, including bank websites, social media, and third-party review sites. Data preprocessing steps, such as tokenization, stemming, and feature extraction using TF-IDF, were employed to prepare the text for analysis. Various machine learning algorithms, including Logistic Regression, Random Forest, Support Vector Machine (SVM), Long Short-Term Memory (LSTM), and Naïve Bayes, were implemented and evaluated using metrics such as accuracy, precision, recall, and F1-score. The results show that LSTM outperformed all models with a 91% accuracy, followed closely by SVM at 89%. These findings demonstrate the potential of advanced machine learning techniques in accurately classifying sentiments and provide valuable insights into customer satisfaction and areas for improvement within the banking sector. Future work aims to further optimize models for better classification of neutral feedback and explore more advanced deep learning models, such as BERT.

ACGAN: adaptive conditional generative adversarial network architecture predicting skin lesion using collaboration of transfer learning models

Skin cancer has become a serious disease which has the potential to scale up if it is not identified earlier. It is imperative to detect and give treatment to skin cancer promptly. Diagnosing skin cancer manually takes a lot of time and it is costly, and the probability of false diagnosis has increased due to the outstanding resemblances among various skin lesions. Enhancing the classification of multi-class lesions of skin needs the development of investigative systems which should be automated. Data augmentation with GANs and Adaptive Conditional Generative Adversarial Network strategies improves performance. The performance is tested using balanced and unbalanced datasets. Using a proper process of augmentation of data, the suggested system attains a 94% accuracy for the VGG16, 93% for the ResNet50 and 94.25% for ResNet101. The process of collaboration of all such methods improves accuracy further to 95%. In summary, the novelty of the work lies in its holistic approach to automated skin lesion classification, incorporating advanced deep learning models, novel data augmentation techniques and comprehensive performance evaluation on real-world datasets. These contributions collectively advance the field of computer-aided diagnosis for the detection of skin cancer and treatment.

Advance deep learning for soil type classification in space informatics

Accurate soil type categorization is very important for resource management in space exploration. Using a complete system including a space station, rovers, and a deep learning framework, this study proposes an advanced deep learning model for soil type categorization in space informatics. Gathering and preprocessing multispectral and hyperspectral soil data, the rovers send it to the space station for in-depth study. Our model had a test accuracy of about 80%. For space informatics, the suggested method guarantees strong and accurate soil categorization, therefore enabling efficient decision-making.

Development of a novel light-sensitive PPG model using PPG scalograms and PPG-NET learning for non-invasive hypertension monitoring

Background and objectivePhotoplethysmography (PPG) signals provide a non-invasive method for monitoring cardiovascular health, including blood pressure levels, which are critical for the early detection and management of hypertension. This study leverages wavelet transformation and special purpose deep learning model, enhanced by signal processing and normalization, to classify blood pressure stages from PPG signals. The primary objective is to advance non-invasive hypertension monitoring, improving the accuracy and efficiency of these assessments. MethodsThe study employed continuous wavelet transform (CWT) to prepare PPG signals for analysis using a special purpose PPG-NET designed by applying advanced deep-learning models. PPG-NET was verified by applying several pre-trained models, including Inception, MobileNetV2, InceptionResNetV2, and others to the PPG data. Rigorously five-fold cross-validated models were conducted to obtain the models' performance to ensure robustness and repeatability of results. ResultsThe PPG-NET model demonstrated superior performance, achieving a perfect accuracy of 100 % in classifying the four stages of hypertension—normal, prehypertension, stage 1, and stage 2. The evaluation metrics reported include precision, sensitivity, and specificity, with the PPG-NET model achieving 100 % across all metrics. Other models showed varying levels of accuracy, with InceptionV3 also reaching 91.5 %, while some, like VGG-19, underperformed significantly. ConclusionsIntegrating CWT and PPG-NET offers a promising avenue for enhancing non-invasive blood pressure monitoring. The PPG-NET model, in particular, showed potential for clinical application due to its high accuracy and reliability. This study showed the effectiveness of combining advanced computational techniques with traditional PPG analysis, potentially leading to more personalized and accessible hypertension management strategies.