Attention Weights Research Articles

ENSO predictive analytics based on layered rendering

AbstractEl Niño-Southern Oscillation (ENSO) is a periodic climate phenomenon in the equatorial Pacific that significantly influences global climate patterns. Accurate prediction and monitoring of ENSO events are essential for meteorological agencies and governmental institutions. This study introduces a content-guided attention module within a convolutional neural network to improve prediction accuracy. This module models inter-channel relationships and enhances information interaction by integrating channel and spatial attention weights. These advancements substantially improve prediction accuracy and help overcome the spring prediction barrier in ENSO forecasting. The research emphasizes global feature modeling and proposes a novel content-guided ENSO prediction model. It also includes an ocean data generation model utilizing global attention. Furthermore, a layered rendering technique is employed to invert ocean data, facilitating detailed analysis and contributing to the development of an ocean synthetic dataset.

YOLOv8-seg-CP: a lightweight instance segmentation algorithm for chip pad based on improved YOLOv8-seg model.

Real-time detection and accurate segmentation of chip pads are important tasks to ensure chip alignment and position correction. To address the challenges of small target chip pad detection, segmentation accuracy and model lightweight, this paper proposes a lightweight chip pad instance segmentation algorithm based on an improved YOLOv8-seg, named YOLOv8-seg-CP (chip pad). Firstly, we integrate the next-generation lightweight StarNet into the original backbone network to enhance fine feature capture capabilities while reducing the number of parameters. Then, we construct the C2f-Star module in the neck network, which enhances the feature extraction performance for small chip pad targets. This maintains accuracy, reduces computational load, and improves detection and segmentation speed. On this basis, we introduce a lightweight shared convolution segmentation head (LSCSH), significantly reducing both parameter count and computational load while enhancing segmentation performance. Additionally, we propose a CGCAFusion convolutional attention fusion module. This module uses a content-guided convolutional attention fusion mechanism to dynamically adjust attention weights based on the content of input features, capturing both global and local feature information and enhancing multimodal feature fusion. Experiments on the chip pad dataset demonstrate that our algorithm achieves a detection and segmentation accuracy of 89.8%. The model size, parameters, and FLOPs are 3.7M, 1.7M, and 8.2G respectively, representing reductions of 45.6%, 50%, and 31.7% compared to the baseline model. The FPS is 1399.3, an improvement of 25.8% over the baseline model. The inference time is 0.72ms, which is 0.18ms less than the baseline model. Extensive experimental results on COCO, carparts-seg, and crack-seg datasets further show that the improved YOLOv8n-seg model outperforms many existing advanced methods in terms of performance. This approach holds significant industrial application value for fully automated chip testing and sorting integrated production.

Multi-Head Attention Affinity Diversity Sharing Network for Facial Expression Recognition

Facial expressions exhibit inherent similarities, variability, and complexity. In real-world scenarios, challenges such as partial occlusions, illumination changes, and individual differences further complicate the task of facial expression recognition (FER). To further improve the accuracy of FER, a Multi-head Attention Affinity and Diversity Sharing Network (MAADS) is proposed in this paper. MAADS comprises a Feature Discrimination Network (FDN), an Attention Distraction Network (ADN), and a Shared Fusion Network (SFN). To be specific, FDN first integrates attention weights into the objective function to capture the most discriminative features by using the proposed sparse affinity loss. Then, ADN employs multiple parallel attention networks to maximize diversity within spatial attention units and channel attention units, which guides the network to focus on distinct, non-overlapping facial regions. Finally, SFN deconstructs facial features into generic parts and unique parts, which allows the network to learn the distinctions between these features without having to relearn complete features from scratch. To validate the effectiveness of the proposed method, extensive experiments were conducted on several widely used in-the-wild datasets including RAF-DB, AffectNet-7, AffectNet-8, FERPlus, and SFEW. MAADS achieves the accuracy of 92.93%, 67.14%, 64.55%, 91.58%, and 62.41% on these datasets, respectively. The experimental results indicate that MAADS not only outperforms current state-of-the-art methods in recognition accuracy but also has a relatively low computational complexity.

Rumor detection model with weighted GraphSAGE focusing on node location

While social media platforms promote people’s information exchange and dissemination, they also make rumors spread rapidly on online platforms. Therefore, how to detect rumors quickly, timely and accurately has become a hot topic for scholars in related fields. Traditional deep learning models ignore the relationship and topology between nodes in the rumor detection task and use fixed weights or mean aggregation strategies in the feature aggregation process, which fail to capture the complex interactions between nodes and the dynamics of information propagation, limiting the accuracy and robustness of the rumor detection model. To address these problems, we propose a location-aware weighted GraphSAGE rumor detection model GSMA. We first introduce an attention mechanism that dynamically assigns different attention weights to different neighboring nodes for different degrees of aggregation, improving GraphSAGE’s strategy of using mean-value aggregation for all neighboring nodes during the aggregation process; second, we introduce a modulated position encoding into the model and encode the position information of nodes into the features to improve the model’s ability to perceive the relative position and order of nodes; finally, the post text sentiment is incorporated into the features to provide additional semantic information for the model as a way to achieve rumor detection in microblogging platforms. Experiments show that the accuracy of the GSMA model on Ma-Weibo and Weibo23 reaches 97.43% and 97.55%, which is an improvement of 1.11% and 0.77% compared to the benchmark GraphSAGE, and all the evaluation metrics are also improved compared to other optimal rumor detection models.

Identifying Performance Issues in Cloud Service Systems Based on Relational-Temporal Features

Cloud systems, typically comprised of various components ( e.g. , microservices), are susceptible to performance issues, which may cause service-level agreement violations and financial losses. Identifying performance issues is thus of paramount importance for cloud vendors. In current practice, crucial metrics, i.e. , key performance indicators (KPIs), are monitored periodically to provide insight into the operational status of components. Identifying performance issues is often formulated as an anomaly detection problem, which is tackled by analyzing each metric independently. However, this approach overlooks the complex dependencies existing among cloud components. Some graph neural network-based methods take both temporal and relational information into account, however, the correlation violations in the metrics that serve as indicators of underlying performance issues are difficult for them to identify. Furthermore, a large volume of components in a cloud system results in a vast array of noisy metrics. This complexity renders it impractical for engineers to fully comprehend the correlations, making it challenging to identify performance issues accurately. To address these limitations, we propose Identifying Performance Issues based on Relational-Temporal Features (ISOLATE ), a learning-based approach that leverages both the relational and temporal features of metrics to identify performance issues. In particular, it adopts a graph neural network with attention to characterizing the relations among metrics and extracts long-term and multi-scale temporal patterns using a GRU and a convolution network, respectively. The learned graph attention weights can be further used to localize the correlation-violated metrics. Moreover, to relieve the impact of noisy data, ISOLATE utilizes a positive unlabeled learning strategy that tags pseudo labels based on a small portion of confirmed negative examples. Extensive evaluation on both public and industrial datasets shows that ISOLATE outperforms all baseline models with 0.945 F1-score and 0.920 Hit rate@3. The ablation study also proves the effectiveness of the relational-temporal features and the PU-learning strategy. Furthermore, we share the success stories of leveraging ISOLATE to identify performance issues in Huawei Cloud, which demonstrates its superiority in practice.

ARMNet: A Network for Image Dimensional Emotion Prediction Based on Affective Region Extraction and Multi-Channel Fusion.

Compared with discrete emotion space, image emotion analysis based on dimensional emotion space can more accurately represent fine-grained emotion. Meanwhile, this high-precision representation of emotion requires dimensional emotion prediction methods to sense and capture emotional information in images as accurately and richly as possible. However, the existing methods mainly focus on emotion recognition by extracting the emotional regions where salient objects are located while ignoring the joint influence of objects and background on emotion. Furthermore, in the existing literature, when fusing multi-level features, no consideration has been given to the varying contributions of features from different levels to emotional analysis, which makes it difficult to distinguish valuable and useless features and cannot improve the utilization of effective features. This paper proposes an image emotion prediction network named ARMNet. In ARMNet, a unified affective region extraction method that integrates eye fixation detection and attention detection is proposed to enhance the combined influence of objects and backgrounds. Additionally, the multi-level features are fused with the consideration of their different contributions through an improved channel attention mechanism. In comparison to the existing methods, experiments conducted on the CGnA10766 dataset demonstrate that the performance of valence and arousal, as measured by Mean Squared Error (MSE), Mean Absolute Error (MAE), and Coefficient of Determination (R²), has improved by 4.74%, 3.53%, 3.62%, 1.93%, 6.29%, and 7.23%, respectively. Furthermore, the interpretability of the network is enhanced through the visualization of attention weights corresponding to emotional regions within the images.

Vulnerability detection using BERT based LLM model with transparency obligation practice towards trustworthy AI

Vulnerabilities in the source code are one of the main causes of potential threats in software-intensive systems. There are a large number of vulnerabilities published each day, and effective vulnerability detection is critical to identifying and mitigating these vulnerabilities. AI has emerged as a promising solution to enhance vulnerability detection, offering the ability to analyse vast amounts of data and identify patterns indicative of potential threats. However, AI-based methods often face several challenges, specifically when dealing with large datasets and understanding the specific context of the problem. Large Language Model (LLM) is now widely considered to tackle more complex tasks and handle large datasets, which also exhibits limitations in terms of explaining the model outcome and existing works focus on providing overview of explainability and transparency. This research introduces a novel transparency obligation practice for vulnerability detection using BERT based LLMs. We address the black-box nature of LLMs by employing XAI techniques, unique combination of SHAP, LIME, heat map. We propose an architecture that combines the BERT model with transparency obligation practices, which ensures the assurance of transparency throughout the entire LLM life cycle. An experiment is performed with a large source code dataset to demonstrate the applicability of the proposed approach. The result shows higher accuracy of 91.8 % for the vulnerability detection and model explainability outcome is highly influenced by “vulnerable”, “function”, "mysql_tmpdir_list", “strmov” tokens using both SHAP and LIME framework. Heatmap of attention weights, highlights the local token interactions that aid in understanding the model's decision points.

GradToken: Decoupling tokens with class-aware gradient for visual explanation of Transformer network

Transformer networks have been widely used in the fields of computer vision, natural language processing, graph-structured data analysis, etc. Subsequently, explanations of Transformer play a key role in helping humans understand and analyze its decision-making and working mechanism, thereby improving the trustworthiness in its real-world applications. However, it is difficult to apply the existing explanation methods for convolutional neural networks to Transformer networks, due to the significant differences between their structures. How to design a specific and effective explanation method for Transformer poses a challenge in the explanation area. To address this challenge, we first analyze the semantic coupling problem of attention weight matrices in Transformer, which puts obstacles in providing distinctive explanations for different categories of targets. Then, we propose a gradient-decoupling-based token relevance method (i.e., GradToken) for the visual explanation of Transformer’s predictions. GradToken exploits the class-aware gradient to decouple the tangled semantics in the class token to the semantics corresponding to each category. GradToken further leverages the relations between the class token and spatial tokens to generate relevance maps. As a result, the visual explanation results generated by GradToken can effectively focus on the regions of selected targets. Extensive quantitative and qualitative experiments are conducted to verify the validity and reliability of the proposed method.

Correlation-Based Attention Weights Mechanism in Multimodal Depression Detection: A Two-Stage Approach

Correlation-Based Attention Weights Mechanism in Multimodal Depression Detection: A Two-Stage Approach

Review-Based Explainable Recommendations: A Transparency Perspective

With the prevalence of deep learning based embedding approaches, recommender systems have become a proven and indispensable tool in various information filtering applications. However, many of them remain difficult to diagnose what aspects of the deep models’ input drive the final ranking decision, thus, they cannot often be understood by human stakeholders. In this paper, we investigate the dilemma between recommendation and explainability, and show that by utilizing the contextual features (e.g., item reviews from users), we can design a series of explainable recommender systems without sacrificing their performance. In particular, we propose three types of explainable recommendation strategies with gradual changes in model transparency: whitebox, graybox, and blackbox. Each strategy explains its ranking decisions via different mechanisms: attention weights, adversarial perturbations, and counterfactual perturbations. We apply these explainable models on five real-world data sets under the contextualized setting where the users provided explicit reviews of the items. The empirical results show that our models can achieve highly competitive ranking performance, and meanwhile, generate accurate and effective explanations in terms of numerous quantitative metrics and qualitative visualizations.

Design of intelligent algorithm for object search based on IoT digital images

With the development of artificial intelligence, traditional object search and image recognition have been replaced by the Internet of Things and artificial intelligence. However, traditional object search algorithms often lack accuracy and low precision. Therefore, this study proposes a new intelligent encryption algorithm to address the issues of insufficient accuracy in object search algorithms and image recognition algorithms. The new algorithm ensures the security of user data and the response efficiency of the model during the conversation process by integrating fully homomorphic encryption technology and dynamic sparse attention mechanism. The dynamic sparse attention mechanism introduced simultaneously improves the model's ability to handle long sequence data by dynamically adjusting attention weights. Experimental results showed that the precision of the proposed algorithm was 0.05 % higher than that of random algorithms and 0.19 % higher than that of sorting algorithms. The recall rate of the proposed algorithm was 0.14 % higher than that of random algorithms and 0.16 % higher than that of sorting algorithms. The research algorithm can identify objects with certain characteristics and is suitable for specific environments, greatly reducing the probability of data leakage in object search and providing new ideas for research in this field.

Autism Identification Based on the Intelligent Analysis of Facial Behaviors: An Approach Combining Coarse- and Fine-Grained Analysis

Background: Facial behavior has emerged as a crucial biomarker for autism identification. However, heterogeneity among individuals with autism poses a significant obstacle to traditional feature extraction methods, which often lack the necessary discriminative power. While deep-learning methods hold promise, they are often criticized for their lack of interpretability. Methods: To address these challenges, we developed an innovative facial behavior characterization model that integrates coarse- and fine-grained analyses for intelligent autism identification. The coarse-grained analysis provides a holistic view by computing statistical measures related to facial behavior characteristics. In contrast, the fine-grained component uncovers subtle temporal fluctuations by employing a long short-term memory (LSTM) model to capture the temporal dynamics of head pose, facial expression intensity, and expression types. To fully harness the strengths of both analyses, we implemented a feature-level attention mechanism. This not only enhances the model’s interpretability but also provides valuable insights by highlighting the most influential features through attention weights. Results: Upon evaluation using three-fold cross-validation on a self-constructed autism dataset, our integrated approach achieved an average recognition accuracy of 88.74%, surpassing the standalone coarse-grained analysis by 8.49%. Conclusions: This experimental result underscores the improved generalizability of facial behavior features and effectively mitigates the complexities stemming from the pronounced intragroup variability of those with autism, thereby contributing to more accurate and interpretable autism identification.

Graph Neural Network-Based Speech Emotion Recognition: A Fusion of Skip Graph Convolutional Networks and Graph Attention Networks

In speech emotion recognition (SER), our research addresses the critical challenges of capturing and evaluating node information and their complex interrelationships within speech data. We introduce Skip Graph Convolutional and Graph Attention Network (SkipGCNGAT), an innovative model that combines the strengths of skip graph convolutional networks (SkipGCNs) and graph attention networks (GATs) to address these challenges. SkipGCN incorporates skip connections, enhancing the flow of information across the network and mitigating issues such as vanishing gradients, while also facilitating deeper representation learning. Meanwhile, the GAT in the model assigns dynamic attention weights to neighboring nodes, allowing SkipGCNGAT to focus on both the most relevant local and global interactions within the speech data. This enables the model to capture subtle and complex dependencies between speech segments, thus facilitating a more accurate interpretation of emotional content. It overcomes the limitations of previous single-layer graph models, which were unable to effectively represent these intricate relationships across time and in different speech contexts. Additionally, by introducing a pre-pooling SkipGCN combination technique, we further enhance the ability of the model to integrate multi-layer information before pooling, improving its capacity to capture both spatial and temporal features in speech. Furthermore, we rigorously evaluated SkipGCNGAT on the IEMOCAP and MSP-IMPROV datasets, two benchmark datasets in SER. The results demonstrated that SkipGCNGAT consistently achieved state-of-the-art performance. These findings highlight the effectiveness of the proposed model in accurately recognizing emotions in speech, offering valuable insights and a solid foundation for future research on capturing complex relationships within speech signals for emotion recognition.

A dual-branch convolutional neural network with domain-informed attention for arrhythmia classification of 12-lead electrocardiograms

The automatic classification of arrhythmia is an important task in the intelligent auxiliary diagnosis of an electrocardiogram. Its efficiency and accuracy are vital for practical deployment and applications in the medical field. For the 12-lead electrocardiogram, we know that the comprehensive utilization of lead characteristics is key to enhancing diagnostic accuracy. However, existing classification methods (1) neglect the similarities and differences between the limb lead group and the precordial lead group; (2) the commonly adopted attention mechanisms struggle to capture the domain characteristics in an electrocardiogram. To address these issues, we propose a new dual-branch convolutional neural network with domain-informed attention, which is novel in two ways. First, it adopts a dual-branch network to extract intra-group similarities and inter-group differences of limb and precordial leads. Second, it proposes a domain-informed attention mechanism to embed the critical domain knowledge of electrocardiogram, multiple RR (R wave to R wave) intervals, into coordinated attention to adaptively assign attention weights to key segments, thereby effectively capturing the characteristics of the electrocardiogram domain. Experimental results show that our method achieves an F1-score of 0.905 and a macro area under the curve of 0.936 on two widely used large-scale datasets, respectively. Compared to state-of-the-art methods, our method shows significant performance improvements with a drastic reduction in model parameters.

Utilizing graph neural networks for adverse health detection and personalized decision making in sensor-based remote monitoring for dementia care

Background:Sensor-based remote health monitoring is increasingly used to detect adverse health in people living with dementia (PLwD) at home, aiming to prevent hospitalizations and reduce caregiver burden. However, home sensor data is often noisy, overly granular, and suffers from unreliable labeling, data drift and high variability between households. Current anomaly detection methods lack generalizability and personalization, often requiring anomaly-free training data and frequent model updates. Objective:To develop a lightweight, explainable, self-supervised approach with personalized alert thresholds to detect adverse health events in PLwD, using changes in home activity. Methods:We hypothesized that health downturns manifest as detectable shifts in household movement patterns. Our approach leverages a Graph Barlow Twins contrastive model, which uses granular activity data and a macroscopic view to extract noise-robust, high-level and low-level discriminative features that represent daily activity patterns. Household-personalized alert thresholds are calculated based on clinician-set target alert rates, and daily anomaly scores are compared against these thresholds, triggering alerts for the clinical monitoring team. Model attention weights support explainability. Data were collected from a real-world dataset by the UK Dementia Research Institute (August 2019-April 2022). Results:Our model outperformed state-of-the-art temporal graph algorithms in detecting agitation and fall events across three patient cohorts, achieving 81% average recall and 88% generalizability at a target alert rate of 7%. Conclusion:We developed a novel, lightweight, explainable, and personalized Graph Barlow Twins model for real-world remote health monitoring in dementia care, with potential for broader applications in healthcare and sensor-based environments.

MPSA-Conformer-CTC/Attention: A High-Accuracy, Low-Complexity End-to-End Approach for Tibetan Speech Recognition.

This study addresses the challenges of low accuracy and high computational demands in Tibetan speech recognition by investigating the application of end-to-end networks. We propose a decoding strategy that integrates Connectionist Temporal Classification (CTC) and Attention mechanisms, capitalizing on the benefits of automatic alignment and attention weight extraction. The Conformer architecture is utilized as the encoder, leading to the development of the Conformer-CTC/Attention model. This model first extracts global features from the speech signal using the Conformer, followed by joint decoding of these features through CTC and Attention mechanisms. To mitigate convergence issues during training, particularly with longer input feature sequences, we introduce a Probabilistic Sparse Attention mechanism within the joint CTC/Attention framework. Additionally, we implement a maximum entropy optimization algorithm for CTC, effectively addressing challenges such as increased path counts, spike distributions, and local optima during training. We designate the proposed method as the MaxEnt-Optimized Probabilistic Sparse Attention Conformer-CTC/Attention Model (MPSA-Conformer-CTC/Attention). Experimental results indicate that our improved model achieves a word error rate reduction of 10.68% and 9.57% on self-constructed and open-source Tibetan datasets, respectively, compared to the baseline model. Furthermore, the enhanced model not only reduces memory consumption and training time but also improves generalization capability and accuracy.

An Enhanced Fault Localization Technique for Distribution Networks Utilizing Cost-Sensitive Graph Neural Networks

Accurate and timely fault diagnosis is of great significance for the stable operation of a distribution network. Traditional artificial intelligence-based localization methods rely heavily on large-scale labeled datasets, making them prone to being affected by distributed generators. To address this issue, this study proposes a fault location method for distribution systems using a cost-sensitive graph attention network model. In this approach, the physical structure of the distribution network is considered a crucial constraint for model training, thereby enhancing its information perception capabilities. Specifically, the electrical nodes and lines of the distribution network are mapped to the vertices and edges within the graph attention network, with the attention weights determined by the correlation of adjacent node fault characteristics, thus improving the sensitivity to grounding faults. Additionally, a cost-sensitive matrix is used to balance class distribution, enhancing the robustness and generalization ability of the model. Fault localization experiments were conducted on the IEEE-33 bus distribution system to validate the effectiveness of the proposed fault localization method. Factors such as data disturbance, varying fault grounding resistances, and distributed power supply access were employed to assess the model’s anti-interference performance. The experimental results demonstrate that the fault location method exhibits high positioning accuracy and excellent robustness.

Distributed Denial of Service Attack Detection and Prevention Using Pipit Fox-Attentional Deep Learning in Software-Defined Networking

The Software-Defined Network (SDN) remains the innovative model that assists in satisfying the new application requirements of future networks. However, destructive attacks, particularly Distributed Denial of Service (DDoS) attacks, are aimed primarily at the SDN control panel, presenting a significant risk to network security. The traditional approaches for DDoS attack detection exhibit several limitations including interpretability challenges, overfitting problems, and false predictions. To mitigate these drawbacks, the research proposed a Pipit Flying Fox Optimized Deep Neural Network (PPF-DNN) and Intelligent Pipit Forage Optimized-Attentional Convolutional Neural Network (IPFO-ACNN) model. The channel attention is employed to enable the network to focus selectively on relevant information, improving the model’s sensitivity to subtle anomalies associated with DDoS attacks. By dynamically adjusting the attention weights across channels, the mechanism enhances the capability of the model to discriminate the normal network traffic and malicious patterns, which results in improved precision and reduced false positives. This approach harnesses the power of CNN to automatically extract hierarchical features from network data, enhancing the efficacy of the ACNN model in attack detection. IPFO and PPF algorithms integrate the unique strengths of bio-inspired algorithms, creating a synergistic approach for efficient model parameter tuning. In the case of Training Percentage (TP) 80, the accuracy, sensitivity, and specificity of the IPFO-ACNN model are evaluated as 98.87%, 99.24% and 97.91%, respectively. Similarly, the PPF-DNN model’s performance is assessed as 98.49%, 92.50% and 98.55%, which presents a promising avenue for bolstering network security infrastructure, ensuring more robust DDoS detection and mitigation capabilities in an increasingly complex cyber threat landscape.

CosGCTFormer: An end-to-end driver state recognition framework

Driver state is critical for safety in mixed traffic environments with automated and human-driven vehicles. Recognizing and monitoring the driver’s state is essential to provide early warnings of abnormal driving performance. This paper proposes a driver state recognition framework called cosGCTFormer for end-to-end recognition of the driver’s emotional state and mental load. First, we employ an inverted residual convolution module to extract local features from physiological signal data. Second, the cosGCT module is introduced to extract global features. This module uses cos attention to address the high computational complexity of the Transformer in calculating attention weights. Additionally, it employs Gated Channel Transformation (GCT) to establish relationships between different channel dimensions, something the Transformer alone cannot do effectively. Finally, a simulated driving experiment was designed to collect data for training and testing the proposed method. Results demonstrate that our framework achieves higher accuracy and a more lightweight design compared to prevalent Convolution neural network-based (CNN-based) and Transformer-based methods. Additionally, the effectiveness of both modules was verified.

Novel state of charge estimation method of containerized Lithium–Ion battery energy storage system based on deep learning

State of charge (SOC) is a critical indicator for lithium–ion battery energy storage system. However, model-driven SOC estimation is challenging due to the coupling of internal charging and discharging processes, ion diffusion, and chemical reactions in the electrode materials. These factors result in frequent changes in their internal characteristics. Therefore, we devise a novel deep learning SOC estimation method based on the attention mechanism (A) and convolutional neural networks-long short term memory (CNN–LSTM) network model. The CNN layers are employed to extract the critical features of the battery dataset. The LSTM layers are utilized to capture the long-term dependencies in the time series data. The attention mechanism can assign attention weights to different features within the output of the convolution layer. The A–CNN–LSTM method not only amplifies the positive influence of critical knowledge but also provides a global reference. The comparative experiments of three datasets and ablation study are carried out. The root mean square error, mean absolute error, and mean absolute percentage error obtained from the A–CNN–LSTM model decreased by 32.75 %, 45.88 %, and 53.36 %, respectively, compared with the best results obtained from the existing CNN, LSTM, gate recurrent unit (GRU), CNN–LSTM, and CNN–GRU models. The proposed model demonstrates greater stability and consistency than other models, underscoring its efficacy in SOC estimation.