Attention Weights Research Articles

Multiscale regional calibration network for crowd counting

Crowd counting aims to estimate the number, density, and distribution of crowds in an image. While CNN-based crowd counting methods have been effective, head-scale variation and complex background remain two major challenges for crowd counting. Therefore, we propose a multiscale region calibration network called MRCNet to effectively address these challenges. To address the former challenge, we design a multiscale aware module that utilizes multi-branch dilated convolutional parallelism to obtain multiscale receptive fields and cope with drastic changes in head size. For the latter challenge, we design a regional calibration module that calibrates the attention weights of each region after obtaining the attention map to effectively handle challenges in complex contexts. Additionally, we improve the loss function by combining L2 loss and binary cross-entropy loss to help MRCNet achieve excellent results. Extensive experiments were conducted on three mainstream datasets to demonstrate the robustness and competitiveness of our approach.

Package Positioning Based on Point Registration Network DCDNet-Att

The application of robot technology in the automatic transportation process of packaging bags is becoming increasingly common. Point cloud registration is the key to applying industrial robots to automatic transportation systems. However, current point cloud registration models cannot effectively solve the registration of deformed targets like packaging bags. In this study, a new point cloud registration network, DCDNet-Att, is proposed, which uses a variable weight dynamic graph convolution module to extract point cloud features. A feature interaction module is used to extract common features between the source point cloud and the template point cloud. The same geometric features between the two pairs of point clouds are strengthened through a bottleneck module. A channel attention model is used to obtain the channel attention weights. The attention weight of each spatial position is calculated, and a rotation translation structure is used to sequentially obtain quaternions and translation vectors. A feature fitting loss function is used to constrain the parameters of the neural network model to have a larger receptive field. Compared with seven methods, including the ICP algorithm, GO-ICP algorithm, and FGR algorithm, the proposed method had rotation errors (MAE, RMSE, and Error of 1.458, 2.541, and 1.024 in the ModelNet40 dataset, respectively) and translation errors (MAE, RMSE, and Error of 0.0048, 0.0114, and 0.0174, respectively). When registering the ModelNet40 dataset with Gaussian noise, the rotation errors (MAE, RMSE, and Error) were 2.028, 3.437, and 2.478, respectively, and the translation errors (MAE, RMSE, and Error) were 0.0107, 0.0327, and 0.0285, respectively. The experimental results were superior to those of the other methods, and the model was effective at registering packaging bag point clouds.

DFCNformer: A Transformer Framework for Non-Stationary Time-Series Forecasting Based on De-Stationary Fourier and Coefficient Network

Time-series data are widely applied in real-world scenarios, but the non-stationary nature of their statistical properties and joint distributions over time poses challenges for existing forecasting models. To tackle this challenge, this paper introduces a forecasting model called DFCNformer (De-stationary Fourier and Coefficient Network Transformer), designed to mitigate accuracy degradation caused by non-stationarity in time-series data. The model initially employs a stabilization strategy to unify the statistical characteristics of the input time series, restoring their original features at the output to enhance predictability. Then, a time-series decomposition method splits the data into seasonal and trend components. For the seasonal component, a Transformer-based encoder–decoder architecture with De-stationary Fourier Attention (DSF Attention) captures temporal features, using differentiable attention weights to restore non-stationary information. For the trend component, a multilayer perceptron (MLP) is used for prediction, enhanced by a Dual Coefficient Network (Dual-CONET) that mitigates distributional shifts through learnable distribution coefficients. Ultimately, the forecasts of the seasonal and trend components are combined to generate the overall prediction. Experimental findings reveal that when the proposed model is tested on six public datasets, in comparison with five classic models it reduces the MSE by an average of 9.67%, with a maximum improvement of 40.23%.

In-Vehicle Network Anomaly Detection Based on a Graph Attention Network

<div>The increased connectivity of vehicles expands the attack surface of in-vehicle networks, enabling attackers to infiltrate through external interfaces and inject malicious traffic. These malicious flows often contain anomalous semantic information, potentially leading to misleading control instructions or erroneous decisions. While most semantic-based anomaly detection methods for in-vehicle networks focus on extracting semantic context, they often overlook interactions and associations between multiple semantics, resulting in a high false positive rate (FPR). To address these challenges, the Adaptive Structure Graph Attention Network Model (AS-GAT) is proposed for in-vehicle network anomaly detection. Our approach combines a semantic extractor with a continuously updated graph structure learning method based on attention weight similarity constraints. The semantic extractor identifies semantic features within messages, while the graph structure learning module adaptively updates the graph structure based on attention weights between semantics. This model effectively learns relationships between multiple semantics in in-vehicle network packets, thereby enhancing anomaly detection accuracy. A case study on a CAN-FD dataset from real vehicles demonstrates that using AS-GAT achieves an F1 score of 97.56% in anomaly detection, outperforming baseline methods by effectively identifying attack packets causing abnormal semantic time series changes, such as fuzzing, spoofing, and replay attacks. Additional experiments on two public datasets, SWaT and WADI, further validate AS-GAT’s superior anomaly detection performance compared to baseline models, highlighting the universal applicability of our approach.</div>

Regression study on fruit-setting days of purple eggplant fruit based on in situ VIS-NIRS and attention cycle neural network.

In the intelligent harvesting of eggplant, the lack of in situ identification technology makes it challenging to determine the maturity of purple eggplant fruit. The length of the fruit-setting date can determine when the eggplant is ready to be harvested. This study uses deep learning techniques to predict the date of fruit maturity. First, we proposed a fruit-setting days prediction method based on fruit spectroscopy and neural networks. Second, we collected the field in situ spectral data of purple eggplant fruit during 15-33 days of fruit setting using a portable spectrometer, covering 500-1000nm. A fruit-setting time regression network combining multi-scale convolution, multi-head attention mechanism, and long short-term memory recurrent neural network was constructed using the collected in situ spectral data. The model demonstrated better fitting performance than traditional machine learning models such as backpropagation neural network, random forest, support vector machine, and partial least squares regression in the regression task of fruit-setting days. After testing various spectral preprocessing methods, the best fitting effect was found on the standard normal variate-processed dataset, with R2 of 0.876 and RMSE(root mean square error) of 2.148 days. Furthermore, the feasibility of each model module was analyzed in depth through ablation experiments, confirming each component's role in improving the model's performance. The network attention weight was also analyzed, and the model has strong detail mining ability in a specific spectral interval. In summary, the combination of visible and near-infrared spectroscopy and attention cycle neural network is an effective method to predict the fruit-setting days of purple eggplant fruit. PRACTICAL APPLICATION: A prediction method of fruit-setting days based on fruit spectral characteristics and recurrent neural network regression was proposed. A novel approach to detecting and disclosing in situ surface VIS-NIRS reflectance data of eggplant fruit during ripening is presented for the first time. A set of long-term and short-term memory networks based on multi-scale convolution and multi-head attention mechanisms was constructed for spectral data fitting. Through the ablation test method and attention weight analysis, the function of each module in the network and the interpretability of feature extraction are explored.

Deep learning for NAD/NADP cofactor prediction and engineering using transformer attention analysis in enzymes

Understanding and manipulating the cofactor preferences of NAD(P)-dependent oxidoreductases, the most widely distributed enzyme group in nature, is increasingly crucial in bioengineering. However, large-scale identification of the cofactor preferences and the design of mutants to switch cofactor specificity remain as complex tasks. Here, we introduce DISCODE (Deep learning-based iterative pipeline to analyze Specificity of Cofactors and to Design Enzyme), a novel transformer-based deep learning model to predict NAD(P) cofactor preferences. For model training, a total of 7,132 NAD(P)-dependent enzyme sequences were collected. Leveraging whole-length sequence information, DISCODE classifies the cofactor preferences of NAD(P)-dependent oxidoreductase protein sequences without structural or taxonomic limitation. The model showed 97.4% and 97.3% of accuracy and F1 score, respectively. A notable feature of DISCODE is the interpretability of its transformer layers. Analysis of attention layers in the model enables identification of several residues that showed significantly higher attention weights. They were well aligned with structurally important residues that closely interact with NAD(P), facilitating the identification of key residues for determining cofactor specificities. These key residues showed high consistency with verified cofactor switching mutants. Integrated into an enzyme design pipeline, DISCODE coupled with attention analysis, enables a fully automated approach to redesign cofactor specificity.

Human intention recognition for trauma resuscitation: An interpretable deep learning approach for medical process data.

Trauma resuscitation is the initial evaluation and management of injured patients in the emergency department. This time-critical process requires the simultaneous pursuit of multiple resuscitation goals. Recognizing whether the required goal is being pursued can reduce errors in goal-related task performance and improve patient outcomes. The intention to pursue a goal can often be inferred from ongoing and completed treatment activities, but monitoring goal pursuit is cognitively demanding and prone to errors. We introduced an interpretable deep learning-based approach to aid decision making by automatically recognizing goal pursuit during trauma resuscitation. We developed a predictive model to recognize the pursuit of two resuscitation goals: airway stabilization and circulatory support. We used event logs of 381 pediatric trauma resuscitations from August 2014 to November 2022 to train a neural network model with a dual-GRU structure that learns from both time-level and activity-type-level features. Our model makes predictions based on a sequence of activities and corresponding timestamps. To enhance the model and facilitate interpretation of predictions, we used the attention weights assigned by our model to represent the importance of features. These weights identified the critical time points and contributing activities during a goal pursuit. Our model achieved an average area under the receiver operating characteristic curve (AUC) score of 0.84 for recognizing airway stabilization and 0.83 for recognizing circulatory support. The most contributing activities and timestamps were aligned with domain knowledge. Our interpretable predictive model can recognize provider intention based on a limited number of treatment activities. The model outperformed existing predictive models for medical events in accuracy and in interpretability. Integrating our model into a decision-support system would automate the tracking of provider actions, optimizing workflow to ensure timely delivery of care.

Adaptive Neural Message Passing for Inductive Learning on Hypergraphs.

Graphs are the most ubiquitous data structures for representing relational datasets and performing inferences in them. They model, however, only pairwise relations between nodes and are not designed for encoding the higher-order relations. This drawback is mitigated by hypergraphs, in which an edge can connect an arbitrary number of nodes. Most hypergraph learning approaches convert the hypergraph structure to that of a graph and then deploy existing geometric deep learning methods. This transformation leads to information loss, and sub-optimal exploitation of the hypergraph's expressive power. We present HyperMSG, a novel hypergraph learning framework that uses a modular two-level neural message passing strategy to accurately and efficiently propagate information within each hyperedge and across the hyperedges. HyperMSG adapts to the data and task by learning an attention weight associated with each node's degree centrality. Such a mechanism quantifies both local and global importance of a node, capturing the structural properties of a hypergraph. HyperMSG is inductive, allowing inference on previously unseen nodes. Further, it is robust and outperforms state-of-the-art hypergraph learning methods on a wide range of tasks and datasets. Finally, we demonstrate the effectiveness of HyperMSG in learning multimodal relations through detailed experimentation on a challenging multimedia dataset.

MITIGATING POISONING ATTACKS TO FEDERATED LEARNING IN IOTs ANOMALY DETECTION WITH ATTENTION AGGREGATION

Federated Learning (FL) is a privacy-preserving approach for training deep neural networks across decentralized devices without sharing raw data. Thus, FL has been popularly applied in domains like anomaly detection in Internet of Things (IoTs). However, IoT networks/ devices have limited protection capabilities, resulting in the vulnerability of FL to data poisoning attacks. In order to address this challenge, we propose a new robust FL system designed to counter data poisoning attacks. Our approach, named as Federated Learning with Attention Aggregation (FedAA), leverages AutoEncoder (AE) models for local anomaly detection in IoT networks. In FedAA, we design to aggregate the global model from local models using a novel aggregation method, named as Attention Aggregation (AA). This method is specifically designed to mitigate the impact of data poisoning attacks, which often lead to high values of the loss functions in the local models. More precisely, the local models with high loss values are assigned lower attention weights when contributing to the global model aggregation, and vice versa. As a result, the proposed AA method enhances the robustness of FedAA against data poisoning attacks. We have conducted extensive experiments on three datasets, i.e., N-BaIoT, NSL-KDD, and UNSW, of IoT anomaly detection. The results show that FedAA is more robust than other FL systems in mitigating data poisoning attacks.

Top-down instructions influence the attentional weight on color and shape dimensions during bidimensional search

Efficient searches are guided by target-distractor distinctiveness: the greater the distinctiveness, the faster the search. Previous research showed that when the target and distractors differ along both color and shape dimensions (i.e., bidimensional search), distinctiveness along individual dimensions combine collinearly to guide the search, following a city-block metric. This result was found when participants expected the target and distractors to differ along both dimensions. In the present study, we used an instruction manipulation to investigate how bidimensional search varies in response to different top-down instructions. Using unidimensional search performance observed in Experiment 1, we predicted bidimensional search performance under three conditions: when participants were instructed to attend to color (Experiment 2), shape (Experiment 3), or both (Experiment 4). Results showed that instructions influenced how distinctiveness along color and shape combine to guide attention: when instructed to search for a target color, participants allocated more attentional weight to the color dimension (and less weight to the shape dimension) compared to when instructed to search for a target shape. Our study presents a novel technique to quantify how top-down instructions change attentional weighting to different features during bidimensional visual searches.

Bidirectional interaction directional variance attention model based on increased-transformer for thyroid nodule classification

Malignant thyroid nodules are closely linked to cancer, making the precise classification of thyroid nodules into benign and malignant categories highly significant. However, the subtle differences in contour between benign and malignant thyroid nodules, combined with the texture features obscured by the inherent noise in ultrasound images, often result in low classification accuracy in most models. To address this, we propose a Bidirectional Interaction Directional Variance Attention Model based on Increased-Transformer, named IFormer-DVNet. This paper proposes the Increased-Transformer, which enables global feature modeling of feature maps extracted by the Convolutional Feature Extraction Module (CFEM). This design maximally alleviates noise interference in ultrasound images. The Bidirectional Interaction Directional Variance Attention module (BIDVA) dynamically calculates attention weights using the variance of input tensors along both vertical and horizontal directions. This allows the model to focus more effectively on regions with rich information in the image. The vertical and horizontal features are interactively combined to enhance the model's representational capability. During the model training process, we designed a Multi-Dimensional Loss function (MD Loss) to stretch the boundary distance between different classes and reduce the distance between samples of the same class. Additionally, the MD Loss function helps mitigate issues related to class imbalance in the dataset. We evaluated our network model using the public TNCD dataset and a private dataset. The results show that our network achieved an accuracy of 76.55% on the TNCD dataset and 93.02% on the private dataset. Compared to other state-of-the-art classification networks, our model outperformed them across all evaluation metrics.

Deep learning-based metabolomics data study of prostate cancer

As a heterogeneous disease, prostate cancer (PCa) exhibits diverse clinical and biological features, which pose significant challenges for early diagnosis and treatment. Metabolomics offers promising new approaches for early diagnosis, treatment, and prognosis of PCa. However, metabolomics data are characterized by high dimensionality, noise, variability, and small sample sizes, presenting substantial challenges for classification. Despite the wide range of applications of deep learning methods, the use of deep learning in metabolomics research has not been extensively explored. In this study, we propose a hybrid model, TransConvNet, which combines transformer and convolutional neural networks for the classification of prostate cancer metabolomics data. We introduce a 1D convolution layer for the inputs to the dot-product attention mechanism, enabling the interaction of both local and global information. Additionally, a gating mechanism is incorporated to dynamically adjust the attention weights. The features extracted by multi-head attention are further refined through 1D convolution, and a residual network is introduced to alleviate the gradient vanishing problem in the convolutional layers. We conducted comparative experiments with seven other machine learning algorithms. Through five-fold cross-validation, TransConvNet achieved an accuracy of 81.03% and an AUC of 0.89, significantly outperforming the other algorithms. Additionally, we validated TransConvNet's generalization ability through experiments on the lung cancer dataset, with the results demonstrating its robustness and adaptability to different metabolomics datasets. We also proposed the MI-RF (Mutual Information-based random forest) model, which effectively identified key biomarkers associated with prostate cancer by leveraging comprehensive feature weight coefficients. In contrast, traditional methods identified only a limited number of biomarkers. In summary, these results highlight the potential of TransConvNet and MI-RF in both classification tasks and biomarker discovery, providing valuable insights for the clinical application of prostate cancer diagnosis.

Single visual model based on transformer for digital instrument reading recognition

Abstract Digital instrument reading recognition (DIRR) technology is crucial for industrial digital transformation and the advancement of industrialisation. However, digital instruments differ in character fonts, styles, spacing, and aspect ratios, as well as the scarcity of data pose significant challenges to current recognition technologies. To address these challenges, this study proposed a novel single visual model based on transformer for digital instrument recognition (SVDIR). The SVDIR model primarily comprised a scaled cosine attention mechanism (SC-attention) and a local Transformer block. First, the SC-attention was designed to calculate the cosine similarity of two image patches. It rendered the attention calculation independent of the input amplitude and produced milder attention weights to alleviate overconcentration issues. Second, a local Transformer block module was proposed for extracting the internal stroke features and dependencies between character components. Fine-grained characteristic features were obtained using this method. In addition, a post-norm structure was introduced into the local Transformer block module to reduce the accumulation of activation values following the deepening of the network. Finally, experimental results demonstrated the effectiveness and superiority of the proposed model on two digital instrument datasets.

Probabilistic Attention Map: A Probabilistic Attention Mechanism for Convolutional Neural Networks.

The attention mechanism is essential to convolutional neural network (CNN) vision backbones used for sensing and imaging systems. Conventional attention modules are designed heuristically, relying heavily on empirical tuning. To tackle the challenge of designing attention mechanisms, this paper proposes a novel probabilistic attention mechanism. The key idea is to estimate the probabilistic distribution of activation maps within CNNs and construct probabilistic attention maps based on the correlation between attention weights and the estimated probabilistic distribution. The proposed approach consists of two main components: (i) the calculation of the probabilistic attention map and (ii) its integration into existing CNN architectures. In the first stage, the activation values generated at each CNN layer are modeled by using a Laplace distribution, which assigns probability values to each activation, representing its relative importance. Next, the probabilistic attention map is applied to the feature maps via element-wise multiplication and is seamlessly integrated as a plug-and-play module into existing CNN architectures. The experimental results show that the proposed probabilistic attention mechanism effectively boosts image classification accuracy performance across various CNN backbone models, outperforming both baseline and other attention mechanisms.

GLMAFuse: A Dual-Stream Infrared and Visible Image Fusion Framework Integrating Local and Global Features with Multi-Scale Attention

Integrating infrared and visible-light images facilitates a more comprehensive understanding of scenes by amalgamating dual-sensor data derived from identical environments. Traditional CNN-based fusion techniques are predominantly confined to local feature emphasis due to their inherently limited receptive fields. Conversely, Transformer-based models tend to prioritize global information, which can lead to a deficiency in feature diversity and detail retention. Furthermore, methods reliant on single-scale feature extraction are inadequate for capturing extensive scene information. To address these limitations, this study presents GLMAFuse, an innovative dual-stream encoder–decoder network, which utilizes a multi-scale attention mechanism to harmoniously integrate global and local features. This framework is designed to maximize the extraction of multi-scale features from source images while effectively synthesizing local and global information across all layers. We introduce the global-aware and local embedding (GALE) module to adeptly capture and merge global structural attributes and localized details from infrared and visible imagery via a parallel dual-branch architecture. Additionally, the multi-scale attention fusion (MSAF) module is engineered to optimize attention weights at the channel level, facilitating an enhanced synergy between high-frequency edge details and global backgrounds. This promotes effective interaction and fusion of dual-modal features. Extensive evaluations using standard datasets demonstrate that GLMAFuse surpasses the existing leading methods in both qualitative and quantitative assessments, highlighting its superior capability in infrared and visible image fusion. On the TNO and MSRS datasets, our method achieves outstanding performance across multiple metrics, including EN (7.15, 6.75), SD (46.72, 47.55), SF (12.79, 12.56), MI (2.21, 3.22), SCD (1.75, 1.80), VIF (0.79, 1.08), Qbaf (0.58, 0.71), and SSIM (0.99, 1.00). These results underscore its exceptional proficiency in infrared and visible image fusion.

Contributions of attention to learning in multidimensional reward environments.

Real-world choice options have many features or attributes, whereas the reward outcome from those options only depends on a few features or attributes. It has been shown that humans learn and combine feature-based with more complex conjunction-based learning to tackle challenges of learning in naturalistic reward environments. However, it remains unclear how different learning strategies interact to determine what features or conjunctions should be attended to and control choice behavior, and how subsequent attentional modulations influence future learning and choice. To address these questions, we examined the behavior of male and female human participants during a three-dimensional learning task in which reward outcomes for different stimuli could be predicted based on a combination of an informative feature and conjunction. Using multiple approaches, we found that both choice behavior and reward probabilities estimated by participants were most accurately described by attention-modulated models that learned the predictive values of both the informative feature and the informative conjunction. Specifically, in the reinforcement learning model that best fit choice data, attention was controlled by the difference in the integrated feature and conjunction values. The resulting attention weights modulated learning by increasing the learning rate on attended features and conjunctions. Critically, modulating decision making by attention weights did not improve the fit of data, providing little evidence for direct attentional effects on choice. These results suggest that in multidimensional environments, humans direct their attention not only to selectively process reward-predictive attributes, but also to find parsimonious representations of the reward contingencies for more efficient learning.Significance Statement From trying exotic recipes to befriending new social groups, outcomes of real-life actions depend on many factors, but how do we learn the predictive values of those factors based on feedback we receive? It has been shown that humans simplify this problem by focusing on individual features that are most predictive of the outcomes but can extend their learning strategy to include combinations of features when necessary. Here, we examined interaction between attention and learning in a multidimensional reward environment that requires learning about individual features and their conjunctions. Using multiple approaches, we found that learning about features and conjunctions control attention in a cooperative manner and that the ensuing attentional modulations mainly affects future learning and not decision making.

A novel cross domain diagnosis method based on physical feature weighting and deep residual shrinkage network

Abstract Traditional intelligent diagnostic methods often exhibit poor generalization ability when faced with data from different experimental platforms due to their differing distributions. Current transfer learning methods typically require fine-tuning of the model with some target domain data to adapt to different data distributions. However, in practical applications, it is often difficult to obtain sufficient target domain data. To address this issue, this paper proposes an innovative cross-domain diagnostic method that can diagnose faults on a target platform without relying on its data. The method introduces fault information of bearings into the attention weights of a Deep Residual Shrinkage Network (DRSN) through a novel physical feature weighting layer. This effectively extracts the hidden fault information from the signals, which is further refined and classified by DRSN to obtain an accurate fault diagnosis result. Experiments conducted on three bearing datasets validate the effectiveness of the proposed method, demonstrating its promising application prospects for bearing fault diagnosis under different operating conditions.

Detection of Defective Apples Using Learnable Residual Multi-Head Attention Networks Integrated with CNNs

Many traditional fruit vendors still rely on manual sorting to pick out high-quality apples. This process is not only time-consuming but can also damage the apples. Meanwhile, automated detection technology is still in its early stage and lacks full reliability. To improve this technology, we propose a novel method, which incorporates a learnable scaling factor and residual connection to enhance the Multi-Head Attention mechanism. In our approach, a learnable scaling factor is first applied to adjust the attention weights dynamically, and then a residual connection combines the scaled attention output with the original input to preserve essential features from the initial data. By integrating Multi-Head Attention with Convolutional Neural Networks (CNNs) using this method, we propose a lightweight deep learning model called “Learnable Residual Multi-Head Attention Networks Fusion with CNNs” to detect defective apples. Compared to existing models, our proposed model has lower memory usage, shorter training time, and higher detection precision. On the test set, the model achieves an accuracy of 97.5%, a recall of 98%, and a specificity of 97%, along with the lowest detection time of 46 ms. Experimental results show that the proposed model using our method is highly promising for commercial sorting, as it reduces labor costs, increases the supply of high-quality apples, and boosts consumer satisfaction.

Mean pulmonary artery pressure prediction with explainable multi-view cardiac MR cine series deep learning model

BackgroundPulmonary hypertension (PH) is a heterogeneous condition and regardless of aetiology impacts negatively on survival. Diagnosis of PH is based on hemodynamic parameters measured invasively at right heart catheterization (RHC), however, a non-invasive alternative would be clinically valuable. Our aim was to estimate RHC parameters non-invasively from cardiac MR data using deep learning models and to identify key contributing imaging features. MethodsWe constructed an explainable convolutional neural network (CNN) taking cardiac MR cine series from 4 different views as input to predict mean pulmonary artery pressure (mPAP). The model was trained and evaluated on 1646 examinations. The model’s attention weight and predictive performance associated with each frame, view, or phase was used to judge its importance. Additionally, the importance of each cardiac chamber was inferred by perturbing part of the input pixels. ResultsThe model achieved a Pearson Correlation Coefficient (PCC) of 0.80 and R2 of 0.64 in predicting mPAP, and identified the right ventricle (RV) region on short-axis (SAX) view to be especially informative. ConclusionsHemodynamic parameters can be estimated non-invasively with a CNN, using MR cine series from 4 views, revealing key contributing features at the same time.

A novel spatio-temporal attention mechanism model for car-following in autonomous driving

Car following is critical for the overall safety, efficiency, and smooth operation of autonomous vehicles in traffic. However, existing car-following model primarily focuses on local feature extraction, overlooking the spatial–temporal relationships between vehicles and key information within the time series. This limitation negatively impacts prediction accuracy and generalization capabilities. To address these issues, a novel Spatio-Temporal Car-Following model based on Graph Convolution Network and Attention mechanism is proposed to enhance the safety, efficiency, and comfort of autonomous driving systems. Firstly, the spatial–temporal structure of the car-following model is constructed using the GCN network. This approach efficiently captures the topological relationships between vehicles. Secondly, we interleave spatial and temporal blocks to extract feature information from both spatial and temporal dimensions, which allows us to perceive spatial interactions and temporal motion patterns of the car. Additionally, a self-attention module assigns different attention weights to the various neighbors of a node in the car-following model, which can facilitate the capture of diverse aspects of node relationships and enhance expressive power. Finally, a Multi-Layer Perceptron (MLP) layer predicts the future behaviors of following vehicles. The proposed car-following model (CF) was trained and evaluated using five real-world datasets: HighD, SPMD, Waymo, NGSIM, and Lyft. The results indicate that our approach surpasses current models in terms of Mean Squared Error (MSE) for spacing, while also maintaining a zero collision rate across every dataset. These findings indicate that the Spatio-Temporal Attention Model (GSTAM-CF) proposed in this paper enhances safety, efficiency, and operational comfort compared to existing models.