Encoder Module Research Articles

Enhancing brain tumor MRI classification with an ensemble of deep learning models and transformer integration

Brain tumors are widely recognized as the primary cause of cancer-related mortality globally, necessitating precise detection to enhance patient survival rates. The early identification of brain tumor is presented with significant challenges in the healthcare domain, necessitating the implementation of precise and efficient diagnostic methodologies. The manual identification and analysis of extensive MRI data are presented as a challenging and laborious task, compounded by the importance of early tumor detection in reducing mortality rates. Prompt initiation of treatment hinges upon identifying the specific tumor type in patients, emphasizing the urgency for a dependable deep learning methodology for precise diagnosis. In this research, a hybrid model is presented which integrates the strengths of both transfer learning and the transformer encoder mechanism. After the performance evaluation of the efficacy of six pre-existing deep learning model, both individually and in combination, it was determined that an ensemble of three pretrained models achieved the highest accuracy. This ensemble, comprising DenseNet201, GoogleNet (InceptionV3), and InceptionResNetV2, is selected as the feature extraction framework for the transformer encoder network. The transformer encoder module integrates a Shifted Window-based Self-Attention mechanism, sequential Self-Attention, with a multilayer perceptron layer (MLP). These experiments were conducted on three publicly available research datasets for evaluation purposes. The Cheng dataset, BT-large-2c, and BT-large-4c dataset, each designed for various classification tasks with differences in sample number, planes, and contrast. The model gives consistent results on all three datasets and reaches an accuracy of 99.34%, 99.16%, and 98.62%, respectively, which are improved compared to other techniques.

A proton exchange membrane fuel cells degradation prediction method based on multi-scale temporal information merging network

Performance degradation prediction is considered as an effective method to improve the service life of proton exchange membrane fuel cells (PEMFCs). A hybrid method Multi-scale Temporal Information Merging Network (MTIMN) is then proposed for PEMFC degradation prediction, which is designed to merge information from both macro and micro perspectives in PEMFC historical working data. Firstly, a data preprocessing method based on P-M scores and min-max wavelet denoising is designed to eliminate redundant variables and noise considering the structure of PEMFC and data statistics. Secondly, aiming at the voltage recovery phenomenon during the aging process, a Multi-scale Exponential Decomposition Encoding Module (MEDEM) is presented to extract and encode different time scales of sequences from the PEMFC degradation data. Thirdly, a two-layer Historical Information Integration Module (HIIM) is established, which consists of the trend extraction module and the local extraction module, to extract deep dynamic nonlinear characteristics. Finally, the Merging Prediction Module (MPM) merges information from trend and local modules for the final output. The experimental results show that Root Mean Square Error (RMSE) and Mean Absolute Error (MAE) are 0.0107 and 0.0079, respectively. Therefore, MTIMN can effectively extract deep dynamic nonlinear characteristics in PEMFC degradation data and improve prediction accuracy greatly.

Multi-Scale Adaptive Skeleton Transformer for action recognition

Transformer has demonstrated remarkable performance in various computer vision tasks. However, its potential is not fully explored in skeleton-based action recognition. On one hand, existing methods primarily utilize fixed function or pre-learned matrix to encode position information, while overlooking the sample-specific position information. On the other hand, these approaches focus on single-scale spatial relationships, while neglecting the discriminative fine-grained and coarse-grained spatial features. To address these issues, we propose a Multi-Scale Adaptive Skeleton Transformer (MSAST), including Adaptive Skeleton Position Encoding Module (ASPEM), Multi-Scale Embedding Module (MSEM), and Adaptive Relative Location Module (ARLM). ASPEM decouples spatial–temporal information in the position encoding procedure, which acquires inherent dependencies of skeleton sequences. ASPEM is also designed to be dependent on input tokens, which can learn sample-specific position information. The MSEM employs multi-scale pooling to generate multi-scale tokens that contain multi-grained features. Then, the spatial transformer captures multi-scale relations to address the subtle differences between various actions. Another contribution of this paper is that ARLM is presented to mine suitable location information for better recognition performance. Extensive experiments conducted on three benchmark datasets demonstrate that the proposed model achieves Top-1 accuracy of 94.9%/97.5% on NTU-60 C-Sub/C-View, 88.7%/91.6% on NTU-120 X-Sub/X-Set and 97.4% on NW-UCLA, respectively.

FRRL: A reinforcement learning approach for link failure recovery in a hybrid SDN

Network failures, especially link failures, happen frequently in Internet Service Provider (ISP) networks. When link failures occur, the routing policies need to be re-computed and failure recovery usually takes a few minutes, which degrades the network performance to a great extent. Therefore, a proper failure recovery scheme that can realize a fast and timely routing policy computation needs to be designed. In this paper, we propose FRRL, a Reinforcement Learning (RL) approach to intelligently perceive network failures and timely compute the routing policy for improving the network performance when link failure happens. Specifically, to perceive the link failures, we design a Topology Difference Vector (TDV) encoder module in FRRL for encoding the topology structure with link failures. To efficiently compute the routing policy when link failures happen, we integrate the TDV in the agent training for learning the map between the encoded failure topology structure and routing policies. To evaluate the performance of our proposed method, we conduct experiments on three network topologies and the experimental results demonstrate that our proposed method has superior performance when link failures happen compared to other methods.

DPD (DePression Detection) Net: a deep neural network for multimodal depression detection.

Depression is one of the most prevalent mental conditions which could impair people's productivity and lead to severe consequences. The diagnosis of this disease is complex as it often relies on a physician's subjective interview-based screening. The aim of our work is to propose deep learning models for automatic depression detection by using different data modalities, which could assist in the diagnosis of depression. Current works on automatic depression detection mostly are tested on a single dataset, which might lack robustness, flexibility and scalability. To alleviate this problem, we design a novel Graph Neural Network-enhanced Transformer model named DePressionDetect Net (DPD Net) that leverages textual, audio and visual features and can work under two different application settings: the clinical setting and the social media setting. The model consists of a unimodal encoder module for encoding single modality, a multimodal encoder module for integrating the multimodal information, and a detection module for producing the final prediction. We also propose a model named DePressionDetect-with-EEG Net (DPD-E Net) to incorporate Electroencephalography (EEG) signals and speech data for depression detection. Experiments across four benchmark datasets show that DPD Net and DPD-E Net can outperform the state-of-the-art models on three datasets (i.e., E-DAIC dataset, Twitter depression dataset and MODMA dataset), and achieve competitive performance on the fourth one (i.e., D-vlog dataset). Ablation studies demonstrate the advantages of the proposed modules and the effectiveness of combining diverse modalities for automatic depression detection.

LSTGCN: Inductive Spatial Temporal Imputation Using Long Short-Term Dependencies

Spatial temporal forecasting of urban sensors is essentially important for many urban systems, such as intelligent transportation and smart cities. However, due to the problem of hardware failure or network failure, there are some missing values or missing monitoring sensors that need to be interpolated. Recent research on deep learning has made substantial progress on imputation problem, especially temporal aspect (i.e., time series imputation), while little attention has been paid to spatial aspect (both dynamic and static) and long-term temporal dependencies. In this article, we proposed a spatial temporal imputation model, named Long Short-Term Graph Convolution Networks (LSTGCN), which includes gated temporal extraction (GTE) module, multi-head attention-based temporal capture (MHAT) module, long-term periodic temporal encoding (LPTE) module, and bidirectional spatial graph convolution (BSGC) module. The GTE adopts a gated mechanism to filter short-term temporal information, while the MHAT utilizes position encoding to enhance the difference of each timestamps, then use multi-head attention to capture short-term temporal dependency. The BSGC is adopted to handle with spatial relationships between sensor nodes. And we design a periodic encoding technique to process long-term temporal dependencies. The BSGC handles spatial relationships between sensor nodes, and a periodic encoding technique is used to process long-term temporal dependencies. Our experimental analysis includes completion and forecasting tasks, as well as transfer and ablation analyses. The results show that our proposed model outperforms state-of-the-art baselines on real-world datasets.

Astrocytic modulation of population encoding in mouse visual cortex via GABA transporter 3 revealed by multiplexed CRISPR/Cas9 gene editing.

Astrocytes, which are increasingly recognized as pivotal constituents of brain circuits governing a wide range of functions, express GABA transporter 3 (Gat3), an astrocyte-specific GABA transporter responsible for maintenance of extra-synaptic GABA levels. Here, we examined the functional role of Gat3 in astrocyte-mediated modulation of neuronal activity and information encoding. First, we developed a multiplexed CRISPR construct applicable for effective genetic ablation of Gat3 in the visual cortex of adult mice. Using in vivo two-photon calcium imaging of visual cortex neurons in Gat3 knockout mice, we observed changes in spontaneous and visually driven single neuronal response properties such as response magnitudes and trial-to-trial variability. Gat3 knockout exerted a pronounced influence on population-level neuronal activity, altering the response dynamics of neuronal populations and impairing their ability to accurately represent stimulus information. These findings demonstrate that Gat3 in astrocytes profoundly shapes the sensory information encoding capacity of neurons and networks within the visual cortex.

A Dual-Stream Fusion Network for Human Energy Expenditure Estimation with Wearable Sensor

With the increasing awareness of health, using wearable sensors to monitor individual activities and accurately estimate energy expenditure has become a current research focus. However, existing research encounters challenges including low estimation accuracy, a deficiency of frequency domain features, and difficulty in integrating time domain and frequency domain features. To address these issues, we propose an innovative framework called the Dual-Stream Fusion Network (DSFN). This framework combines the Time Domain Encoding (TDE) module, the Frequency Domain Hierarchical-Split Encoding (FDHSE) module, and a Two-Stage Feature Fusion (TSF) module. Specifically, the temporal stream of the framework employs the TDE module to capture deep temporal features that reflect the complex dynamic variations in time-series data. The frequency domain stream introduces the FDHSE module, which extracts frequency domain features using a multi-level, multi-scale approach, ensuring a comprehensive and diverse representation of frequency information. Through this dual-stream architecture, our model effectively learns both time and frequency domain features, addressing the limitations of frequency domain features observed in prior studies. Additionally, we propose the TSF module to fully integrate time and frequency domain features, effectively overcoming the challenge of fusing these two types of features. We conducted experiments on two public datasets, namely the GOTOV dataset (elderly people) and the JSI dataset (young people). Experimental results demonstrate that our method achieves excellent performance across different age groups. Compared to the baseline models, the proposed DSFN significantly improves the accuracy of human energy expenditure estimation.

Riemannian manifold-based disentangled representation learning for multi-site functional connectivity analysis

Functional connectivity (FC), derived from resting-state functional magnetic resonance imaging (rs-fMRI), has been widely used to characterize brain abnormalities in disorders. FC is usually defined as a correlation matrix that is a symmetric positive definite (SPD) matrix lying on the Riemannian manifold. Recently, a number of learning-based methods have been proposed for FC analysis, while the geometric properties of Riemannian manifold have not yet been fully explored in previous studies. Also, most existing methods are designed to target one imaging site of fMRI data, which may result in limited training data for learning reliable and robust models. In this paper, we propose a novel Riemannian Manifold-based Disentangled Representation Learning (RM-DRL) framework which is capable of learning invariant representations from fMRI data across multiple sites for brain disorder diagnosis. In RM-DRL, we first employ an SPD-based encoder module to learn a latent unified representation of FC from different sites, which can preserve the Riemannian geometry of the SPD matrices. In latent space, a disentangled representation module is then designed to split the learned features into domain-specific and domain-invariant parts, respectively. Finally, a decoder module is introduced to ensure that sufficient information can be preserved during disentanglement learning. These designs allow us to introduce four types of training objectives to improve the disentanglement learning. Our RM-DRL method is evaluated on the public multi-site ABIDE dataset, showing superior performance compared with several state-of-the-art methods.

Design, Simulation, and Implementation of MRAC Controller with PMAC Target Hardware for Controlling 7.5 m Antenna

The article presents the design of an adaptive controller, simulation, code generation, and porting of the algorithm into target hardware, movement of the 7.5 m antenna, testing, verification, and validation of theoretical and experimental results of the antenna system. The conventional servo control methods based on PID controllers are being used for tracking X-band missions. As the need for higher data rates is increasing, missions are switching to the Ka band, as a result the satellite ground stations will have to track the satellite in the Ka band where the antenna band beam width is narrowed by a factor of 3 when compared to the X-band antenna beamwidth. These challenges had driven the designer to design and implement a Model Reference Adaptive Controller over PID to improve and achieve the tracking accuracy of less than 25m° from the present 30 m°. Thus, consistent and constant performance is achieved throughout the trajectory tracking of the mission using PID + MRAC (Model Reference Adaptive Control). MRAC uses a reference model that specifies the desired response of the antenna system and adjusts the controller parameters such that the plant follows the reference to achieve the desired performance. The entire drive chain modules and their parameters are considered in the design of the plant, namely MRAC controller, PID controller, Power Drive electronic (Rectifier and Inverter), Servo Motor, mechanical torque coupling, Gear reducer, Slew Ring Bearing (SRB), and absolute encoder modules of each axis for position feedback from the antenna. Model Order Reduction of higher-order actual Plant model to the standard second-order system is optimized. The modelled and analysed outcomes of the MRAC scheme using the Modified MIT are demonstrated. The developed model is tested against step, ramp, parabolic, satellite trajectory, velocity and acceleration inputs in MATLAB/Simulink and validated with practical/ experimental test results with antenna load at the satellite ground Station. The achieved results indicate that the PID + MRAC controller has significant and consistent performance improvements when compared to the PID controller alone.

KAN-Transformer Model for UltraShort-Term Wind Power Prediction Based on EWMA Data Processing

When using the Transformer model for wind power prediction, the presence of noise in wind power data and the model’s final layer relying solely on a simple linear output reduces the model’s ability to capture nonlinear relationships, leading to a decrease in prediction accuracy. To address these issues, this paper proposes an ultrashort-term wind power prediction model based on exponential weighted moving average (EWMA) data processing and Kolmogorov–Arnold Network (KAN)-Transformer. First, multiple variable features are smoothed using EWMA, which suppresses noise while preserving the original data trends. Then, the EWMA-processed data is input into the Encoder and Decoder modules of the Transformer model to extract features. The output from the Decoder layer is then passed through the KAN layer, built using a cubic B-spline function, to enhance the model’s ability to capture nonlinear relationships, thereby improving the prediction accuracy of the Transformer model for wind power. Finally, experimental analysis is conducted, and it shows that the proposed model achieves the highest prediction accuracy, with a mean absolute error of 4.38 MW, a root mean squared error of 7.37 MW, and a coefficient of determination of 98.73%.

Improving crop type mapping by integrating LSTM with temporal random masking and pixel-set spatial information

Accurate and timely crop type classification is essential for effective agricultural monitoring, cropland management, and yield estimation. Unfortunately, the complicated temporal patterns of different crops, combined with gaps and noise in satellite observations caused by clouds and rain, restrict crop classification accuracy, particularly during early seasons with limited temporal information. Although deep learning-based methods have exhibited great potential for improving crop type mapping, insufficient and noisy training data may lead them to overlook more generalizable features and derive inferior classification performance. To address these challenges, we developed a Mask Pixel-set SpatioTemporal Integration Network (Mask-PSTIN), which integrates a temporal random masking technique and a novel PSTIN model. Temporal random masking augments the training data by selectively removing certain temporal information to improve data variability, enforcing the model to learn more generalized features. The PSTIN, comprising a pixel-set aggregation encoder (PSAE) and long short-term memory (LSTM) module, effectively captures comprehensive spatiotemporal features from time-series satellite images. The effectiveness of Mask-PSTIN was evaluated across three regions with different landscapes and cropping systems. Results demonstrated that the addition of PSAE in PSTIN significantly improved crop classification accuracy compared to a basic LSTM, with average overall accuracy (OA) increasing from 80.9% to 83.9%, and the mean F1-Score (mF1) rising from 0.781 to 0.818. Incorporating temporal random masking in training led to further improvements, increasing average OA and mF1 to 87.4% and 0.865, respectively. The Mask-PSTIN significantly outperformed traditional machine learning and deep learning methods (i.e., RF, SVM, Transformer, and CNN-LSTM) in crop type mapping across all three regions. Furthermore, Mask-PSTIN enabled earlier and more accurate crop type identification before or during their developing stages compared to machine learning models. Feature importance analysis based on the gradient backpropagation algorithm revealed that Mask-PSTIN effectively leveraged multi-temporal features, exhibiting broader attention across various time steps and capturing critical crop phenological characteristics. These results suggest that Mask-PSTIN is a promising approach for improving both post-harvest and early-season crop type classification, with potential applications in agricultural management and monitoring.

A Deep Learning-Based Two-Branch Generative Adversarial Network for Image De-Raining

Raindrops can scatter and absorb light, causing images to become blurry or distorted. To improve image quality by reducing the impact of raindrops, this paper proposes a novel generative adversarial network for image de-raining. The network comprises two parts: a generative network and an adversarial network. The generative network performs image de-raining. The adversarial network determines whether the input image is rain-free or de-rained. The generative network comprises two branches: the A branch, which follows a traditional convolutional network structure, and the U branch, which utilizes a U-Net architecture. The A branch includes a multi-scale module for extracting information at different scales and a residual attention module to reduce redundant information interference. The U branch contains an encoder module designed to address the loss of details and local information caused by conventional down-sampling. To improve the performance of the generative network in image de-raining, this paper employs a relative discriminator incorporating a mean squared error loss. This discriminator better measures the differences between rainy and rain-free images while effectively preventing the occurrence of gradient vanishing. Finally, this study performs visual and quantitative comparisons of the proposed method and existing methods on three established rain image datasets. In the quantitative experiments, the proposed method outperforms existing methods regarding PSNR, SSIM, and VIF metrics. Specifically, our method achieves an average PSNR, SSIM, and VIF of approximately 5%, 3%, and 4% higher than the MFAA-GAN method, respectively. These results indicate that the de-rained images generated via the proposed method are closer to rain-free images.

Decoupling Deep Learning for Enhanced Image Recognition Interpretability

The quest for enhancing the interpretability of neural networks has become a prominent focus in recent research endeavors. Prototype-based neural networks have emerged as a promising avenue for imbuing models with interpretability by gauging the similarity between image components and category prototypes to inform decision-making. However, these networks face challenges as they share similarity activations during both the inference and explanation processes, creating a tradeoff between accuracy and interpretability. To address this issue and ensure that a network achieves high accuracy and robust interpretability in the classification process, this article introduces a groundbreaking prototype-based neural network termed the “Decoupling Prototypical Network” (DProtoNet). This novel architecture comprises encoder, inference, and interpretation modules. In the encoder module, we introduce decoupling feature masks to facilitate the generation of feature vectors and prototypes, enhancing the generalization capabilities of the model. The inference module leverages these feature vectors and prototypes to make predictions based on similarity comparisons, thereby preserving an interpretable inference structure. Meanwhile, the interpretation module advances the field by presenting a novel approach: a “multiple dynamic masks decoder” that replaces conventional upsampling similarity activations. This decoder operates by perturbing images with mask vectors of varying sizes and learning saliency maps through consistent activation. This methodology offers a precise and innovative means of interpreting prototype-based networks. DProtoNet effectively separates the inference and explanation components within prototype-based networks. By eliminating the constraints imposed by shared similarity activations during the inference and explanation phases, our approach concurrently elevates accuracy and interpretability. Experimental evaluations on diverse public natural datasets, including CUB-200-2011, Stanford Cars, and medical datasets like RSNA and iChallenge-PM, corroborate the substantial enhancements achieved by our method compared to previous state-of-the-art approaches. Furthermore, ablation studies are conducted to provide additional evidence of the effectiveness of our proposed components.

Precision Segmentation of Subretinal Fluids in OCT Using Multiscale Attention-Based U-Net Architecture.

This paper presents a deep-learning architecture for segmenting retinal fluids in patients with Diabetic Macular Oedema (DME) and Age-related Macular Degeneration (AMD). Accurate segmentation of multiple fluid types is critical for diagnosis and treatment planning, but existing techniques often struggle with precision. We propose an encoder-decoder network inspired by U-Net, processing enhanced OCT images and their edge maps. The encoder incorporates Residual and Inception modules with an autoencoder-based multiscale attention mechanism to extract detailed features. Our method shows superior performance across several datasets. On the RETOUCH dataset, the network achieved F1 Scores of 0.82 for intraretinal fluid (IRF), 0.93 for subretinal fluid (SRF), and 0.94 for pigment epithelial detachment (PED). The model also performed well on the OPTIMA and DUKE datasets, demonstrating high precision, recall, and F1 Scores. This architecture significantly enhances segmentation accuracy and edge precision, offering a valuable tool for diagnosing and managing retinal diseases. Its integration of dual-input processing, multiscale attention, and advanced encoder modules highlights its potential to improve clinical outcomes and advance retinal disease treatment.

RS-Net: Hyperspectral Image Land Cover Classification Based on Spectral Imager Combined with Random Forest Algorithm

Recursive neural networks and transformers have recently become dominant in hyperspectral (HS) image classification due to their ability to capture long-range dependencies in spectral sequences. Despite the success of these sequential architectures, mainstream deep learning methods primarily handle two-dimensional structured data. However, challenges such as the curse of dimensionality, spectral variability, and confounding factors in hyperspectral remote sensing images limit their effectiveness, especially in remote sensing applications. To address this issue, this paper proposes a novel land cover classification algorithm that integrates random forests with a spectral transformer network structure (RS-Net). Firstly, this paper presents a combination of the Gramian Angular Field (GASF) and Gramian Angular Difference Field (GADF) algorithms, which effectively maps the multidimensional time series constructed for each pixel onto two-dimensional image features, enabling precise extraction and recognition in the backend network algorithms and improving the classification accuracy of land cover types. Secondly, to capture the relationships between features at different scales, this paper proposes a SpectralFormer network architecture using the Context and Structure Encoding (CASE) module to effectively learn dependencies between channels. This architecture enhances important features and suppresses unimportant ones, thereby addressing the semantic gap and improving the recognition capability of land cover features. Finally, the final prediction results are determined by a voting mechanism from the Random Forest algorithm, which synthesizes predictions from multiple decision trees to enhance classification stability and accuracy. To better compare the performance of RS-Net, this paper conducted extensive experiments on three benchmark HS datasets obtained from satellite and airborne imagers, comparing various classic neural network models. Surprisingly, the RS-Net algorithm achieves high performance and efficiency, offering a new and effective tool for land cover classification.

Predicting functional connectivity network from routinely acquired T1-weighted imaging-based brain network by generative U-GCNet

Predicting the function magnetic resonance imaging (fMRI)-based brain network (fMRI-BN) from structure MRI-based brain network is imperative in clinical practice because fMRIs are not routinely acquired in a vast majority of hospitals. In this study, a generative U-GCNet (U-shaped graph convolutional Network) was proposed to predict fMRI-BN from radiomics-based morphological brain network (radMBN) derived from routinely acquired T1-WI image. Specifically, the U-GCNet consisted of a graph convolutional network (GCN) encoder module (En-GCN), a deep feature connectivity construction module (DF2C), and a GCN decoder module (De-GCN). Both En-GCN and De-GCN employed mixed local-and-long distance node feature aggregation strategy to enhance the graph encoding and decoding ability, and the DF2C reshaped the deep feature matrix into the connectivity matrix for outputting the brain network prediction. Additionally, a multi-scale network similarity loss function was conducted on full values, upper triangular values, and each row values of connectivity matrix. Experiments on 3169 subjects from three publicly available databases demonstrated that the U-GCNet could predict the fMRI-BN from radMBN with a promising performance (MSE [0.0002 0.0025], PCC [0.956 0.991]) over eight alternative methods under comparison. The results exhibited a significant correlation (PCC [0.796, 0.897], P<0.05) between the estimated and real radiomics-function coupling values. The individual-level and group-level brain network visualization was displayed with high consistency. The TOP brain regions identified by four graph-based metrics also exhibited with consistency. These results demonstrated that the proposed U-GCNet could achieve promising prediction of fMRI-BN from radMBN which could alleviate the limited availability of fMRI and boost its usage in clinical practice.

Spatial–Temporal-Correlation-Constrained Dynamic Graph Convolutional Network for Traffic Flow Forecasting

Accurate traffic flow prediction in road networks is essential for intelligent transportation systems (ITS). Since traffic data are collected from the road network with spatial topological and time series sequences, the traffic flow prediction is regarded as a spatial–temporal prediction task. With the powerful ability to model the non-Euclidean data, the graph convolutional network (GCN)-based models have become the mainstream framework for traffic forecasting. However, existing GCN-based models either use the manually predefined graph structure to capture the spatial features, ignoring the heterogeneity of road networks, or simply perform 1-D convolution with fixed kernel to capture the temporal dependencies of traffic data, resulting in insufficient long-term temporal feature extraction. To solve those issues, a spatial–temporal correlation constrained dynamic graph convolutional network (STC-DGCN) is proposed for traffic flow forecasting. In STC-DGCN, a spatial–temporal embedding encoder module (STEM) is first constructed to encode the dynamic spatial relationships for road networks at different time steps. Then, a temporal feature encoder module with heterogeneous time series correlation modeling (TFE-HCM) and a spatial feature encoder module with dynamic multi-graph modeling (SFE-DCM) are designed to generate dynamic graph structures for effectively capturing the dynamic spatial and temporal correlations. Finally, a spatial–temporal feature fusion module based on a gating fusion mechanism (STM-GM) is proposed to effectively learn and leverage the inherent spatial–temporal relationships for traffic flow forecasting. Experimental results from three real-world traffic flow datasets demonstrate the superior performance of the proposed STC-DGCN compared with state-of-the-art traffic flow forecasting models.

Multimodal vehicle trajectory prediction and integrated threat assessment algorithm based on adaptive driving intention

As autonomous driving and connected communication technologies advance swiftly, vehicle trajectory prediction has become increasingly significant. The motion of a vehicle is contingent not only on its historical trajectory but is also subject to the influence of surrounding vehicles, thereby exhibiting intricate social and temporal interdependencies. Furthermore, the inherent randomness and uncertainty in driver behavior render vehicle trajectory prediction inherently multimodal, a factor that is frequently neglected in current research. Against this backdrop, a multimodal vehicle trajectory prediction (MTP) model based on an encoder-decoder architecture is proposed to hierarchically extract historical features of vehicles. The model consists of five key components: temporal feature encoder module, spatial interaction module, spatial-temporal dependence module, driving intention fusion module and multimodal trajectory output module. Experiments on the NGSIM dataset show that the predictive performance of the model has been improved to varying degrees, especially at 3–5 s, where the improvement is more significant. Compared with state-of-the-art models, the Root Mean Square Error (RMSE) error at 5 s time horizon is 3.38 m on NGSIM dataset, which represents a 25 % improvement. To measure the safety of predicted trajectories, we propose a comprehensive threat assessment model that combines collision time (TTC), headway (TH) and time to lateral collision (TLC) metrics based on safe distance theory. This model not only evaluates the longitudinal collision threat in the following state, but also evaluates the lateral collision threat during driving maneuvers in multi lane scenarios, thereby comprehensively improving the safety of vehicle driving. This research also offers new perspectives and insights for the development of autonomous driving.

TC-DTA: Predicting Drug-Target Binding Affinity With Transformer and Convolutional Neural Networks.

Bioinformatics is a rapidly evolving field that applies computational methods to analyze and interpret biological data. A key task in bioinformatics is identifying novel drug-target interactions (DTIs), which plays a crucial role in drug discovery. Most computational approaches treat DTI prediction as a binary classification problem, determining whether drug-target pairs interact. However, with the growing availability of drug-target binding affinity data, this binary task can be reframed as a regression problem focused on drug-target affinity (DTA). DTA quantifies the strength of drug-target binding, offering more detailed insights than DTI and serving as a valuable tool for virtual screening in drug discovery. Accurately predicting compound interactions with targets can accelerate the drug development process. In this study, we introduce a deep learning model named TC-DTA for DTA prediction, leveraging convolutional neural networks (CNN) and the encoder module of the transformer architecture. We begin by extracting raw drug SMILES strings and protein amino acid sequences from the dataset, which are then represented using various encoding methods. Subsequently, we employ CNN and the transformer's encoder module to extract features from the drug SMILES strings and protein sequences, respectively. Finally, the feature information is concatenated and input into a multi-layer perceptron to predict binding affinity scores. We evaluated our model on two benchmark DTA datasets, Davis and KIBA, comparing it with methods such as KronRLS, SimBoost, and DeepDTA. Our model, TC-DTA, outperformed these baseline methods based on evaluation metrics like Mean Squared Error (MSE), Concordance Index (CI), and Regression towards the Mean Index ( rm2 ). These results highlight the effectiveness of the Transformer's encoder and CNN in extracting meaningful representations from sequences, thereby enhancing DTA prediction accuracy. This deep learning model can accelerate drug discovery by identifying drug candidates with high binding affinity to specific targets. Compared to traditional methods, machine learning technology offers a more effective and efficient approach to drug discovery.