Spatial Feature Extraction Research Articles

Autonomous Underwater Vehicle Trajectory Prediction with the Nonlinear Kepler Optimization Algorithm–Bidirectional Long Short-Term Memory–Time-Variable Attention Model

Autonomous underwater vehicles (AUVs) have been widely used in ocean missions. When they fail in the ocean, it is important to predict their trajectory. Existing methods rely heavily on historical trajectory data while overlooking the influence of the ocean environment on an AUV’s trajectory. At the same time, these methods fail to use the dependency between variables in the trajectory. To address these challenges, this paper proposes an AUV trajectory prediction model known as the nonlinear Kepler optimization algorithm–bidirectional long short-term memory–time-variable attention (NKOA-BiLSTM-TVA) model. This paper introduces opposition-based learning during the initialization process of the KOA and improves the algorithm by incorporating a nonlinear factor into the planet position update process. We designed an attention mechanism layer that spans both time and variable dimensions, called TVA. TVA can extract features from both the time and variable dimensions of the trajectory and use the dependency between trajectory variables to predict the trajectory. First, the model uses a convolutional neural network (CNN) to extract spatial features from the trajectory. Next, it combines a BiLSTM network with TVA to predict the AUV’s trajectory. Finally, the improved NKOA is used to optimize the model’s hyperparameters. Experimental results show that the NKOA-BiLSTM-TVA model has an excellent parameter optimization effect and higher prediction accuracy in AUV trajectory prediction tasks. It also achieves excellent results in ship trajectory prediction.

Short-term wind farm cluster power point-interval prediction based on graph spatio-temporal features and S-Stacking combined reconstruction

Wind energy is becoming increasingly competitive, Accurate and reliable multi-engine wind power forecasts can reduce power system operating costs and improve wind power consumption capacity. Existing research on wind power forecasting has neglected the importance of interval forecasting using clusters of wind farms to capture spatial characteristics and the objective selection of forecasting sub-learners, leading to increased uncertainty and risk in system operation. This paper proposes a new “decomposition-aggregation-multi-model parallel prediction” method. The data set is pre-processed by a decomposition-aggregation strategy and spatial feature extraction, and then a Stacking model with multiple parallel sub-learners selected by bootstrap method is used for point and interval forecasting. Experiments and discussions are conducted based on 15-min resolution wind power data from a cluster dataset of a wind farm in northwest China. The experimental results indicate that the method achieves higher accuracy and reliability in both point prediction and interval prediction than other comparative models, with a root mean square error value of 7.47 and an average F value of 1.572, which can provide a reliable reference for power generation planning from wind farm clusters.

Symmetric spatiotemporal learning network with sparse meter graph for short-term energy-consumption prediction in manufacturing systems

Short-term energy-consumption prediction is the basis of anomaly detection, real-time scheduling, and energy-saving control in manufacturing systems. Most existing methods focus on single-node energy-consumption prediction and suffer from difficult parameter collection and modelling. Although several methods have been presented for multinode energy-consumption prediction, their prediction performance needs to be improved owing to a lack of appropriate knowledge guidance and learning networks for complex spatiotemporal relationships. This study presents a symmetric spatiotemporal learning network (SSTLN) with a sparse meter graph (SMG) (SSTLN-SMG) that aims to predict multiple nodes based on energy-consumption time series and general process knowledge. The SMG expresses process knowledge by abstracting production nodes, material flows, and energy usage, and provides initial guidance for the SSTLN to extract spatial features. SSTLN, a symmetrical stack of graph convolutional networks (GCN) and gated linear units (GLU), is devised to achieve a trade-off not only between spatial and temporal feature extraction but also between detail capture and noise suppression. Extensive experiments were performed using datasets from an aluminium profile plant. The experimental results demonstrate that the proposed method allows multinode energy-consumption prediction with less prediction error than state-of-the-art methods, methods with deformed meter graphs, and methods with deformed learning networks.

Multi-station collaborative wave height prediction based on multi-feature identification and interpretable analysis

This study proposes a novel deep learning model, the graph convolutional gated recurrent unit (GC-GRU), to address the critical challenge of accurate forecasting of ocean wave heights due to the complex nonlinear spatiotemporal variability of wave dynamics. The proposed model, which integrates the strengths of graph convolutional networks (GCNs) for spatial feature extraction and gated recurrent units (GRUs) for temporal feature extraction, allows for effective capture of complex spatiotemporal patterns in wave height data and is evaluated on a dataset of 666 observation stations in the Gulf of Mexico, forecasting wave heights up to 36 h in advance. Comparative experiments with traditional CNN and GRU models demonstrate the superior predictive performance of the GC-GRU approach. Additionally, we introduce the shapley additive explanation (SHAP) values to provide physical insights into the key physical variables and historical patterns driving the model's predictions. The results show that wind speed and mean wave period are the most influential factors related to wave height variations. It is expected that this work presents a significant advancement in wave height forecasting by introducing the innovative GC-GRU architecture and leveraging SHAP analysis to interpret the model's inner workings. The findings are expected to have important implications for enhancing coastal and maritime operations as well as informing our understanding of complex ocean wave dynamics.

Machine vision-based gait scan method for identifying cognitive impairment in older adults.

Early identification of cognitive impairment in older adults could reduce the burden of age-related disabilities. Gait parameters are associated with and predictive of cognitive decline. Although a variety of sensors and machine learning analysis methods have been used in cognitive studies, a deep optimized machine vision-based method for analyzing gait to identify cognitive decline is needed. This study used a walking footage dataset of 158 adults named West China Hospital Elderly Gait, which was labelled by performance on the Short Portable Mental Status Questionnaire. We proposed a novel recognition network, Deep Optimized GaitPart (DO-GaitPart), based on silhouette and skeleton gait images. Three improvements were applied: short-term temporal template generator (STTG) in the template generation stage to decrease computational cost and minimize loss of temporal information; depth-wise spatial feature extractor (DSFE) to extract both global and local fine-grained spatial features from gait images; and multi-scale temporal aggregation (MTA), a temporal modeling method based on attention mechanism, to improve the distinguishability of gait patterns. An ablation test showed that each component of DO-GaitPart was essential. DO-GaitPart excels in backpack walking scene on CASIA-B dataset, outperforming comparison methods, which were GaitSet, GaitPart, MT3D, 3D Local, TransGait, CSTL, GLN, GaitGL and SMPLGait on Gait3D dataset. The proposed machine vision gait feature identification method achieved a receiver operating characteristic/area under the curve (ROCAUC) of 0.876 (0.852-0.900) on the cognitive state classification task. The proposed method performed well identifying cognitive decline from the gait video datasets, making it a prospective prototype tool in cognitive assessment.

A Spatial Semantic Feature Extraction Method for Urban Functional Zones Based on POIs

Accurately extracting semantic features of urban functional zones is crucial for understanding urban functional zone types and urban functional spatial structures. Points of interest provide comprehensive information for extracting the semantic features of urban functional zones. Many researchers have used topic models of natural language processing to extract the semantic features of urban functional zones from points of interest, but topic models cannot consider the spatial features of points of interest, which leads to the extracted semantic features of urban functional zones being incomplete. To consider the spatial features of points of interest when extracting semantic features of urban functional zones, this paper improves the Latent Dirichlet Allocation topic model and proposes a spatial semantic feature extraction method for urban functional zones based on points of interest. In the proposed method, an assumption (that points of interest belonging to the same semantic feature are spatially correlated) is introduced into the generation process of urban functional zones, and then, Gibbs sampling is combined to carry out the parameter inference process. We apply the proposed method to a simulated dataset and the point of interest dataset for Chaoyang District, Beijing, and compare the semantic features extracted by the proposed method with those extracted by the Latent Dirichlet Allocation. The results show that the proposed method sufficiently considers the spatial features of points of interest and has a higher capability of extracting the semantic features of urban functional zones than the Latent Dirichlet Allocation.

An Improved Method for Detecting Crane Wheel-Rail Faults Based on YOLOv8 and the Swin Transformer.

In the realm of special equipment, significant advancements have been achieved in fault detection. Nonetheless, faults originating in the equipment manifest with diverse morphological characteristics and varying scales. Certain faults necessitate the extrapolation from global information owing to their occurrence in localized areas. Simultaneously, the intricacies of the inspection area's background easily interfere with the intelligent detection processes. Hence, a refined YOLOv8 algorithm leveraging the Swin Transformer is proposed, tailored for detecting faults in special equipment. The Swin Transformer serves as the foundational network of the YOLOv8 framework, amplifying its capability to concentrate on comprehensive features during the feature extraction, crucial for fault analysis. A multi-head self-attention mechanism regulated by a sliding window is utilized to expand the observation window's scope. Moreover, an asymptotic feature pyramid network is introduced to augment spatial feature extraction for smaller targets. Within this network architecture, adjacent low-level features are merged, while high-level features are gradually integrated into the fusion process. This prevents loss or degradation of feature information during transmission and interaction, enabling accurate localization of smaller targets. Drawing from wheel-rail faults of lifting equipment as an illustration, the proposed method is employed to diagnose an expanded fault dataset generated through transfer learning. Experimental findings substantiate that the proposed method in adeptly addressing numerous challenges encountered in the intelligent fault detection of special equipment. Moreover, it outperforms mainstream target detection models, achieving real-time detection capabilities.

DFFNet: a lightweight approach for efficient feature-optimized fusion in steel strip surface defect detection

Steel surface defect detection is crucial in manufacturing, but achieving high accuracy and real-time performance with limited computing resources is challenging. To address this issue, this paper proposes DFFNet, a lightweight fusion network, for fast and accurate steel surface defect detection. Firstly, a lightweight backbone network called LDD is introduced, utilizing partial convolution to reduce computational complexity and extract spatial features efficiently. Then, PANet is enhanced using the Efficient Feature-Optimized Converged Network and a Feature Enhancement Aggregation Module (FEAM) to improve feature fusion. FEAM combines the Efficient Layer Aggregation Network and reparameterization techniques to extend the receptive field for defect perception, and reduce information loss for small defects. Finally, a WIOU loss function with a dynamic non-monotonic mechanism is designed to improve defect localization in complex scenes. Evaluation results on the NEU-DET dataset demonstrate that the proposed DFFNet achieves competitive accuracy with lower computational complexity, with a detection speed of 101 FPS, meeting real-time performance requirements in industrial settings. Furthermore, experimental results on the PASCAL VOC and MS COCO datasets demonstrate the strong generalization capability of DFFNet for object detection in diverse scenarios.

A Spatio-temporal Graph Transformer driven model for recognizing fine-grained data human activity

Human activity recognition(HAR) is an important research focus in ubiquitous computing. It has been widely applied in various domains, such as smart homes, healthcare assistance, and sports training. Accurate human activity recognition directly impacts the performance of downstream tasks. Existing methods for human activity recognition primarily rely on extracting temporal or spatial features from the data. These features are used for the task of human activity recognition. The existing methods mainly use CNN or RNN models to model the Euclidean correlations among spatially adjacent sensors or channels. However, non-Euclidean pairwise correlations among all sensors or channels are even critical for accurate classification, which has been ignored by the existing methods. In this paper, we incorporate fine-grained spatial structural data into the model to overcome these limitations. A novel deep learning model for human activity recognition is proposed, which is called the fine-grained data-oriented Spatial-Temporal Graph Transformer network (STGT). The introduction of the STGT model can eliminate the limitations of existing spatial feature extraction methods by leveraging a novel data organization approach proposed in this study. This model enhances the spatial features within the time feature extraction module for effective spatio-temporal feature extraction. We conducted experiments on four large-scale real-world HAR datasets to evaluate its performance. The experimental results demonstrate the superiority of our method over state-of-the-art approaches.

Estimating high spatio-temporal resolution XCO2 using spatial features deep fusion model

The high temporal-spatial resolution estimation of XCO2 data is foundational for precision quantification of carbon dioxide sources and sinks at a regional scale. This study proposed an advanced XCO2 data estimation method, using spatial features deep fusion. Leveraging convolutional neural network (CNN) principles, SpatialFusionNet—a module was designed to amalgamate geographical features within a defined spatial range. This module captures and integrates the spatial characteristics of meteorological and surface environmental factors, enhancing its application to the XCO2 estimation model. Building upon machine learning methods including Extreme Gradient Boosting (XGBoost), Support Vector Machine (SVM), and Deep Neural Network (DNN), combined with the SpatialFusionNet module, the spatial features deep fusion models were constructed utilizing relationships among OCO-2 satellite XCO2 trajectory monitoring data in China, Copernicus Atmospheric Monitoring Service (CAMS) XCO2 reanalysis data, and meteorological factors, surface vegetation, and meteorological factors. Model performance improvements were significant, with SVM, DNN, and XGBoost showing respective RMSE reductions of 1.297 ppm, 0.480 ppm, and 0.200 ppm in ten-fold cross-validation based on OCO-2 trajectory samples. In data validation with TCCON Hefei station, the correlation between inversion data and ground-based data reached 0.85, affirming the method's high accuracy. Employing the spatial feature extraction module combined with DNN, the 2015 XCO2 annual spatial distribution of China, analyzing temporal-spatial distribution characteristics in China was generated. The DNN model, combining the SpatialFusionNet module, significantly contributes to the estimation of high temporal-spatial resolution XCO2 datasets, facilitating fine-scale quantification of regional carbon cycling.

Spatial-temporal memory enhanced multi-level attention network for origin-destination demand prediction

Origin-destination demand prediction is a critical task in the field of intelligent transportation systems. However, accurately modeling the complex spatial-temporal dependencies presents significant challenges, which arises from various factors, including spatial, temporal, and external influences such as geographical features, weather conditions, and traffic incidents. Moreover, capturing multi-scale dependencies of local and global spatial dependencies, as well as short and long-term temporal dependencies, further complicates the task. To address these challenges, a novel framework called the Spatial-Temporal Memory Enhanced Multi-Level Attention Network (ST-MEN) is proposed. The framework consists of several key components. Firstly, an external attention mechanism is incorporated to efficiently process external factors into the prediction process. Secondly, a dynamic spatial feature extraction module is designed that effectively captures the spatial dependencies among nodes. By incorporating two skip-connections, this module preserves the original node information while aggregating information from other nodes. Finally, a temporal feature extraction module is proposed that captures both continuous and discrete temporal dependencies using a hierarchical memory network. In addition, multi-scale features cascade fusion is incorporated to enhance the performance of the proposed model. To evaluate the effectiveness of the proposed model, extensively experiments are conducted on two real-world datasets. The experimental results demonstrate that the ST-MEN model achieves excellent prediction accuracy, where the maximum improvement can reach to 19.1%.

Transformer-enhanced two-stream complementary convolutional neural network for hyperspectral image classification

Convolutional neural networks (CNNs) and Transformers have been widely employed in hyperspectral image (HSI) classification, due to their advantages in extracting local-range and global-range features respectively. However, some hybrid CNNs and Transformer-based methods still face some shortcomings in effectively exploiting local-global features in HSI. In this article, a Transformer-enhanced two-stream complementary convolutional neural network is proposed for HSI classification. It mainly comprises the spectral feature extraction stream, spatial feature extraction stream, and a spectral-spatial weight feature complementary module. Firstly, in spectral feature extraction stream, a hybrid convolution block is proposed to extract high-resolution spectral features. And then an attention mechanism, which is based on a Transformer encoder, is used to learn key informative features in spectral channel. The spatial feature extraction stream is designed for spatial feature extraction, and it has a similar architecture to the spectral feature extraction stream. Secondly, a spectral-spatial weight feature complementary module is developed to fully exploit the features which are based on the feature weight in spectral tokens and spatial tokens, they are from spectral and spatial feature extraction streams separately, and then the spectral-spatial tokens with different semantic features are derived. Finally, the spectral-spatial tokens are fed into the linear layer to produce classification outcomes. Furthermore, simplified convolution kernels are employed in our proposed method to decrease the total parameters. Results from experiments on four extensively used HSI datasets highlight the superior performance of our proposed method over other cutting-edge methods.

Fine particulate matter concentration prediction based on hybrid convolutional network with aggregated local and global spatiotemporal information: A case study in Beijing and Chongqing

Air pollution is a highly concerned environmental issue that have serious impacts on human health and the ecological environment. Accurate air quality prediction can help people effectively deal with the threats posed by air pollution. Most previous work mainly focused on temporal modeling of air quality data at monitoring stations. In recent years, a number of works have used graph convolution to model the spatial dependencies between neighboring sites to extract spatial features and combined temporal model to extract temporal features for predicting PM2.5. However, these models only considered the local spatial relationships of adjacent sites and ignored sites with longer geographic distance. Therefore, this study proposes a spatio-temporal hybrid model based on convolution and attention (named Attentive Graph Convolution and 1D Convolution Network, AGCC) to predict multi-site and multi-step PM2.5 concentration. This method not only models the local spatial relationships of adjacent sites, but also combines spatial attention and graph convolutional network (GCN) to model the global spatial relationships of sites. The local and global spatial features are obtained respectively and are fused to obtain spatial features with richer semantic information. Meanwhile, the temporal dependency relationship is modeled through a temporal module composed of temporal attention and one-dimensional convolutional neural network (Conv 1D). AGCC was compared to six baseline models at different prediction horizon based on the air quality datasets of Beijing and Chongqing. The experimental results demonstrate that the model proposed by our achieves the best performance and verifies the feasibility of the model.

Classification of Grapevine Varieties Using UAV Hyperspectral Imaging

Classifying grapevine varieties is crucial in precision viticulture, as it allows for accurate estimation of vineyard row growth for different varieties and ensures authenticity in the wine industry. This task can be performed with time-consuming destructive methods, including data collection and analysis in the laboratory. In contrast, unmanned aerial vehicles (UAVs) offer a markedly more efficient and less restrictive method for gathering hyperspectral data, even though they may yield data with higher levels of noise. Therefore, the first task is the processing of these data to correct and downsample large amounts of data. In addition, the hyperspectral signatures of grape varieties are very similar. In this study, we propose the use of a convolutional neural network (CNN) to classify seventeen different varieties of red and white grape cultivars. Instead of classifying individual samples, our approach involves processing samples alongside their surrounding neighborhood for enhanced accuracy. The extraction of spatial and spectral features is addressed with (1) a spatial attention layer and (2) inception blocks. The pipeline goes from data preparation to dataset elaboration, finishing with the training phase. The fitted model is evaluated in terms of response time, accuracy and data separability and is compared with other state-of-the-art CNNs for classifying hyperspectral data. Our network was proven to be much more lightweight by using a limited number of input bands (40) and a reduced number of trainable weights (560 k parameters). Hence, it reduced training time (1 h on average) over the collected hyperspectral dataset. In contrast, other state-of-the-art research requires large networks with several million parameters that require hours to be trained. Despite this, the evaluated metrics showed much better results for our network (approximately 99% overall accuracy), in comparison with previous works barely achieving 81% OA over UAV imagery. This notable OA was similarly observed over satellite data. These results demonstrate the efficiency and robustness of our proposed method across different hyperspectral data sources.

An integrated navigation algorithm assisted by CNN-Informer during short-time GNSS outages

Abstract Inertial Navigation System (INS) and Global Navigation Satellite System (GNSSs) are commonly used in integrated navigation systems. The GNSS navigation system, however, is not autonomous, and its signals are susceptible to environmental influences leading to poor navigation results. In order to improve the positioning accuracy of the INS/GNSS integrated navigation system during GNSS outages, a hybrid algorithm of Convolutional Neural Network (CNN)-Informer model-assisted INS is proposed. The CNN-Informer model combines the advantages of each of the convolutional neural network and the Informer network. It utilizes CNN to rapidly extract spatial features from input data, employs the Informer network to establish relationships between inputs and outputs, and acquires pseudo-GNSS signals to compensate for position errors in the INS. Lastly, we evaluate the proposed algorithm through six sets of experiments conducted on public datasets and real road scenarios. The comparative results demonstrate that the proposed method offers a more accurate and reliable solution during GNSS outages. Specifically, CNN-Informer can improve navigation accuracy by 81.46% over MLP and by 58.91% over LSTM during 20-second GNSS outages.

Deep Learning Wind Power Prediction Model Based on Attention Mechanism-Based Convolutional Neural Network and Gated Recurrent Unit Neural Network

Accurate prediction of wind power is crucial for the efficient operation and risk management of wind farms. This paper introduces a deep learning model for wind power prediction that integrates an Attention mechanism with a convolutional neural network (CNN) and a gated recurrent unit (GRU) neural network. Addressing the randomness, intermittency, volatility and uncertainty of wind speed, we first apply swarm decomposition (SWD) to preprocess the original wind power data into subsequences. Subsequently, the CNN extracts spatial features, and the GRU identifies temporal correlations. The Attention mechanism enhances feature significance, further optimizing prediction accuracy. Complex error sequences generated by the CNN–GRU–Attention (CGA) model are corrected using the autoregressive integrated moving average (ARIMA). We evaluated the model’s performance using three wind power datasets against 16 other models, employing six evaluation indices (MSE, RMSE, MAPE, Theil’s [Formula: see text], TIC and SPL) and the Diebold–Mariano (DM) test and model confidence set (MCS) for model testing. Our results demonstrate the proposed model’s superior accuracy and efficiency in predicting wind power.

Real-Time Arabic Sign Language Recognition Using a Hybrid Deep Learning Model.

Sign language is an essential means of communication for individuals with hearing disabilities. However, there is a significant shortage of sign language interpreters in some languages, especially in Saudi Arabia. This shortage results in a large proportion of the hearing-impaired population being deprived of services, especially in public places. This paper aims to address this gap in accessibility by leveraging technology to develop systems capable of recognizing Arabic Sign Language (ArSL) using deep learning techniques. In this paper, we propose a hybrid model to capture the spatio-temporal aspects of sign language (i.e., letters and words). The hybrid model consists of a Convolutional Neural Network (CNN) classifier to extract spatial features from sign language data and a Long Short-Term Memory (LSTM) classifier to extract spatial and temporal characteristics to handle sequential data (i.e., hand movements). To demonstrate the feasibility of our proposed hybrid model, we created a dataset of 20 different words, resulting in 4000 images for ArSL: 10 static gesture words and 500 videos for 10 dynamic gesture words. Our proposed hybrid model demonstrates promising performance, with the CNN and LSTM classifiers achieving accuracy rates of 94.40% and 82.70%, respectively. These results indicate that our approach can significantly enhance communication accessibility for the hearing-impaired community in Saudi Arabia. Thus, this paper represents a major step toward promoting inclusivity and improving the quality of life for the hearing impaired.

Wavelet–pixel domain progressive fusion network for underwater image enhancement

Due to the complexity of the underwater environment, underwater images suffer from low-illumination, color deviation, and blurred details. Conventional underwater image enhancement (UIE) approaches have achieved remarkable success in improving color and illumination in the pixel domain. However, they ignore fine-grained details during the enhancement process. To solve these problems, we propose a novel wavelet–pixel domain progressive fusion network (WPFNet) to improve details while enhancing illumination and reducing color deviation for degraded underwater images. Unlike the UIE method, we propose a wavelet domain module (WDM) to obtain different scale frequency features with fine-grained details. Concretely, we develop a residual attention block (RAB) that optimizes low-frequency sub-images to maintain rich details, and enhance color and illumination information for low-frequency features. Transformer block (T_Block) is proposed to convert three high-frequency sub-images into high-frequency features operated by self-attention mechanism. Frequency features are obtained by adding low- and high-frequency features from the same scale, suffering from insufficient color and illumination information. Thus, the pixel domain module (PDM) is introduced to extract spatial features of different scales with rich color and illumination information in the pixel domain. Reconstruction module (REM) is proposed to fuse frequency and spatial features from small to large scale to restructure clear underwater images. The dual-domain fusion block (DFB) is proposed as a key element of REM, which exploits the constraint signal to fuse superior information of frequency and spatial features. Finally, our WPFNet obtains high-quality underwater images in four benchmark datasets and achieves superior performance compared to state-of-the-art UIE methods. The code is available at: https://github.com/LiuShiBen/WPFNet.

STVANet: A spatio-temporal visual attention framework with large kernel attention mechanism for citywide traffic dynamics prediction

Enhancing the efficiency and safety of the Intelligent Transportation System requires effective modeling and prediction of citywide traffic dynamics. Most studies employ convolutional neural networks (CNNs) with a 3D convolutional structure or spatio-temporal models with self-attention mechanisms to capture the spatio-temporal information of traffic distribution. Although 3D CNNs excel at capturing local contextual information, they are computationally complex due to the large number of parameters and cannot capture long-range dependence. By contrast, although self-attention mechanisms originally designed to address challenges in natural language processing can capture long-range dependence, their application to 2D image structures requires breaking down the inherent 2D context into a 1D sequence, increasing the computational complexity and neglecting the adaptability between local contextual information and channels. Accordingly, we propose a spatio-temporal visual attention neural network (STVANet), a novel spatio-temporal visual attention 2D CNN, which integrates a unique visual attention module with a large kernel attention (LKA) mechanism, a squeeze-and-excitation (SE) mechanism and a feedforward component to capture long-range dependence and channel information in urban traffic data while preserving the 2D image structure. LKA-based spatio-temporal attention networks extract spatial and temporal features from weekly, daily, and recent hourly periods, and aggregate them with weighted consideration of external features to make predictions. Evaluation of real-world datasets demonstrates STVANet’s superiority over baseline models, showcasing its potential in citywide traffic prediction.

MISP-Fuse: A progressive fusion network guided by Multi-Information supervision

The fusion of infrared and visible images aims to synthesize an image that combines the advantageous characteristics of both image sources. Nevertheless, existing image fusion algorithms often struggle to extract relevant feature information from the source images and find it challenging to balance the significance of different modality information within the source images. This leads to suboptimal fusion results and inadequately serve advanced downstream visual tasks. In order to confront this challenge, a novel image fusion algorithm, denoted as Multiple Information Supervised Progressive Fusion Network (MISP-Fuse), has been proposed. Specifically, MISP-Fuse employs an innovative and robust multi-stage encoder-decoder network called Full Scale Feature Residual network (FSFR) to extract spatial context and detailed feature information corresponding to infrared and visible images. Within this encoder-decoder network, source image feature information is progressively extracted at different stages, which are recombined at multiple scales and distilled for crucial information. Finally, the fusion image is generated through a spatial localization module named as Spatial Localization Network (SLNet). MISP-Fuse incorporates a multi-information supervision mechanism to establish a linkage between different modality information in the infrared and visible source images. It ensures that the resulting fused image not only aligns with human visual perception but also effectively serves advanced downstream visual tasks. The comparative experiments utilizing diverse image fusion benchmark datasets has been conducted. In comparison to other algorithms, MISP-Fuse demonstrated significant enhancements in comprehensive image fusion metrics, including Average Gradient (AG), Sum of Correlated Differences (SCD), and Correlation Coefficient (CC).