Spatial Modulator Research Articles

Convolutional Neural Network Incorporating Multiple Attention Mechanisms for MRI Classification of Lumbar Spinal Stenosis.

Lumbar spinal stenosis (LSS) is a common cause of low back pain, especially in the elderly, and accurate diagnosis is critical for effective treatment. However, manual diagnosis using MRI images is time consuming and subjective, leading to a need for automated methods. This study aims to develop a convolutional neural network (CNN)-based deep learning model integrated with multiple attention mechanisms to improve the accuracy and robustness of LSS classification via MRI images. The proposed model is trained on a standardized MRI dataset sourced from multiple institutions, encompassing various lumbar degenerative conditions. During preprocessing, techniques such as image normalization and data augmentation are employed to enhance the model's performance. The network incorporates a Multi-Headed Self-Attention Module, a Slot Attention Module, and a Channel and Spatial Attention Module, each contributing to better feature extraction and classification. The model achieved 95.2% classification accuracy, 94.7% precision, 94.3% recall, and 94.5% F1 score on the validation set. Ablation experiments confirmed the significant impact of the attention mechanisms in improving the model's classification capabilities. The integration of multiple attention mechanisms enhances the model's ability to accurately classify LSS in MRI images, demonstrating its potential as a tool for automated diagnosis. This study paves the way for future research in applying attention mechanisms to the automated diagnosis of lumbar spinal stenosis and other complex spinal conditions.

Short-Term Prediction of Origin–Destination Passenger Flow in Urban Rail Transit Systems with Multi-Source Data: A Deep Learning Method Fusing High-Dimensional Features

Short-term origin–destination (OD) passenger flow forecasting is crucial for urban rail transit enterprises aiming to optimise transportation products and increase operating income. As there are large-scale OD pairs in an urban rail transit system, OD passenger flow cannot be obtained in real time (temporal hysteresis). Additionally, the distribution characteristics are also complex. Previous studies mainly focus on passenger flow prediction at metro stations, while few methods solve the OD passenger flow prediction problems of an urban rail transit system. In view of this, we propose a novel deep learning method fusing high-dimensional features (HDF-DL) with multi-source data. The HDF-DL method is combined with three modules. The temporal module incorporates the time-varying, trend, and cyclic characteristics of OD passenger flow, while the latest OD passenger flow time sequence (within 1 h) is excluded from the time-varying characteristics. In the spatial module, the K-means and K-shape algorithms are used to classify OD pairs from multiple perspectives and capture the spatial features, reducing the difficulty of OD passenger flow predictions with large-scale and complex characteristics. Weather factors are considered in the external feature module. The HDF-DL method is tested on a large-scale metro system in China, in which eight baseline models are designed. The results show that the HDF-DL method achieves high prediction accuracy across multiple time granularities, with a mean absolute percentage error of about 10%. OD passenger flow in every departure time interval can be predicted with high and stable accuracy, effectively capturing temporal characteristics. The modular design of HDF-DL, which fuses high-dimensional features and employs appropriate neural networks for different data types, significantly reduces prediction errors and outperforms baseline models.

Wolf and wild boar in the Alps: Trophic, temporal and spatial interactions in an Alpine protected area

In predator-prey systems, interspecific interactions at the spatial and temporal scale may affect different components of the ecosystem. Despite a growing interest in the topic, little is known about these interactions in many regions, particularly in areas recently recolonized by apex predators, such as the Alps. Using a multidimensional approach, we assessed wolf-wild boar interactions in a recently recolonized protected area of the Western Alps (Gran Paradiso National Park, Italy) by combining camera trapping and wolf scat analyses. First, we quantified the importance of wild boar in the wolf diet by comparing two different periods (October 2018–September 2020 vs. October 2021–September 2022). Second, we evaluated prey selection in 2021–2022, when data on prey availability were collected. Then, we assessed spatiotemporal interactions between the wild boar and the wolf. Finally, we calculated combined spatiotemporal associations between the wolf and the wild boar in comparison to other ungulate prey. We found a remarkable increase in wild boar consumption between the two periods; roe deer and wild boar were consumed in proportion to their availability, while the chamois was positively selected and the red deer and the ibex were less consumed in relation to their availability. Although we found no negative association between wolf and wild boar detection rates, in winter the wild boar limited its temporal overlap with the predator by being less active in sites more frequently used by the wolf, suggesting some spatial modulation of a temporal antipredator response. Among ungulate species, the wild boar had the highest combined spatiotemporal association with the wolf, particularly in autumn-winter. Our results suggest complex interspecific interactions in alpine areas recently recolonized by the wolf.

EENet: An effective and efficient network for single image dehazing

While numerous solutions leveraging convolutional neural networks and Transformers have been proposed for image dehazing, there remains significant potential to improve the balance between efficiency and reconstruction performance. In this paper, we introduce an efficient and effective network named EENet, designed for image dehazing through enhanced spatial–spectral learning. EENet comprises three primary modules: the frequency processing module, the spatial processing module, and the dual-domain interaction module. Specifically, the frequency processing module handles Fourier components individually based on their distinct properties for image dehazing while also modeling global dependencies according to the convolution theorem. Additionally, the spatial processing module is designed to enable multi-scale learning. Finally, the dual-domain interaction module promotes information exchange between the frequency and spatial domains. Extensive experiments demonstrate that EENet achieves state-of-the-art performance on seven synthetic and real-world datasets for image dehazing. Moreover, the network’s generalization ability is validated by extending it to image desnowing, image defocus deblurring, and low-light image enhancement.

MultiScale spectral–spatial convolutional transformer for hyperspectral image classification

AbstractDue to the powerful ability in capturing the global information, transformer has become an alternative architecture of CNNs for hyperspectral image classification. However, general transformer mainly considers the global spectral information while ignores the multiscale spatial information of the hyperspectral image. In this paper, we propose a multiscale spectral–spatial convolutional transformer (MultiFormer) for hyperspectral image classification. First, the developed method utilizes multiscale spatial patches as tokens to formulate the spatial transformer and generates multiscale spatial representation of each band in each pixel. Second, the spatial representation of all the bands in a given pixel are utilized as tokens to formulate the spectral transformer and generate the multiscale spectral–spatial representation of each pixel. Besides, a modified spectral–spatial CAF module is constructed in the MultiFormer to fuse cross‐layer spectral and spatial information. Therefore, the proposed MultiFormer can capture the multiscale spectral–spatial information and provide better performance than most of other architectures for hyperspectral image classification. Experiments are conducted over commonly used real‐world datasets and the comparison results show the superiority of the proposed method.

Multistage Spatial-Spectral Fusion Network for Spectral Super-Resolution.

Spectral super-resolution (SSR) aims to restore a hyperspectral image (HSI) from a single RGB image, in which deep learning has shown impressive performance. However, the majority of the existing deep-learning-based SSR methods inadequately address the modeling of spatial-spectral features in HSI. That is to say, they only sufficiently capture either the spatial correlations or the spectral self-similarity, which results in a loss of discriminative spatial-spectral features and hence limits the fidelity of the reconstructed HSI. To solve this issue, we propose a novel SSR network dubbed multistage spatial-spectral fusion network (MSFN). From the perspective of network design, we build a multistage Unet-like architecture that differentially captures the multiscale features of HSI both spatialwisely and spectralwisely. It consists of two types of the self-attention mechanism, which enables the proposed network to achieve global modeling of HSI comprehensively. From the perspective of feature alignment, we innovatively design the spatial fusion module (SpatialFM) and spectral fusion module (SpectralFM), aiming to preserve the comprehensively captured spatial correlations and spectral self-similarity. In this manner, the multiscale features can be better fused and the accuracy of reconstructed HSI can be significantly enhanced. Quantitative and qualitative experiments on the two largest SSR datasets (i.e., NTIRE2022 and NTIRE2020) demonstrate that our MSFN outperforms the state-of-the-art SSR methods. The code implementation will be uploaded at https://github.com/Matsuri247/MSFN-for-Spectral-Super-Resolution.

Error performance of DF cooperative SMTs with I/Q imbalance over Beckmann fading channels

The direct down-conversion principle, which has generally been used in the design of multiple-input multiple-output (MIMO) schemes, including space modulation techniques (SMTs), is attractive to researchers because of its low cost, low power consumption with fewer components, flexible and simple structure. However, hardware imperfections such as in-phase (I) and quadrature-phase (Q) imbalance (IQI) negatively affect the performance of the systems with direct down-conversion in practice. On the other hand, cooperative communication is a promising technology that can be utilized in the design of future wireless networks due to its significant advantages such as increasing system reliability, extending network coverage, reducing channel degradation, and providing high quality of service. In this study, SMT-based methods are integrated into cooperative systems, and a flexible and comprehensive model is presented that is applicable to many channel structures. Specifically, the error performance analysis of space shift keying (SSK), spatial modulation (SM), and quadrature SM (QSM) systems in the presence of IQI in decode-and-forward (DF) cooperative communication is carried out by analytical derivations and computer simulations over generalized Beckmann fading channels. The obtained results show that the performance of SMT-based DF cooperative systems is superior to the conventional schemes, and the effects of receiver IQI can be eliminated by optimal detector designs.

A lightweight object detection approach based on edge computing for mining industry

AbstractCoal Mining enterprises deploy numerous monitoring devices to ensure safe and efficient production using target detection technologies. However, deploying deep detection models on edge devices poses challenges due to high computational loads, impacting detection speed and accuracy. A mining target detection dataset has been created to address these issues, featuring key targets in coal mining scenes such as miners, safety helmets, and coal gangue. A model is proposed to improve real‐time performance for edge mining detection tasks. Detection performance is enhanced by incorporating a Pixel‐wise Normalization Spatial Attention Module (PN‐SAM) into the MobileNet‐v3 bneck structure and replacing the h‐swish activation function with Mish, providing more prosperous gradient information transfer. The proposed model, YOLO‐v4‐LSAM, shows a 3.2% mAP improvement on the VOC2012 dataset and a 2.4% improvement on the mining target dataset compared to YOLO‐v4‐Tiny, demonstrating its effectiveness in mining environments. These enhancements enable more accurate and efficient detection in resource‐constrained edge environments, contributing to safer and more reliable monitoring in coal mining operations.

Equilibrium Optimization-Based Ensemble CNN Framework for Breast Cancer Multiclass Classification Using Histopathological Image.

Background: Breast cancer is one of the most lethal cancers among women. Early detection and proper treatment reduce mortality rates. Histopathological images provide detailed information for diagnosing and staging breast cancer disease. Methods: The BreakHis dataset, which includes histopathological images, is used in this study. Medical images are prone to problems such as different textural backgrounds and overlapping cell structures, unbalanced class distribution, and insufficiently labeled data. In addition to these, the limitations of deep learning models in overfitting and insufficient feature extraction make it extremely difficult to obtain a high-performance model in this dataset. In this study, 20 state-of-the-art models are trained to diagnose eight types of breast cancer using the fine-tuning method. In addition, a comprehensive experimental study was conducted to determine the most successful new model, with 20 different custom models reported. As a result, we propose a novel model called MultiHisNet. Results: The most effective new model, which included a pointwise convolution layer, residual link, channel, and spatial attention module, achieved 94.69% accuracy in multi-class breast cancer classification. An ensemble model was created with the best-performing transfer learning and custom models obtained in the study, and model weights were determined with an Equilibrium Optimizer. The proposed ensemble model achieved 96.71% accuracy in eight-class breast cancer detection. Conclusions: The results show that the proposed model will support pathologists in successfully diagnosing breast cancer.

Spatial–Temporal-Correlation-Constrained Dynamic Graph Convolutional Network for Traffic Flow Forecasting

Accurate traffic flow prediction in road networks is essential for intelligent transportation systems (ITS). Since traffic data are collected from the road network with spatial topological and time series sequences, the traffic flow prediction is regarded as a spatial–temporal prediction task. With the powerful ability to model the non-Euclidean data, the graph convolutional network (GCN)-based models have become the mainstream framework for traffic forecasting. However, existing GCN-based models either use the manually predefined graph structure to capture the spatial features, ignoring the heterogeneity of road networks, or simply perform 1-D convolution with fixed kernel to capture the temporal dependencies of traffic data, resulting in insufficient long-term temporal feature extraction. To solve those issues, a spatial–temporal correlation constrained dynamic graph convolutional network (STC-DGCN) is proposed for traffic flow forecasting. In STC-DGCN, a spatial–temporal embedding encoder module (STEM) is first constructed to encode the dynamic spatial relationships for road networks at different time steps. Then, a temporal feature encoder module with heterogeneous time series correlation modeling (TFE-HCM) and a spatial feature encoder module with dynamic multi-graph modeling (SFE-DCM) are designed to generate dynamic graph structures for effectively capturing the dynamic spatial and temporal correlations. Finally, a spatial–temporal feature fusion module based on a gating fusion mechanism (STM-GM) is proposed to effectively learn and leverage the inherent spatial–temporal relationships for traffic flow forecasting. Experimental results from three real-world traffic flow datasets demonstrate the superior performance of the proposed STC-DGCN compared with state-of-the-art traffic flow forecasting models.

Automated Recognition of Snow-Covered and Icy Road Surfaces Based on T-Net of Mount Tianshan

The Tianshan Expressway plays a crucial role in China’s “Belt and Road” strategy, yet the extreme climate of the Tianshan Mountains poses significant traffic safety risks, hindering local economic development. Efficient detection of hazardous road surface conditions (RSCs) is vital to address these challenges. The complexity and variability of RSCs in the region, exacerbated by harsh weather, make traditional surveillance methods inadequate for real-time monitoring. To overcome these limitations, a vision-based artificial intelligence approach is urgently needed to ensure effective, real-time detection of dangerous RSCs in the Tianshan road network. This paper analyzes the primary structures and architectures of mainstream neural networks and explores their performance for RSC recognition through a comprehensive set of experiments, filling a research gap. Additionally, T-Net, specifically designed for the Tianshan Expressway engineering project, is built upon the optimal architecture identified in this study. Leveraging the split-transform-merge structure paradigm and asymmetric convolution, the model excels in capturing detailed information by learning features across multiple dimensions and perspectives. Furthermore, the integration of channel, spatial, and multi-head attention modules enhances the weighting of key features, making the T-Net particularly effective in recognizing the characteristics of snow-covered and icy road surfaces. All models presented in this paper were trained on a custom RSC dataset, compiled from various sources. Experimental results indicate that the T-Net outperforms fourteen once state-of-the-art (SOTA) models and three models specifically designed for RSC recognition, with 97.44% accuracy and 9.79% loss on the validation set.

A Lightweight Model Enhancing Facial Expression Recognition with Spatial Bias and Cosine-Harmony Loss

This paper proposes a facial expression recognition network called the Lightweight Facial Network with Spatial Bias (LFNSB). The LFNSB model effectively balances model complexity and recognition accuracy. It has two key components: a lightweight feature extraction network (LFN) and a Spatial Bias (SB) module for aggregating global information. The LFN introduces combined channel operations and depth-wise convolution techniques, effectively reducing the number of parameters while enhancing feature representation capability. The Spatial Bias module enables the model to focus on local facial features while capturing the dependencies between different facial regions. Additionally, a new loss function called Cosine-Harmony Loss is designed. This function optimizes the relative positions of feature vectors in high-dimensional space, resulting in better feature separation and clustering. Experimental results on the AffectNet and RAF-DB datasets demonstrate that the LFNSB model achieves competitive recognition accuracy, with 63.12% accuracy on AffectNet-8, 66.57% accuracy on AffectNet-7, and 91.07% accuracy on RAF-DB, while significantly reducing the model complexity.

Secrecy Rate Bounds in Spatial Modulation-Based Visible Light Communications under Signal-Dependent Noise Conditions

This study examines the physical-layer security of an indoor visible light communication (VLC) system using spatial modulation (SM), which consists of several transmitters, an authorized receiver, and a passive adversary. The SM technique is applied at the transmitters so that only one transmitter is operational at any given time. A uniform selection (US) strategy is employed to choose the active transmitter. The two scenarios under examination encompass the conditions of non-negativity and average optical intensity, as well as the conditions of non-negativity, average optical intensity, and peak optical intensity. The secrecy rate is then obtained for these two scenarios while accounting for both signal-independent noise and signal-dependent noise. Additionally, the high signal-to-noise ratio (SNR) asymptotic behavior of the derived secrecy rate constraints is investigated. A channel-adaptive selection (CAS) strategy and a greedy selection (GS) scheme are utilized to select the active transmitter, aiming to enhance the secrecy performance. The current numerical findings affirm a pronounced convergence between the lower and upper bounds characterizing the secrecy rate. Notably, marginal asymptotic differentials in performance emerge at elevated SNRs. Furthermore, the GS system outperforms the CAS scheme and the US method, in that order. Additionally, the impact of friendly optical jamming on the secrecy rate is investigated. The results show that optical jamming significantly enhances the secrecy rate, particularly at higher power levels.

Multimodal vehicle trajectory prediction and integrated threat assessment algorithm based on adaptive driving intention

As autonomous driving and connected communication technologies advance swiftly, vehicle trajectory prediction has become increasingly significant. The motion of a vehicle is contingent not only on its historical trajectory but is also subject to the influence of surrounding vehicles, thereby exhibiting intricate social and temporal interdependencies. Furthermore, the inherent randomness and uncertainty in driver behavior render vehicle trajectory prediction inherently multimodal, a factor that is frequently neglected in current research. Against this backdrop, a multimodal vehicle trajectory prediction (MTP) model based on an encoder-decoder architecture is proposed to hierarchically extract historical features of vehicles. The model consists of five key components: temporal feature encoder module, spatial interaction module, spatial-temporal dependence module, driving intention fusion module and multimodal trajectory output module. Experiments on the NGSIM dataset show that the predictive performance of the model has been improved to varying degrees, especially at 3–5 s, where the improvement is more significant. Compared with state-of-the-art models, the Root Mean Square Error (RMSE) error at 5 s time horizon is 3.38 m on NGSIM dataset, which represents a 25 % improvement. To measure the safety of predicted trajectories, we propose a comprehensive threat assessment model that combines collision time (TTC), headway (TH) and time to lateral collision (TLC) metrics based on safe distance theory. This model not only evaluates the longitudinal collision threat in the following state, but also evaluates the lateral collision threat during driving maneuvers in multi lane scenarios, thereby comprehensively improving the safety of vehicle driving. This research also offers new perspectives and insights for the development of autonomous driving.

Exploring nonlinear optical properties of RbPbI3 nanorods using SSPM technique: Implications for all-optical switching applications

Nowadays, nanomaterials' nonlinear optical (NLO) properties are a fascinating topic for developing a novel all-optical switching application. Spatial self-phase modulation (SSPM) is an important technique to determine the laser-induced coherent light-matter interaction to determine the NLO properties of nanomaterials. This study successfully synthesized RbPbI3 nanorods (NRs) via a low-cost, less complicated chemical route with all the basic characterizations to confirm its structural and compositional analysis. For the first time, we conducted a comprehensive analysis of the nonlinear refractive index (n2) and third-order susceptibility (χ(3)) for RbPbI3 NRs dissolved in two different solvents, namely N-Methyl-2-pyrrolidone (NMP) and 2-propanol, utilizing the SSPM technique. Our investigation revealed significant nonlinear optical (NLO) responses with n2 = 6.19 × 10−5 cm2 W−1 and χmonolaer(3) = 2.38 × 10−8 esu for RbPbI3. We elucidated the factors contributing to this pronounced NLO behaviour, including variations in parameters such as solvent type, viscosity, wavelength, and cuvette length. Additionally, we successfully demonstrated optical information transfer through an all-optical switching application. This research sheds light on the NLO properties of Perovskite materials and holds promise for the development of advanced all-optical device applications.

Retinal fundus image super-resolution based on generative adversarial network guided with vascular structure prior

Many ophthalmic and systemic diseases can be screened by analyzing retinal fundus images. The clarity and resolution of retinal fundus images directly determine the effectiveness of clinical diagnosis. Deep learning methods based on generative adversarial networks are used in various research fields due to their powerful generative capabilities, especially image super-resolution. Although Real-ESRGAN is a recently proposed method that excels in processing real-world degraded images, it suffers from structural distortions when super-resolving retinal fundus images are rich in structural information. To address this shortcoming, we first process the input image using a pre-trained U-Net model to obtain a structural segmentation map of the retinal vessels and use the segmentation map as the structural prior. The spatial feature transform layer is then used to better integrate the structural prior into the generation process of the generator. In addition, we introduce channel and spatial attention modules into the skip connections of the discriminator to emphasize meaningful features and accordingly enhance the discriminative power of the discriminator. Based on the original loss functions, we introduce the L1 loss function to measure the pixel-level differences between the segmentation maps of retinal vascular structures in the high-resolution images and the super-resolution images to further constrain the super-resolution images. Simulation results on retinal image datasets show that our improved algorithm results have a better visual performance by suppressing structural distortions in the super-resolution images.

Reconfigurable Intelligent Surface-Assisted Generalized Spatial Modulation With Phase Offset: Efficient Designs for Single-RF MIMO Systems

Reconfigurable Intelligent Surface-Assisted Generalized Spatial Modulation With Phase Offset: Efficient Designs for Single-RF MIMO Systems

RIS-Assisted Receive Generalized Space-Shift Keying and Receive Generalized Spatial Modulation

RIS-Assisted Receive Generalized Space-Shift Keying and Receive Generalized Spatial Modulation

Deep Learning Based Signal Detection for Quadrature Spatial Modulation System

Deep Learning Based Signal Detection for Quadrature Spatial Modulation System

Golden Angle Modulation Aided Fractional Spatial Modulation: System Design and Performance Optimization

Golden Angle Modulation Aided Fractional Spatial Modulation: System Design and Performance Optimization