Multi-modal fusion for millimeter-wave communication systems: A spatio-temporal enabled approach

Abstract

In millimeter-wave (mmWave) massive multi-input multi-output (MIMO) systems, beam selection can enhance channel capacity and reduce error rate. However, existing beam selection methods for MIMO systems rely on traditional optimization techniques, which may not be feasible for real-time data transmission. Hence, this paper proposes a novel beam prediction approach via multi-modal fusion and spatio-temporal-enabled features. Specifically, the proposed method can improve MIMO system performance by integrating information from multiple perspectives, such as temporal, spatial, and frequency domains. Moreover, the proposed approach is based on a deep learning framework utilizing 3-dimensional convolutional neural networks (3DCNNs) and transformer module to capture spatio-temporal data correlations. Extensive simulation results show that the proposed mechanism outperforms massive benchmarks in terms of beam prediction accuracy from 2.63% to 11%.