Traffic prediction is the core task of intelligent transportation system (ITS) and accurate traffic prediction can greatly improve the utilization of public resources. Dynamic interaction of multiple spatial relationships will influence the accuracy of traffic prediction. However, many existing methods only consider static spatial relationships, which restricts the accuracy of the prediction. To address the above problem, in this article, we propose the Dynamic Multi-Graph Fusion Network (DMGF-Net) to model the spatial-temporal correlations in traffic network. In the DMGF-Net, the fusion graph is designed to leverage and extract the various spatial correlations between different regions by fusing spatial graph, semantic graph, and spatial-semantic graph. Further, to dynamically learn the importance of different neighbors, we design the Dynamic Spatial-Temporal Unit (DSTU), which can adjust the aggregation weights of different neighbors by combining the convolution operation and the attention mechanism. It can selectively aggregate spatial-temporal features from different neighbors. Extensive experiments on three datasets demonstrate that effectiveness of our model, especially on PEMS08, our model achieves an increase of about 8.55% and 7.55% in terms of MAE and RMSE than the static model STGCN.