In this paper, we propose a convolutional symmetric compressed look-up-table (CSC-LUT) method to accelerate computer-generated hologram (CGH) computation based on the Fresnel diffraction theory and LUT. The proposed method can achieve one-time high-quality fast generation of color holograms by utilizing dynamic convolution operation, which is divided three processes. Firstly, the pre-calculated data of maximum horizontal modulation factor is compressed in 1D array by coordinate symmetry. Then, the test object is resampled to satisfy convolutional translation invariance. Finally, the dynamic convolution operation is used to simplify CGH computation process rather than the point-by-point computation. Numerical simulation and optical experimental results show that our proposed method can achieve faster computation speed, higher reconstruction quality and wider application compared to conventional SC-LUT method. The further optimization method for parallel acceleration on the GPU framework can achieve real-time (>24fps) color holographic display corresponding to three perspectives of a 3D scene.