Low-velocity impacts can cause microscopic and invisible damage to carbon fiber reinforced polymer (CFRP) structures, potentially compromising their integrity and leading to catastrophic failures. Therefore, obtaining precise information about the impact location is crucial for monitoring the health of CFRP structures. In this paper, an impact localization system for CFRP structures was developed by using fiber Bragg grating (FBG) sensors, and impact signals detected by FBG sensors are demodulated by edge-filtering at high speed. An impact localization method of CFRP structure based on CNN-LSTM-Attention is proposed. The time difference of arrival (TDOA) between signals from different FBG sensors are collected to characterize the impact location, and attention mechanism is introduced into the CNN-LSTM model to augment the significance of TDOA of impact signal detected by proximal FBG sensors. The model is trained using the training set, its parameters are optimized using the validation set and the localization performance of different models are compared by the test set. The proposed impact localization method based on CNN-LSTM-Attention model was verified on a CFRP plate with an experiment area of 400 mm*400 mm. Experimental results prove the effectiveness and satisfactory performance of the proposed method.