This work provides an error compensation strategy based on deep learning to address the temperature error of a fiber optic gyroscope (FOG). The Attention structure was used to improve the Long Short-Term Memory Network (LSTM), and the improved network was utilized to establish the prediction and compensation model of FOG error. From the two perspectives of the learning ability of the neural network and the improvement of FOG performance, this paper selects three indicators: prediction accuracy, mean square error, and FOG bias stability to comprehensively evaluate the performance of the compensation model. Through comparison, the compensation effect is significantly enhanced.