Feature extraction method of a phase-sensitive optical time-domain reflectometer distributed optical fiber vibration detection system requires a priori knowledge. A lack of feature evaluation methods leads to a low pattern recognition accuracy. Traditional pattern recognition methods cannot be widely applied. This paper presents the implementation of a deep learning-based method to identify vibration signal categories. First, the vibration signal was reconstructed in the time and space domain, which is regarded as a pulse scanning image. Secondly, moving average was used to reduce noise, and seeking the signal envelope surface as an image sample. Finally, the image sample was inputted into the trained convolutional neural network (CNN) to obtain recognition results. Experiments showed that the phase-sensitive optical time-domain reflectometer pulse scanning imaging pattern recognition method based on deep learning proposed in this paper improved recognition accuracy while ensuring recognition efficiency. The algorithm is easy to implement and apply and satisfies the requirements of real-time online monitoring.