Since the emergence of human beings on the earth, various disasters have been accompanied. Among many natural disasters, the earthquake is undoubtedly one of the most threatening disasters. This project uses Res Net-50 model for deep learning and image recognition of building structural damage. Through the program to assess the local earthquake damage, given the feasible standards to facilitate a unified understanding of the earthquake situation, thereby improving the efficiency of disaster relief. Through experiments, the accuracy of the training set of the two classifications finally reached about 89.3 %, and the prediction accuracy of the test set finally reached about 71.4 %, Through the identification of post-earthquake building damage in Songyuan area, it can be learned that the accuracy of the software identification binary classification task is 73.21 %. Experiments show that taking photos can be used to predict the damage level of buildings in a certain area, and seismic damage identification can provide basis and support for post-disaster rescue and reconstruction and economic loss assessment.