To ensure the safety and durability of concrete structures, timely detection and classification of concrete cracks using a low-cost and high-efficiency method is necessary. In this study, a concrete surface crack damage detection method based on the ResNet-18 residual network was developed. This method was implemented by training a model with images to extract the cracks, where the image processing algorithms and deep learning were combined. The results show that the computational accuracy can meet the requirements by utilizing the established image dataset and appropriate model hyperparameters. The trained model had high recognition accuracy when the 256 × 256 resolution images were adopted, and the worst accuracy of crack recognition in the test set was over 90%. The average accuracy in the test set was 91.3% when considering environmental interference generated by processing the images with a brightness adjustment, salt-and-pepper noise, and localized interference. Then, it was demonstrated that the present model possesses good robustness for crack identification in different environments. The average recognition accuracy when dealing with images of a real bridge, which are outside the training dataset, was 99.7%. The residual network model developed in this study has the advantages of low cost, high efficiency, and practicality compared to traditional detection methods. Compared to the existing deep learning methods, the model created in this study requires less computational resources and storage space, and shows a faster training speed and higher accuracy.