The detection of internal damage characteristics of concrete is an important aspect of damage evolution mechanism in concrete meso-structure. In this paper, the improved Faster R-CNN is used to detect the porosity and cracks in concrete CT images. Based on the Faster R-CNN, ResNet-101 and ResNet-50 are used as the main framework. Feature pyramid network (FPN) and ROI Align are introduced to improve the performance of the model. FPN can generate high quality feature maps. ROI Align solves the region mismatch caused by the quantization operation. Experiments show that the detection accuracy of ResNet-101[Formula: see text]+[Formula: see text]FPN[Formula: see text]+[Formula: see text]ROI Align reaches 87.08%, which is 4.74 higher than that of ResNet-101. The detection accuracy of ResNet-50 [Formula: see text]+[Formula: see text] FPN [Formula: see text]+[Formula: see text] ROI Align reached 81.36%, which is 3.12% points higher than ResNet-50. These two improved algorithms are slower than the original algorithm for the detection time of a single picture. An effective method is provided to analyze concrete meso-damage evolution through the research.