With more and more in-depth research on deep learning algorithms in recent years, how to use deep learning method to detect remote sensing images is the key to improving the utilization efficiency of remote sensing data and realizing the transformation from data to knowledge. In this paper, an improved YOLO V3 algorithm is proposed to solve the problems of missed detection and false detection of the original YOLOv3 algorithm in remote sensing image target detection with different size and wide disparity in length and width ratio. first of all, K-means algorithm is used for clustering analysis of data set to obtain the position of anchor box; Secondly, the dilated convolution with expansion rate of 2 is used to replace the general convolution in the feature extraction part; Then four scales are used for prediction; Finally, the improved algorithm is applied to the recognition of bridges, harbors and airports. The results show that the detection performance of the algorithm is improved by about 2% compared with the original algorithm.