Ship detection of synthetic aperture radar (SAR) images is one of the research hotspots in the field of marine surveillance. Fusing salient features to detection network can effectively improve the precision of ship detection. However, how to effectively fuse the salient features of SAR images is still a difficult task. In this paper, to improve the ship detection precision, we design a novel one-stage ship detection network to fuse salient features and deep convolutional neural network (CNN) features. Firstly, a saliency map extraction algorithm is proposed. The algorithm is applied to generate saliency map by using multi-scale pyramid features and frequency domain features. Secondly, the backbone of the ship detection network contains a two-stream network. The upper-stream network uses the original SAR image as input to extract multi-scale deep CNN features. The lower-stream network uses the corresponding saliency map as input to acquire multi-scale salient features. Thirdly, for integrating the salient features to deep CNN features, a novel salient feature fusion method is designed. Finally, an improved bi-directional feature pyramid network is applied to the ship detection network for reducing the computational complexity and network parameters. The proposed methods are evaluated on the public ship detection dataset and the experimental results shows that it can make a significant improvement in the precision of SAR image ship detection.