With the explosion of the number of meteoroid/orbital debris in terrestrial space in recent years, the detection environment of spacecraft becomes more complex. This phenomenon causes most current detection methods based on machine learning intractable to break through the two difficulties of solving scale transformation problem of the targets in image and accelerating detection rate of high-resolution images. To overcome the two challenges, we propose a novel non-cooperative target detection method using the framework of deep convolutional neural network.Firstly, a specific spacecraft simulation dataset using over one thousand images to train and test our detection model is built. The deep separable convolution structure is applied and combined with the residual network module to improve the network’s backbone. To count the different shapes of the spacecrafts in the dataset, a particular prior-box generation method based on K-means cluster algorithm is designed for each detection head with different scales. Finally, a comprehensive loss function is presented considering category confidence, box parameters, as well as box confidence. The experimental results verify that the proposed method has strong robustness against varying degrees of luminance change, and can suppress the interference caused by Gaussian noise and background complexity. The mean accuracy precision of our proposed method reaches 93.28%, and the global loss value is 13.252. The comparative experiment results show that under the same epoch and batchsize, the speed of our method is compressed by about 20% in comparison of YOLOv3, the detection accuracy is increased by about 12%, and the size of the model is reduced by nearly 50%.