Remote sensing images are usually obtained from high-altitude observation. The spatial resolution of the images varies greatly and there are scale differences both between and within object classes, resulting in a diversified distribution of object scales. In order to solve these problems, we propose a novel object detection algorithm that maintains adaptability to multi-scale object detection based on feature interaction and Gaussian distribution in remote sensing images. The proposed multi-scale feature interaction model constructs feature interaction modules in the feature layer and spatial domain and combines them to fully utilize the spatial and semantic information of multi-level features. The proposed regression loss algorithm based on Gaussian distribution takes the normalized generalized Jensen–Shannon divergence with Gaussian angle loss as the regression loss function to ensure the scale invariance of the model. The experimental results demonstrate that our method achieves 77.29% mAP on the DOTA-v1.0 dataset and 97.95% mAP on the HRSC2016 dataset, which are, respectively, 1.12% and 1.41% higher than that of the baseline. These experimental results indicate the effectiveness of our method for object detection in remote sensing images.