Safety helmet detection is a hot topic of research in the field of industrial safety for object detection technology. Existing object detection methods still face great challenges for the detection of small-scale safety helmet object. In this paper, we propose a safety helmet detection method based on the fusion of semantic guidance and feature selection. The method is able to consider the balance between detection performance and efficiency. First, a multi-scale non-local module is proposed to establish internal correlations between different scales of deep image features as well as to aggregate semantic context information to guide the information recovery of decoder network features. Then the feature selection fusion structure is proposed to adaptively select deep features and underlying key features for fusion to make up for the missing semantic and spatial detail information of the decoding network and improve the spatial location expression capability of the decoding network. Experimental analysis shows that the method in this paper has good detection performance on the expanded safety helmet wearing dataset with 5.12% improvement in mAP compared to the baseline method CenterNet, and 6.11% improvement in AP for the safety helmet object.