There are problems of low model detection accuracy, low detection speed and difficulty in deploying online inspection in industrial surface defect detection relying on deep learning object detection algorithms. In order to effectively solve this problem, an efficient channel attention-enhanced lightweight neural network model named as EMV2-YOLOX is proposed in this paper. The algorithm incorporates the ECA module into the lightweight backbone extraction network MobileNetV2 to achieve adaptive adjustment of channel information weights, which can improve the extraction capability of the algorithm. The YOLOX model is also introduced to enhance the model’s identification and localization of tiny defects. The improved algorithm can guarantee the model’s accuracy and improve the model detection performance, as well as the carrying capacity of hardware devices. The experimental results show that the highest accuracy is achieved on the GCT10 and NEU public defect datasets with mean Average Precision values of 0.86 and 0.68, respectively, which is higher than the accuracy of the EMV2yoloV4 model. The parametric model number is only 10.24[Formula: see text]M in size, and the detection rate is 54.25[Formula: see text]f/s, which is the highest performance in embedded devices. EMV2-YOLOX, combined with the attention mechanism, can efficiently extract the location and semantic information of hard-to-detect defects and plays a vital role in the intelligent detection methods.