In industrial production, automated surface defect detection holds paramount importance. While current techniques can handle complex background power adapter surface defect detection with high precision, they have many parameters, low speed, slow convergence, and complex post-processing (such as non-maximum suppression). To this end, this study aims to develop a more efficient and effective method for surface defect detection. The primary objective of this research is to enhance the speed of detection without sacrificing accuracy. This paper designs the Separable-reparam Feature Module (SFM), which is based on depth-wise convolution, to fuse multi-branch features through reparameterization techniques. SFM has low parameters and excels in feature extraction without increasing the model’s Memory Access Cost (MAC). SFM combines cross-stage fusion to construct the backbone, enabling the extraction of multi-scale features. Following the standard query decoder architecture, a QRNet was built to learn the query set of objects, and predict the surface defect set for power adapter. Additionally, a novel two-stage mixed-supervised transfer-learning approach is proposed to expedite the convergence process. Experimental results on the power adapter defect dataset demonstrate QRNet’s remarkable performance. It achieved a mean average precision (mAP) of 98.94 % with just 0.841 M parameters and a latency of 5.72 ms. In conclusion, QRNet represents an advancement in surface defect detection, offering a balance between accuracy and efficiency.