The fluorescent magnetic particle inspection technique is often used for surface crack detection of bearing rings due to its advantages of simple operation and high sensitivity. With the development of computer vision technology, more and more visual algorithms are used in magnetic particle inspection for defect detection. However, most of these current algorithm models have low detection accuracy and poor efficiency, making it difficult to meet the precision requirements of production testing and affecting the overall pace of production processes. To address this problem, this paper proposes an improved algorithm model based on Yolov5. Firstly, MobileNetV3-small is utilized to construct the backbone feature extraction network, reducing the network’s parameter count and enhancing its detection speed. In addition, Bidirectional Feature Pyramid Network is implemented to facilitate swift and efficient multi-scale feature fusion, while the C3 module in the neck is replaced with C2f to enhance detection precision. Finally, Focal-Loss EIoU is adopted as the loss function to improve the model’s accuracy in positioning the crack borders. Experimental results demonstrate that the precision of this model in detecting surface cracks in bearing rings achieves an impressive 95.1%, while the recall reaches 90.4%. The mAP stands at 0.946. When compared to the original Yolov5s network, this model showcases a reduction in network parameters by 32.1% and a significant increase in frames per second by 40.0%. These improvements effectively fulfill the production process’s demands for crack detection tasks, providing a balance between accuracy and efficiency.