To solve the problem of the difficult and slow detection of small defects in magnetic flux leakage (MFL), we propose a fast magnetic flux leakage small defect detection network (FSDDNet). First, we introduce COMSOL data augmentation method that utilizes COMSOL simulation software to obtain high-resolution images of defects, which allows the network to capture complete defect features. Furthermore, this method addresses the issue of the relatively monotonic nature of the MFL defect data set used in the experiment. Deep learning networks usually use stride = 2 or max pooling to downsample the feature map, but this method will make the feature map lose some information, and small targets will lose more fine-grained information. Therefore, we introduce an SPD-Conv method to downsample the feature map, which can effectively avoid the loss of information. Meanwhile, an improved C3 network is introduced in the backbone network of FSDDNet. It greatly decreases the computational effort of the network and improves the detection speed. Finally, we add a small target detection head, which effectively improves the small target accuracy. FSDDNet is improved on the basis of YOLOv5, and after the above improvements, FSDDNet obtains very good results in the problem of MFL small defect detection. Experiments show that the accuracy of this algorithm is 95.2% when IOU = 0.5, and the latency is 7.9 ms.