Rice leaf diseases is a threat to sustainable rice production. Many methods via machine learning or artificial intelligence have been used to detect rice leaf diseases. However, these methods may fail to identify them or are slow in recognition. Therefore, a YOLO V5-IMPROVEMENT model is proposed to overcome these issues. Based on YOLO V5s, the improved K-Means is utilized to generate initial anchor sizes which more fit in the experimental dataset. An attention mechanism is added to let the feature extraction layer focus more on Regions of Interest (ROIs). This improvement enhances the efficiency of feature information extraction from objects of different scales. To better detect occluded targets, the loss function is improved to strengthen the regression effect of the predicted bounding boxes. Main branch gradient flow BottleNeck Block module of Cross Stage Partial (CSP) in the Neck is improved. The Spatial Pyramid Pooling-Fast (SPPF) is replaced with the S-SPPF module to optimize the model structure and ensure real-time performance. An ablation study was conducted and the proposed YOLO V5-EFFICIENT was compared with other models. Compared with YOLO V5, the detection result of YOLO V5-EFFICIENT increased by 9.90% in terms of mAP. Compared with other approaches, the improved model can better balance detection accuracy and speed when recognizing fine features of lesions. And it is capable of handling rice leaf disease detection tasks with numerous small targets. It is suggested that YOLO V5-EFFICIENT is of high accuracy and robustness.