Single image deraining, as a low-level computer vision task, has been drawn extensive attention in recent years. Rain streaks can degrade subjective visibility quality, meanwhile, bring significant difficulties to subsequent high-level computer vision tasks such as object detection. Nowadays, deep-learning based methods, specifically Convolutional Neural Networks (CNN) based ones are adopted to remove the rain streaks and become the state-of-the-art. However, existing popular deep-networks have complicated branches and numerous layers, which strengthen the ability of removing rain streaks and result in high memory and computational cost inevitably. This restricts many applications in real-time and limited computation resource scene, especially on mobile or edge devices. To handle this issue, this paper proposes a novel Lightweight Recursive Pyramid network (LRP-Net) with a small number of parameters for single image deraining. To begin with, we propose a novel Lightweight Pyramid Deraining (LPD) block which consists of a multi-scale pyramid convolution for sufficient feature extraction and a pointwise convolution for feature fusion. Meanwhile, we also design a novel group convolution strategy in LPD for the sake of remarkable parameter reduction. Secondly, we combine a recursive deraining mechanism, a critical component that serves as a feature fusion iterator to construct our LRPNet a powerful and lightweight multi-stage model. In the benefit of the combination between the LPD block and recursive mechanism, the total number of parameters in LRP-Net is only 130 k, which is nearly a 40- reduction compared with the latest state-of-the-art models. The extensive experiments demonstrate the superiority of LRP-Net in both quantitative assessments and visual quality.