Rain adversely affects the performance of collaborative robots in outdoor applications. In machine vision, single image rain removal is an extremely difficult problem due to the disordered and irregular rain streaks in the image. Existing methods either fail to achieve satisfactory rain removal results or destroy image details. In this paper, we propose a novel multi-scale rain removal model to address these problems by decomposing images into base layers and detail layers. The proposed method adapts a two-branch squeeze-and-excitation residual network architecture that learns the basic structure and texture details of the corresponding clean image. By decomposing the image into multiple layers and merging these layers, the network can effectively remove rain streaks from an image to restore its structural information and details. Extensive experiments on synthetic and real datasets demonstrate that the proposed method significantly outperforms recent state-of-the-art algorithms in terms of both qualitative and quantitative measures.