Capsule network is a promising network architecture in computer vision that integrates feature information and feature relationships of images. However, the original capsule network performs poorly with complex datasets, and the network is computationally intensive, which is not conducive to deployment in end-side devices. In this paper, a novel hardware-friendly capsule network named MRCapsNet is proposed to improve the performance of the capsule network and make it deployable in end-side devices. In the proposed network, the multi-level residual capsule block is constructed to extract features with multi-granularity from images. Moreover, a novel reconstruction subnetwork is designed to facilitate network training. Experiments show that MRCapsNet achieved competitive results on the CIFAR-10 (90.34%) and SVHN (96.59%) datasets compared with other variants of CapsNet. Further, the hardware implementation scheme of MRCapsNet based on memristor crossbars is designed to provide a nano-scale, low-power capsule network deployment scheme. Finally, the power consumption of the core computing unit built by memristor elements is analyzed and calculated, and the maximum power consumption of a memristive neuron is only 6.4μW during the inference of the network, which is significantly less than the CMOS-based circuit.