The gross mechanical efficiency of the manual wheelchair propulsion movement is particularly low compared to other movements. The energy losses in the manual wheelchair propulsion movement are partly due to energy losses associated with the wheelchair, and especially to the rolling resistance of the wheels. The distribution of mass between the front rear wheels and the caster wheels has a significant impact on the rolling resistance. The study of the caster wheels cannot therefore be neglected due to their involvement in rolling resistance. Thus, this study aimed to evaluate the power dissipated due to rolling resistance by different caster wheels, at different speeds and under different loadings on various terrains. Four caster wheels of different shapes, diameters, and materials were tested on two surfaces representative of indoor sports surfaces at four different speeds and under four loadings. The results showed a minimal dissipated power of 0.4 ± 0.2 W for the skate caster, on the parquet, at 0.5 m/s and under a loading of 50 N. The maximal mean power dissipated was 43.3 ± 27.6 W still for the skate caster, but on the Taraflex, at 1.5 m/s and under loading of 200 N. The power dissipated on the parquet was lower than the one on the Taraflex. The Spherical and Omniwheel caster wheels dissipated less power than the two other casters. This study showed that caster wheels cannot be neglected in the assessment of gross mechanical efficiency, particularly in light of the power dissipated by athletes during propulsion.