The charge dynamics of the spin ladder compound ${\mathrm{Sr}}_{14\ensuremath{-}x}{\mathrm{Ca}}_{x}{\mathrm{Cu}}_{24}{\mathrm{O}}_{41}$ with $x=0,6,13.6$ has been investigated using wavelength- and temperature-dependent phonon Raman scattering on single crystals. In the unsubstituted, $x=0$, compound, a set of sharp two-phonon lines shows a strong increase in intensity below $T\ensuremath{\sim}200\phantom{\rule{0.3em}{0ex}}\mathrm{K}$ for light polarized along the ladder layer and with excitation energy close to the charge transfer gap $\ensuremath{\sim}1.9\phantom{\rule{0.3em}{0ex}}\mathrm{eV}$. The temperature dependence of the strongly enhanced two-phonon bands below $200\phantom{\rule{0.3em}{0ex}}\mathrm{K}$ closely follows the recently reported formation of a standing charge density wave in the ladders [P. Abbamonte et al., Nature 431, 1078 (2004)]. Upon calcium substitution the polarized resonant Raman response rapidly decreases, signaling an increase of hole mobility in the ladder units. Temperature-dependent measurements of the $x=13.6$ sample indicate mobility of holes down to $<8\phantom{\rule{0.3em}{0ex}}\mathrm{K}$.