We study the sliding friction for viscoelastic solids, e.g., rubber, on hard flat substrate surfaces. We consider first the fluctuating shear stress inside a viscoelastic solid which results from the thermal motion of the atoms or molecules in the solid. At the nanoscale the thermal fluctuations are very strong and give rise to stress fluctuations in the MPa-range, which is similar to the depinning stresses which typically occur at solid-rubber interfaces, indicating the crucial importance of thermal fluctuations for rubber friction on smooth surfaces. We develop a detailed model which takes into account the influence of thermal fluctuations on the depinning of small contact patches (stress domains) at the rubber-substrate interface. The theory predicts that the velocity dependence of the macroscopic shear stress has a bell-shaped form, and that the low-velocity side exhibits the same temperature dependence as the bulk viscoelastic modulus, in qualitative agreement with experimental data. Finally, we discuss the influence of small-amplitude substrate roughness on rubber sliding friction.