A theory is developed of Brownian motion in granular gases (systems of many macroscopic particles undergoing inelastic collisions), where the energy loss in inelastic collisions is determined by a restitution coefficient ɛ. Whereas previous studies used a simplified model with ɛ = const, the present analysis takes into account the dependence of the restitution coefficient on relative impact velocity. The granular temperature and the Brownian diffusion coefficient are calculated for a granular gas in the homogeneous cooling state and a gas driven by a thermostat force, and their variation with grain mass and size and the restitution coefficient is analyzed. Both equipartition principle and fluctuation-dissipation relations are found to break down. One manifestation of this behavior is a new phenomenon of “relative heating” of Brownian particles at the expense of cooling of the ambient granular gas.