Gear rolling contact fatigue displays itself with many failure modes such as micro-pitting, pitting or tooth flank fracture. Competing mechanism exists between these failure modes. Tooth surface micro-topography plays an important role in contact behavior and contact fatigue performance of gears. In particular, the root of mean square (RMS) of surface roughness is extensively characterized as one of the main parameters influencing the service performance. In this work an elastic-plastic finite element contact fatigue model is proposed in which the surface roughness is explicitly measured through an optical profiler. The relationship between the dimensionless normal loads and the contact area ratio is plotted. The critical planes and the characteristic shear strain amplitudes are captured at each material point within the contact area. Then the Brown-Miller multi-axial fatigue criterion is applied to evaluate the fatigue life of rough surface contacts. Results reveal that the surface roughness significantly influences the contact fatigue life of gears, and the competing mechanism between the micro-pitting from near-surface area and pitting from sub-surface area. When the RMS value of surface roughness is considerably small, the pitting failure risk and the micro-pitting risk coexist since the minimum fatigue life appears both at the near-surface area and the sub-surface area.