Apart from morphological differences, male and female faces also vary in color, especially in overall lightness and facial contrast, i.e., the contrast between the luminance and color of facial features (eyes, lips, or brows) and luminance and color of the surrounding skin. In many populations, it has been demonstrated that women tend to be lighter than men. Other differences were found in facial contrast: women have a higher contrast between the lightness of their eyes and lips and the surrounding skin. Manipulation of this contrast in an artificial genderless face can result in a masculine or feminine appearance. So far, however, this phenomenon has been studied mostly in Euro-American and East Asian samples, with little evidence from populations with darker facial tone. We explored natural sexual dimorphism in both facial contrast and lightness in an African, namely Cameroonian, sample, and compared it with results for a European, in particular Czech, population. Our findings showed that sexual differences in luminance contrast of eyes and brows were in both studied populations similar but in the Cameroonian sample, significant difference in lips contrast was absent. These results indicate that sex differences in facial contrast are a side effect of the sex differences in skin color and can be used as a proxy for skin color perception.