Sex identification of faces without any cultural or conventional sex cue is primarily based on two independent components: a) shape or facial structure, and b) surface reflectance (skin texture and color). The present work studied the relative contribution of each component by means of two experiments based on 3D face models created with different degrees of masculinity-femininity within a sex continuum. The first experiment utilized totally artificial faces created ex novo by computer. The second employed face models created from photographs of real people. The results of both experiments were consistent. As expected, when both components were present in a face, sex was correctly classified in almost all the cases. More interestingly, the contribution of the “pure” facial structure to the sex perception (with no surface reflectance) was about 80%, whereas 20% of the total information was provided by the surface reflectance. Furthermore, examination of the psychometric curves suggests that the information provided by surface reflectance contributes to a categorical perception of facial sex, since when it is removed the sex is perceived in a more continuous / less categorical way. On the other hand, our stimuli presented a certain “male” bias, repeatedly found in the literature on facial sex perception.