People differ in their tendency to infer others' personalities and abilities from their faces. An extreme form of such face-based trait inference (FBTI) is problematic because of its unwarranted impact on real-world decision making. Evolutionary perspectives on FBTI suggest that its inter-individual variation would be trait-specific: e.g. those who make extreme face-based inferences about trustworthiness may not necessarily do so about dominance. However, there are several psychological variables that can increase the FBTI extremity across traits. Here, we show that there is a generalized individual tendency to make extreme FBTI across traits, in support of the latter view. We found that the degrees of extremity of face-based inferences about seven traits had high cross-trait correlations, constituting a general factor. This generalized FBTI extremity had good test-retest reliability and was neither an artefact of extreme nor socially desirable response biases. Moreover, it was positively associated with facial emotion recognition ability and tendencies to believe physiognomy and endorse stereotypes. Our results demonstrate that there are individuals who have a temporally stable disposition to draw extreme conclusions about various traits of others from facial appearance as well as their psychological characteristics.