Although the term “diversity” is ubiquitous in modern day society, the meaning of the term is not well understood, and it is unclear when people perceive groups to be more or less diverse. Across five experiments, we examined perceptions of racial diversity and preferences for living in diverse contexts related to visual representations of groups. While recent theorizing suggests that a greater number of low-status racial minorities may contribute to perceiving more diversity and determine living preferences, our findings indicate the importance of the race of the perceiver in these processes. Although White and Black participants rated majority Black compared to majority Asian groups as more diverse, followed by majority White targets (cf. Experiment 4), Asian participants did not differ in their diversity ratings across the different racial compositions. Notably, White and Asian participants rated majority White and majority Asian compared to majority Black neighborhoods as more desirable. Black participants, in contrast, consistently rated majority Black compared to majority White and majority Asian neighborhoods as more desirable and did not distinguish between the latter two contexts. Together these findings provide new evidence about how people define diversity and the importance of target-level factors, perceiver-level factors, and the interplay between these factors on perceptions of racial diversity and inclusivity.