Perceivers' ability to use multiple sources of information when forming impressions—including top-down, perceiver-level features, and bottom-up, target-level features—is a hallmark of social cognition. We investigate this primary foundation by examining the role of perceiver-level and target-level variation in perceived racial prototypicality in the U.S. In Study 1 (200 unique faces; 2608 raters), we quantified contributions of perceiver- and target-level effects to perceived racial prototypicality. Perceiver- and target-level contributions varied across racial category (Asian, Black, Latine, Middle Eastern, and Multiracial), with Multiracial and Middle Eastern prototypicality being more perceiver-driven. Although several appearance features (e.g., perceived ambiguity, skin tone) related to perceived prototypicality, there were distinctions in how perceivers used them (e.g., some people strongly used skin tone to infer Black prototypicality, while others used this less or not at all). A second study (N = 511) experimentally manipulated race essentialist beliefs. While there was no impact on perceived racial prototypicality, regardless of the category (Asian, Black, Latine, Middle Eastern, Multiracial, and Native American), Middle Eastern, Multiracial, and Native American prototypicality were generally more perceiver-driven than other categories, converging with Study 1. Further, perceivers' social dominance orientation, but not several other individual differences, were associated with less use of each of these categories. Taken together, findings suggest perceived racial prototypicality may originate less from stable individual differences like attitudes and instead reflects both i) differences in perceptions of target features and ii) differences in how people use particular target features in making racial prototypicality judgments.