Recent studies have revealed that human position sense relies on a massively distorted representation of hand size and shape. By comparing the judged location of landmarks on an occluded hand, Longo and Haggard (2010) constructed implicit perceptual maps of represented hand structure, showing large underestimation of finger length and overestimation of hand width. Here, we investigated the contribution of two potential sources of distortions to such effects: perceptual distortions reflecting spatial warping of the representation of bodily tissue itself, perhaps reflecting distortions of somatotopic cortical maps, and conceptual distortions reflecting mistaken beliefs about the locations of different landmarks within the body. In Experiment 1 we compared distorted hand maps to a task in which participants explicitly judged the location of their knuckles in a hand silhouette. The results revealed the conceptual distortions are responsible for at least part of the underestimation of finger length, but cannot explain overestimation of hand width. Experiment 2 compared distortions of the participant’s own hand based on position sense with a prosthetic hand based on visual memory. Underestimation of finger length was found for both hands, providing further evidence that it reflects a conceptual distortion. In contrast, overestimation of hand width was specific to representation of the participant’s own hand, confirming it reflects a perceptual distortion. Together, these results suggest that distorted body representations do not reflect a single underlying cause. Rather, both perceptual and conceptual distortions contribute to the overall configuration of the hand representation.