Semantic dementia (SD) patients including semantic variant primary progressive aphasia (svPPA) and semantic behavioral variant frontotemporal dementia (sbvFTD) patients show semantic difficulties identifying faces and known people related to right anterior temporal lobe (ATL) atrophy. However, it remains unclear whether they also have perceptual deficits in face recognition. We selected 74 SD patients (54 with svPPA and predominant left ATL atrophy and 20 with sbvFTD and predominant right ATL atrophy) and 36 cognitively healthy controls (HC) from UCSF Memory and Aging Center. They underwent a perceptual face processing test (Benton facial recognition test-short version; BFRT-S), and semantic face processing tests (UCSF Famous people battery - Recognition, Naming, Semantic associations - pictures and words subtests), as well as structural magnetic resonance imaging (MRI). Neural correlates with the task's performance were conducted with a Voxel-based morphometry approach using CAT12. svPPA and sbvFTD patients were impaired on all semantic face processing tests, with sbvFTD patients performing significantly lower on the famous faces' recognition task in comparison to svPPA, and svPPA performing significantly lower on the naming task in comparison to sbvFTD. These tasks predominantly correlated with gray matter (GM) volumes in the right and left ATL, respectively. Compared to HC, both svPPA and sbvFTD patients showed preserved performance on the perceptual face processing test (BFRT-S), and performance on the BFRT-S negatively correlated with GM volume in the right posterior superior temporal sulcus (pSTS). Our results suggest that early in the disease, with the atrophy mostly restricted to the anterior temporal regions, SD patients do not present with perceptual deficits. However, more severe SD cases with atrophy in right posterior temporal regions might show lower performance on face perception tests, in addition to the semantic face processing deficits. Early sparing of face perceptual deficits in SD patients, regardless of hemispheric lateralization, furthers our understanding of clinical phenomenology and therapeutical approaches of this complex disease.