The quartz fields of the Greater Cape Floristic Region (GCFR) are arid and island-like special habitats, hosting ~142 habitat-specialized plant species, of which 81 % are local endemics, characterized by a rapid turnover of species between and among sites. We use several phylogenetic community metrics: (1) to examine species diversity and phylogenetic structure within and among quartz fields; (2) to investigate whether quartz field specialists are evolutionarily drawn from local species pools, whereas the alternative hypothesis posits that there is no significant evolutionary connection between quartz field specialists and the local species pools; and (3) to determine whether there is an association between certain traits and the presence of species in quartz fields. We sampled and developed dated phylogenies for six species-rich angiosperm families (Aizoaceae, Asteraceae, Crassulaceae, Cyperaceae, Fabaceae and Santalaceae) represented in the quartz field floras of southern Africa. Specifically, we focused on the flora of three quartz field regions in South Africa (Knersvlakte, Little Karoo and Overberg) and their surrounding species pools to address our research questions by scoring traits associated with harsh environments. We found that the Overberg and Little Karoo had the highest level of species overlap for families Aizoaceae and Fabaceae, whereas the Knersvlakte and the Overberg had the highest species overlap for families Asteraceae, Crassulaceae and Santalaceae. Although our phylogenetic community structure and trait analyses showed no clear patterns, relatively low pairwise phylogenetic distances between specialists and their local species pools for Aizoaceae suggest that quartz species could be drawn evolutionarily from their surrounding areas. We also found that families Aizoaceae and Crassulaceae in Knersvlakte and Little Karoo were phylogenetically even. Despite their proximity to one another within the GCFR, the studied areas differ in their species pools and the phylogenetic structure of their specialists. Our work provides further justification for increased conservation focus on these unique habitats under future scenarios of global change.