Despite the widespread perception that non-native species threaten biodiversity, there are few documented cases of non-native species displacing rare or specialized native species. Here, I examined changes in plant species composition over 5 years during patch expansion of a non-native grass, Imperata cylindrica, in longleaf pine flatwoods in Mississippi, USA. I used a multivariate approach to quantify the degree of habitat specialization and geographic range of all species encountered. I examined losses of species collectively as a function of plant height (controlling for initial frequency) and then the relationship between height and the degree of association with longleaf pine flatwoods, disturbed habitats, and the outer Gulf Coastal Plain of the southeastern USA. Patch expansion resulted in dramatic declines in species richness and increases in ground-level shade at both sites in just 3 years. Most tall saplings, shrubs, and vines were not endemic to longleaf pine communities and were less likely to be displaced than short herbs, most of which were indicative of longleaf pine communities. These results suggest that invasion of longleaf pine communities by I. cylindrica will likely cause significant losses of short, habitat-specialists and reduce the distinctiveness of the native flora of these threatened ecosystems.