The rarity of species has multiple facets. Functional rarity is an integrated index that can quantify species’ sparseness and functional traits’ rarity. However, a comprehensive evaluation of various dimensions of functional rarity at different scales in urban ecosystems is under-researched. Here, we addressed two hypotheses: (1) functional rarity will not be consistent with conventional rarity at local and regional scales, and (2) land use types influence the distribution of all forms of functionally rare plants in urban ecosystems. We first selected ten functional traits to quantify the functional rarity components of 170 plant species. Second, we assessed the distribution of functional rarity and all forms of functionally rare plants across seven land use types in the megacity of Tokyo. We found that functional rarity is not consistent with conventional rarity at the local and regional scales. Several native species and two non-native species were functionally distinct and not scarce at the local scale. In addition, natural remnants and anthropogenic habitats could protect functionally rare species in urban ecosystems. We thus argue for incorporating functional rarity in the current biodiversity conservation framework and considering natural remnants and anthropogenic habitats in urban functionally rare species conservation.