Habitat loss and fragmentation per se have been shown to be a major threat to global biodiversity and ecosystem function. However, little is known about how habitat loss and fragmentation per se alters the relationship between biodiversity and ecosystem function (BEF relationship) in the natural landscape context. Based on 130 landscapes identified by a stratified random sampling in the agro-pastoral ecotone of northern China, we investigated the effects of landscape context (habitat loss and fragmentation per se) on plant richness, above-ground biomass, and the relationship between them in grassland communities using a structural equation model. We found that habitat loss directly decreased plant richness and hence decreased above-ground biomass, while fragmentation per se directly increased plant richness and hence increased above-ground biomass. Fragmentation per se also directly decreased soil water content and hence decreased above-ground biomass. Meanwhile, habitat loss decreased the magnitude of the positive relationship between plant richness and above-ground biomass by reducing the percentage of grassland specialists in the community, while fragmentation per se had no significant modulating effect on this relationship. These results demonstrate that habitat loss and fragmentation per se have inconsistent effects on BEF, with the BEF relationship being modulated by landscape context. Our findings emphasise that habitat loss rather than fragmentation per se can weaken the positive BEF relationship by decreasing the degree of habitat specialisation of the community.