The Pliocene warm period is regarded as an analogue of future warming, and quantitative reconstruction of palaeovegetation and palaeoclimate during this period is helpful for predicting vegetation and climate changes in the future. Here, we quantitatively reconstructed the late Pliocene vegetation and climate based on phytolith assemblages from the Upper Zhangcun Formation in the Yushe Basin, Northern China. The results show that the vegetation was dominated by grasses with few trees, which conflicts with the pollen assemblage results. The possible reason for this contradiction is that phytoliths tend to mirror local, shoreline vegetation (grasses with few aquatic plants), while pollen tend to mirror both local and regional vegetation (trees in surrounding mountains and piedmont, grasses on lakeshores and aquatic plants in wetlands) in a small structurally confined lake. The phytolith-based quantitative reconstruction of mean annual precipitation (812–1673 mm) and warm season precipitation (417–1197 mm) provides values that are higher than those at present, revealing an increase in Asian monsoon rainfall during the late Pliocene, which is consistent with the pollen results, although the reconstructed vegetation is different. This study highlights the usefulness of phytoliths in reconstructing Cenozoic vegetation and climate and underlines the importance of landforms in determining phytolith and pollen assemblages in sediments.