Identifying the multi-scale spatial relationship between land use and water quality is critical for determining the priorities and key areas of river management. To more accurately identify the scale effect of land-use patterns on water quality and quantitatively distinguish the difference in the impacts of land-use composition and configuration on water quality, we used 94 sites to extract the upstream catchment and riparian buffer zone with different widths. The results showed that the ability of land use variables with different buffer widths to explain water quality differed slightly from the ability of these variables at the catchment scale, and the joint explanatory ability of land use composition and configuration was greater than that of each individually. The patch density and landscape shape index of cultivated land, shrubland, and built-up land in the buffer area close to the water bodies were the main factors for the increase in the concentration of total nitrogen, nitrate nitrogen, total phosphorus, and suspended solids. As the width of the buffer increased, the role of the percent of land use increased. Our research indicates that water quality management needs to adopt a multi-scale perspective and focus on key local areas while coordinating at a broader scale.