Surface water is a crucial part of terrestrial ecosystems and is crucial to maintaining ecosystem health, ensuring social stability, and promoting high-quality regional economic development. The surface water in the Yellow River Basin (YRB) has a high sediment content and spatially heterogeneous sediment distribution, presenting a significant challenge for surface water extraction. In this study, we first analyze the applicability of nine water indexes in the YRB by using the Landsat series images (Landsat 5, 7, 8) and then examine the correlation between the accuracy of the water indexes and suspended particulate matter (SPM) concentrations. On this basis, we propose a surface water extraction method considering the SPM concentrations (SWE-CSPM). Finally, we examine the dynamic variations in the surface water in the YRB at four scales: the global scale, the secondary water resource zoning scale, the provincial scale, and the typical water scale. The results indicate that (1) among the nine water indexes, the MBWI has the highest water extraction accuracy, followed by the AWEInsh and WI2021, while the NDWI has the lowest. (2) Compared with the nine water indexes and the multi-index water extraction rule method (MIWER), the SWE-CSPM can effectively reduce the commission errors of surface water extraction, and the water extraction accuracy is the highest (overall accuracy 95.44%, kappa coefficient 90.62%). (3) At the global scale, the maximum water area of the YRB shows a decreasing trend, but the change amount is small. The permanent water area shows an uptrend, whereas the seasonal water area shows a downtrend year by year. The reason may be that the increase in surface runoff and the construction of reservoir projects have led to the transformation of some seasonal water into permanent water. (4) At the secondary water resource zoning scale, the permanent water area of other secondary water resource zonings shows an increasing trend in different degrees, except for the Interior Drainage Area. (5) At the provincial scale, the permanent water area of all provinces shows an uptrend, while the seasonal water areas show a fluctuating downtrend. The maximum water area of Shandong, Inner Mongolia Autonomous Region, and Qinghai increases slowly, while the other provinces show a decreasing trend. (6) At the typical water scale, there are significant differences in the water area variation process in Zhaling Lake, Eling Lake, Wuliangsuhai, Hongjiannao, and Dongping Lake, but the permanent water area and maximum water area of these waters have increased over the past decade. This study offers significant technical support for the dynamic monitoring of surface water and helps to deeply understand the spatiotemporal variations in surface water in the YRB.