The 25 counties along the Shandong section of the Yellow River are the core areas for promoting the ecological protection and high-quality development of the Yellow River in Shandong Province. Moreover, it is of great significance to study the current situation, sources, and potential risks of heavy metal pollution in the topsoil in this region. In this study, 103 soil samples were collected from the 25 counties along the Shandong section of the Yellow River, and the contents of eight heavy metals (As, Cu, Pb, Cr, Zn, Ni, Cd, and Hg) were determined. The pollution characteristics of heavy metals were analyzed and evaluated using the geological accumulation index and potential ecological risk index. Correlation analysis and the positive matrix factorization (PMF) model were used to analyze the sources of heavy metals. The results showed that the average contents of Cu and Cr were lower than that of the background values of soils, whereas the average contents of As, Pb, Zn, Ni, Cd, and Hg were 1.16, 1.42, 1.05, 1.14, 2.29, and 1.85 times higher than that of the background values, respectively, and the average contents of all eight elements were lower than the screening value of soil pollution risk in agricultural land. In terms of different heavy metal variations, the coefficient of variation (CV) of Cu and Cd was higher than 0.500, indicating high variations, whereas As, Pb, Cr, Zn, Ni, and Hg showed moderate variation. Cd and Hg were slightly polluted, whereas the other six elements were not polluted. Cd and Hg had a moderate potential ecological risk level, whereas the other six elements were at a low level. Correlation analysis and PMF model showed that the sources of heavy metals in the study area were influenced by four factors, i.e., agricultural activities, natural sources, industrial emissions, and atmospheric dust from coal combustion and vehicle exhaust emissions, and the relative contribution rates were 32.4%, 34.9%, 16.5%, and 16.2%, respectively.