To ensure the ecological security of the basin, data on the heavy metal content in the sediments of the Weihe River Basin were collected to analyze the spatial distribution characteristics of heavy metals based on descriptive statistics and geostatistics. The geoaccumulation index and potential ecological risk assessment were conducted, and the number of pollution sources and the contribution rate were determined by combining correlation analysis, principal component analysis, and APCS-MLR. The results showed that the mean values of As, Pb, Zn, Ni, Cr, Hg, Cu, and Cd were 15.42, 27.27, 88.05, 31.05, 75, 0.13, 29.47, and 1.05 mg·kg−1, respectively. Igeo is in the order of Cd > Hg > As > Cu > Pb > Zn > Cr > Ni. Cd had the highest potential ecological risk factor, followed by Hg with a critical risk proportion of 12.6%. The traceability analysis showed that the heavy metals come from mixed, industrial, and agricultural sources, among which Pb, Zn, Ni, Cr, and Cu are affected by soil-forming parent materials and industrial comprehensive sources, As is affected by agricultural sources, and Hg and Cd are affected by industrial sources. Industrial and living sources are the main sources of the heavy metals. The results of the study can provide a basis for formulating relevant pollution prevention and control measures in the Weihe River Basin.