Soil environmental quality related to the residents’ life, health, and safety, has been the hotspot issues in science of ecological environment protection. Evaluating the distribution characteristics, ecological risk, and source of heavy metals in farmland is important for protecting soil resources. The agricultural area of Lianhua town, Gongcheng County, Guilin is a typical karst landform. In response to the problem of heavy metal pollution and complex sources in the soil of this area, the characteristics and sources of heavy metal pollution in the soil profiles from farmland, abandoned land, and forest were studied using the single-factor index method, the geoaccumulation index (Igeo) principal component analysis (PCA) and positive matrix factorization (PMF) model. The results showed that: (1) that the contents of cadmium (Cd), chromium (Cr), copper (Cu), nickel (Ni), lead (Pb), and zinc (Zn) in the soil profile of the study area were higher than that of the soil elements background values in Guangxi. The total and available forms contents of all heavy metal elements exhibited the characteristics of accumulation in the surface profile; (2) among the six heavy elements, the contents of Cd were in a moderately to heavily polluted state. The contents of Cd in some soil profiles exceeded the control standard for agricultural land soil pollution. The contents of Zn and Ni were from slightly to moderately polluted in areas with frequent agricultural activities; (3) according to the PCA and PMF model, there were three main sources of heavy metals in the study area. Among them, Cd, Cu, Ni, and Zn are related to agricultural activities; the elements As, Cd, Cr, and Hg are closely related to geological background; Pb and Zn are mainly affected by atmospheric sedimentation of transportation. Agricultural activities and natural geological background are the main contribution sources of heavy metals in soil. Human activities are the main factors that cause the accumulation of heavy metals in soil. This research has theoretical guidance and practical significance for the prevention and control of soil heavy metal pollution and the protection of farmland environmental quality in the region.