This research was conducted in the typical rural water-level-fluctuating zone (TRWLFZ) in Zhong County, located in the core region of Three Gorges Reservoir. In July 2010, sediment samples of the TRWLFZ were collected after the dam water level was mostly kept at 155 m. Toxic metals and arsenic in the sediments were analyzed for potential risk assessment and source analysis, which supplied basic data for non-point pollutant control. The index of geo-accumulation (I geo) showed that the pollution levels of the sediment followed the order Cd > Co > Mn > As > Cu in the area with a height of 155–160 m. At a height of 170–175 m, the order was Co > Mn > Cd > As. The sediment pollution index values show that the potential ecological risk posed by such sediment pollution in the area at 155–160 m is significantly higher than that at 170–175 m, where most analyzed elements were in relatively low concentrations. At a height of 155–160 m, As was significantly correlated with Co, Cr, and Ni, suggesting the same pollution source. There was also a considerable correlation between Cd and Cu as well as Pb and Zn. Similarly, this could be observed for Mn and Cu. At 170–175 m, As was closely related to Cr and Cu. There was a significant correlation among Co, Pb, Ni, Cu, and Cr, which suggests that these metals are derived from the same pollution source. At 155–160 m, Cr, Cu, Pb, and Zn were mainly derived from natural weathering processes, while As, Ni, Cd, Co, and Mn were derived from inputs from the upper reaches. At 170–175 m, Cr, Ni, Pb, and Zn were mainly derived from natural weathering processes, whereas Cd, As, Co, Mn, and Cu levels were a result of agricultural activities and inputs from the upper reaches.