The present study aims to develop a new potentially low-cost and efficient approach to removing soluble inorganic phosphorus from acid leaching wastewater. This wastewater was of high acidity and high phosphorus content. Low-grade oolitic hematite with high phosphorus (LGOHWHP) was chosen as an economic adsorbent and was also used to adjust the acidity of the solution. The adsorption isotherms, adsorption thermodynamics, and effect of various parameters such as pH value, contact time, temperature and adsorbent dosage on the phosphorus removal from wastewater were investigated. The results showed that pH value and adsorbent dosage have a significant impact on the phosphorus removal. The phosphorus adsorption results fitted very well to Langmuir and Freundlich adsorption isotherm models, and the adsorption process was an endothermic process. At the optimum parameters pH 5.5, reaction temperature of 302 K with 20 g L−1 LGOHWHP, the phosphorus removal percentage of about 95% and the phosphorus concentration in the wastewater of about 0.27 mg L−1 are achieved after 60 min. The results indicate that the phosphorus concentration in wastewater after dephosphorization by the LGOHWHP completely meets the requirements of the national wastewater discharge standard in China. This research provides an efficient and environmentally friendly technology to remove phosphorus from wastewater.