Heavy metals (HMs) in acid mine drainage (AMD) pose serious threats to aquatic life and soil. Biosorbents are promising materials to remove HMs from wastewater. Herein, food waste compost was used as a low-cost and sustainable biosorbent to remove Fe3+, Cu2+, Ni2+, Pb2+, and Zn2+ from an AMD and other model solutions with an initial pH of 2.24. Batch adsorption tests were conducted to systematically investigate the influences of initial pH, compost dose, the presence of Fe3+ and SO42− ions, and the pH-induced Fe precipitates on the removal performance. The involved removal mechanisms were explored by conducting stepwise precipitation tests, sequential extraction tests, and Fourier transform infrared spectroscopy (FTIR) characterization. It was found that the addition of the compost removed over 90% of Zn, Ni, Cu, and Pb at pH 5.80, 5.50, 4.50, and 3.00, respectively. At low compost doses, the presence of Fe3+ and SO42− ions hindered the removal since the competition effect of Fe3+ ion and lower absorbability of metal-sulfate complexes, respectively. Sequential extraction tests revealed that higher fractions of Fe, Cu, and Pb were strongly immobilized through complexation and precipitation relative to Ni and Zn. Additionally, the pH-induced Fe precipitates did not favor the removal of the HMs, probably due to the adsorption of organic components released from the compost. This study suggests that compost have the potential to remove HMs from acidic wastewater without pre-neutralization.