The excessive presence of phosphate in water leads to eutrophication, prompting us to develop a novel material named PC/SP/CA-H (Portland cement/slag powder/coal ash-hydrate). The experimental data accorded with the pseudo-second-order kinetics and Langmuir model, suggesting that chemical interactions predominated in the adsorption process. Within 4 hours, PC/SP/CA-H rapidly removed phosphate at a rate of 65%, which exceeded 90% after 12 hours with a material dosage of 30 g/L and a phosphate concentration of 30 mg/L. The maximum phosphate adsorption capacity of PC/SP/CA-H was 8.84 mg/g. The material exhibited effective phosphate removal capabilities across a wide pH range of 3.0–11.0. It demonstrated remarkable selectivity to phosphate in the presence of competitive anions, such as CO32-, HCO3-, NO3-, SO42-, and Cl-. Characterization analyses including SEM, BET, XRD, XRF, FTIR, and XPS were conducted to investigate the performance of PC/SP/CA-H and mechanisms of phosphate removal. The findings indicated that phosphate replaced hydroxyl groups on the material to form inner-sphere complexes through ligand exchange. Besides, phosphate was adsorbed onto the positively charged material via electrostatic attraction with protonated hydroxyl groups (-OH2+). It also reacted with metal cations (Ca2+) released by the material to form insoluble crystalline substances (Ca-P) through surface precipitation. This material holds promising prospects for efficiently removing phosphate from water or wastewater, while simultaneously enhancing the resource utilization of coal ash and slag powder.