The purpose of this work was to improve the wear resistance of an alumina ceramic grinding medium used in water-based ball milling procedures. High alumina ceramics (>97wt% Al2O3) composed of Al2O3–CaCO3–SiO2–MgO–Y2O3 and designated as ACSMY have been investigated. The content of Y2O3 was varied from 0.0 to 1.6wt%. Tests were performed in a ball milling apparatus in a water environment, and according to a Chinese standard test method. The compositions and microstructure of four compositions, as well as the effects of sintering temperature on wear rate are discussed. It is found that Y2O3 can refine grain size, enhance density, improve bonding strength of grain boundaries, and promote the crystallization of the CaAl12O19 phase. In addition, Y3Al5O12 and alkaline earth metal ions (Ca2+or Mg2+) can form solid solutions to purify the grain boundaries. The role of Y2O3 is discussed in detail, and the mechanisms responsible for wear resistance of ACSMY ceramics are revealed.