Mild alkali treatment can potentially be developed as a greener alternative to the traditional alkali treatment of starch, but the effect of mild alkali on starch is still understudied. Normal and waxy rice starches were subjected to mild alkali combined with hydrothermal treatment to investigate their changes in physicochemical properties. After mild alkali treatment, the protein content of normal and waxy rice starches decreased from 0.76% to 0.23% and from 0.89% to 0.23%, respectively. Mild alkali treatment decreased gelatinization temperature but increased the swelling power and solubility of both starches. Mild alkali treatment also increased the gelatinization enthalpy of waxy rice starch from 20.01 J/g to 25.04 J/g. Mild alkali treatment at room temperature increased the pasting viscosities of both normal and waxy rice starches, whereas at high temperature, it decreased pasting viscosities during hydrothermal treatment. Alkali treatment significantly changed the properties of normal and waxy rice starch by the ionization of hydroxyl groups and the removal of starch granule-associated proteins. Hydrothermal conditions promoted the effect of alkali. The combination of hydrothermal and alkali treatment led to greater changes in starch properties.