A robot belt grinding system provides promising prospects for relieving hand grinders from their noisy work environment, as well as for improving machining accuracy and product consistency. However, for a manufacturing system with a flexible grinder, controlling the robot to perform precise material removal from free-form surfaces is a challenge. In the belt grinding process, material removal is related to a variety of factors, such as workpiece shape, contact force, and robot velocity. Some factors of the grinding process, such as belt wear, are time-variant. To achieve the desired removal in the grinding process, an intelligent control method for the industrial robot is proposed in this paper. First, an adaptive grinding process model that can track discontinuous changes in working conditions is constructed to precisely predict material removal in accordance with in situ measurement data. With incorporated prior knowledge, the method considerably improves model accuracy, which worsens when new samples from an in situ measurement are insufficient or are unevenly distributed under new working conditions. After this, an online trajectory generation method for the robot control parameters is proposed. By calculating the optimal control parameters in real time, the control transition process is shortened and its negative effect on grinding quality is reduced. Finally, the preliminary grinding experiments validate the workability and effectiveness of the proposed control method.