The position accuracy of the polishing tool affects the surface quality of the polished aspheric surface. The contact deformation among the polishing tool, abrasives, and aspheric part can cause a displacement, which, in turn, will cause a position error of the polishing tool, which will lead to a significant change in the polishing force. In order to resolve this error, this paper proposed a method of normal displacement compensation for a computer numerical controlled (CNC) polishing system by controlling the polishing force. Firstly, the coupling principle between the polishing force and the position of the polishing tool is expounded, and the relationship between normal displacement and deformation is analyzed. Based on Hertz’s theory, a model of normal displacement is established. Then, on the basis of the decoupled polishing system developed, a normal displacement compensation method was proposed. Finally, a group of comparative experiments was carried out to verify the effectiveness of the proposed method. Compared with no displacement compensation, when the part was polished with the normal displacement compensation method, the value of roughness decreased from 0.4 µm to 0.21 µm, and the unevenness coefficient of surface roughness decreased from 112.5% to 19%. The experimental results show that the polishing quality is improved greatly, and the aspheric surfaces can be polished more uniformly with the method proposed in this paper.