Nowadays, large-aperture optical components are increasingly used in high-power laser systems, remote-sensing satellites, and space-based astronomical telescopes. Fabricating these surfaces with submicron-scale shape accuracy and a nanoscale surface finish has been a great challenge for the optical industry, especially for hard and difficult-to-machine materials. Thus, to achieve the high-efficiency and high-precision polishing of large-aperture aspherical optical parts, this study combined robotic machining technology with computer-controlled optical surfacing (CCOS) technology and investigated the effect of robot motion accuracy on the surface topography of workpieces during polishing. First, a material removal model considering the normal error of the polishing tool was developed based on contact mechanics, kinematic theory, and the abrasion mechanism. Next, in combination with the polishing trajectory, the surface morphology and form accuracy after polishing were predicted under different normal-error conditions. Then, preliminary experiments were conducted to verify the model. The experimental data agreed with the simulation results, showing that as the normal error increased from 0° to 0.5° and 1°, the peak-to-valley (PV) values of the surface profile of the optical element decreased from 5.42, 5.28, and 4.68 μm to 3.97, 4.09, and 4.43 μm, respectively. The corresponding convergence rates were 26.8%, 22.5%, and 5.3%. The root mean square (RMS) values decreased from 0.754, 0.895, and 0.678 μm to 0.593, 0.620, and 0.583 μm, with corresponding convergence rates of 21.4%, 30.7% and 14.0%, respectively. Moreover, a higher motion accuracy enabled the polishing robot to reduce the mid- and high-frequency errors of the optical element.