During the milling of a curved surface, the shape of the curved surface changes continuously along the milling path, and the direction of the angle between the axis of the milling tool and the normal vector of the machined surface also changes. In turn, these changes greatly affect the quality of the machined surface. Therefore, to improve the surface quality of machined workpieces, the milling tool inclination angle should be optimized. In the present study, a three-dimensional model of the milling tool path and the workpiece surface was created. The three-dimensional surface model of the workpiece at each machining moment was iteratively built based on the Z-map method. From the macroscopic perspective, the influences of different milling tool inclination angles on surface roughness were analyzed. The difference in surface roughness generated by different inclination angles under the same feed rate and pitch was obtained. When cutting with the top of the milling tool, the roughness was larger compared to other inclination angles, and there was a roughness deviation of about 8% when other inclination angles were selected. Moreover, from the microscopic perspective, the material removal rate from a single feed cycle and the optimal inclination angle range for milling stability were explored. An optimal inclination range of 15–30% was obtained, considering both the total material removal volume and unit-angle material removal volume. Finally, a spherical machining verification experiment was conducted on an ultra-precision machining tool, and the experimental findings agreed well with the simulation results, demonstrating the reliability of the proposed model. The theoretical model of this study can provide a basis for optimal cutting tool inclination angle selection during complex surface milling.