Titanium alloy structural components are widely used in the aerospace industry due to their excellent physical properties. However, their susceptibility to machining deformation during milling, attributed to the presence of initial residual stress, poses a significant challenge. Therefore, this paper proposes a finite element model for predicting milling deformation in Ti6Al4V titanium alloy structural parts, considering initial residual stress. The methodology involves establishing the initial residual stress field of titanium alloy components based on a mathematical model of annealing heat transfer and a theoretical study of thermal elasto-plastic constitutive relation, with experimental confirmation of model accuracy. Based on the initial residual stress, a finite element model was developed to predict the maximum deformation under different toolpaths. Finally, a comparative analysis with milling experiments validated the proposed models. The experimental results demonstrated a remarkable alignment with the finite element model, showcasing the correctness of the developed models. The maximum deviation in deformation observed was within the range of 0.003 mm–0.011 mm. This study is the first to comprehensively consider the effects of initial residual stress and toolpaths on titanium alloy milling deformation, and to experimentally verify the proposed model’s accuracy and practicality. It provides a new theoretical basis and technical guidelines for titanium alloy milling deformation.