A white layer will be generated on the machined surface when milling titanium alloy. Its existence will make the surface of the workpiece more brittle, and it is easy to produce cracks during the machining process, which greatly harms machining. Aiming at this problem, this paper takes the ball-end milling cutter with mesoscopic geometric characteristics as the research object, establishes the theoretical model of the white layer thickness of the workpiece under the micro-texture and cutting edge factors, analyzes the single factor of micro-texture parameters and cutting edge parameters, and the interaction between them and cutting parameters on the milling force-thermal characteristics and the white layer of the workpiece. The influence law reveals its mechanism from the macro and micro perspectives. It uses the white layer thickness as the evaluation index to optimize the mesoscopic geometric characteristic parameters by using the simulated annealing algorithm. The results demonstrate that the mesoscopic geometric feature of the ball-end milling cutter effectively reduces the thickness of the white layer, achieving a 38% reduction compared to that observed with a non-textured milling cutter. The thickness of the white layer exhibits a positive correlation with the distance L from the blade and a negative correlation with the edge radius R. The interaction between the cutting edge radius R and the cutting depth a p influences the milling force and heat, alters the micro-texture of the workpiece surface, induces the α + β phase transformation, and affects the white layer thickness. Based on the predictive model for white layer thickness, the optimization results for comprehensive milling performance are as follows: R is 59.39 μm, L is 110.01 μm, D is 58.68 μm, L1 is 130.59 μm, a p is 0.30 mm, V is 147.39 mm/min, f is 0.08 mm/z.