This study focuses on systematically revealing how cutting parameters influence the surface integrity of ball-end hard milled surface of AISI D2 steel and proposing optimization scheme from surface integrity, wear resistance and fatigue resistance perspective based on response surface methodology respectively. Results can be summarized into three aspects. Firstly, radial depth of cut with percent contribution ratio (PCR) 62.05% has a decisive influence on surface roughness, followed by spindle speed 13.25% and feed per tooth 6.63%. The work hardening degree was raised from 12.5% to 38.4% when spindle speed changed from 8000 rpm to 2000 rpm. Spindle speed and radial depth of cut are the most significant factor influencing residual stress. The PCR of spindle speed and radial depth of cut reached 73.47% and 18.63% for residual stress in feed direction, 47.11% and 37.51% in step-over direction, respectively. High residual compressive stress can be generated by lowering spindle speed and radial depth of cut benefiting from the aggravated squeeze between ball-end milling cutter and workpiece. Secondly, too small feed per tooth or too small radial depth of cut should be avoided from wear resistance point because though the surface microhardness can be improved, the surface quality will also be deteriorated. The combination of high spindle speed, small feed per tooth together with small radial depth of cut can meet the wear resistance and the machining efficiency requirement. Finally, a medium-sized cutting parameter combination should be adopted to realize satisfying material removal rate and fatigue resistance. This study can be used to guide the selection of cutting parameters during ball-end milling of hardened AISI D2 steel for dies/molds manufacturing industries.