ADC12 aluminum alloy has been widely used in the aerospace, ship, and automotive fields because of its high specific strength, excellent die-casting performance, and wear resistance. Adhesion wear is the main wear mechanism of high-speed milling ADC12 aluminum alloy. The most important factor affecting adhesion wear is the tool–chip interface friction, which is directly manifested in the tool–chip interface temperature. Therefore, the temperature variation during the milling of aluminum alloy is analyzed using a temperature field model and infrared temperature measurement technology. Then, the tool wear morphology and the tool wear land width are observed using a scanning electron microscope. Finally, the tool wear mechanism considering the tool–chip interface temperature is discussed. The tool–chip interface temperature is related to the friction angle, tool–chip contact length, and friction force at the rake face, which increases first and then decreases as the cutting speed and feed rate increase. During the formation of the adhesive layer, the tool–chip interface temperature increases, the change rate of the cutting force and the tool wear rate increase, and adhesion, oxidation, and abrasive and delamination wear are generated on the tool surface. With the increase in temperature, the tool wear rate increases, the molten adhesive layer on the tool surface is accompanied by crack propagation, and adhesion wear, oxidation wear, and abrasive wear occur on the tool surface.