Product quality is an important indicator for determining the quality of industrial products. Defects on the surface of aluminum profiles are inevitably caused in the actual production process due to the influence of various factors such as environment and equipment, and these defects seriously affect the quality of aluminum profiles. The focus and difficulty of research have shifted to how to quickly and accurately identify and classify surface defects in aluminum profiles. To address this issue, this paper proposes an aluminum defect classification algorithm that uses an attention mechanism in conjunction with the traditional Inception V4 network model for deep learning image classification, to accurately identify and classify aluminum defect areas. Experiments and comparative analysis are performed on the aluminum defect recognition dataset from the Alias Tianchi platform, and the results show that the algorithm with the addition of the attention mechanism improves accuracy by 1.24% over the original model.