The surface appearance of milk powders is a crucial quality property since the roughness of the milk powder determines its functional properties, and especially the purchaser perception of the milk powder. Unfortunately, powder produced from similar spray dryers, or even the same dryer but in different seasons, produces powder with a wide variety of surface roughness. To date, professional panelists are used to quantify this subtle visual metric, which is time-consuming and subjective. Consequently, developing a fast, robust, and repeatable surface appearance classification method is essential. This study proposes a three-dimensional digital photogrammetry technique for quantifying the surface roughness of milk powders. A contour slice analysis and frequency analysis of the deviations were performed on the three-dimensional models to classify the surface roughness of milk powder samples. The result shows that the contours for smooth-surface samples are more circular than those for rough-surface samples, and the smooth-surface samples had a low standard deviation; thus, milk powder samples with the smoother surface have lower Q (the energy of the signal) values. Lastly, the performance of the nonlinear support vector machine (SVM) model demonstrated that the technique proposed in this study is a practicable alternative technique for classifying the surface roughness of milk powders.