The growth of the mobile Internet, smartphones and social networks has brought in huge amounts of picture information, and traditional manual identification is not able to meet the demand well enough. Therefore, the automatical image recognition [1] has been proposed which can help us recognize the image efficiently and get the corresponding information. Although traditional machine learning methods [2] have already been widely used in the field of image recognition, most of these methods are designed to handle one-dimensional vector information. Thus, we should first stretch image matrix to one-dimensional vector or extract features from images to employ traditional image recognition methods, which would lose the adjacent information in images and miss some important features. With the development of computer technology, deep learning [3] is gradually applied to the field of image recognition. It can deal with two-dimensional image data naturally and extract features automatically. Compared with the traditional machine learning methods, deep learning is popular for its good learning ability and low generalization error. In this paper, we compare the differences between SVM [4] and deep learning on image recognition, with an application to handwritten digital images recognition. The results show that the deep learning method is more accurate and more stable in image recognition.