An End-to-End Method to Extract Information from Vietnamese ID Card Images

Abstract

Information extraction from ID cards plays an important role in many daily activities, such as legal, banking, insurance, or health services. However, in many developing countries, such as Vietnam, it is mostly carried out manually, which is time-consuming, tedious, and may be prone to errors. Therefore, in this paper, we propose an end-to-end method to extract information from Vietnamese ID card images. The proposed method contains three steps with four neural networks and two image processing techniques, including U-Net, VGG16, Contour detection, and Hough transformation to pre-process input card images, CRAFT, and Rebia neural network for Optical Character Recognition, and Levenshtein distance and regular expression to post-process extracted information. In addition, a dataset, including 3.256 Vietnamese ID cards, 400k manual annotated text, and more than 500k synthetic text, was built for verifying our methods. The results of an empirical experiment conducted on our self-collected dataset indicate that the proposed method achieves a high accuracy of 94%, 99.5%, and 98.3% for card segmentation, classification, and text recognition.