Scene text recognition using a Hough forest implicit shape model and semi-Markov conditional random fields

Abstract

Most of the scene text recognition methods utilize character models only in the character recognition phase, the last stage of the process. In former phases such as text detection, only abstracted features of text regions are used, which might cause loss of information. In this paper, we propose a novel scene text recognition method which fully utilizes model of target characters throughout the process. Each of the target character set is modeled with a part-based object model called implicit shape model (ISM) to achieve robustness for the partial degradation of characters. Towards this end, we trained a Hough forest which localizes and aggregates character parts to detect character candidates in the image. The detected character candidates are verified by organizing the most plausible text lines in a semi-Markov conditional random field (semi-CRF) framework. As concrete character models are utilized throughout the process, even extremely deformed texts are detected and recognized.