On-line mathematical expression recognition using flexible structural matching and hierarchical decomposition parsing

Abstract

With the recent advances in pen-based computing technologies, we already have all the necessary hardware to provide an input device for entering mathematical expressions into computers in a natural way, i.e., we simply write the expressions on an electronic tablet for the computer to recognize them automatically. The key problem that remains is of course the automatic recognition of mathematical expressions, which is more on the software side. Mathematical expressions are generally two-dimensional structural patterns. They typically consist of special symbols and Greek letters in addition to English letters and digits. Moreover, characters and symbols may appear in various positions, possibly of different sizes. All these together make the recognition process very complicated even when all the individual characters and symbols can be recognized correctly. Mathematical expression recognition consists of two major stages: symbol recognition and structural analysis. Character recognition, as the most common type of symbol recognition problems, has been an active research area for more than three decades. Structural analysis of two-dimensional patterns also has a long history. However, very few papers have addressed specific problems related to mathematical expression recognition. In this thesis, we tackle various issues related to mathematical expression recognition. In particular, we propose two methods to solve problems in different stages of the recognition process, i.e., flexible structural matching for symbol recognition and hierarchical decomposition parsing for structural analysis. In addition, we incorporate some error detection and correction mechanisms in both stages so that the overall recognition rate can be improved. To show the effectiveness of the proposed methods, we also suggest some schemes for evaluating recognition performance. Experiments have been done on 600 mathematical expressions written by 10 writers. The results show that the recognition rates obtained are fairly high and the recognition speed for a single expression ranges from 0.73 second to 6 seconds over different sizes of expressions, with the system running in Prolog on a modest Sun SPARC 10 Unix workstation. This makes mathematical expression recognition more feasible for real-world applications.