Mathematical Symbol Hypothesis Recognition with Rejection Option

Abstract

In the context of handwritten mathematical expressions recognition, a first step consist on grouping strokes (segmentation) to form symbol hypotheses: groups of strokes that might represent a symbol. Then, the symbol recognition step needs to cope with the identification of wrong segmented symbols (false hypotheses). However, previous works on symbol recognition consider only correctly segmented symbols. In this work, we focus on the problem of mathematical symbol recognition where false hypotheses need to be identified. We extract symbol hypotheses from complete handwritten mathematical expressions and train artificial neural networks to perform both symbol classification of true hypotheses and rejection of false hypotheses. We propose a new shape context-based symbol descriptor: fuzzy shape context. Evaluation is performed on a publicly available dataset that contains 101 symbol classes. Results show that the fuzzy shape context version outperforms the original shape context. Best recognition and false acceptance rates were obtained using a combination of shape contexts and online features: 86% and 17.5% respectively. As false rejection rate, we obtained 8.6% using only online features.