Abstract
Given the similarity of handwritten formula symbols and various handwriting styles, this paper proposes a squeeze-extracted multi-feature convolution neural network (SE-MCNN) to improve the recognition rate of handwritten formula symbols. The system proposed in this paper integrates the eight-directional feature of the original sequence in the convolutional layer, which significantly compensates for the lost dynamic trajectory information in the handwritten formula symbol. Meanwhile, the joint loss is constructed to improve the discriminability of features in the way of supervised learning, which enlarges the inter-class difference and decreases inner-class similarity. The standard mathematical formula symbol library provided by the Competition Organization on Recognition of Online Handwritten Mathematical Expression (CROHME) is used to verify the effectiveness of the proposed algorithm. Experiments show that the proposed SE-MCNN approach outperforms the state-of-the-art methods even at the condition of without using the data augmentation.
Highlights
With the wide applications of handwriting input in electronic devices, the research of handwriting recognition technology has attracted attentions both from the academia and the industry [1]–[6]
This paper summarizes the three major problems of isolated handwritten formula symbol recognition (IHFSR): (1) Compared with only 10 categories of MNIST datasets, the datasets have hundreds of formula symbols, including numbers, Latin letters, and arithmetic symbols, such as ‘‘∇’’, ‘‘ ’’. (2) There are a number of similar symbols in the formula symbol library, resulting in lots of confusion among categories, such as ‘‘O’’ and ‘‘o’’, ‘‘S’’ and ‘‘s’’. (3) For the same type of symbol, handwriting style for formula symbols
The joint training makes the convolutional neural network extract more discriminative features to represent handwritten formula symbol images, and various loss functions are conceived in IHFSR based on the squeeze-extracted multi-feature convolution neural network (SE-MCNN) model
Summary
With the wide applications of handwriting input in electronic devices, the research of handwriting recognition technology has attracted attentions both from the academia and the industry [1]–[6]. Handwriting mathematical formula recognition can divide into two steps [5], [6], including symbol recognition and structural analysis. Paper focuses on designing a method to improve the performance of isolated handwritten formula symbol recognition (IHFSR). Zhang: Multi-Feature Learning by Joint Training for Handwritten Formula Symbol Recognition. To address these problems, this paper proposes a multifeature learning network model to improve the accuracy of IHFSR. The main novelties and contributions of the proposed method originate from the following aspects: (1) In the online mode, the eight-directional features are designed for the first time to compensate for the missed trajectory information in the image formation process, and the eight-directional Gabor feature is extracted in the offline mode to obtain the change of multi-directional gradient.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
