Online Handwritten Chinese Character Recognition Research Articles

Online Handwritten Chinese Character Recognition Based on 1-D Convolution and Two-Streams Transformers

Online Handwritten Chinese Character Recognition Based on 1-D Convolution and Two-Streams Transformers

Fast and Robust Online Handwritten Chinese Character Recognition With Deep Spatial and Contextual Information Fusion Network

Deep convolutional neural networks have achieved fairly high accuracy for single online handwritten Chinese character recognition (SOLHCCR). However, in real application scenarios, users always write multiple characters to form a complete sentence, and previous contextual information holds significant potential for improving the accuracy, robustness and efficiency of recognition. In this work, we first propose a simple and straightforward model named the vanilla compositional network (VCN) by coupling convolutional neural network with a sequence modeling architecture (i.e., a recurrent neural network or Transformer), which exploits the handwritten character's previous contextual information. Although VCN performs much better than the previous state-of-the-art SOLHCCR models, it is a two-stage architecture in nature. It suffers from high fragility when confronting with poorly written characters such as sloppy writing, and missing or broken strokes, due to relying heavily on contextual information. To improve the robustness of the OLHCCR model, we further propose a novel deep spatial & contextual information fusion network (DSCIFN). It utilizes an autoregresssive framework pre-trained on a large-scale sentence corpora as the backbone component, and highly integrates the spatial features of handwritten characters and their previous contextual information in a multi-layer fusion module. To verify the effectiveness of models, we reorganize a new form of online Chinese handwritten character with its previous context dataset, named OHCCC. Extensive experimental results demonstrate that DSCIFN achieves state-of-the-art performance and has increased strong robustness compared to VCN and previous SOLHCCR models. The in-depth empirical analysis and case study indicate that DSCIFN can significantly improve the efficiency of handwriting input because it does not need complete strokes to recognize a handwritten Chinese character precisely.

An End-to-End Classifier Based on CNN for In-Air Handwritten-Chinese-Character Recognition

A convolutional neural network (CNN) has been successfully applied to in-air handwritten-Chinese-character recognition (IAHCCR). However, the existing models based on CNN for IAHCCR need to convert the coordinate sequence of a character into images. This conversion process increases training and classifying time, and leads to the loss of information. In order to solve this problem, we propose an end-to-end classifier based on CNN for IAHCCR in this paper, which, to knowledge, is novel for online handwritten-Chinese-character recognition (OLHCCR). Specifically, our model based on CNN directly takes the original coordinate sequence of an in-air handwritten-Chinese-character as input, and the output of the full connection layer is pooled by global average pooling to form a fixed-size feature vector, which is sent to softmax for classification. Our model can not only directly process coordinate sequences such as the models based on recurrent neural network (RNN), but can also obtain the global structure information of characters. We conducted experiments on two datasets, IAHCC-UCAS2016 and SCUT-COUCH2009. The experimental results show a comparison with existing CNN models based on image processing or RNN-based methods; our method does not require data augmentation techniques nor an ensemble of multiple trained models, and only uses a more compact structure to obtain higher recognition accuracy.

Stroke constrained attention network for online handwritten mathematical expression recognition

In this paper, we propose a novel stroke constrained attention network (SCAN) which treats stroke as the basic unit for encoder-decoder based online handwritten mathematical expression recognition (HMER). Unlike previous methods which use trace points or image pixels as basic units, SCAN makes full use of stroke-level information for better alignment and representation. The proposed SCAN can be adopted in both single-modal (online or offline) and multi-modal HMER. For single-modal HMER, SCAN first employs a CNN-GRU encoder to extract point-level features from input traces in online mode and employs a CNN encoder to extract pixel-level features from input images in offline mode, then use stroke constrained information to convert them into online and offline stroke-level features. Using stroke-level features can explicitly group points or pixels belonging to the same stroke, therefore reduces the difficulty of symbol segmentation and recognition via the decoder with attention mechanism. For multi-modal HMER, other than fusing multi-modal information in decoder, SCAN can also fuse multi-modal information in encoder by utilizing the stroke based alignments between online and offline modalities. The encoder fusion is a better way for combining multi-modal information as it implements the information interaction one step before the decoder fusion so that the advantages of multiple modalities can be exploited earlier and more adequately. Besides, we propose an approach combining the encoder fusion and decoder fusion, namely encoder-decoder fusion, which can further improve the performance. Evaluated on a benchmark published by CROHME competition, the proposed SCAN achieves the state-of-the-art performance. Furthermore, by conducting experiments on an additional task: online handwritten Chinese character recognition (HCCR), we demonstrate the generality of our proposed method.

Stroke Sequence-Dependent Deep Convolutional Neural Network for Online Handwritten Chinese Character Recognition.

We propose a novel model, called stroke sequence-dependent deep convolutional neural network (SSDCNN), which uses the stroke sequence information and eight-directional features of Chinese characters for online handwritten Chinese character recognition (OLHCCR). SSDCNN learns the representation of OLHCCs by incorporating the natural sequence information of the strokes. Furthermore, it naturally incorporates the eight-directional features. First, SSDCNN inputs the stroke sequence and transforms it into stacks of feature maps following the writing order of the strokes. Second, the fixed-length, stroke sequence-dependent representations of OLHCC are derived through convolutional, residual, and max-pooling operations. Third, the stroke sequence-dependent representation is combined with the eight-directional features via a number of fully connected neural network layers. Finally, the Chinese characters are recognized using a softmax classifier. The SSDCNN is trained in two stages: 1) the whole architecture is pretrained using the training data until the performance converges to an acceptable degree. 2) The stroke sequence-dependent representation is combined with the eight-directional features by a fully connected neural network and a softmax layer for further training. The model was experimentally evaluated on the OLHCCR competition tasks of International Conference on Document Analysis and Recognition (ICDAR) 2013. The recognition error was a maximum 58.28% lower in SSDCNN than in a model using the eight-directional features alone (5.13% versus 2.14%). Owing to its high accuracy (97.86%), the proposed SSDCNN reduced the recognition error by approximately 18.0% as compared with that of the winning system in the ICDAR 2013 competition. SSDCNN integrated with an adaptation mechanism, called the SSDCNN+Adapt model, and reached a new state-of-the-art (SOTA) standard with an accuracy of 97.94%. The SSDCNN exploits the stroke sequence information to learn high-quality OLHCC representations. Moreover, the learned representation and the classical eight-directional features complement each other within the SSDCNN architecture.

A new hybrid-parameter recurrent neural network for online handwritten chinese character recognition

Bidirectional RNN (BRNN) is a common method of modeling time series and has been widely applied in many areas, including speech recognition, machine translation, natural language processing, scene text recognition. Compared with the unidirectional RNN, the bidirectional RNN usually obtains better performance but higher computational cost due to additional backward processing of an input sequence. In this paper, we present a new hybrid-parameter RNN which consists of two virtual unidirectional recurrent neural networks. The computational cost of the proposed RNN is only three-fourths of that of the bidirectional RNNs. In addition, we accumulate the feature vectors from different layers to obtain the output of the RNN system, which is an efficient way to combine all the feature vectors without increasing the model size. The experimental results on IAHCC-UCAS2016 dataset and ICDAR2013 competition database show that the hybrid-parameter RNN obtains a better recognition performance with lower computational cost, compared with the bidirectional RNN.

A new perspective: Recognizing online handwritten Chinese characters via 1-dimensional CNN

For online handwritten Chinese character recognition (OLHCCR), it has become a popular choice to employ the 2-dimensional convolutional neural network (2-D CNN) in recognizing the extracted feature images or utilize the recurrent neural network (RNN) to classify handwriting trajectories directly. Instead, here we propose to apply the 1-dimensional CNN (1-D CNN), which, to our knowledge, is novel in the context of OLHCCR. Specifically, a 1-D CNN is engaged upon the sequential handwriting trajectories. Each output sequence is then averaged over time to form a fixed-size vector representation, upon which the final classification is made via a softmax operation. Compared with the 2-D CNN architecture, our approach is capable of delivering better results with a more compact model. This is achieved without adopting the computationally demanding techniques that are necessary when working with the 2-D CNN, including data augmentation and feature image extraction. Furthermore, our method attains a faster test time speed compared with the RNN, and this is more pronounced in processing long sequences. Empirically, our method yields the near-state-of-the-art accuracy of 98.11% on ICDAR 2013 competition dataset, and the state-of-the-art accuracy of 97.14% on in-air handwriting dataset IAHCC-UCAS2016, respectively.

Toward high-performance online HCCR: A CNN approach with DropDistortion, path signature and spatial stochastic max-pooling

This paper presents an investigation of several techniques that increase the accuracy of online handwritten Chinese character recognition (HCCR). We propose a new training strategy named DropDistortion to train a deep convolutional neural network (DCNN) with distorted samples. DropDistortion gradually lowers the degree of character distortion during training, which allows the DCNN to better generalize. Path signature is used to extract effective features for online characters. Further improvement is achieved by employing spatial stochastic max-pooling as a method of feature map distortion and model averaging. Experiments were carried out on three publicly available datasets, namely CASIA-OLHWDB 1.0, CASIA-OLHWDB 1.1, and the ICDAR2013 online HCCR competition dataset. The proposed techniques yield state-of-the-art recognition accuracies of 97.67%, 97.30%, and 97.99%, respectively.

Writer adaptation via deeply learned features for online Chinese handwriting recognition

This paper proposes a novel framework of writer adaptation based on deeply learned features for online handwritten Chinese character recognition. Our motivation is to further boost the state-of-the-art deep learning-based recognizer by using writer adaptation techniques. First, to perform an effective and flexible writer adaptation, we propose a tandem architecture design for the feature extraction and classification. Specifically, a deep neural network (DNN) or convolutional neural network (CNN) is adopted to extract the deeply learned features which are used to build a discriminatively trained prototype-based classifier initialized by Linde---Buzo---Gray clustering techniques. In this way, the feature extractor can fully utilize the useful information of a DNN or CNN. Meanwhile, the prototype-based classifier could be designed more compact and efficient as a practical solution. Second, the writer adaption is performed via a linear transformation of the deeply learned features which is optimized with a sample separation margin-based minimum classification error criterion. Furthermore, we improve the generalization capability of the previously proposed discriminative linear regression approach for writer adaptation by using the linear interpolation of two transformations and adaptation data perturbation. The experiments on the tasks of both the CASIA-OLHWDB benchmark and an in-house corpus with a vocabulary of 20,936 characters demonstrate the effectiveness of our proposed approach.

Discovering similar Chinese characters in online handwriting with deep convolutional neural networks

A primary reason for performance degradation in unconstrained online handwritten Chinese character recognition is the subtle differences between similar characters. Various methods have been proposed in previous works to address the problem of generating similar characters. These methods are basically comprised of two components--similar character discovery and cascaded classifiers. The goal of similar character discovery is to make similar character pairs/sets cover as many misclassified samples as possible. It is observed that the confidence of convolutional neural network (CNN) is output by an end-to-end manner and it can be understood as one type of probability metric. In this paper, we propose an algorithm by leveraging CNN confidence for discovering similar character pairs/sets. Specifically, a deep CNN is applied to output the top ranked candidates and the corresponding confidence scores, followed by an accumulating and averaging procedure. We experimentally found that the number of similar character pairs for each class is diverse and the confusion degree of similar character pairs is varied. To address these problems, we propose an entropy- based similarity measurement to rank these similar character pairs/sets and reject those with low similarity. The experimental results indicate that by using 30,000 similar character pairs, our method achieves the hit rates of 98.44 and 98.05 % on CASIA-OLHWDB1.0 and CASIA-OLHWDB1.0---1.2 datasets, respectively, which are significantly higher than corresponding results produced by MQDF-based method (95.42 and 94.49 %). Furthermore, recognition of ten randomly selected similar character subsets with a two-stage classification scheme results in a relative error reduction of 30.11 % comparing with traditional single stage scheme, showing the potential usage of the proposed method.

DropSample: A new training method to enhance deep convolutional neural networks for large-scale unconstrained handwritten Chinese character recognition

Inspired by the theory of Leitner׳s learning box from the field of psychology, we propose DropSample, a new method for training deep convolutional neural networks (DCNNs), and apply it to large-scale online handwritten Chinese character recognition (HCCR). According to the principle of DropSample, each training sample is associated with a quota function that is dynamically adjusted on the basis of the classification confidence given by the DCNN softmax output. After a learning iteration, samples with low confidence will have a higher frequency of being selected as training data; in contrast, well-trained and well-recognized samples with very high confidence will have a lower frequency of being involved in the ongoing training and can be gradually eliminated. As a result, the learning process becomes more efficient as it progresses. Furthermore, we investigate the use of domain-specific knowledge to enhance the performance of DCNN by adding a domain knowledge layer before the traditional CNN. By adopting DropSample together with different types of domain-specific knowledge, the accuracy of HCCR can be improved efficiently. Experiments on the CASIA-OLHDWB 1.0, CASIA-OLHWDB 1.1, and ICDAR 2013 online HCCR competition datasets yield outstanding recognition rates of 97.33%, 97.06%, and 97.51% respectively, all of which are significantly better than the previous best results reported in the literature.

An irrelevant variability normalization approach to discriminative training of multi-prototype based classifiers and its applications for online handwritten Chinese character recognition

This paper presents an irrelevant variability normalization (IVN) approach to jointly discriminative training of feature transforms and multi-prototype based classifier for recognition of online handwritten Chinese characters. A sample separation margin based minimum classification error criterion is adopted in IVN-based training, while an Rprop algorithm is used for optimizing the objective function. For the IVN approach based on piecewise linear transforms, the corresponding recognizer can be made both compact and efficient by using a two-level fast-match tree whose internal nodes coincide with the labels of feature transforms. Furthermore, the IVN system using weighted sum of linear transforms outperforms that based on piecewise linear transforms. The effectiveness of the proposed approach is first confirmed using an in-house developed online Chinese handwriting corpus with a vocabulary of 9306 characters, and then further verified on a standard benchmark database for an online handwritten character recognition task with a vocabulary of 3755 characters.

Efficient online handwritten Chinese character recognition system using a two-dimensional functional relationship model

Efficient online handwritten Chinese character recognition system using a two-dimensional functional relationship modelThis paper presents novel feature extraction and classification methods for online handwritten Chinese character recognition (HCCR). TheX-graph andY-graph transformation is proposed for deriving a feature, which shows useful properties such as invariance to different writing styles. Central to the proposed method is the idea of capturing the geometrical and topological information from the trajectory of the handwritten character using theX-graph and theY-graph. For feature size reduction, the Haar wavelet transformation was applied on the graphs. For classification, the coefficient of determination (R2p) from the two-dimensional unreplicated linear functional relationship model is proposed as a similarity measure. The proposed methods show strong discrimination power when handling problems related to size, position and slant variation, stroke shape deformation, close resemblance of characters, and non-normalization. The proposed recognition system is applied to a database with 3000 frequently used Chinese characters, yielding a high recognition rate of 97.4% with reduced processing time of 75.31%, 73.05%, 58.27% and 40.69% when compared with recognition systems using the city block distance with deviation (CBDD), the minimum distance (MD), the compound Mahalanobis function (CMF) and the modified quadratic discriminant function (MQDF), respectively. High precision rates were also achieved.

Modeling inverse covariance matrices by expansion of tied basis matrices for online handwritten Chinese character recognition

The state-of-the-art modified quadratic discriminant function (MQDF) based approach for online handwritten Chinese character recognition (HCCR) assumes that the feature vectors of each character class can be modeled by a Gaussian distribution with a mean vector and a full covariance matrix. In order to achieve a high recognition accuracy, enough number of leading eigenvectors of the covariance matrix have to be retained in MQDF. This paper presents a new approach to modeling each inverse covariance matrix by basis expansion, where expansion coefficients are character-dependent while a common set of basis matrices are shared by all the character classes. Consequently, our approach can achieve a much better accuracy–memory tradeoff. The usefulness of the proposed approach to designing compact HCCR systems has been confirmed and demonstrated by comparative experiments on popular Nakayosi and Kuchibue Japanese character databases.

A new linguistic decoding method for online handwritten Chinese character recognition

This paper presents a new linguistic decoding method for online handwritten Chinese character recognition. The method employs a hybrid language model which combinesN-gram and linguistic rules by rule quantification technique. The linguistic decoding algorithm consists of three stages: word lattice construction, the optimal sentence hypothesis search and self-adaptive learning mechanism. The technique has been applied to palmtop computer’s online handwritten Chinese character recognition. Samples containing millions of characters were used to test the linguistic decoder. In the open experiment, accuracy rate up to 92% is achieved, and the error rate is reduced by 68%.

Large scale on-line handwritten Chinese character recognition using successor method based on stochastic regular grammar

In this paper, we propose an original method for the recognition of on-line handwritten Chinese characters using the successor method based on the stochastic regular grammar. We use Kohonen’s self-organizing feature map for feature extraction to get optimal sets of prototypical waveforms of peaks from sample data automatically. The strings of symbols are converted into matrices using the stochastic successor method, and are analyzed by simple calculation between matrices. In order to symbolize and analyze input patterns efficiently and accurately in a large scale, we employ a hierarchical approach. Using unrestricted handwritten characters, we obtained 94.34% recognition rate for the test patterns.

On-Line Handwritten Chinese Character Recognition Directed by Components with Dynamic Templates

Structural matching has been recognized as a promising approach for on-line Chinese character recognition. In order to reduce its great computational complexity and improve its performance, people have been seeking for ways to direct the matching of a whole character by the result of partial matching. In this paper, the authors proposed 45 basic components for 3,755 categories of daily-used Chinese characters to direct the stroke segment matching of whole characters. Since they are always located at either the beginning or the end of the stroke segment string of characters, these components are easy to be extracted and separated from other parts of a character. Besides, in our approach, the reference templates of these components are extracted dynamically from the corresponding segment string of characters when a specific matching is carried out. This strategy avoids building multiple templates for the components of the same kind but at different places of characters. The experiments show that the segment matching computation has been reduced greatly without reducing the correctness of matching.

Bipartite weighted matching for on-line handwritten Chinese character recognition

The matching of line segments between input and prototype characters can be formulated as bipartite weighted matching problem. Under the assumption that the distance of the two line segments and the unmatched penalty of any line segment are given, the matching goal is to find a matching such that the sum of the weights of matching edges and the penalties of unmatched vertices is minimum. In this paper, the Hungarian method is applied to solve the matching problem by a reduction algorithm. Moreover, a greedy algorithm based on the Hungarian method is proposed by restricting the above matching which satisfies the constraints of geometric relation. For each iteration in the greedy algorithm, a matched pair is deleted if the relation of their neighbors does not match and a new matching is then found by applying Hungarian method. Finally, we can find a stable matching that preserves the geometric relation. We have implemented this method to recognize on-line Chinese handwritten characters permitting both stroke-order variation and stroke-number variation and a 91% recognition rate is attained.

On-line handwritten Chinese character recognition via a fuzzy attribute representation

Chinese characters are constructed by strokes according to structural rules. Therefore, the geometric configurations of characters are important features for character recognition. In handwritten characters, stroke shapes and their spatial relations may vary to some extent. The attribute value of a structural identification is then a fuzzy quantity rather than a binary quantity. Recognizing these facts, we propose a fuzzy attribute representation (FAR) to describe the structural features of handwritten Chinese characters for an on-line Chinese character recognition (OLCCR) system. With a FAR. a fuzzy attribute graph for each handwritten character is created, and the character recognition process is thus transformed into a simple graph matching problem. This character representation and our proposed recognition method allow us to relax the constraints on stroke order and stroke connection. The graph model provides a generalized character representation that can easily incorporate newly added characters into an OLCCR system with an automatic learning capability. The fuzzy representation can describe the degree of structural deformation in handwritten characters. The character matching algorithm is designed to tolerate structural deformations to some extent. Therefore, even input characters with deformations can be recognized correctly once the reference dictionary of the recognition system has been trained using a few representative learning samples. Experimental results are provided to show the effectiveness of the proposed method.

On-Line Hand-Written Chinese Character Recognition by an Improved Perceptron Considering the Order of Classification.

On-Line Hand-Written Chinese Character Recognition by an Improved Perceptron Considering the Order of Classification.