Online Handwritten Character Recognition Research Articles

Improving accuracy and explainability of online handwritten character recognition

Improving accuracy and explainability of online handwritten character recognition

Comparison of Various Algorithms for Handwritten Character Recognition of Indian Languages

Abstract: In this paper, we present a comparison of various pre-processor, feature extraction methods and algorithms for handwritten character recognition of various Indian languages. Comparison of classifier, feature set and accuracy of offline handwritten character recognition of Gujarati, Devanagari, Gurmukhi, Kannada, Malayalam, Bangla and Hindi Indian languages. Comparison of classifier, feature set and accuracy of online handwritten character recognition of Assamese, Tamil, Devanagari, Malayalam, Gurmukhi, and Bangla Indian languages. Indian language wise best performance of each language is compared for both offline and online handwritten character recognition systems.

A Review of online handwritten character and word recognition of Indian Languages

A Review of online handwritten character and word recognition of Indian Languages

Off-Line and Online Handwritten Character Recognition Using RNN-GRU Algorithm

Abstract: Recognizing handwritten characters is an extremely difficult task in the domains of pattern recognition and computer vision. It involves the use of a process that enables computers to identify and convert handwritten or printed characters, such as letters and numbers, into a digital format that is usable by the computer. Currently, the RNN-CNN hybrid algorithm is employed to predict handwritten text in images with an accuracy rate of 91.5%. However, the existing system can only recognize characters and words character-by-character and word-by-word. The proposed system aims to address this limitation by enabling line-byline recognition and the conversion of handwritten text to OCR. To achieve this, the system utilizes the GRU algorithm to predict the next letter in incomplete words. Furthermore, the IAM dataset, consisting of 135,000 annotated sentences, is utilized to detect and rectify spelling errors in texts.

Fast and Robust Online Handwritten Chinese Character Recognition With Deep Spatial and Contextual Information Fusion Network

Deep convolutional neural networks have achieved fairly high accuracy for single online handwritten Chinese character recognition (SOLHCCR). However, in real application scenarios, users always write multiple characters to form a complete sentence, and previous contextual information holds significant potential for improving the accuracy, robustness and efficiency of recognition. In this work, we first propose a simple and straightforward model named the vanilla compositional network (VCN) by coupling convolutional neural network with a sequence modeling architecture (i.e., a recurrent neural network or Transformer), which exploits the handwritten character's previous contextual information. Although VCN performs much better than the previous state-of-the-art SOLHCCR models, it is a two-stage architecture in nature. It suffers from high fragility when confronting with poorly written characters such as sloppy writing, and missing or broken strokes, due to relying heavily on contextual information. To improve the robustness of the OLHCCR model, we further propose a novel deep spatial & contextual information fusion network (DSCIFN). It utilizes an autoregresssive framework pre-trained on a large-scale sentence corpora as the backbone component, and highly integrates the spatial features of handwritten characters and their previous contextual information in a multi-layer fusion module. To verify the effectiveness of models, we reorganize a new form of online Chinese handwritten character with its previous context dataset, named OHCCC. Extensive experimental results demonstrate that DSCIFN achieves state-of-the-art performance and has increased strong robustness compared to VCN and previous SOLHCCR models. The in-depth empirical analysis and case study indicate that DSCIFN can significantly improve the efficiency of handwriting input because it does not need complete strokes to recognize a handwritten Chinese character precisely.

Recognizing on-line hand written alphanumeric character

Speed, accuracy, and flexibility are crucial to the practical use of on - line handwriting recognition. Besides, extensibility is also an important concern as we move from one domain to another which requires the character set to be extended. This research, produce a study for simple yet robust structural approach for recognizing on - line handwriting. In this approach is designed to achieve reasonable speed, high accuracy and sufficient tolerance to variations. At the same time, it maintains a high degree of reusability and hence facilitates extensibility. Experimental for study results show that the recognition rates are 98.60% for digits, 98.49% for the uppercase letters, 97.44% for lowercase letters, and 97.40% for the combined set. When the rejected cases are excluded from the calculation, the rates can be increased to 99.93%, 99.53%, 89.55% and 98.07%, respectively. Keywords: Structure extraction, structural primitives; Flexible structural matching, and On-line handwritten character recognition.

Online recognition of handwritten characters from scalp-recorded brain activities during handwriting

Objective. Brain–computer interfaces aim to build an efficient communication with the world using neural signals, which may bring great benefits to human society, especially to people with physical impairments. To date, the ability to translate brain signals to effective communication outcome remains low. This work explores whether the handwriting process could serve as a potential interface with high performance. To this end, we first examined how much the scalp-recorded brain signals encode information related to handwriting and whether it is feasible to precisely retrieve the handwritten content solely from the scalp-recorded electrical data. Approach. Five participants were instructed to write the sentence ‘HELLO, WORLD!’ repeatedly on a tablet while their brain signals were simultaneously recorded by electroencephalography (EEG). The EEG signals were first decomposed by independent component analysis for extracting features to be used to train a convolutional neural network (CNN) to recognize the written symbols. Main results. The accuracy of the CNN-based classifier trained and applied on the same participant (training and test data separated) ranged from 76.8% to 97.0%. The accuracy of cross-participant application was more diverse, ranging from 14.7% to 58.7%. These results showed the possibility of recognizing the handwritten content directly from the scalp level brain signal. A demonstration of the recognition system in an online mode was presented. The major factor that grounded the recognition was the close association between the rich dynamics of electroencephalogram source activities and the kinematic information during the handwriting movements. Significance. This work revealed an explicit and precise mapping between scalp-level electrophysiological signals and linguistic information conveyed by handwriting, which provided a novel approach to developing brain computer interfaces that focus on semantic communication.

Stroke constrained attention network for online handwritten mathematical expression recognition

In this paper, we propose a novel stroke constrained attention network (SCAN) which treats stroke as the basic unit for encoder-decoder based online handwritten mathematical expression recognition (HMER). Unlike previous methods which use trace points or image pixels as basic units, SCAN makes full use of stroke-level information for better alignment and representation. The proposed SCAN can be adopted in both single-modal (online or offline) and multi-modal HMER. For single-modal HMER, SCAN first employs a CNN-GRU encoder to extract point-level features from input traces in online mode and employs a CNN encoder to extract pixel-level features from input images in offline mode, then use stroke constrained information to convert them into online and offline stroke-level features. Using stroke-level features can explicitly group points or pixels belonging to the same stroke, therefore reduces the difficulty of symbol segmentation and recognition via the decoder with attention mechanism. For multi-modal HMER, other than fusing multi-modal information in decoder, SCAN can also fuse multi-modal information in encoder by utilizing the stroke based alignments between online and offline modalities. The encoder fusion is a better way for combining multi-modal information as it implements the information interaction one step before the decoder fusion so that the advantages of multiple modalities can be exploited earlier and more adequately. Besides, we propose an approach combining the encoder fusion and decoder fusion, namely encoder-decoder fusion, which can further improve the performance. Evaluated on a benchmark published by CROHME competition, the proposed SCAN achieves the state-of-the-art performance. Furthermore, by conducting experiments on an additional task: online handwritten Chinese character recognition (HCCR), we demonstrate the generality of our proposed method.

SVM and HMM Classifier Combination Based Approach for Online Handwritten Indic Character Recognition

Background: The growing use of smart hand-held devices in the daily lives of the people urges for the requirement of online handwritten text recognition. Online handwritten text recognition refers to the identification of the handwritten text at the very moment it is written on a digitizing tablet using some pen-like stylus. Several techniques are available for online handwritten text recognition in English, Arabic, Latin, Chinese, Japanese, and Korean scripts. However, limited research is available for Indic scripts. Objective: This article presents a novel approach for online handwritten numeral and character (simple and compound) recognition of three popular Indic scripts - Devanagari, Bengali and Tamil. Methods: The proposed work employs the Zone wise Slopes of Dominant Points (ZSDP) method for feature extraction from the individual characters. Support Vector Machine (SVM) and Hidden Markov Model (HMM) classifiers are used for recognition process. Recognition efficiency is improved by combining the probabilistic outcomes of the SVM and HMM classifiers using Dempster-Shafer theory. The system is trained using separate as well as combined dataset of numerals, simple and compound characters. Results: The performance of the present system is evaluated using large self-generated datasets as well as public datasets. Results obtained from the present work demonstrate that the proposed system outperforms the existing works in this regard. Conclusion: This work will be helpful to carry out researches on online recognition of handwritten character in other Indic scripts as well as recognition of isolated words in various Indic scripts including the scripts used in the present work.

Segmentation of Malayalam Handwritten Characters into Pattern Primitives and Recognition using SVM

This paper describes a lexical analysis (segmentation) approach in Pattern Recognition for Online Handwritten Character Recognition (OHCR) in Malayalam. The subunits (Pattern Primitives) in the single stroke vowel characters in Malayalam are identified and marked with pattern primitives to obtain a reference set of characters. Segmentation of the handwritten character samples into pattern primitives is made using a Combined Approach of Ramer Douglas Peucker algorithm and Eight Direction Freeman Code as per reference set. Features that are unique in the primitives of a character are extracted. The discriminating features identified are the direction of first primitive, segment count, cusp in second primitive, crossing in third primitive, and cusp in seventh primitive. The experiments were conducted on 100 samples per character that showed exact segmentation as per the reference set. With a five dimension feature set, the study achieved a recognition rate of 95.77% for five-fold cross-validation using Support Vector Machine with RBF kernel. The study shows that the segmentation of characters into pattern primitives is an effective method to realize accurate Malayalam OHCR systems for real-time applications.

Stroke Sequence-Dependent Deep Convolutional Neural Network for Online Handwritten Chinese Character Recognition.

We propose a novel model, called stroke sequence-dependent deep convolutional neural network (SSDCNN), which uses the stroke sequence information and eight-directional features of Chinese characters for online handwritten Chinese character recognition (OLHCCR). SSDCNN learns the representation of OLHCCs by incorporating the natural sequence information of the strokes. Furthermore, it naturally incorporates the eight-directional features. First, SSDCNN inputs the stroke sequence and transforms it into stacks of feature maps following the writing order of the strokes. Second, the fixed-length, stroke sequence-dependent representations of OLHCC are derived through convolutional, residual, and max-pooling operations. Third, the stroke sequence-dependent representation is combined with the eight-directional features via a number of fully connected neural network layers. Finally, the Chinese characters are recognized using a softmax classifier. The SSDCNN is trained in two stages: 1) the whole architecture is pretrained using the training data until the performance converges to an acceptable degree. 2) The stroke sequence-dependent representation is combined with the eight-directional features by a fully connected neural network and a softmax layer for further training. The model was experimentally evaluated on the OLHCCR competition tasks of International Conference on Document Analysis and Recognition (ICDAR) 2013. The recognition error was a maximum 58.28% lower in SSDCNN than in a model using the eight-directional features alone (5.13% versus 2.14%). Owing to its high accuracy (97.86%), the proposed SSDCNN reduced the recognition error by approximately 18.0% as compared with that of the winning system in the ICDAR 2013 competition. SSDCNN integrated with an adaptation mechanism, called the SSDCNN+Adapt model, and reached a new state-of-the-art (SOTA) standard with an accuracy of 97.94%. The SSDCNN exploits the stroke sequence information to learn high-quality OLHCC representations. Moreover, the learned representation and the classical eight-directional features complement each other within the SSDCNN architecture.

Learning Enhancement of Online Handwritten Telugu Character Modeling for Various Features Sets

Feature extraction plays vital role in online hand written character recognition. Local Features captured through co-ordinate system approach plays significant role in determining the online telugu character recognition. In this paper, we have instigated the performance of various features using Artificial Neural Networks( ANNs). ANN model is tested with various combination such as (x,y) co-ordinates , pen-up and pen-down ) , ( y x  , ( ) y x 2 2 , . Finally it is observed that ( ) y x 2 2 , features have given better accuracy. 95.18 % performance is obtained for 300 epochs for 52 Telugu characters. The database used for the study is HP-online Telugu database.

A new hybrid-parameter recurrent neural network for online handwritten chinese character recognition

Bidirectional RNN (BRNN) is a common method of modeling time series and has been widely applied in many areas, including speech recognition, machine translation, natural language processing, scene text recognition. Compared with the unidirectional RNN, the bidirectional RNN usually obtains better performance but higher computational cost due to additional backward processing of an input sequence. In this paper, we present a new hybrid-parameter RNN which consists of two virtual unidirectional recurrent neural networks. The computational cost of the proposed RNN is only three-fourths of that of the bidirectional RNNs. In addition, we accumulate the feature vectors from different layers to obtain the output of the RNN system, which is an efficient way to combine all the feature vectors without increasing the model size. The experimental results on IAHCC-UCAS2016 dataset and ICDAR2013 competition database show that the hybrid-parameter RNN obtains a better recognition performance with lower computational cost, compared with the bidirectional RNN.

Augmentation of Local, Global Feature Analysis for online Character Recognition System for Telugu Language using Feed Forward Neural Networks (FFNN)

In this paper, we propose ANN based online handwritten character recognition for Telugu Language. In literature review, it is observed that Size of the database and preprocessing approaches plays prominent role in the recognition performance. Preprocessing techniques like normalization, interpolation, Uniformization, Smoothing, Slant Correction and resampling techniques are performed for better recognition performance. Local features like(x,y)co-ordinates, (x,y) (2 x,2 y) and the global features like tan( ) are considered as features for ANN modeling and Classification of 52 Telugu vowels and consonants. Recognition performance is evaluated by augmentation the local, global features and and tan ( ) Features. Theperformance is evaluated in terms of precision, recall and F-measure. Significant Improvement is reported by augmentation andby adopting preprocessing techniques. The database used for the study is HP-online Telugu database.

A new perspective: Recognizing online handwritten Chinese characters via 1-dimensional CNN

For online handwritten Chinese character recognition (OLHCCR), it has become a popular choice to employ the 2-dimensional convolutional neural network (2-D CNN) in recognizing the extracted feature images or utilize the recurrent neural network (RNN) to classify handwriting trajectories directly. Instead, here we propose to apply the 1-dimensional CNN (1-D CNN), which, to our knowledge, is novel in the context of OLHCCR. Specifically, a 1-D CNN is engaged upon the sequential handwriting trajectories. Each output sequence is then averaged over time to form a fixed-size vector representation, upon which the final classification is made via a softmax operation. Compared with the 2-D CNN architecture, our approach is capable of delivering better results with a more compact model. This is achieved without adopting the computationally demanding techniques that are necessary when working with the 2-D CNN, including data augmentation and feature image extraction. Furthermore, our method attains a faster test time speed compared with the RNN, and this is more pronounced in processing long sequences. Empirically, our method yields the near-state-of-the-art accuracy of 98.11% on ICDAR 2013 competition dataset, and the state-of-the-art accuracy of 97.14% on in-air handwriting dataset IAHCC-UCAS2016, respectively.

Toward high-performance online HCCR: A CNN approach with DropDistortion, path signature and spatial stochastic max-pooling

This paper presents an investigation of several techniques that increase the accuracy of online handwritten Chinese character recognition (HCCR). We propose a new training strategy named DropDistortion to train a deep convolutional neural network (DCNN) with distorted samples. DropDistortion gradually lowers the degree of character distortion during training, which allows the DCNN to better generalize. Path signature is used to extract effective features for online characters. Further improvement is achieved by employing spatial stochastic max-pooling as a method of feature map distortion and model averaging. Experiments were carried out on three publicly available datasets, namely CASIA-OLHWDB 1.0, CASIA-OLHWDB 1.1, and the ICDAR2013 online HCCR competition dataset. The proposed techniques yield state-of-the-art recognition accuracies of 97.67%, 97.30%, and 97.99%, respectively.

Writer adaptation via deeply learned features for online Chinese handwriting recognition

This paper proposes a novel framework of writer adaptation based on deeply learned features for online handwritten Chinese character recognition. Our motivation is to further boost the state-of-the-art deep learning-based recognizer by using writer adaptation techniques. First, to perform an effective and flexible writer adaptation, we propose a tandem architecture design for the feature extraction and classification. Specifically, a deep neural network (DNN) or convolutional neural network (CNN) is adopted to extract the deeply learned features which are used to build a discriminatively trained prototype-based classifier initialized by Linde---Buzo---Gray clustering techniques. In this way, the feature extractor can fully utilize the useful information of a DNN or CNN. Meanwhile, the prototype-based classifier could be designed more compact and efficient as a practical solution. Second, the writer adaption is performed via a linear transformation of the deeply learned features which is optimized with a sample separation margin-based minimum classification error criterion. Furthermore, we improve the generalization capability of the previously proposed discriminative linear regression approach for writer adaptation by using the linear interpolation of two transformations and adaptation data perturbation. The experiments on the tasks of both the CASIA-OLHWDB benchmark and an in-house corpus with a vocabulary of 20,936 characters demonstrate the effectiveness of our proposed approach.

Discovering similar Chinese characters in online handwriting with deep convolutional neural networks

A primary reason for performance degradation in unconstrained online handwritten Chinese character recognition is the subtle differences between similar characters. Various methods have been proposed in previous works to address the problem of generating similar characters. These methods are basically comprised of two components--similar character discovery and cascaded classifiers. The goal of similar character discovery is to make similar character pairs/sets cover as many misclassified samples as possible. It is observed that the confidence of convolutional neural network (CNN) is output by an end-to-end manner and it can be understood as one type of probability metric. In this paper, we propose an algorithm by leveraging CNN confidence for discovering similar character pairs/sets. Specifically, a deep CNN is applied to output the top ranked candidates and the corresponding confidence scores, followed by an accumulating and averaging procedure. We experimentally found that the number of similar character pairs for each class is diverse and the confusion degree of similar character pairs is varied. To address these problems, we propose an entropy- based similarity measurement to rank these similar character pairs/sets and reject those with low similarity. The experimental results indicate that by using 30,000 similar character pairs, our method achieves the hit rates of 98.44 and 98.05 % on CASIA-OLHWDB1.0 and CASIA-OLHWDB1.0---1.2 datasets, respectively, which are significantly higher than corresponding results produced by MQDF-based method (95.42 and 94.49 %). Furthermore, recognition of ten randomly selected similar character subsets with a two-stage classification scheme results in a relative error reduction of 30.11 % comparing with traditional single stage scheme, showing the potential usage of the proposed method.

DropSample: A new training method to enhance deep convolutional neural networks for large-scale unconstrained handwritten Chinese character recognition

Inspired by the theory of Leitner׳s learning box from the field of psychology, we propose DropSample, a new method for training deep convolutional neural networks (DCNNs), and apply it to large-scale online handwritten Chinese character recognition (HCCR). According to the principle of DropSample, each training sample is associated with a quota function that is dynamically adjusted on the basis of the classification confidence given by the DCNN softmax output. After a learning iteration, samples with low confidence will have a higher frequency of being selected as training data; in contrast, well-trained and well-recognized samples with very high confidence will have a lower frequency of being involved in the ongoing training and can be gradually eliminated. As a result, the learning process becomes more efficient as it progresses. Furthermore, we investigate the use of domain-specific knowledge to enhance the performance of DCNN by adding a domain knowledge layer before the traditional CNN. By adopting DropSample together with different types of domain-specific knowledge, the accuracy of HCCR can be improved efficiently. Experiments on the CASIA-OLHDWB 1.0, CASIA-OLHWDB 1.1, and ICDAR 2013 online HCCR competition datasets yield outstanding recognition rates of 97.33%, 97.06%, and 97.51% respectively, all of which are significantly better than the previous best results reported in the literature.

Allograph modeling for online handwritten characters in devanagari using constrained stroke clustering

Writer-specific character writing variations such as those of stroke order and stroke number are an important source of variability in the input when handwriting is captured “online” via a stylus and a challenge for robust online recognition of handwritten characters and words. It has been shown by several studies that explicit modeling of character allographs is important for achieving high recognition accuracies in a writer-independent recognition system. While previous approaches have relied on unsupervised clustering at the character or stroke level to find the allographs of a character, in this article we propose the use of constrained clustering using automatically derived domain constraints to find a minimal set of stroke clusters. The allographs identified have been applied to Devanagari character recognition using Hidden Markov Models and Nearest Neighbor classifiers, and the results indicate substantial improvement in recognition accuracy and/or reduction in memory and computation time when compared to alternate modeling techniques.