Unconstrained Handwritten Text Research Articles

DAN: A Segmentation-Free Document Attention Network for Handwritten Document Recognition.

Unconstrained handwritten text recognition is a challenging computer vision task. It is traditionally handled by a two-step approach, combining line segmentation followed by text line recognition. For the first time, we propose an end-to-end segmentation-free architecture for the task of handwritten document recognition: the Document Attention Network. In addition to text recognition, the model is trained to label text parts using begin and end tags in an XML-like fashion. This model is made up of an FCN encoder for feature extraction and a stack of transformer decoder layers for a recurrent token-by-token prediction process. It takes whole text documents as input and sequentially outputs characters, as well as logical layout tokens. Contrary to the existing segmentation-based approaches, the model is trained without using any segmentation label. We achieve competitive results on the READ 2016 dataset at page level, as well as double-page level with a CER of 3.43% and 3.70%, respectively. We also provide results for the RIMES 2009 dataset at page level, reaching 4.54% of CER. We provide all source code and pre-trained model weights at https://github.com/FactoDeepLearning/DAN.

End-to-End Handwritten Paragraph Text Recognition Using a Vertical Attention Network.

Unconstrained handwritten text recognition remains challenging for computer vision systems. Paragraph text recognition is traditionally achieved by two models: the first one for line segmentation and the second one for text line recognition. We propose a unified end-to-end model using hybrid attention to tackle this task. This model is designed to iteratively process a paragraph image line by line. It can be split into three modules. An encoder generates feature maps from the whole paragraph image. Then, an attention module recurrently generates a vertical weighted mask enabling to focus on the current text line features. This way, it performs a kind of implicit line segmentation. For each text line features, a decoder module recognizes the character sequence associated, leading to the recognition of a whole paragraph. We achieve state-of-the-art character error rate at paragraph level on three popular datasets: 1.91% for RIMES, 4.45% for IAM and 3.59% for READ 2016. Our code and trained model weights are available at https://github.com/FactoDeepLearning/VerticalAttentionOCR.

Computer multimedia assisted language and literature teaching using Heuristic hidden Markov model and statistical language model

Computer technology has been used for decades in secondary education and foreign language preparation. Still, attempts to incorporate technology have presented educators with various challenges due to rapid progress in technology and occasional changes in language education methods. One way of improving student engagement is to provide connectivity throughout the teaching and learning process in a classroom, allowing students to improve their English language skills. The multimedia classroom allows students to interact with various texts, giving them a solid background intasks and material of mainstream college classes. This paper introduces a method to recognise large unconstrained handwritten text vocabulary using Heuristic Hidden Markov Model and Statistical Language model (HHMM-SLM).This provides statistical language models to be applied to enhance our system's performance. Numerous experiments with single and multiple writer data have been conducted. Language models have been used to improve system accuracy. The variable size Lexica is used (between 10,000 and 50,000 words). Further, a lexicon for English with a very precise width has been developed, which makes its contribution. Our approach is comprehensive and compared to other literature approaches to deal with the same issue.

Baseline detection of multi-lingual unconstrained handwritten text lines

Many handwritten text recognition systems use the baseline information for better recognition of text line characters. Improper baseline detection reduces the performance of the recognition. In this paper we propose a novel baseline detection scheme for unconstrained handwritten text lines of multilingual documents. For baseline detection of a text line, at first, we detect the set of significant contour points (S-points) of the text line. Every non-singleton subsets of S-points forms a curve. The orientation invariant features of the curve determine whether the curve can construct a probable baseline of the input text line or not. It is determined by an SVM, trained using the orientation invariant features of the curves. The curves classified as probable baselines, are sorted according to their relative positions in ascending order to get the optimal baseline. We tested our method on different handwritten text lines of Bangla(Bengali), English(Roman), Kannada, Oriya, Devnagari and Persian scripts and obtained encouraging results.

Unsupervised language model adaptation for handwritten Chinese text recognition

This paper presents an effective approach for unsupervised language model adaptation (LMA) using multiple models in offline recognition of unconstrained handwritten Chinese texts. The domain of the document to recognize is variable and usually unknown a priori, so we use a two-pass recognition strategy with a pre-defined multi-domain language model set. We propose three methods to dynamically generate an adaptive language model to match the text output by first-pass recognition: model selection, model combination and model reconstruction. In model selection, we use the language model with minimum perplexity on the first-pass recognized text. By model combination, we learn the combination weights via minimizing the sum of squared error with both L2-norm and L1-norm regularization. For model reconstruction, we use a group of orthogonal bases to reconstruct a language model with the coefficients learned to match the document to recognize. Moreover, we reduce the storage size of multiple language models using two compression methods of split vector quantization (SVQ) and principal component analysis (PCA). Comprehensive experiments on two public Chinese handwriting databases CASIA-HWDB and HIT-MW show that the proposed unsupervised LMA approach improves the recognition performance impressively, particularly for ancient domain documents with the recognition accuracy improved by 7 percent. Meanwhile, the combination of the two compression methods largely reduces the storage size of language models with little loss of recognition accuracy.

Transcript mapping for handwritten Chinese documents by integrating character recognition model and geometric context

Creating document image datasets with ground-truths of regions, text lines and characters is a prerequisite for document analysis research. However, ground-truthing large datasets is not only laborious and time consuming but also prone to errors due to the difficulty of character segmentation and the large variability of character shape, size and position. This paper describes an effective recognition-based annotation approach for ground-truthing handwritten Chinese documents. Under the Bayesian framework, the alignment of text line images with text transcript, which is the crucial step of annotation, is formulated as an optimization problem by incorporating geometric context of characters and character recognition model. We evaluated the alignment performance on a Chinese handwriting database CASIA-HWDB, which contains nearly four million character samples of 7356 classes and 5091 pages of unconstrained handwritten texts. The experimental results demonstrate the superiority of recognition-based text line alignment and the benefit of integrating geometric context. On a test set of 1015 handwritten pages (10,449 text lines), the proposed approach achieved character level alignment accuracy 92.32% when involving under-segmentation errors and 99.04% when excluding under-segmentation errors. The tool based on the proposed approach has been practically used for labeling handwritten Chinese documents.

Handwritten Chinese Text Recognition by Integrating Multiple Contexts

This paper presents an effective approach for the offline recognition of unconstrained handwritten Chinese texts. Under the general integrated segmentation-and-recognition framework with character oversegmentation, we investigate three important issues: candidate path evaluation, path search, and parameter estimation. For path evaluation, we combine multiple contexts (character recognition scores, geometric and linguistic contexts) from the Bayesian decision view, and convert the classifier outputs to posterior probabilities via confidence transformation. In path search, we use a refined beam search algorithm to improve the search efficiency and, meanwhile, use a candidate character augmentation strategy to improve the recognition accuracy. The combining weights of the path evaluation function are optimized by supervised learning using a Maximum Character Accuracy criterion. We evaluated the recognition performance on a Chinese handwriting database CASIA-HWDB, which contains nearly four million character samples of 7,356 classes and 5,091 pages of unconstrained handwritten texts. The experimental results show that confidence transformation and combining multiple contexts improve the text line recognition performance significantly. On a test set of 1,015 handwritten pages, the proposed approach achieved character-level accurate rate of 90.75 percent and correct rate of 91.39 percent, which are superior by far to the best results reported in the literature.

Piece-wise painting technique for line segmentation of unconstrained handwritten text: a specific study with Persian text documents

The most important and difficult task in text document analysis is to achieve line segmentation accurately, particularly when the document is composed of unconstrained handwritten text. To accomplish this objective a painting scheme is proposed in this research work. Being motivated by the fact that the handwritten Persian texts offer the most critical challenges in the process of text-line segmentation, the new method has been devised by studying the cursive Persian text scripts extensively; yet, in general the proposed line segmentation algorithm is applicable to handwritten text in any language/script. The text block is vertically decomposed into parallel pipe structures called as strip. Each row in each strip is painted by a gray intensity, which is the average intensity value of gray values of all pixels present in that row-strip. Subsequently, the painted pipes are converted into two-tone painting and it is smoothed. The white/black spaces in each pipe of the smoothed image are analyzed to get a short line of separation, phrased as Piece-wise Potential Separating Line (PPSL), between two consecutive black spaces. The PPSLs are concatenated to produce the segmentation of text lines. Some additional procedures are built to handle certain anomalies, which may occur. The scheme is validated by extensive experimentation. We tested the proposed algorithm with 52 pages of Persian text documents containing totally 823 lines and correct line segmentation of 92.35% is achieved. Moreover, the proposed algorithm was also tested with two different datasets of 152 and 200 handwritten text-pages of different languages. Efficiency and script independency of the proposed algorithm were proved when compared with various approaches presented in recent literature.

Offline handwritten Amharic word recognition

This paper describes two approaches for Amharic word recognition in unconstrained handwritten text using HMMs. The first approach builds word models from concatenated features of constituent characters and in the second method HMMs of constituent characters are concatenated to form word model. In both cases, the features used for training and recognition are a set of primitive strokes and their spatial relationships. The recognition system does not require segmentation of characters but requires text line detection and extraction of structural features, which is done by making use of direction field tensor. The performance of the recognition system is tested by a dataset of unconstrained handwritten documents collected from various sources, and promising results are obtained.

A new scheme for unconstrained handwritten text-line segmentation

Variations in inter-line gaps and skewed or curled text-lines are some of the challenging issues in segmentation of handwritten text-lines. Moreover, overlapping and touching text-lines that frequently appear in unconstrained handwritten text documents significantly increase segmentation complexities. In this paper, we propose a novel approach for unconstrained handwritten text-line segmentation. A new painting technique is employed to smear the foreground portion of the document image. The painting technique enhances the separability between the foreground and background portions enabling easy detection of text-lines. A dilation operation is employed on the foreground portion of the painted image to obtain a single component for each text-line. Thinning of the background portion of the dilated image and subsequently some trimming operations are performed to obtain a number of separating lines, called candidate line separators. By using the starting and ending points of the candidate line separators and analyzing the distances among them, related candidate line separators are connected to obtain segmented text-lines. Furthermore, the problems of overlapping and touching components are addressed using some novel techniques. We tested the proposed scheme on text-pages of English, French, German, Greek, Persian, Oriya and Bangla and remarkable results were obtained.

Text independent writer recognition using redundant writing patterns with contour-based orientation and curvature features

We propose an effective method for automatic writer recognition from unconstrained handwritten text images. Our method relies on two different aspects of writing: the presence of redundant patterns in the writing and its visual attributes. Analyzing small writing fragments, we seek to extract the patterns that an individual employs frequently as he writes. We also exploit two important visual attributes of writing, orientation and curvature, by computing a set of features from writing samples at different levels of observation. Finally we combine the two facets of handwriting to characterize the writer of a handwritten sample. The proposed methodology evaluated on two different data sets exhibits promising results on writer identification and verification.

A Novel Connectionist System for Unconstrained Handwriting Recognition

Recognizing lines of unconstrained handwritten text is a challenging task. The difficulty of segmenting cursive or overlapping characters, combined with the need to exploit surrounding context, has led to low recognition rates for even the best current recognizers. Most recent progress in the field has been made either through improved preprocessing or through advances in language modeling. Relatively little work has been done on the basic recognition algorithms. Indeed, most systems rely on the same hidden Markov models that have been used for decades in speech and handwriting recognition, despite their well-known shortcomings. This paper proposes an alternative approach based on a novel type of recurrent neural network, specifically designed for sequence labeling tasks where the data is hard to segment and contains long-range bidirectional interdependencies. In experiments on two large unconstrained handwriting databases, our approach achieves word recognition accuracies of 79.7 percent on online data and 74.1 percent on offline data, significantly outperforming a state-of-the-art HMM-based system. In addition, we demonstrate the network's robustness to lexicon size, measure the individual influence of its hidden layers, and analyze its use of context. Last, we provide an in-depth discussion of the differences between the network and HMMs, suggesting reasons for the network's superior performance.

Contour Vs Non-Contour based Word Segmentation from Handwritten Text Lines: an Experimental Analysis

This paper compares contour based and noncontours based techniques for extracting words from unconstrained handwritten text lines. Proposed novel approach is based on contours of the words rather only considering threshold for inter-word gaps as previous studies. In this approach, contour of each word is examined along with threshold for inter-word gaps to extract words with high confidence. Unlike previous studies, preprocessing technique is not applied, that enhance the speed significantly. Furthermore, a simple technique for punctuation detection is proposed to increase accuracy of word extraction. For fair comparison text lines are taken randomly from IAM benchmark database and threshold calculation is kept same for all techniques. Experiments thus performed, exhibit improved results and speed over the conventional word extraction methods. Furthermore, developed techniques and results are compared with the other approaches available in the literature using same benchmark database.

INTEGRATION OF n-GRAM LANGUAGE MODELS IN MULTIPLE CLASSIFIER SYSTEMS FOR OFFLINE HANDWRITTEN TEXT LINE RECOGNITION

Current multiple classifier systems for unconstrained handwritten text recognition do not provide a straightforward way to utilize language model information. In this paper, we describe a generic method to integrate a statistical n-gram language model into the combination of multiple offline handwritten text line recognizers. The proposed method first builds a word transition network and then rescores this network with an n-gram language model. Experimental evaluation conducted on a large dataset of offline handwritten text lines shows that the proposed approach improves the recognition accuracy over a reference system as well as over the original combination method that does not include a language model.

Toward automatic video-based whiteboard reading

The increasing popularity of whiteboards as a popular tool in meeting rooms has been accompanied by a growing interest in making use of the whiteboard as a user interface for human-computer interaction. Therefore, systems based on electronic whiteboards have been developed in order to serve as meeting assistants for, e.g., collaborative working. However, as special pens and erasers are required, natural interaction is restricted. In order to render this communication method more natural, it was proposed to retain ordinary whiteboard and pens and to visually observe the writing process using a video camera [22, 25]. In this paper a prototype system for automatic video-based whiteboard reading is presented. The system is designed for recognizing unconstrained handwritten text and is further characterized by an incremental processing strategy in order to facilitate recognizing portions of text as soon as they have been written on the board. We will present the methods employed for extracting text regions, preprocessing, feature extraction, and statistical modeling and recognition. Evaluation results on a writer-independent unconstrained handwriting recognition task demonstrate the feasibility of the proposed approach.

Off-line cursive handwriting recognition using multiple classifier systems—on the influence of vocabulary, ensemble, and training set size

Unconstrained handwritten text recognition is one of the most difficult problems in the field of pattern recognition. Recently, a number of classifier creation and combination methods, known as ensemble methods, have been proposed in the field of machine learning. They have shown improved recognition performance over single classifiers. In this paper, we examine the influence of the vocabulary size, the number of training samples, and the number of classifiers on the performance of three ensemble methods in the context of cursive handwriting recognition. All experiments were conducted using an off-line handwritten word recognizer based on hidden Markov models (HMMs).

Offline recognition of unconstrained handwritten texts using HMMs and statistical language models

This paper presents a system for the offline recognition of large vocabulary unconstrained handwritten texts. The only assumption made about the data is that it is written in English. This allows the application of Statistical Language Models in order to improve the performance of our system. Several experiments have been performed using both single and multiple writer data. Lexica of variable size (from 10,000 to 50,000 words) have been used. The use of language models is shown to improve the accuracy of the system (when the lexicon contains 50,000 words, the error rate is reduced by approximately 50 percent for single writer data and by approximately 25 percent for multiple writer data). Our approach is described in detail and compared with other methods presented in the literature to deal with the same problem. An experimental setup to correctly deal with unconstrained text recognition is proposed.

USING A STATISTICAL LANGUAGE MODEL TO IMPROVE THE PERFORMANCE OF AN HMM-BASED CURSIVE HANDWRITING RECOGNITION SYSTEM

In this paper, a system for the reading of totally unconstrained handwritten text is presented. The kernel of the system is a hidden Markov model (HMM) for handwriting recognition. This HMM is enhanced by a statistical language model. Thus linguistic knowledge beyond the lexicon level is incorporated in the recognition process. Another novel feature of the system is that the HMM is applied in such a way that the difficult problem of segmenting a line of text into individual words is avoided. A number of experiments with various language models and large vocabularies have been conducted. The language models used in the system were also analytically compared based on their perplexity.

External word segmentation of off-line handwritten text lines

Techniques are described to separate a line of unconstrained (written in a natural manner) handwritten text into words. When the writing style is unconstrained, recognition of individual components may be unreliable so they must be grouped together into word hypotheses, before recognition algorithms (which may require dictionaries) can be used. The system uses original algorithms to determine distances between components in a text line and to detect punctuation. The algorithms are tested on nearly 3000 handwritten text lines extracted from postal address blocks. A detailed performance analysis is given of the complete system and its components.