Text Line Images Research Articles

Automatic Visual Features for Writer Identification: A Deep Learning Approach

Identification of a person from his writing is one of the challenging problems; however, it is not new. No one can repudiate its applications in a number of domains, such as forensic analysis, historical documents, and ancient manuscripts. Deep learning-based approaches have proved as the best feature extractors from massive amounts of heterogeneous data and provide promising and surprising predictions of patterns as compared with traditional approaches. We apply a deep transfer convolutional neural network (CNN) to identify a writer using handwriting text line images in English and Arabic languages. We evaluate different freeze layers of CNN (Conv3, Conv4, Conv5, Fc6, Fc7, and fusion of Fc6 and Fc7) affecting the identification rate of the writer. In this paper, transfer learning is applied as a pioneer study using ImageNet (base data-set) and QUWI data-set (target data-set). To decrease the chance of over-fitting, data augmentation techniques are applied like contours, negatives, and sharpness using text-line images of target data-set. The sliding window approach is used to make patches as an input unit to the CNN model. The AlexNet architecture is employed to extract discriminating visual features from multiple representations of image patches generated by enhanced pre-processing techniques. The extracted features from patches are then fed to a support vector machine classifier. We realized the highest accuracy using freeze Conv5 layer up to 92.78% on English, 92.20% on Arabic, and 88.11% on the combination of Arabic and English, respectively.

Ground Truth for training OCR engines on historical documents in German Fraktur and Early Modern Latin

In this paper we describe a dataset of German and Latin \textit{ground truth} (GT) for historical OCR in the form of printed text line images paired with their transcription. This dataset, called \textit{GT4HistOCR}, consists of 313,173 line pairs covering a wide period of printing dates from incunabula from the 15th century to 19th century books printed in Fraktur types and is openly available under a CC-BY 4.0 license. The special form of GT as line image/transcription pairs makes it directly usable to train state-of-the-art recognition models for OCR software employing recurring neural networks in LSTM architecture such as Tesseract 4 or OCRopus. We also provide some pretrained OCRopus models for subcorpora of our dataset yielding between 95\% (early printings) and 98\% (19th century Fraktur printings) character accuracy rates on unseen test cases, a Perl script to harmonize GT produced by different transcription rules, and give hints on how to construct GT for OCR purposes which has requirements that may differ from linguistically motivated transcriptions.

Human Reading Knowledge Inspired Text Line Extraction

Text in images contains exact semantic information and the text knowledge can be utilized in many image cognition and understanding applications. The human reading habits can provide the clues of text line structure for text line extraction. In this paper, we propose a novel human reading knowledge inspired text line extraction method based on k-shortest paths global optimization. Firstly, the candidate character extraction is reformulated as Maximal Stable Extremal Region (MSER) algorithm on gray, red, blue, and green channels of the target images, and the extracted MSERs are fed into Convolutional Neural Network (CNN) to remove the noise components. Then, the directed graph is built upon the character component nodes with edges inspired by human reading sense. The directed graph can automatically construct the relationship to eliminate the disorder of candidate text components. The text line paths optimization is inspired by the human reading ability in planning of a text line path sequentially. Therefore, the text line extraction problem can be solved using the k-shortest paths optimization algorithm by taking advantage of the human reading sense structure of the directed graph. It can extract the text lines iteratively to avoid the exhaustive searching and obtain global optimized text line number. The proposed method achieves the f-measure of 0.820 and 0.812 on public ICDAR2011 and ICDAR2013 dataset, respectively. The experimental results demonstrate the effectiveness of the proposed human reading knowledge inspired text line extraction method in comparison with state-of-the-art methods This paper presents one human reading knowledge inspired text line extraction method, which approves that the human reading knowledge can benefit the text line extraction and image text discovery.

Date-field retrieval in scene image and video frames using text enhancement and shape coding

Text recognition in scene image and video frames is difficult because of low resolution, blur, background noise, etc. Since traditional OCRs do not perform well in such images, information retrieval using keywords could be an alternative way to index/retrieve such text information. Date is a useful piece of information which has various applications including date-wise videos/scene searching, indexing or retrieval. This paper presents a date spotting based information retrieval system for natural scene image and video frames where text appears with complex backgrounds. We propose a line based date spotting approach using Hidden Markov Model (HMM) which is used to detect the date information in a given text. Different date models are searched from a line without segmenting characters or words. Given a text line image in RGB, we apply an efficient gray image conversion to enhance the text information. Wavelet decomposition and gradient sub-bands are used to enhance text information in gray scale. Next, Pyramid Histogram of Oriented Gradient (PHOG) feature has been extracted from gray image and binary images for date-spotting framework. Binary and gray image features are combined by MLP based Tandem approach. Finally, to boost the performance further, a shape coding based scheme is used to combine the similar shape characters in same class during word spotting. For our experiment, three different date models have been constructed to search similar date information having numeric dates that contains numeral values and punctuations and semi-numeric that contains dates with numerals along with months in scene/video text. We have tested our system on 1648 text lines and the results show the effectiveness of our proposed date spotting approach.

Urdu Nastaliq recognition using convolutional–recursive deep learning

Recent developments in recognition of cursive scripts rely on implicit feature extraction methods that provide better results as compared to traditional hand-crafted feature extraction approaches. We present a hybrid approach based on explicit feature extraction by combining convolutional and recursive neural networks for feature learning and classification of cursive Urdu Nastaliq script. The first layer extracts low-level translational invariant features using Convolutional Neural Networks (CNN) which are then forwarded to Multi-dimensional Long Short-Term Memory Neural Networks (MDLSTM) for contextual feature extraction and learning. Experiments are carried out on the publicly available Urdu Printed Text-line Image (UPTI) dataset using the proposed hierarchical combination of CNN and MDLSTM. A recognition rate of up to 98.12% for 44-classes is achieved outperforming the state-of-the-art results on the UPTI dataset.

Word graphs size impact on the performance of handwriting document applications

Two document processing applications are considered: computer-assisted transcription of text images (CATTI) and Keyword Spotting (KWS), for transcribing and indexing handwritten documents, respectively. Instead of working directly on the handwriting images, both of them employ meta-data structures called word graphs (WG), which are obtained using segmentation-free handwritten text recognition technology based on N-gram language models and hidden Markov models. A WG contains most of the relevant information of the original text (line) image required by CATTI and KWS but, if it is too large, the computational cost of generating and using it can become unafordable. Conversely, if it is too small, relevant information may be lost, leading to a reduction of CATTI or KWS performance. We study the trade-off between WG size and performance in terms of effectiveness and efficiency of CATTI and KWS. Results show that small, computationally cheap WGs can be used without loosing the excellent CATTI and KWS performance achieved with huge WGs.

Offline cursive Urdu-Nastaliq script recognition using multidimensional recurrent neural networks

Optical Character Recognition of cursive scripts remains a challenging task due to a large number of character shapes, inter- and intra-word overlaps, context sensitivity and diagonality of text. This paper presents an implicit segmentation based recognition system for Urdu text lines in Nastaliq script. The proposed technique relies on sliding overlapped windows on lines of text and extracting a set of statistical features. The extracted features are fed to a multi-dimensional long short term memory recurrent neural network (MDLSTM RNN) with a connectionist temporal classification (CTC) output layer that labels the character sequences. Experimental study of the proposed technique is carried out on the standard Urdu Printed Text-line Images (UPTI) database which comprises 10,000 text lines in Nastaliq font. Evaluations under different experimental settings realize promising recognition rates with a highest character recognition rate of 96.40%.

Open-vocabulary recognition of machine-printed Arabic text using hidden Markov models

In this paper, we present multi-font printed Arabic text recognition using hidden Markov models (HMMs). We propose a novel approach to the sliding window technique for feature extraction. The size and position of the cells of the sliding window adapt to the writing line of Arabic text and ink-pixel distributions. We employ a two-step approach for mixed-font text recognition, in which the input text line image is associated with the closest known font in the first step, using simple and effective features for font identification. The text line is subsequently recognized by the recognizer that was trained for the particular font in the next step. This approach proves to be more effective than text recognition using a recognizer trained on samples from multiple fonts. We also present a framework for the recognition of unseen fonts, which employs font association and HMM adaptation techniques. Experiments were conducted using two separate databases of printed Arabic text to demonstrate the effectiveness of the presented techniques. The presented techniques can be easily adapted to other scripts, such as Roman script.

Arabic Handwriting Text Offline Recognition Using the HMM Toolkit (HTK)

This Recognition of Arabic text handwritten awaits precise recognition solutions. There are a lot of difficulties facing a good handwritten Arabic recognition system such as unlimited variant in human handwriting, similarities of different character shapes, and their location in the word. This paper presents a handwriting Arabic text recognition system. It decomposes the text image into text line images and extracts a set of simple statistical features from a narrow window which is sliding a long that text line, then it injects the resulting feature vectors to the Hidden Markov Model Toolkit (HTK). HTK is a portable toolkit for speech recognition system. In recognized state, the concatenation of characters to form words is modelled by simple lexical models, each word is modelled by a stochastic finite-state automaton (SFSA), and the concatenation of words into sentences is modelled by an n-gram language model. The proposed system is applied to a data corpus constructed by Text lines examples from the “Arabic-Numbers”, which contains 1905 sentences and 47 words. This phrase is written by 5 different peoples.

Transcript mapping for handwritten Chinese documents by integrating character recognition model and geometric context

Creating document image datasets with ground-truths of regions, text lines and characters is a prerequisite for document analysis research. However, ground-truthing large datasets is not only laborious and time consuming but also prone to errors due to the difficulty of character segmentation and the large variability of character shape, size and position. This paper describes an effective recognition-based annotation approach for ground-truthing handwritten Chinese documents. Under the Bayesian framework, the alignment of text line images with text transcript, which is the crucial step of annotation, is formulated as an optimization problem by incorporating geometric context of characters and character recognition model. We evaluated the alignment performance on a Chinese handwriting database CASIA-HWDB, which contains nearly four million character samples of 7356 classes and 5091 pages of unconstrained handwritten texts. The experimental results demonstrate the superiority of recognition-based text line alignment and the benefit of integrating geometric context. On a test set of 1015 handwritten pages (10,449 text lines), the proposed approach achieved character level alignment accuracy 92.32% when involving under-segmentation errors and 99.04% when excluding under-segmentation errors. The tool based on the proposed approach has been practically used for labeling handwritten Chinese documents.

Automatic writer identification from text line images

In the present article, new techniques have been introduced for revealing the individual features of a person’s handwriting pattern from the scanned images of handwritten text lines to facilitate text-independent writer identification. These techniques are aimed at designing a dynamic model which can be formalized according to any handwritten text line. Various combinations of the extracted features are applied to three well known classifiers for evaluating the contribution of features to define the correct identification rate. The K-NN, GMM, and Normal Density Discriminant Function Bayes classifiers are used in the present identification model. The experimental studies are conducted using two datasets obtained from the IAM database. The first dataset has already been proposed and used in the literature, whereas the second dataset is an expanded version of the first dataset and has been constituted for the first time in this study to analyze the performance of the extracted features under conditions such as an increased number of writers to discriminate in the database and a decreased number of text lines per writer. The remarkable identification rates obtained from the three classifiers on both datasets clearly indicate that the proposed feature extraction techniques can be effectively used in writer identification systems.

Offline recognition of omnifont Arabic text using the HMM ToolKit (HTK)

This paper presents a cursive Arabic text recognition system. The system decomposes the document image into text line images and extracts a set of simple statistical features from a narrow window which is sliding a long that text line. It then injects the resulting feature vectors to the Hidden Markov Model Toolkit (HTK). HTK is a portable toolkit for speech recognition system. The proposed system is applied to a data corpus which includes Arabic text of more than 600 A4-size sheets typewritten in multiple computer-generated fonts.

Lexicon-driven segmentation and recognition of handwritten character strings for Japanese address reading

This paper describes a handwritten character string recognition system for Japanese mail address reading on a very large vocabulary. The address phrases are recognized as a whole because there is no extra space between words. The lexicon contains 111,349 address phrases, which are stored in a trie structure. In recognition, the text line image is matched with the lexicon entries (phrases) to obtain reliable segmentation and retrieve valid address phrases. The paper first introduces some effective techniques for text line image preprocessing and presegmentation. In presegmentation, the text line image is separated into primitive segments by connected component analysis and touching pattern splitting based on contour shape analysis. In lexicon matching, consecutive segments are dynamically combined into candidate character patterns. An accurate character classifier is embedded in lexicon matching to select characters matched with a candidate pattern from a dynamic category set. A beam search strategy is used to control the lexicon matching so as to achieve real-time recognition. In experiments on 3,589 live mail images, the proposed method achieved correct rate of 83.68 percent while the error rate is less than 1 percent.

Recognition-based handwritten Chinese character segmentation using a probabilistic Viterbi algorithm

This paper presents a recognition-based character segmentation method for handwritten Chinese characters. Possible non-linear segmentation paths are initially located using a probabilistic Viterbi algorithm. Candidate segmentation paths are determined by verifying overlapping paths, between-character gaps, and adjacent-path distances. A segmentation graph is then constructed using candidate paths to represent nodes and two nodes with appropriate distances are connected by an arc. The cost in each arc is a function of character recognition distances, squareness of characters and internal gaps in characters. After the shortest path is detected from the segmentation graph, the nodes in the path represent optimal segmentation paths. In addition, 125 text-line images are collected from seven form documents. Cumulatively, these text-lines contain 1132 handwritten Chinese characters. The average segmentation rate in our experiments is 95.58%. Moreover, the probabilistic Viterbi algorithm is modified slightly to extract text-lines from document pages by obtaining non-linear paths while gaps between text-lines are not obvious. This algorithm can also be modified to segment characters from printed text-line images by adjusting parameters used to represent costs of arcs in the segmentation graph.