Handwritten Text Recognition System Research Articles

Enhancing handwritten text recognition accuracy with gated mechanisms

Handwritten Text Recognition (HTR) is a challenging task due to the complex structures and variations present in handwritten text. In recent years, the application of gated mechanisms, such as Long Short-Term Memory (LSTM) networks, has brought significant advancements to HTR systems. This paper presents an overview of HTR using a gated mechanism and highlights its novelty and advantages. The gated mechanism enables the model to capture long-term dependencies, retain relevant context, handle variable length sequences, mitigate error propagation, and adapt to contextual variations. The pipeline involves preprocessing the handwritten text images, extracting features, modeling the sequential dependencies using the gated mechanism, and decoding the output into readable text. The training process utilizes annotated datasets and optimization techniques to minimize transcription discrepancies. HTR using a gated mechanism has found applications in digitizing historical documents, automatic form processing, and real-time transcription. The results show improved accuracy and robustness compared to traditional HTR approaches. The advancements in HTR using a gated mechanism open up new possibilities for effectively recognizing and transcribing handwritten text in various domains. This research does a better job than the most recent iteration of the HTR system when compared to five different handwritten datasets (Washington, Saint Gall, RIMES, Bentham and IAM). Smartphones and robots are examples of low-cost computing devices that can benefit from this research.

Albanian Handwritten Text Recognition using Synthetic Datasets and Pre-Trained Models

Handwritten Text Recognition (HTR) has continuously attracted the focus of researchers to enable the integration of technology into our daily lives. Handwritten text recognition (HTR), a technology of considerable importance, takes a leading role in the analysis and digitization of various documents. This technology is important in facilitating the efficient use of handwritten documents, especially within academic, historical, and cultural contexts. The use of artificial intelligence in handwriting recognition offers a very good opportunity to achieve satisfactory results in this field, but to achieve good results a large dataset is needed. Creating a large dataset to train different AI models is a challenge for languages with limited resources such as the Albanian language. This paper aims to present a novel approach to the development of an HTR system for the Albanian language using an attention-based encoder-decoder architecture. The dataset used in the experiments is a synthetic dataset generated using deep learning techniques based on the English language dataset as they are both variants of the Latin alphabet. We enhanced the dataset with two letters specific to Albanian, (“ë” and “ç”). The usage of pre-trained English models for handwriting recognition improved our model’s performance. The results of the experiments are very promising and prove that our approach is efficient in recognizing handwriting in the Albanian language. This shows that the attention-based encoder-decoder architecture can be adopted for different languages with limited resources.

Advancements and Trends in CNN-Based Handwritten Text Recognition: A Comprehensive Survey

Handwritten Text Recognition (HTR) is pivotal in transforming handwritten documents into digital format, enabling efficient search, storage, and information retrieval. In this project, we explore the application of Convolutional Neural Networks (CNNs) for HTR tasks. We comprehensively analyzed existing literature surveys to understand the current state-of-the-art techniques, methodologies, and challenges in HTR using CNNs. The survey encompassed various aspects including network architectures, dataset compositions, preprocessing techniques, and evaluation metrics. Our findings reveal the evolution of CNN-based HTR systems and highlight key trends in research, such as the integration of attention mechanisms and recurrent neural networks to enhance recognition accuracy and contextual understanding. Through this analysis, we provide insights into the advancements and future directions in CNN-based HTR methodologies.

Advancements and Challenges in Handwritten Text Recognition: A Comprehensive Survey.

Handwritten Text Recognition (HTR) is essential for digitizing historical documents in different kinds of archives. In this study, we introduce a hybrid form archive written in French: the Belfort civil registers of births. The digitization of these historical documents is challenging due to their unique characteristics such as writing style variations, overlapped characters and words, and marginal annotations. The objective of this survey paper is to summarize research on handwritten text documents and provide research directions toward effectively transcribing this French dataset. To achieve this goal, we presented a brief survey of several modern and historical HTR offline systems of different international languages, and the top state-of-the-art contributions reported of the French language specifically. The survey classifies the HTR systems based on techniques employed, datasets used, publication years, and the level of recognition. Furthermore, an analysis of the systems' accuracies is presented, highlighting the best-performing approach. We have also showcased the performance of some HTR commercial systems. In addition, this paper presents a summarization of the HTR datasets that publicly available, especially those identified as benchmark datasets in the International Conference on Document Analysis and Recognition (ICDAR) and the International Conference on Frontiers in Handwriting Recognition (ICFHR) competitions. This paper, therefore, presents updated state-of-the-art research in HTR and highlights new directions in the research field.

Handwritten text recognition using deep learning techniques: A survey

HTR (Handwritten Text Recognition) is the automated process of converting handwritten text into digital text, holding immense value in digitizing historical records and facilitating data entry. Through a combination of image processing and HTR systems decode handwritten characters and words. Pre-processing techniques increases image quality by reducing noise and correcting orientation, while models, like “convolutional neural networks” and “recurrent neural networks”, extract features and capture sequence patterns. Effective HTR models demand diverse training datasets and involve supervised learning to align predicted text with actual transcriptions. Post processing tools, including language models and spell-checkers, refine recognition outcomes. HTR's significance spans historical archive digitization, automated form processing, and aiding individuals with disabilities. Challenges persist in deciphering complex handwriting and handling degraded documents. The integration of deep learning advances HTR, enhancing its accuracy and efficiency, thereby expanding access to handwritten texts and enabling their digital search ability and edit ability. The outcome of this endeavor is a robust and user-friendly tool capable of converting handwritten notes, letters, manuscripts, and other textual materials into editable digital text. This project contributes significantly to bridging the gap between analog and digital information, offering immense potential for archival preservation, data accessibility improved productivity across domains.

Gated Convolution and Stacked Self-Attention Encoder–Decoder-Based Model for Offline Handwritten Ethiopic Text Recognition

Offline handwritten text recognition (HTR) is a long-standing research project for a wide range of applications, including assisting visually impaired users, humans and robot interactions, and the automatic entry of business documents. However, due to variations in writing styles, visual similarities between different characters, overlap between characters, and source document noise, designing an accurate and flexible HTR system is challenging. The problem becomes serious when the algorithm has a low learning capacity and when the text used is complex and has a lot of characters in the writing system, such as Ethiopic script. In this paper, we propose a new model that recognizes offline handwritten Ethiopic text using a gated convolution and stacked self-attention encoder–decoder network. The proposed model has a feature extraction layer, an encoder layer, and a decoder layer. The feature extraction layer extracts high-dimensional invariant feature maps from the input handwritten image. Using the extracted feature maps, the encoder and decoder layers transcribe the corresponding text. For the training and testing of the proposed model, we prepare an offline handwritten Ethiopic text-line dataset (HETD) with 2800 samples and a handwritten Ethiopic word dataset (HEWD) with 10,540 samples obtained from 250 volunteers. The experiment results of the proposed model on HETD show a 9.17 and 13.11 Character Error Rate (CER) and Word Error Rate (WER), respectively. However, the model on HEWD shows an 8.22 and 9.17 CER and WER, respectively. These results and the prepared datasets will be used as a baseline for future research.

Arabic Handwritten Text Recognition Systems and Challenges and Opportunities

Arabic Handwritten Text Recognition Systems and Challenges and Opportunities

Handwritten Text Recognition from Image

Abstract: A computer vision program called Handwritten Text Recognition (HTR) attempts to recognize and translate handwritten text from scanned or photographed images. In this project, we suggest implementing an HTR system using Tesseract and OpenCV. English, Chinese, and Arabic are all supported by the popular open-source optical character recognition (OCR) engine known as Tesseract. It is employed to find and identify printed text within photographs. On the other hand, OpenCV is a well-liked computer vision library that offers several tools for processing and analyzing images. The pre-processing step of the proposed system uses OpenCV to increase the input image's quality and OCR accuracy. After that, Tesseract receives the pre-processed image for text recognition. The extracted text is then saved in a text file after being identified. To enhance the quality of the input image, the project will use several pre-processing techniques, including deskewing, noise removal, and binarization. With the help of a sizable dataset of handwritten photographs, the Tesseract OCR engine is taught to recognize handwritten text more accurately. The HTR system can be used in a variety of fields, including document analysis, historical manuscript digitalization, and postal automation. It can also be applied in academic settings to help students translate their notes and assignments. Therefore, it is anticipated that the proposed HTR system employing Tesseract and OpenCV will offer a reliable and effective method for identifying and transcribing handwritten text from photographs.

End-to-End page-Level assessment of handwritten text recognition

The evaluation of Handwritten Text Recognition (HTR) systems has traditionally used metrics based on the edit distance between HTR and ground truth (GT) transcripts, at both the character and word levels. This is very adequate when the experimental protocol assumes that both GT and HTR text lines are the same, which allows edit distances to be independently computed to each given line. Driven by recent advances in pattern recognition, HTR systems increasingly face the end-to-end page-level transcription of a document, where the precision of locating the different text lines and their corresponding reading order (RO) play a key role. In such a case, the standard metrics do not take into account the inconsistencies that might appear. In this paper, the problem of evaluating HTR systems at the page level is introduced in detail. We analyse the convenience of using a two-fold evaluation, where the transcription accuracy and the RO goodness are considered separately. Different alternatives are proposed, analysed and empirically compared both through partially simulated and through real, full end-to-end experiments. Results support the validity of the proposed two-fold evaluation approach. An important conclusion is that such an evaluation can be adequately achieved by just two simple and well-known metrics: the Word Error Rate (WER), that takes transcription sequentiality into account, and the here re-formulated Bag of Words Word Error Rate (bWER), that ignores order. While the latter directly and very accurately assess intrinsic word recognition errors, the difference between both metrics (ΔWER) gracefully correlates with the Normalised Spearman’s Foot Rule Distance (NSFD), a metric which explicitly measures RO errors associated with layout analysis flaws. To arrive to these conclusions, we have introduced another metric called Hungarian Word Word Rate (hWER), based on a here proposed regularised version of the Hungarian Algorithm. This metric is shown to be always almost identical to bWER and both bWER and hWER are also almost identical to WER whenever HTR transcripts and GT references are guarantee to be in the same RO.

An Efficient Hybrid Model for Arabic Text Recognition

In recent years, Deep Learning models have become indispensable in several fields such as computer vision, automatic object recognition, and automatic natural language processing. The implementation of a robust and efficient handwritten text recognition system remains a challenge for the research community in this field, especially for the Arabic language, which, compared to other languages, has a dearth of published works. In this work, we presented an efficient and new system for offline Arabic handwritten text recognition. Our new approach is based on the combination of a Convolutional Neural Network (CNN) and a Bidirectional Long-Term Memory (BLSTM) followed by a Connectionist Temporal Classification layer (CTC). Moreover, during the training phase of the model, we introduce an algorithm of data augmentation to increase the quality of data. Our proposed approach can recognize Arabic handwritten texts without the need to segment the characters, thus overcoming several problems related to this point. To train and test (evaluate) our approach, we used two Arabic handwritten text recognition databases, which are IFN/ENIT and KHATT. The Experimental results show that our new approach, compared to other methods in the literature, gives better results.

A User Perspective on HTR Methods for the Automatic Transcription of Rare Scripts: The Case of Codex Runicus

Recent breakthroughs in Artificial Intelligence, Deep Learning, and Document Image Analysis and Recognition have significantly eased the creation of digital libraries and the transcription of historical documents. However, for documents in rare scripts with few labelled training data available, current Handwritten Text Recognition (HTR) systems are too constraining. Moreover, research on HTR often focuses on technical aspects only, and rarely puts emphasis on implementing software tools for scholars in Humanities. In this article, we describe, compare, and analyse different transcription methods for rare scripts. We evaluate their performance in a real-use case of a medieval manuscript written in the runic script ( Codex Runicus ) and discuss advantages and disadvantages of each method from the user perspective. From this exhaustive analysis and comparison with a fully manual transcription, we raise conclusions and provide recommendations to scholars interested in using automatic transcription tools.

A Survey on Digitization of Handwritten Notes in Kannada

Recognition of handwritten text is still an unresolved research problem in the field of optical character recognition. This article suggests an efficient method for creating handwritten text recognition systems. This is a challenging subject that has received a lot of attention recently. A discipline known as optical character recognition makes it possible to convert many kinds of texts or photos into editable, searchable, and analyzable data. Researchers have been using artificial intelligence and machine learning methods to automatically evaluate printed and handwritten documents during the past ten years in order to digitize them. This review paper's goals are to present research directions and a summary of previous studies on character recognition in handwritten texts. Since different people have different handwriting styles, handwritten characters might be challenging to read. Our "Digitization of handwritten notes" research and effort is to categorize and identify characters in the south Indian language of Kannada. The characters are extracted from printed texts and pre-processed using NumPy and OpenCV before being fed through a CNN

Lexicon and attention based handwritten text recognition system

The handwritten text recognition problem is widely studied by the researchers of computer vision community due to its scope of improvement and applicability to daily lives. It is a sub-domain of pattern recognition. Due to advancement of computational power of computers since last few decades neural networks based systems heavily contributed towards providing the state-of-the-art handwritten text recognizers. In the same direction, we have taken two state-of-the art neural networks systems and merged the attention mechanism with it. The attention technique has been widely used in the domain of neural machine translations and automatic speech recognition and now is being implemented in text recognition domain. In this study, we are able to achieve 4.15% character error rate and 9.72% word error rate on IAM dataset, 7.07% character error rate and 16.14% word error rate on GW dataset after merging the attention and word beam search decoder with existing Flor et al. architecture. To analyse further, we have also used system similar to Shi et al. neural network system with greedy decoder and observed 23.27% improvement in character error rate from the base model.

Textline alignment on the image domain

Editing and publishing a historical manuscript involves a research phase to recover the original manuscript and reconstruct the transmission of its text based on the relations between its surviving copies. Manuscript alignment, which aims to locate the shared and the different text among a set of copies of the same manuscript, is essential for this phase. In this paper, we present an alignment algorithm for historical handwritten documents that works directly on the image domain due to the absence of an accurate handwritten text recognition (HTR) system for handwritten historical documents and the necessity to visualize the original manuscripts in parallel to examine features beyond the transcribed text. Our approach extracts subwords, estimates the similarity among these subwords, and establishes an alignment among them. We extract subwords from textlines images and convert them into sequences of subword images. It estimates the similarity between two subwords using a Siamese network model and applies Longest Common Subsequence (LCS) to establish the alignment between two image sequences. We have implemented our algorithm, trained the Siamese model, and evaluate its performance using textline images from historical documents. Our algorithm outperformed the state-of-the-art by large margins. Unlike the state-of-the-art, the framework builds the alignment from scratch without requiring any prior knowledge concern subwords boundaries. In addition, we build a new dataset for textline alignment for historical documents, which include ten pairs of pages taken from two copies of two Arabic manuscripts and annotated at the subword level.

Generative adversarial network based adaptive data augmentation for handwritten Arabic text recognition.

Training deep learning based handwritten text recognition systems needs a lot of data in terms of text images and their corresponding annotations. One way to deal with this issue is to use data augmentation techniques to increase the amount of training data. Generative Adversarial Networks (GANs) based data augmentation techniques are popular in literature especially in tasks related to images. However, specific challenges need to be addressed in order to effectively use GANs for data augmentation in the domain of text recognition. Text data is inherently imbalanced in terms of frequency of different characters appearing in training samples and the training data as a whole. GANs trained on the imbalanced dataset leads to augmented data that does not represent the minority characters well. In this paper, we present an adaptive data augmentation technique using GANs that deals with the issue of class imbalance arising in text recognition problems. We show, using experimental evaluations on two publicly available datasets for handwritten Arabic text recognition, that the GANs trained using the presented technique is effective in dealing with class imbalanced problem by generating augmented data that is balanced in terms of character frequencies. The resulting text recognition systems trained on the balanced augmented data improves the text recognition accuracy as compared to the systems trained using standard techniques.

Automatic Estimation of Age Distributions from the First Ottoman Empire Population Register Series by Using Deep Learning

Recently, an increasing number of studies have applied deep learning algorithms for extracting information from handwritten historical documents. In order to accomplish that, documents must be divided into smaller parts. Page and line segmentation are vital stages in the Handwritten Text Recognition systems; it directly affects the character segmentation stage, which in turn determines the recognition success. In this study, we first applied deep learning-based layout analysis techniques to detect individuals in the first Ottoman population register series collected between the 1840s and the 1860s. Then, we employed horizontal projection profile-based line segmentation to the demographic information of these detected individuals in these registers. We further trained a CNN model to recognize automatically detected ages of individuals and estimated age distributions of people from these historical documents. Extracting age information from these historical registers is significant because it has enormous potential to revolutionize historical demography of around 20 successor states of the Ottoman Empire or countries of today. We achieved approximately 60% digit accuracy for recognizing the numbers in these registers and estimated the age distribution with Root Mean Square Error 23.61.

Improvement in Efficiency of The State-Of-The-Art Handwritten Text Recognition Models

In the past few years, the research in the discipline of Handwritten Text Recognition (HTR) has been fast-tracked as many researchers in computer vision are pursuing this discipline. Most of the deep learning models are likely to have vanishing gradient errors when processing paragraph images like scanned images. The most crucial problem with these models is that they have many parameters, which require a large amount of data and resources. So, the most recent offline HTR follows the Convolutional Recurrent Neural Network (CRNNs). Recently developed neural network architecture was used to get better results, namely Gated Convolutional Neural Network (Gated-CNN), which has fewer layers and parameters. The HTR based on Gated-CNN can outperform the CRNN based HTR. This research surpasses the state-of-the-art HTR system on five different handwritten datasets: Bentham, IAM, RIMES, Saint Gall, and Washington. This research needs low computational resources to use in real life, such as smartphones and robots.

Recognition and Digitization of Handwritten Text using Histogram of Gradients and Artificial Neural Network

Handwriting recognition is one of the most persuasive and interesting projects as it is required in many real-life applications such as bank-check processing, postal-code recognition, handwritten notes or question paper digitization etc. Machine learning and deep learning methods are being used by developers to make computers more intelligent. A person learns how to execute a task by learning and repeating it over and over before it memorises the steps. The neurons in his brain will then be able to easily execute the task that he has mastered. This is also very close to machine learning. It employs a variety of architectures to solve various problems. Handwritten text recognition systems are models that capture and interpret handwritten numeric and character data from sources such as paper documents and photographs. For this application, a variety of machine learning algorithms were used. However, several limitations have been found, such as a large number of iterations, high training costs, and so on. Even though the other models have given impressive accuracy, it still has some drawbacks. In an unsupervised way, the Artificial Neural Network is used to learn effective data coding. For recognising real-world data, we built a model using Histogram of Oriented Gradients (HOG) and Artificial Neural Networks (ANN).

Towards the Natural Language Processing as Spelling Correction for Offline Handwritten Text Recognition Systems

The increasing portability of physical manuscripts to the digital environment makes it common for systems to offer automatic mechanisms for offline Handwritten Text Recognition (HTR). However, several scenarios and writing variations bring challenges in recognition accuracy, and, to minimize this problem, optical models can be used with language models to assist in decoding text. Thus, with the aim of improving results, dictionaries of characters and words are generated from the dataset and linguistic restrictions are created in the recognition process. In this way, this work proposes the use of spelling correction techniques for text post-processing to achieve better results and eliminate the linguistic dependence between the optical model and the decoding stage. In addition, an encoder–decoder neural network architecture in conjunction with a training methodology are developed and presented to achieve the goal of spelling correction. To demonstrate the effectiveness of this new approach, we conducted an experiment on five datasets of text lines, widely known in the field of HTR, three state-of-the-art Optical Models for text recognition and eight spelling correction techniques, among traditional statistics and current approaches of neural networks in the field of Natural Language Processing (NLP). Finally, our proposed spelling correction model is analyzed statistically through HTR system metrics, reaching an average sentence correction of 54% higher than the state-of-the-art method of decoding in the tested datasets.

Urdu handwritten text recognition: a survey

Work on the problem of handwritten text recognition in Urdu script has been an active research area. A significant progress is made in this interesting and challenging field in the last few years. In this study, the authors presented a comprehensive survey for a number of offline and online handwritten text recognition systems for Urdu script written in Nastaliq font style from 2004 to 2019. Following features make their contribution worthwhile and unique among the reviews of a similar kind: (i) their review classifies the existing studies based on types of recognition systems used for Urdu handwritten text, (ii) it covers a very different outlook of the recognition process of the Urdu handwritten text at different granularity levels (e.g. character, word, ligature, or sentence level), (iii) this review article also presents each of surveyed articles in following dimensions: the task performed, its granularity level, dataset used, results obtained, and future dimensions, and (iv) lastly it gives the summary of the surveyed articles according to the granularity levels, publishing years, related tasks or subtasks, and types of classifiers used. In the end, major challenges and tasks related to Urdu handwritten text recognition approaches are also discussed in detail.