Text Recognition Model Research Articles

General Models for Handwritten Text Recognition: Feasibility and State-of-the Art. German Kurrent as an Example

Existing text recognition engines enables to train general models to recognize not only one specific hand but a multitude of historical hands within a particular script, and from a rather large time period (more than 100 years). This paper compares different text recognition engines and their performance on a test set independent of the training and validation sets. We argue that both, test set and ground truth, should be made available by researchers as part of a shared task to allow for the comparison of engines. This will give insight into the range of possible options for institutions in need of recognition models. As a test set, we provide a data set consisting of 2,426 lines which have been randomly selected from meeting minutes of the Swiss Federal Council from 1848 to 1903. To our knowledge, neither the aforementioned text lines, which we take as ground truth, nor the multitude of different hands within this corpus have ever been used to train handwritten text recognition models. In addition, the data set used is perfect for making comparisons involving recognition engines and large training sets due to its variability and the time frame it spans. Consequently, this paper argues that both the tested engines, HTR+ and PyLaia, can handle large training sets. The resulting models have yielded very good results on a test set consisting of unknown but stylistically similar hands.

TSRGAN: Real-world text image super-resolution based on adversarial learning and triplet attention

The text in a low-resolution (LR) image is usually hard to read. Super-resolution (SR) is an intuitive solution to this issue. Existing single image super-resolution (SISR) models are mainly trained on synthetic datasets whose LR images are obtained by performing bicubic interpolation or gaussian blur on high-resolution (HR) images. However, these models can hardly generalize to practical scenarios because real-world LR images are more difficult to super-resolve. The newly proposed TextZoom dataset is the first dataset for real-world text image super-resolution. We propose a new model termed TSRGAN trained on this dataset. First, a discriminator is designed to prevent the SR network from generating over-smoothed images. Second, we introduce triplet attention into the SR network for better representational ability. Moreover, besides L2 loss and adversarial loss, wavelet loss is incorporated to help reconstruct sharper character edges. Since TextZoom provides text labels, the recognition accuracy of scene text recognition (STR) model can be used to evaluate the quality of SR images. It can reflect the performance of text image SR models better than traditional SR evaluation metrics such as PSNR and SSIM. Comprehensive experiments show the superiority of our TSRGAN. Compared with the state-of-the-art method, the proposed TSRGAN improves the average recognition accuracy of ASTER, MORAN and CRNN by 0.8%, 1.5% and 3.2% on TextZoom respectively.

Improvement in Efficiency of The State-Of-The-Art Handwritten Text Recognition Models

In the past few years, the research in the discipline of Handwritten Text Recognition (HTR) has been fast-tracked as many researchers in computer vision are pursuing this discipline. Most of the deep learning models are likely to have vanishing gradient errors when processing paragraph images like scanned images. The most crucial problem with these models is that they have many parameters, which require a large amount of data and resources. So, the most recent offline HTR follows the Convolutional Recurrent Neural Network (CRNNs). Recently developed neural network architecture was used to get better results, namely Gated Convolutional Neural Network (Gated-CNN), which has fewer layers and parameters. The HTR based on Gated-CNN can outperform the CRNN based HTR. This research surpasses the state-of-the-art HTR system on five different handwritten datasets: Bentham, IAM, RIMES, Saint Gall, and Washington. This research needs low computational resources to use in real life, such as smartphones and robots.

Automated Sensing System for Real-Time Recognition of Trucks in River Dredging Areas Using Computer Vision and Convolutional Deep Learning.

Sand theft or illegal mining in river dredging areas has been a problem in recent decades. For this reason, increasing the use of artificial intelligence in dredging areas, building automated monitoring systems, and reducing human involvement can effectively deter crime and lighten the workload of security guards. In this investigation, a smart dredging construction site system was developed using automated techniques that were arranged to be suitable to various areas. The aim in the initial period of the smart dredging construction was to automate the audit work at the control point, which manages trucks in river dredging areas. Images of dump trucks entering the control point were captured using monitoring equipment in the construction area. The obtained images and the deep learning technique, YOLOv3, were used to detect the positions of the vehicle license plates. Framed images of the vehicle license plates were captured and were used as input in an image classification model, C-CNN-L3, to identify the number of characters on the license plate. Based on the classification results, the images of the vehicle license plates were transmitted to a text recognition model, R-CNN-L3, that corresponded to the characters of the license plate. Finally, the models of each stage were integrated into a real-time truck license plate recognition (TLPR) system; the single character recognition rate was 97.59%, the overall recognition rate was 93.73%, and the speed was 0.3271 s/image. The TLPR system reduces the labor force and time spent to identify the license plates, effectively reducing the probability of crime and increasing the transparency, automation, and efficiency of the frontline personnel’s work. The TLPR is the first step toward an automated operation to manage trucks at the control point. The subsequent and ongoing development of system functions can advance dredging operations toward the goal of being a smart construction site. By intending to facilitate an intelligent and highly efficient management system of dredging-related departments by providing a vehicle LPR system, this paper forms a contribution to the current body of knowledge in the sense that it presents an objective approach for the TLPR system.

RETRACTED ARTICLE: Effective offline handwritten text recognition model based on a sequence-to-sequence approach with CNN–RNN networks

Automatic text recognition system might serve as an important factor in creating a paperless environment through digitizing and processing the existing paper documents in the upcoming days. Handwritten recognition using deep learning methods has been widely explored by many researchers. The existence of large quantity of data and a variety of algorithmic innovations enable the ease of training deep neural networks. Different techniques have been initiated in the literature for recognizing text from handwritten documents. This paper proposes a hybrid handwritten text recognition (H2TR) model using deep neural networks that use the sequence-to-sequence (Seq2Seq) approach. This hybrid model makes use of the salient features of convolution neural network (CNN) and recurrent neural network (RNN) with long–short-term memory network (LSTM). It uses CNN to extract the features from the handwritten image. The features that are extracted are later modelled with a sequence-to-sequence approach and fed to RNN–LSTM for encoding the visual features and decoding the sequence of letters that are available in the handwritten image. The proposed model is tested with IAM and RIMES handwritten databases, which shows competitive letter accuracy and word accuracy results.

Morphological Feature Aware Multi-CNN Model for Multilingual Text Recognition

Text recognition is a crucial and challenging task, which aims at translating a cropped text instance image into a target string sequence. Recently, Convolutional neural networks (CNN) have been widely used in text recognition tasks as it can effectively capture semantic and structural information in text. However, most existing methods are usually based on contextual clues. If only recognize a single character, the accuracy of these approaches can be reduced. For example, it is difficult to distinguish 0 and O in the traditional CNN network because they are very similar in composition and structure. To solve this problem, we propose a novel neural network model called Morphological Feature Aware Multi-CNN Model for Multilingual Text Recognition (MFAM-CNN) in this article. We introduce a contour extraction model to enrich the representation ability, which can distinguish characters with similar shapes. Self-adaptive text density classification module is designed to recognize characters of different densities, improving the accuracy of character recognition. In general, the model is more sensitive to the overall size of the text, which improves the recognition rate of similar text. To evaluate the effectiveness of our approach, we make a dataset containing Chinese, numbers, and letters called SC dataset. Extensive experiments are conducted on the above SC dataset and ARTI-TEXT dataset, the results demonstrate that our model significantly improves the performance, achieving 98.03% and 97.77% respectively.

Realistic Text Replacement With Non-Uniform Style Conditioning

In this work, we study the possibility of realistic text replacement. The goal of realistic text replacement is to replace text present in the image with user-supplied text. The replacement should be performed in a way that will not allow distinguishing the resulting image from the original one. We achieve this goal by developing a novel non-uniform style conditioning layer and apply it to an encoder-decoder ResNet based architecture. The resulting model is a single-stage model, with no post-processing. We train the model with a combination of adversarial, style, content and $L_{1}$ losses. Qualitative and quantitative evaluations show that the model achieves realistic text replacement and outperforms existing approaches in multilingual and challenging scenarios. Quantitative evaluation is performed with direct metrics, like SSIM and PSNR, and proxy metrics based on the performance of a text recognition model. The proposed model has several potential applications in augmented reality.

Simulation of English text recognition model based on ant colony algorithm and genetic algorithm

The difficulty of English text recognition lies in fuzzy image text classification and part-of-speech classification. Traditional models have a high error rate in English text recognition. In order to improve the effect of English text recognition, guided by machine learning ideas, this paper combines ant colony algorithm and genetic algorithm to construct an English text recognition model based on machine learning. Moreover, based on the characteristics of ant colony intelligent algorithm optimization, a method of using ant colony algorithm to solve the central node is proposed. In addition, this paper uses the ant colony algorithm to obtain the characteristic points in the study area and determine a reasonable number, and then combine the uniform grid to select some non-characteristic points as the central node of the core function, and finally use the central node with a reasonable distribution for modeling. Finally, this paper designs experiments to verify the performance of the model constructed in this paper and combines mathematical statistics to visually display the experimental results using tables and graphs. The research results show that the performance of the model constructed in this paper is good.

Scene text spotting based on end-to-end

Aiming at the problem that the traditional OCR processing method ignores the inherent connection between the text detection task and the text recognition task, This paper propose a novel end-to-end text spotting framework. The framework includes three parts: shared convolutional feature network, text detector and text recognizer. By sharing convolutional feature network, the text detection network and the text recognition network can be jointly optimized at the same time. On the one hand, it can reduce the computational burden; on the other hand, it can effectively use the inherent connection between text detection and text recognition. This model add the TCM (Text Context Module) on the basis of Mask RCNN, which can effectively solve the negative sample problem in text detection tasks. This paper propose a text recognition model based on the SAM-BiLSTM (spatial attention mechanism with BiLSTM), which can more effectively extract the semantic information between characters. This model significantly surpasses state-of-the-art methods on a number of text detection and text spotting benchmarks, including ICDAR 2015, Total-Text.

A Holistic Model for Recognition of Handwritten Arabic Text Based on the Local Binary Pattern Technique

<p class="0abstract">In this paper, we introduce a multi-stage offline holistic handwritten Arabic text recognition model using the Local Binary Pattern (LBP) technique and two machine-learning approaches; Support Vector Machines (SVM) and Artificial Neural Network (ANN). In this model, the LBP method is utilized for extracting the global text features without text segmentation. The suggested model was tested and utilized on version II of the IFN/ENIT database applying the polynomial, linear, and Gaussian SVM and ANN classifiers. Performance of the ANN was assessed using the Levenberg-Marquardt (LM), Bayesian Regularization (BR), and Scaled Conjugate Gradient (SCG) training methods. The classification outputs of the herein suggested model were compared and verified with the results obtained from two benchmark Arabic text recognition models (ATRSs) that are based on the Discrete Cosine Transform (DCT) and Principal Component Analysis (PCA) methods using various normalization sizes of images of Arabic text. The classification outcomes of the suggested model are promising and better than the outcomes of the examined benchmarks models. The best classification accuracies of the suggested model (97.46% and 94.92%) are obtained using the polynomial SVM classifier and the BR ANN training methods, respectively.</p>

An attention-based row-column encoder-decoder model for text recognition in Japanese historical documents

This paper presents an attention-based row-column encoder-decoder (ARCED) model for recognizing an input image of multiple text lines from Japanese historical documents without explicit segmentation of lines. The recognition system has three main parts: a feature extractor, a row-column encoder, and a decoder. We introduce a row-column BLSTM in the encoder and a residual LSTM network in the decoder. The whole system is trained end-to-end by a standard cross-entropy loss function, requiring only document images and their ground-truth text. We experimentally evaluate the performance of ARCED on the dataset of Japanese historical documents: Kana-PRMU. The results of the experiments show that ARCED outperforms the state-of-the-art recognition methods on the dataset. Furthermore, we demonstrate that the row-column BLSTM in the encoder and the residual LSTM in the decoder improves the performance of the encoder-decoder model for the recognition of Japanese historical document.

Cursive-Text: A Comprehensive Dataset for End-to-End Urdu Text Recognition in Natural Scene Images.

Reading text in natural scene images is an active research area in the fields of computer vision and pattern recognition as text detection, text recognition and script identification are required. In this data article, a comprehensive dataset for Urdu text detection and recognition in natural scene images is presented and analysed. To develop the dataset, more than 2500 natural scene images were captured using a digital camera and a built-in mobile phone camera. Three separate datasets for isolated Urdu character images, cropped word images and end-to-end text spotting were developed. The isolated Urdu character and cropped word images dataset contain a much larger number of samples than existing Arabic natural scene text datasets. The Urdu text spotting dataset contains images with Urdu, English and Sindhi text instances. However, the focus has been given to the Urdu text instances. The ground truths for each image in the isolated character, cropped word or text spotting datasets are provided separately. The proposed datasets can be used to perform Urdu text detection and recognition or end-to-end recognition in natural scenes. These datasets can also be helpful to develop Arabic and Persian natural scene text detection and recognition systems, as Urdu is a derived language of these scripts and has many similar letters. The datasets can also be helpful to develop multi-language translation systems, which can facilitate foreign tourists to read and translate multilingual text in natural scene images. To evaluate the datasets, state-of-the-art machine learning and deep neural networks were used to build the text detection and recognition models, where the best classification accuracies are achieved. To the best of the authors’ knowledge, this is the first dataset proposed for Urdu text detection, recognition or end-to-end text recognition in natural scene images. The aim of this data article is to present a benchmark work in the field of document analysis and recognition.Computer ScienceComputer Vision and Pattern RecognitionTablesFiguresImagesText FilesUsing a digital camera with a 20 megapixels (MP) sensor, an iPhone with a 12 MP back camera and a Samsung mobile with a 16MP back camera.RawAnalyzedEnvironmental factors such as illuminations, blurring and lighting conditions were considered while capturing images. The focus was given to the text within an image.The images in the dataset were obtained from the advertisement banners, sign-boards along the road side and streets, shop name boards, text written on the passing vehicles and walls.The images provided in this dataset were collected in different cities of Sindh, Pakistan.Summarized data are hosted with the article.The datasets and their related files are hosted in a Mendeley public data repository.DOI: https://data.mendeley.com/datasets/k5fz57zd9z/1URL: http://dx.doi.org/10.17632/k5fz57zd9z.1

Depression Tendency Screening Use Text Based Emotional Analysis Technique

This paper proposes a text recognition model for semantic analysis of the interview records related to depression, which can effectively identify whether the interviewee is a patient with depression tendency. It mainly consists of two components: 1) The framework of Support Vector Machine (SVM) for Classification of depression related questions; 2) the framework of Doc2vec and Text Convolutional Neural Network (TextCNN) for classification of whether the interviewee has a tendency to depression. Finally, the results obtained by the two classification methods are combined to establish a text classification model that is easy to analyze the tendency of depression.

Deep neural network with attention model for scene text recognition

The authors present a deep neural network (DNN) with attention model for scene text recognition. The proposed model does not require any segmentation of the input text image. The framework is inspired by the attention model presented recently for speech recognition and image captioning. In the proposed framework, feature extraction, feature attention and sequence recognition are integrated in a jointly trainable network. Compared with previous approaches, the following contributions are mainly made. (i) The attention model is applied into DNN to recognise scene text, and it can effectively solve the sequence recognition problem caused by variable length labels. (ii) Rigorous experiments are performed across a number of challenging benchmarks, including IIIT5K, SVT, ICDAR2003 and ICDAR2013 datasets. Results in experiments show that the proposed model is comparable or better than the state‐of‐the‐art methods. (iii) This model only contains 6.5 million parameters. Compared with other DNN models for scene text recognition, this model has the least number of parameters so far.

A blind deconvolution model for scene text detection and recognition in video

Text detection and recognition in poor quality video is a challenging problem due to unpredictable blur and distortion effects caused by camera and text movements. This affects the overall performance of the text detection and recognition methods. This paper presents a combined quality metric for estimating the degree of blur in the video/image. Then the proposed method introduces a blind deconvolution model that enhances the edge intensity by suppressing blurred pixels. The proposed deblurring model is compared with other state-of-the-art models to demonstrate its superiority. In addition, to validate the usefulness and the effectiveness of the proposed model, we conducted text detection and recognition experiments on blurred images classified by the proposed model from standard video databases, namely, ICDAR 2013, ICDAR 2015, YVT and then standard natural scene image databases, namely, ICDAR 2013, SVT, MSER. Text detection and recognition results on both blurred and deblurred video/images illustrate that the proposed model improves the performance significantly.

Fractional poisson enhancement model for text detection and recognition in video frames

Performing Laplacian operation on video images is a common technique to improve image contrast to achieve good text detection and recognition accuracies. However, it is a fact that when Laplacian operation enhances contrast, at the same time it introduces too many noises. To alleviate this, the existing methods propose different enhancement methods and filters. In this paper, we propose a generalized enhancement model based on fractional calculus to increase the quality of images obtained by Laplacian operation. The proposed method considers edges and their neighbor information to derive a mathematical model for enhancing low contrast information in video as well as in scene images. Experimental results of text detection and recognition methods on different databases show that the proposed enhancement model improves their accuracies significantly. The enhancement model is compared with standard enhancement models to show that the proposed model outperforms the existing models in terms of quality measures. The usefulness of the proposed model is validated through text detection and recognition experiments.

A robust model for on-line handwritten japanese text recognition

This paper describes a robust context integration model for on-line handwritten Japanese text recognition. Based on string class probability approximation, the proposed method evaluates the likelihood of candidate segmentation–recognition paths by combining the scores of character recognition, unary and binary geometric features, as well as linguistic context. The path evaluation criterion can flexibly combine the scores of various contexts and is insensitive to the variability in path length, and so, the optimal segmentation path with its string class can be effectively found by Viterbi search. Moreover, the model parameters are estimated by the genetic algorithm so as to optimize the holistic string recognition performance. In experiments on horizontal text lines extracted from the TUAT Kondate database, the proposed method achieves the segmentation rate of 0.9934 that corresponds to a f-measure and the character recognition rate of 92.80%.

A language model using variable length tokens for open-vocabulary Hangul text recognition

We propose a novel language model for Hangul text recognition. Without relying on prior linguistic knowledge in training, the proposed model learns variable length Hangul character sequences, which comprise the elementary tokens of Korean language, and their probabilities from statistics of a raw text corpus. Experiments in handwritten Hangul recognition shows that the proposed language model is effective in postprocessing of recognition results.