Text Recognition Model Research Articles

A survey of text detection and recognition algorithms based on deep learning technology

Optical Character Recognition (OCR) poses a crucial challenge within the realm of computer vision research, as it plays a pivotal role in converting vast amounts of unstructured text data into structured formats to support diverse artificial intelligence applications. The OCR process encompasses two core components: text detection and text recognition. Text detection involves identifying and extracting text regions, achieved through either object detection or segmentation techniques, while text recognition focuses on accurately deciphering the content within these identified regions. In recent years, remarkable strides have been made in the domain of text recognition, primarily driven by deep learning-based models. These models eliminate the need for manual feature processing and excel in recognizing text even within complex scenes, surpassing the performance of traditional text recognition methods and subsequently emerging as the dominant approach. The objective of this paper is to present a comprehensive survey of both text detection and text recognition models. Firstly, we systematically categorize and provide an overview of existing off-the-shelf text detection methods. Subsequently, we conduct an in-depth investigation of six distinct text recognition models, taking into account their unique implementations. Additionally, we explore and analyze the principal datasets that currently prevail in the field of text detection and recognition. Furthermore, this research entails a meticulous performance comparison of various text detection algorithms on the CTW1500, TotalText, and ICDAR2015 datasets. Additionally, we evaluate and scrutinize the efficacy of mainstream text recognition algorithms on the IIIT-5K, SVT, ICDAR2013, SVT-P, CUTE80, and ICDAR2015 datasets. Finally, we conclude with a discussion on the future development and research trends concerning text detection and recognition, providing insights that can further drive progress in this crucial area.

A hybrid deep-based model for scene text detection and recognition in meter reading

A hybrid deep-based model for scene text detection and recognition in meter reading

Text Recognition Model Based on Multi-Scale Fusion CRNN.

Scene text recognition is a crucial area of research in computer vision. However, current mainstream scene text recognition models suffer from incomplete feature extraction due to the small downsampling scale used to extract features and obtain more features. This limitation hampers their ability to extract complete features of each character in the image, resulting in lower accuracy in the text recognition process. To address this issue, a novel text recognition model based on multi-scale fusion and the convolutional recurrent neural network (CRNN) has been proposed in this paper. The proposed model has a convolutional layer, a feature fusion layer, a recurrent layer, and a transcription layer. The convolutional layer uses two scales of feature extraction, which enables it to derive two distinct outputs for the input text image. The feature fusion layer fuses the different scales of features and forms a new feature. The recurrent layer learns contextual features from the input sequence of features. The transcription layer outputs the final result. The proposed model not only expands the recognition field but also learns more image features at different scales; thus, it extracts a more complete set of features and achieving better recognition of text. The results of experiments are then presented to demonstrate that the proposed model outperforms the CRNN model on text datasets, such as Street View Text, IIIT-5K, ICDAR2003, and ICDAR2013 scenes, in terms of text recognition accuracy.

Optimal Classification Model for Text Detection and Recognition in Video Frames

Currently, the identification of text from video frames and normal scene images has got amplified awareness amongst analysts owing to its diverse challenges and complexities. Owing to a lower resolution, composite backdrop, blurring effect, color, diverse fonts, alternate textual placement among panels of photos and videos, etc., text identification is becoming complicated. This paper suggests a novel method for identifying texts from video with five stages. Initially, “video-to-frame conversion”, is done during pre-processing. Further, text region verification is performed and keyframes are recognized using CNN. Then, improved candidate text block extraction is carried out using MSER. Subsequently, “DCT features, improved distance map features, and constant gradient-based features” are extracted. These characteristics subsequently provided “Long Short-Term Memory (LSTM)” for detection. Finally, OCR is done to recognize the texts in the image. Particularly, the Self-Improved Bald Eagle Search (SI-BESO) algorithm is used to adjust the LSTM weights. Finally, the superiority of the SI-BESO-based technique over many other techniques is demonstrated.

Mathrm T^2Net: an improved image-based text transfer framework using background inpainting and text conversion

Text, which is regarded as one of the important clues for visual recognition, can provide rich and accurate high-level semantic information. Therefore, the detection and recognition of textual data have become a research hotspot in computer vision and artificial intelligence. However, the difficulty of data collection and the non-uniform distribution of characters still poses challenges for accurate text recognition, especially for recognizing complicated character sets, such as Chinese. To address small-sample text recognition, we propose an improved image-based text transfer framework, named mathrm T^2Net. This work can replace or modify the text content in an image so as to arbitrarily expand a recognition data set. Considering that the main challenge of text transfer lies in decoupling the complex interrelationship between text and background, a text content mask branch is first added into a background inpainting module so as to more realistically restore background textures. Second, a text recognition model is developed to guide the readability of the text transfer results in the text conversion module. Finally, a text fusion module is used to fuse the independent migrations of background and text. We examined the performance of our proposed framework in a real-word scene text recognition data set. Qualitative and quantitative results have proved the efficiency of our method in comparison with previous works.

Text Recognition from Images using CNN and MSER Algorithms

Abstract: The ability to recognize text from images is a great importance in a range of applications, including document analysis, images captioning and augmented reality. The reliability and accuracy of text extraction from images have been completely transformed by text recognition models using Optical Character Recognition (OCR) and Maximally Stable Extremal Regions (MSER) algorithms. In our study, we propose a text recognition model that leverages the advantages of both OCR and MSER algorithms to enhance the reliability and accuracy of the text extraction process. OCR algorithm serve as the fundamental basis for text recognition by utilizing advanced methods to separate and identify individual characters. None the less these algorithms can encounter difficulties when confronted with intricated backgrounds, images of poor quality, or text arranged in irregular layouts. To address these limitations, we integrate the MSER algorithm, which excels in detecting text regions by identifying maximally stable regions across different scales and intensities. Our proposed model follows a multi-stage approach. First, the input image, the MSER method is used to extract probable text locations. These regions are then refined using pre-processing techniques, such as noise removal and image enhancement, to improve OCR performance. Next, the refined regions are passed through the OCR algorithm, which utilizes machine learning and pattern recognition techniques to recognize the text within each regions. The recognized text is subsequently post-processed to refine the results and improve overall accuracy. The text recognition model is implemented using CNN (OCR which is part of CNN) and the Maximally stable Extremal regions (MSER) algorithms

Digital Image Text Recognition Using Machine Learning Algorithms

Abstract: Text recognition is used in various fields like document analysis, picture labeling, and text analysis, etc. The process of recognizing text from image is very crucial.. Employing Maximally Stable Extremal Regions (MSER) and Optical Character Recognition (OCR) algorithms have drastically changed the accuracy and reliability of text detection from photos. Our proposed work suggests a model for text recognition that makes use of both the MSER and OCR algorithms' benefits for the purpose of accuracy improvement and reliability of the digital text extraction procedure. OCR follows cutting-edge techniques to distinguish and identify individual characters, serving as the essential building block for text recognition. However, these algorithms may struggle when dealing with complex backdrops, blurry photos, or text that is structured in an atypical way. We use MSER approach, which excels at recognizing text sections by finding maximally stable regions across various severities and scales, to solve these constraints. The suggested model employs a multi-stage methodology. The MSER algorithm is used for extracting the likely text spots from the input image first. To boost OCR performance, these zones are then fine-tuned using pre-processing techniques including noise reduction and picture enhancement. The OCR system next processes the cleaned-up sections, identifying each region's text using machine learning and pattern recognition methods. The text that is recognized is then further processed to increase the accuracy and refine these findings. Thus, compared to most other models, the text recognition model that is built utilizing the MSER and the CNN (OCR, a component of CNN) algorithm performs better

Scene Text Recognition Based on Improved CRNN

Text recognition is an important research topic in computer vision. Scene text, which refers to the text in real scenes, sometimes needs to meet the requirement of attracting attention, and there is the situation such as deformation. At the same time, the image acquisition process is affected by factors such as occlusion, noise, and obstruction, making scene text recognition tasks more challenging. In this paper, we improve the CRNN model for text recognition, which has relatively low accuracy, poor performance in recognizing irregular text, and only considers obtaining text sequence information from a single aspect, resulting in incomplete information acquisition. Firstly, to address the problems of low text recognition accuracy and poor recognition of irregular text, we add label smoothing to ensure the model’s generalization ability. Then, we introduce the smoothing loss function from speech recognition into the field of text recognition, and add a language model to increase information acquisition channels, ultimately achieving the goal of improving text recognition accuracy. This method was experimentally verified on six public datasets and compared with other advanced methods. The experimental results show that this method performs well in most benchmark tests, and the improved model outperforms the original model in recognition performance.

Towards a general open dataset and model for late medieval Castilian text recognition (HTR/OCR)

Towards a general open dataset and model for late medieval Castilian text recognition (HTR/OCR)

An application of Vietnamese handwriting text recognition for information extraction from high school admission form

This paper presents an effective Vietnamese handwritten text recognition model by applying an improved convolutional recurrent neural networks (CRNNs) model to high school enrollment forms in Tay Ninh province, Vietnam. First, the proposed model extracts data areas containing text characters from forms. Then, we connect text boxes on the same row and divide the fields that containing text into three specific regions. Finally, we detect areas containing text characters for handwritten text recognition. We use word error rate (WER) to evaluate the recognition process and obtain a result of 0.3602. This result is one of the best solutions to the Vietnamese handwritten text recognition problem.

Discrete representation learning for handwritten text recognition

Handwritten text recognition, i.e., the conversion of scanned handwritten documents into machine-readable text, is a complex exercise due to the variability and complexity of handwriting. A common approach in handwritten text recognition consists of a feature extraction step followed by a recognizer. In this paper, we propose a novel DNN architecture for handwritten text recognition that extracts discrete representation from the input text-line image. The proposed model is constructed of an encoder–decoder network with an added quantization layer which applies a dictionary of representative vectors to discretize the latent variables. The dictionary and the network parameters are trained jointly through the k-means algorithm and back propagation, respectively. The performance of the suggested model is evaluated through conducting extensive experiments on five datasets, analyzing the effect of discrete representation on handwriting recognition. The results demonstrate that the use of feature discretization improves the performance of deep handwriting text recognition models when compared to the conventional DNN models with continuous representation. Specifically, the character error rate is decreased by 22% and 21.1% on IAM and ICFHR18 datasets, respectively.

Synthetized Multilanguage OCR Using CRNN and SVTR Models for Realtime Collaborative Tools

Background: Remote diagnosis using collaborative tools have led to multilingual joint working sessions in various domains, including comprehensive health care, and resulting in more inclusive health care services. One of the main challenges is providing a real-time solution for shared documents and presentations on display to improve the efficacy of noninvasive, safe, and far-reaching collaborative models. Classic optical character recognition (OCR) solutions fail when there is a mixture of languages or dialects or in case of the participation of different technical levels and skills. Due to the risk of misunderstandings caused by mistranslations or lack of domain knowledge of the interpreters involved, the technological pipeline also needs artificial intelligence (AI)-supported improvements on the OCR side. This study examines the feasibility of machine learning-supported OCR in a multilingual environment. The novelty of our method is that it provides a solution not only for different speaking languages but also for a mixture of technological languages, using artificially created vocabulary and a custom training data generation approach. Methods: A novel hybrid language vocabulary creation method is utilized in the OCR training process in combination with convolutional recurrent neural networks (CRNNs) and a single visual model for scene text recognition within the patch-wise image tokenization framework (SVTR). Data: In the research, we used a dedicated Python-based data generator built on dedicated collaborative tool-based templates to cover and simulated the real-life variances of remote diagnosis and co-working collaborative sessions with high accuracy. The generated training datasets ranged from 66 k to 8.5 M in size. Twenty-one research results were analyzed. Instruments: Training was conducted by using tuned PaddleOCR with CRNN and SVTR modeling and a domain-specific, customized vocabulary. The Weight & Biases (WANDB) machine learning (ML) platform is used for experiment tracking, dataset versioning, and model evaluation. Based on the evaluations, the training dataset was adjusted by using a different language corpus or/and modifications applied to templates. Results: The machine learning models recognized the multilanguage/hybrid texts with high accuracy. The highest precision scores achieved are 90.25%, 91.35%, and 93.89%. Conclusions: machine learning models for special multilanguages, including languages with artificially made vocabulary, perform consistently with high accuracy.

Towards a general open dataset and model for late medieval Castilian text recognition (HTR/OCR). Paper

Towards a general open dataset and model for late medieval Castilian text recognition (HTR/OCR). Paper

Towards a general open dataset and model for late medieval Castilian text recognition (HTR/OCR)

Towards a general open dataset and model for late medieval Castilian text recognition (HTR/OCR)

Pill Box Text Identification Using DBNet-CRNN.

The recognition process of natural scenes is complicated at present, and images themselves may be complex owing to the special features of natural scenes. In this study, we use the detection and recognition of pill box text as an application scenario and design a deep-learning-based text detection algorithm for such natural scenes. We propose an end-to-end graphical text detection and recognition model and implement a detection system based on the B/S research application for pill box recognition, which uses DBNet as the text detection framework and a convolutional recurrent neural network (CRNN) as the text recognition framework. No prior image preprocessing is required in the detection and recognition processes. The recognition result from the back-end is returned to the front-end display. Compared with traditional methods, this recognition process reduces the complexity of preprocessing prior to image detection and improves the simplicity of the model application. Experiments on the detection and recognition of 100 pill boxes demonstrate that the proposed method achieves better accuracy in text localization and recognition results than the previous CTPN + CRNN method. The proposed method is significantly more accurate and easier to use than the traditional approach in terms of both training and recognition processes.

Dual-Targeted Textfooler Attack on Text Classification Systems

Deep neural networks provide good performance on classification tasks such as those for image, audio, and text classification. However, such neural networks are vulnerable to adversarial examples. An adversarial example is a sample created by adding a small adversarial noise to an original data sample in such a way that it will be correctly classified by a human but misclassified by a deep neural network. Studies on adversarial examples have focused mainly on the image field, but research is expanding into the text field as well. Adversarial examples in the text field that are designed with two targets in mind can be useful in certain situations. In a military scenario, for example, if enemy models A and B use a text recognition model, it may be desirable to cause enemy model A tanks to go to the right and enemy model B self-propelled guns to go to the left by using strategically designed adversarial messages. Such a dual-targeted adversarial example could accomplish this by causing different misclassifications in different models, in contrast to single-target adversarial examples produced by existing methods. In this paper, I propose a method for creating a dual-targeted textual adversarial example for attacking a text classification system. Unlike the existing adversarial methods, which are designed for images, the proposed method creates dual-targeted adversarial examples that will be misclassified as a different class by each of two models while maintaining the meaning and grammar of the original sentence, by substituting words of importance. Experiments were conducted using the SNLI dataset and the TensorFlow library. The results demonstrate that the proposed method can generate a dual-targeted adversarial example with an average attack success rate of 82.2% on the two models.

Text Prior Guided Scene Text Image Super-resolution.

Scene text image super-resolution (STISR) aims to improve the resolution and visual quality of low-resolution (LR) scene text images, while simultaneously boost the performance of text recognition. However, most of the existing STISR methods regard text images as natural scene images, ignoring the categorical information of text. In this paper, we make an inspiring attempt to embed text recognition prior into STISR model. Specifically, we adopt the predicted character recognition probability sequence as the text prior, which can be obtained conveniently from a text recognition model. The text prior provides categorical guidance to recover high-resolution (HR) text images. On the other hand, the reconstructed HR image can refine the text prior in return. Finally, we present a multi-stage text prior guided super-resolution (TPGSR) framework for STISR. Our experiments on the benchmark TextZoom dataset show that TPGSR can not only effectively improve the visual quality of scene text images, but also significantly improve the text recognition accuracy over existing STISR methods. Our model trained on TextZoom also demonstrates certain generalization capability to the LR images in other datasets.

Digitizing Cyrillic Manuscripts for the Historical Dictionary of the Serbian Language Using Handwritten Text Recognition Technology

The paper explores the possibilities of using information technologies based on the principles of machine learning and artificial intelligence in the process of digitizing Cyrillic manuscripts for the purposes of creating a historical dictionary of the Serbian language. Empirical research is based on the use of the Transkribus software platform in the creation of a model for automatic text recognition of the manuscripts by Gavril Stefanović Venclović, the most significant and prolific Serbian cultural enthusiast of the 18th century, whose extensive manuscript legacy in Serbian vernacular represents the most significant primary source for the historical dictionary of the Serbian language of this period. Following the results of conducted research, it can be concluded that the process of digitizing Cyrillic manuscripts for the purposes of creating a historical dictionary of the Serbian language can be significantly accelerated using Transkribus by creating specific and generic models for automatic text recognition. The advantage of automatic text recognition compared to the traditional methods is particularly reflected in the possibility of continuous improvement of the performance of specific and generic models in accordance with the progress of the transcription process and the increase in the amount of digitized text that can be used to train a new version of the model.

Embedded System Based Raspberry Pi 4 for Text Detection and Recognition

Detecting and recognizing text from natural scene images presents a challenge because the image quality depends on the conditions in which the image is captured, such as viewing angles, blurring, sensor noise, etc. However, in this paper, a prototype for text detection and recognition from natural scene images is proposed. This prototype is based on the Raspberry Pi 4 and the Universal Serial Bus (USB) camera and embedded our text detection and recognition model, which was developed using the Python language. Our model is based on the deep learning text detector model through the Efficient and Accurate Scene Text Detector (EAST) model for text localization and detection and the Tesseract-OCR, which is used as an Optical Character Recognition (OCR) engine for text recognition. Our prototype is controlled by the Virtual Network Computing (VNC) tool through a computer via a wireless connection. The experiment results show that the recognition rate for the captured image through the camera by our prototype can reach 99.75% with low computational complexity. Furthermore, our prototype is more performant than the Tesseract software in terms of the recognition rate. Besides, it provides the same performance in terms of the recognition rate with a huge decrease in the execution time by an average of 89% compared to the EasyOCR software on the Raspberry Pi 4 board.

An Auto-Grading Oriented Approach for Off-Line Handwritten Organic Cyclic Compound Structure Formulas Recognition

Auto-grading, as an instruction tool, could reduce teachers’ workload, provide students with instant feedback and support highly personalized learning. Therefore, this topic attracts considerable attentions from researchers recently. To realize the automatic grading of handwritten chemistry assignments, the problem of chemical notations recognition should be solved first. The recent handwritten chemical notations recognition solutions belonging to the end-to-end trainable category suffered from the problem of lacking the accurate alignment information between the input and output. They serve the aim of reading notations into electrical devices to better prepare relevant e-documents instead of auto-grading handwritten assignments. To tackle this limitation to enable the auto-grading of handwritten chemistry assignments at a fine-grained level. In this work, we propose a component-detection-based approach for recognizing off-line handwritten Organic Cyclic Compound Structure Formulas (OCCSFs). Specifically, we define different components of OCCSFs as objects (including graphical objects and text objects), and adopt the deep learning detector to detect them. Then, regarding the detected text objects, we introduce an improved attention-based encoder-decoder model for text recognition. Finally, with these detection results and the geometric relationships of detected objects, this article designs a holistic algorithm for interpreting the spatial structure of handwritten OCCSFs. The proposed method is evaluated on a self-collected data set consisting of 3000 samples and achieves promising results.