Text Line Recognition Research Articles

Text Detection and Recognition for X-ray Weld Seam Images

X-ray weld seam images carry vital information about welds. Leveraging graphic–text recognition technology enables intelligent data collection in complex industrial environments, promising significant improvements in work efficiency. This study focuses on using deep learning methods to enhance the accuracy and efficiency of detecting weld seam information. We began by actively gathering a dataset of X-ray weld seam images for model training and evaluation. The study comprises two main components: text detection and text recognition. For text detection, we employed a model based on the DBNet algorithm and tailored post-processing techniques to the unique features of weld seam images. Through model training, we achieved efficient detection of the text regions, with 91% precision, 92.4% recall, and a 91.7% F1 score on the test dataset. In the text recognition phase, we introduced modules like CA, CBAM, and HFA to capture the character position information and global text features effectively. This optimization led to a remarkable text line recognition accuracy of 93.4%. In conclusion, our study provides an efficient deep learning solution for text detection and recognition in X-ray weld seam images, offering robust support for weld seam information collection in industrial manufacturing.

An approach for handwritten Chinese text recognition unifying character segmentation and recognition

Text line recognition methods can be categorized into explicit segmentation based and implicit segmentation based ones. Explicit segmentation based methods require character-level annotation during training, while implicit segmentation based methods, trained on line-level annotated data, face alignment drift challenges. Though some methods have been proposed to address these challenges using weakly supervised object detection, they often rely on cumbersome pseudo-box generation processes and complex decoding. In this paper, we propose a unified framework to overcome these challenges, achieving high accuracy in text recognition and character segmentation. To eliminate the need of character-level annotated real text line data in training, we introduce a novel training paradigm that utilizes character-level annotated synthetic data and line-level annotated real data jointly. For synthetic data, candidate characters are explicitly aligned with labeled characters to generate hard labels for supervising model training. For real data, implicit alignment is produced by Connectionist Temporal Classification (CTC) mapping to provide soft labels for weakly-supervised model training. And for inference, we propose two decoding strategies leveraging the advantages of Non-Maximum Suppression (NMS) and CTC decoding. Extensive experiments on benchmark datasets demonstrate the superior performance of our method in text recognition and character localization, even with minimal amounts of character-level annotated line data.

Efficient CRNN: Towards end-to-end low resource Urdu text recognition using depthwise separable convolutions and gated recurrent units

In this study, a novel technique is proposed to recognize printed text in images for Urdu, a low-resource language with a scarcity of benchmark datasets. The proposed technique is called Efficient CRNN which uses depthwise separable convolutional and bidirectional gated recurrent unit layers, followed by connectionist temporal classification loss. The proposed technique is computationally more efficient than the existing text recognition techniques, requiring fewer parameters and computations. A multi-font printed Urdu text lines corpus is also presented, consisting of 245,000 text line images rendered using 7 different fonts. The corpus is called the MMU-Extension-22 and is used to train and evaluate existing state-of-the-art end-to-end text recognition techniques. Efficient CRNN is also evaluated using the proposed corpus. The proposed technique is first trained using a total of 196,000 text line images and then tested using 49,000 images. The Efficient CRNN technique achieved the minimum character and word error rates of 0.91% and 1.49% respectively for Urdu text line recognition under different settings, outperforming the existing computationally more complex techniques. The simple nature of the proposed technique not only makes it more efficient but also more robust for Urdu text line recognition, achieving a 2.23% reduced character error rate and a 71%11Percentage Decrease = 100 * (Baseline Value - Changed Value)/Baseline Value. decrease in character error rate as compared to the best performing existing Recurrent Neural Networks based technique. Also, the proposed technique outperforms Vision Transformer-based network achieving a 0.79% reduced character error rate accounting for a 41% decrease in error. Also, the Efficient CRNN has 49.16% reduced parameters compared to the baseline Vision Transformer technique.

DAN: A Segmentation-Free Document Attention Network for Handwritten Document Recognition.

Unconstrained handwritten text recognition is a challenging computer vision task. It is traditionally handled by a two-step approach, combining line segmentation followed by text line recognition. For the first time, we propose an end-to-end segmentation-free architecture for the task of handwritten document recognition: the Document Attention Network. In addition to text recognition, the model is trained to label text parts using begin and end tags in an XML-like fashion. This model is made up of an FCN encoder for feature extraction and a stack of transformer decoder layers for a recurrent token-by-token prediction process. It takes whole text documents as input and sequentially outputs characters, as well as logical layout tokens. Contrary to the existing segmentation-based approaches, the model is trained without using any segmentation label. We achieve competitive results on the READ 2016 dataset at page level, as well as double-page level with a CER of 3.43% and 3.70%, respectively. We also provide results for the RIMES 2009 dataset at page level, reaching 4.54% of CER. We provide all source code and pre-trained model weights at https://github.com/FactoDeepLearning/DAN.

End-to-End Handwritten Paragraph Text Recognition Using a Vertical Attention Network.

Unconstrained handwritten text recognition remains challenging for computer vision systems. Paragraph text recognition is traditionally achieved by two models: the first one for line segmentation and the second one for text line recognition. We propose a unified end-to-end model using hybrid attention to tackle this task. This model is designed to iteratively process a paragraph image line by line. It can be split into three modules. An encoder generates feature maps from the whole paragraph image. Then, an attention module recurrently generates a vertical weighted mask enabling to focus on the current text line features. This way, it performs a kind of implicit line segmentation. For each text line features, a decoder module recognizes the character sequence associated, leading to the recognition of a whole paragraph. We achieve state-of-the-art character error rate at paragraph level on three popular datasets: 1.91% for RIMES, 4.45% for IAM and 3.59% for READ 2016. Our code and trained model weights are available at https://github.com/FactoDeepLearning/VerticalAttentionOCR.

Pay attention to what you read: Non-recurrent handwritten text-Line recognition

The advent of recurrent neural networks for handwriting recognition marked an important milestone reaching impressive recognition accuracies despite the great variability that we observe across different writing styles. Sequential architectures are a perfect fit to model text lines, not only because of the inherent temporal aspect of text, but also to learn probability distributions over sequences of characters and words. However, using such recurrent paradigms comes at a cost at training stage, since their sequential pipelines prevent parallelization. In this work, we introduce a novel method that bypasses any recurrence during the training process with the use of transformer models. By using multi-head self-attention layers both at the visual and textual stages, we are able to tackle character recognition as well as to learn language-related dependencies of the character sequences to be decoded. Our model is unconstrained to any predefined vocabulary, being able to recognize out-of-vocabulary words, i.e. words that do not appear in the training vocabulary. We significantly advance over prior art and demonstrate that satisfactory recognition accuracies are yielded even in few-shot learning scenarios.

Densely connected layer to improve VGGnet-based CRNN for Arabic handwriting text line recognition

In recent years, Deep neural networks (DNNs) have achieved great success in sequence modeling. Several deep models have been used for enhancing Handwriting Text Recognition (HTR). Among these models, Convolutional Neural Networks (CNNs) and Recurrent Neural network especially Long-Short-Term-Memory (LSTM) networks achieve state-of-the-art recognition accuracy. The recognition methods for Arabic text lines have been widely applied in many specific tasks. However, there are still some potential challenges as the lack of available and large Arabic text recognition dataset and the characteristics of Arabic script. In order to address these challenges, we propose an end-to-end recognition method based on convolutional recurrent neural networks (CRNNs), which adds feature reuse network component on the basis of a CRNN. The model is trained and tested on two Arabic text recognition datasets named KHATT and AHTID/MW. The experimental results demonstrate that the proposed method achieves better performance than other methods in the literature.

Text Line Recognition of Dai Language using Statistical Characteristics of Texture Analysis and Deep Gaussian Process

In view of the different recognition methods of Dai in different language, we proposed a novel method of text line recognition for New Tai Lue and Lanna Dai based on statistical characteristics of texture analysis and Deep Gaussian process, which can classify different Dai text lines. First, the Dai text line database is constructed, and the images are preprocessed by de-noise and size standardization. Gabor multi-scale decomposition is carried out on two Dai text line images, and then the statistical features of image entropy and average row variance feature is extracted. The multi-layers Deep Gaussian process classifier is constructed. Experiments show that the accuracy of text line classification of New Tai Lue and Lanna Dai based on Deep Gaussian process is 99.89%, the values of precision, recall and f1-score are 1, 0.9978 and 0.9989, respectively. The combination of Gabor texture analysis average row variance statistical features and Deep Gaussian process model can effectively classify the text line of New Tai Lue and Lanna Dai. Comparative experiments show that the classification accuracy of the model is superior to traditional methods, such as Gaussian Naive Bayes, Random Forest, Decision Tree, and Gaussian Process.

Cyclic Autoencoder for Multimodal Data Alignment Using Custom Datasets

The subtitle recognition under multimodal data fusion in this paper aims to recognize text lines from image and audio data. Most existing multimodal fusion methods tend to be associated with pre-fusion as well as post-fusion, which is not reasonable and difficult to interpret. We believe that fusing images and audio before the decision layer, i.e., intermediate fusion, to take advantage of the complementary multimodal data, will benefit text line recognition. To this end, we propose: (i) a novel cyclic autoencoder based on convolutional neural network. The feature dimensions of the two modal data are aligned under the premise of stabilizing the compressed image features, thus the high-dimensional features of different modal data are fused at the shallow level of the model. (ii) A residual attention mechanism that helps us improve the performance of the recognition. Regions of interest in the image are enhanced and regions of disinterest are weakened, thus we can extract the features of the text regions without further increasing the depth of the model (iii) a fully convolutional network for video subtitle recognition. We choose DenseNet-121 as the backbone network for feature extraction, which effectively enabling the recognition of video subtitles in complex backgrounds. The experiments are performed on our custom datasets, and the automatic and manual evaluation results show that our method reaches the state-of-the-art.

SCUT-HCCDoc: A new benchmark dataset of handwritten Chinese text in unconstrained camera-captured documents

In this paper, we introduce a large-scale dataset, called SCUT-HCCDoc, to address challenging detection and recognition problems of handwritten Chinese text (HCT) in the camera-captured documents. Despite extensive studies of optical character recognition (OCR) and offline handwriting recognition for document images, text detection and recognition in the camera-captured documents remains an unsolved problem that is worth for extensive study and investigation. With recent advances in deep learning, researchers have proposed useful architectures for feature learning, detection, and recognition for the scene text. However, the performance of deep learning methods highly depends on the amount and diversity of training data. Previous OCR and offline HCT datasets were built under specific constraints, and most of the recent scene text datasets are for non-handwritten text. Hence, there is a lack of a comprehensive scene handwritten text benchmark. This study focuses on scenes with handwritten Chinese text. We introduce the SCUT-HCCDoc database for HCT detection, recognition and spotting. SCUT-HCCDoc contains 12,253 camera-captured document images with 116,629 text lines and 1,155,801 characters. The diversity of SCUT-HCCDoc can be described at three levels: (1) image-level diversity: image appearance and geometric variances caused by camera-captured settings (such as perspective, background, and resolution) and different applications (such as note-taking, test papers, and homework); (2) text-level diversity: variances of text line length, rotation, etc.; (3) character-level diversity: variances of character categories (up to 6109 classes with additional English letters, and digits), character size, individual writing style, etc. Three kinds of baseline experiments were conducted, where we used several popular text detection methods for text line detection, CTC-based/attention-based methods for text line recognition, and combine text detectors with CTC-based recognizer to achieve end-to-end text spotting. The results indicate the diversity of SCUT-HCCDoc and the challenges of HCT understanding in document images. The dataset is available at https://github.com/HCIILAB/SCUT-HCCDoc_Dataset_Release.

Two-Step CNN Framework for Text Line Recognition in Camera-Captured Images

In this paper, we introduce an “on the device” text line recognition framework that is designed for mobile or embedded systems. We consider per-character segmentation as a language-independent problem and individual character recognition as a language-dependent one. Thus, the proposed solution is based on two separate artificial neural networks (ANN) and dynamic programming instead of employing image processing methods for the segmentation step or end-to-end ANN. To satisfy the tight constraints on memory size imposed by embedded systems and to avoid overfitting, we employ ANNs with a small number of trainable parameters. The primary purpose of our framework is the recognition of low-quality images of identity documents with complex backgrounds and a variety of languages and fonts. We demonstrate that our solution shows high recognition accuracy on natural datasets even being trained on purely synthetic data. We use MIDV-500 and Census 1961 Project datasets for text line recognition. The proposed method considerably surpasses the algorithmic method implemented in Tesseract 3.05, the LSTM method (Tesseract 4.00), and unpublished method used in the ABBYY FineReader 15 system. Also, our framework is faster than other compared solutions. We show the language-independence of our segmenter with the experiment with Cyrillic, Armenian, and Chinese text lines.

MA-CRNN: a multi-scale attention CRNN for Chinese text line recognition in natural scenes

The recognition methods for Chinese text lines, as an important component of optical character recognition, have been widely applied in many specific tasks. However, there are still some potential challenges: (1) lack of open Chinese text recognition dataset; (2) challenges caused by the characteristics of Chinese characters, e.g., diverse types, complex structure and various sizes; (3) difficulties brought by text images in different scenes, e.g., blur, illumination and distortion. In order to address these challenges, we propose an end-to-end recognition method based on convolutional recurrent neural networks (CRNNs), i.e., multi-scale attention CRNN, which adds three components on the basis of a CRNN: asymmetric convolution, feature reuse network and attention mechanism. The proposed model is mainly aimed at scene text recognition including Chinese characters. Then the model is trained and tested on two Chinese text recognition datasets, i.e., the open dataset MTWI and our constructed large-scale Chinese text line dataset collected from various scenes. The experimental results demonstrate that the proposed method achieves better performance than other methods.

MIDV-500: a dataset for identity document analysis and recognition on mobile devices in video stream

A lot of research has been devoted to identity documents analysis and recognition on mobile devices. However, no publicly available datasets designed for this particular problem currently exist. There are a few datasets which are useful for associated subtasks but in order to facilitate a more comprehensive scientific and technical approach to identity document recognition more specialized datasets are required. In this paper we present a Mobile Identity Document Video dataset (MIDV-500) consisting of 500 video clips for 50 different identity document types with ground truth which allows to perform research in a wide scope of document analysis problems. The paper presents characteristics of the dataset and evaluation results for existing methods of face detection, text line recognition, and document fields data extraction. Since an important feature of identity documents is their sensitiveness as they contain personal data, all source document images used in MIDV-500 are either in public domain or distributed under public copyright licenses. The main goal of this paper is to present a dataset. However, in addition and as a baseline, we present evaluation results for existing methods for face detection, text line recognition, and document data extraction, using the presented dataset.

N-FTRN: Neighborhoods based fully convolutional network for Chinese text line recognition

The convolutional recurrent neural network is one of the most popular text recognition methods. Recurrent structures can extract long-term dependencies, but they are time consuming in computation compared with convolutional structures. We argue that the Chinese text line recognition can be performed based on neighbor rather than entire contextual information, and the information extracted from neighborhoods should only be a supplement to the information extracted from character regions. Therefore, we propose a novel neighborhoods based fully convolutional text recognition network (N-FTRN). It first extracts character-level feature sequences from text lines, then uses residual blocks instead of the recurrent structure to utilize contextual information. A reshape layer is applied to enable the network to recognize both vertical and horizontal text lines. Extensive experiments have been conducted to validate the efficiency and effectiveness of the proposed network. Compared with the state-of-the-art methods, we achieve comparable recognition performances on a Chinese scene text competition dataset (TRW) in ICDAR 2015 with much more compact models.

Zoning Features and 2DLSTM for Urdu Text-line Recognition

Recognition of Urdu cursive script is a challenging task due to the implicit complexities associated with it. The performance of a recognition system is immensely dependent on extracted features. There are various features extraction approaches proposed in recent years. Among many, an approach based on zoning features proved to be efficient and popular. Such zoning features represent significant information with low complexity and high speed. In this paper, we used zoning features for the classification of Urdu Nasta’liq text lines, with a combination of 2-Dimensional Long Short Term Memory networks (2DLSTM) as learning classifier. The proposed model is evaluated on publicly available UPTI dataset and character recognition rate of 93.39% is obtained.

Exploring More Representative States of Hidden Markov Model in Optical Character Recognition: A Clustering-Based Model Pre-Training Approach

Hidden Markov Model (HMM) is an effective method to describe sequential signals in many applications. As to model estimation issue, common training algorithm only focuses on the optimization of model parameters. However, model structure influences system performance as well. Although some structure optimization methods are proposed, they are usually implemented as an independent module before parameter optimization. In this paper, the clustering feature of states in HMM is discussed through comparing the mechanism of Quadratic Discriminant Function (QDF) classifier and HMM. Then, through the clustering effect of Viterbi training and Baum–Welch training, a novel clustering-based model pre-training approach is proposed. It can optimize model parameters and model structure by turns, until the representative states of all models are explored. Finally, the proposed approach is evaluated on two typical OCR applications, printed and handwritten Arabic text line recognition. And it is compared with some other optimization methods. The improvement of character recognition performance proves the proposed approach can make more precise state allocation. And the representative states are benefit to HMM decoding.

Character confidence based on N-best list for keyword spotting in online Chinese handwritten documents

In keyword spotting from handwritten documents by text query, the word similarity is usually computed by combining character similarities, which are desired to approximate the logarithm of the character probabilities. In this paper, we propose to directly estimate the posterior probability (also called confidence) of candidate characters based on the N-best paths from the candidate segmentation-recognition lattice. On evaluating the candidate segmentation-recognition paths by combining multiple contexts, the scores of the N-best paths are transformed to posterior probabilities using soft-max. The parameter of soft-max (confidence parameter) is estimated from the character confusion network, which is constructed by aligning different paths using a string matching algorithm. The posterior probability of a candidate character is the summation of the probabilities of the paths that pass through the candidate character. We compare the proposed posterior probability estimation method with some reference methods including the word confidence measure and the text line recognition method. Experimental results of keyword spotting on a large database CASIA-OLHWDB of unconstrained online Chinese handwriting demonstrate the effectiveness of the proposed method.

An over-segmentation method for single-touching Chinese handwriting with learning-based filtering

The segmentation of touching characters is still a challenging task, posing a bottleneck for offline Chinese handwriting recognition. In this paper, we propose an effective over-segmentation method with learning-based filtering using geometric features for single-touching Chinese handwriting. First, we detect candidate cuts by skeleton and contour analysis to guarantee a high recall rate of character separation. A filter is designed by supervised learning and used to prune implausible cuts to improve the precision. Since the segmentation rules and features are independent of the string length, the proposed method can deal with touching strings with more than two characters. The proposed method is evaluated on both the character segmentation task and the text line recognition task. The results on two large databases demonstrate the superiority of the proposed method in dealing with single-touching Chinese handwriting.

New baseline correction algorithm for text-line recognition with bidirectional recurrent neural networks

Many preprocessing techniques have been proposed for isolated word recognition. However, recently, recognition systems have dealt with text blocks and their compound text lines. In this paper, we propose a new preprocessing approach to efficiently correct baseline skew and fluctuations. Our approach is based on a sliding window within which the vertical position of the baseline is estimated. Segmentation of text lines into subparts is, thus, avoided. Experiments conducted on a large publicly available database (Rimes), with a BLSTM (bidirectional long short-term memory) recurrent neural network recognition system, show that our baseline correction approach highly improves performance.

Handwritten Chinese Text Recognition by Integrating Multiple Contexts

This paper presents an effective approach for the offline recognition of unconstrained handwritten Chinese texts. Under the general integrated segmentation-and-recognition framework with character oversegmentation, we investigate three important issues: candidate path evaluation, path search, and parameter estimation. For path evaluation, we combine multiple contexts (character recognition scores, geometric and linguistic contexts) from the Bayesian decision view, and convert the classifier outputs to posterior probabilities via confidence transformation. In path search, we use a refined beam search algorithm to improve the search efficiency and, meanwhile, use a candidate character augmentation strategy to improve the recognition accuracy. The combining weights of the path evaluation function are optimized by supervised learning using a Maximum Character Accuracy criterion. We evaluated the recognition performance on a Chinese handwriting database CASIA-HWDB, which contains nearly four million character samples of 7,356 classes and 5,091 pages of unconstrained handwritten texts. The experimental results show that confidence transformation and combining multiple contexts improve the text line recognition performance significantly. On a test set of 1,015 handwritten pages, the proposed approach achieved character-level accurate rate of 90.75 percent and correct rate of 91.39 percent, which are superior by far to the best results reported in the literature.