Text Line Detection Research Articles

Enhancing optical character recognition: Efficient techniques for document layout analysis and text line detection

AbstractIn recent years, automatic document and text analysis has gained significant importance, driven by advancements in optical character recognition (OCR) technology and the need for efficient processing of large volumes of printed or handwritten documents. This article specifically focuses on document layout analysis (DLA) and text line detection (TLD), both of which are crucial components of OCR systems. Our objective is to develop an effective method for extracting both textual and non‐textual regions, addressing challenges unique to the Persian (and Persian‐like) language(s). In the DLA stage, we employ deep learning models and a voting system to accurately determine the regions of interest. Additionally, we introduce methods such as optimum font size concepts, angle correction, and a line curvature elimination algorithm in the TLD process to enhance OCR accuracy. Comparative evaluations against state‐of‐the‐art methods demonstrate the superiority of our approach, showcasing a 2.8% improvement in the accuracy of Tesseract‐OCR 5.1.0 (a well‐established commercial OCR system) on the official Iranian newspapers dataset. These findings underscore the importance of addressing DLA and TLD challenges to advance OCR technology for Persian language documents and provide a solid foundation for future research in this domain.

UTTSR: A Novel Non-Structured Text Table Recognition Model Powered by Deep Learning Technology

To prevent the compilation of documents, many table documents are formatted with non-editable and non-structured texts such as PDFs or images. Quickly recognizing the contents of tables is still a challenge due to factors such as irregular formats, uneven text quality, and complex and diverse table content. This article proposes the UTTSR table recognition model, which consists of four parts: text region detection, text line detection and recognition, and table sequence recognition. For table detection, the Cascade Faster RCNN with the ResNeXt105 network is implemented, using TPS (Thin Plate Spline) transformation and affine transformation to correct the image and to improve accuracy. For text line detection, DBNET is used with Do-Conv in FPN (Feature Pyramid Networks) to speed up training. Text lines are recognized using CRNN without the CTC module, enhancing recognition performance. Table sequence recognition is based on the transformer combined with post-processing algorithms that fuse table structure sequences and unit grid content. Experimental results show that the UTTSR model outperforms the compared methods. This upgraded model significantly improves the accuracy of the previous state-of-the-art F1 score on complex tables, reaching 97.8%.

Historical document image analysis using controlled data for pre-training

Using neural networks for semantic labeling has become a dominant technique for layout analysis of historical document images. However, to train or fine-tune appropriate models, large labeled datasets are needed. This paper addresses the case when only limited labeled data are available and promotes a novel approach using so-called controlled data to pre-train the networks. Two different strategies are proposed: The first addresses the real labeling task by using artificial data; the second uses real data to pre-train the networks with a pretext task. To assess these strategies, a large set of experiments has been carried out on a text line detection and classification task using different variants of U-Net. The observations, obtained from two different datasets, show that globally the approach reduces the training time while offering similar or better performance. Furthermore, the effect is bigger on lightweight network architectures.

Large-scale genealogical information extraction from handwritten Quebec parish records

This paper presents a complete workflow designed for extracting information from Quebec handwritten parish registers. The acts in these documents contain individual and family information highly valuable for genetic, demographic and social studies of the Quebec population. From an image of parish records, our workflow is able to identify the acts and extract personal information. The workflow is divided into successive steps: page classification, text line detection, handwritten text recognition, named entity recognition and act detection and classification. For all these steps, different machine learning models are compared. Once the information is extracted, validation rules designed by experts are then applied to standardize the extracted information and ensure its consistency with the type of act (birth, marriage and death). This validation step is able to reject records that are considered invalid or merged. The full workflow has been used to process over two million pages of Quebec parish registers from the 19–20th centuries. On a sample comprising 65% of registers, 3.2 million acts were recognized. Verification of the birth and death acts from this sample shows that 74% of them are considered complete and valid. These records will be integrated into the BALSAC database and linked together to recreate family and genealogical relations at large scale.

A Parallel Text Recognition in Electrical Equipment Nameplate Images Based on Apache Flink

Information on the equipment nameplate is important for the storage, transportation, verification and maintenance of electrical equipment. However, because a natural image of the device on the text nameplate may be multidirectional, curved, noisy or blurry, automatically recognizing the image from the device nameplate can be difficult. Meanwhile, image preprocessing methods are carried out in a serial manner, so the processing speed with regard to the above problems is slower and takes a longer time. Accordingly, this study proposes a parallel and deep-learning-based text automatic recognition method. In the proposed method, a pretreatment method comprising edge detection, morphological manipulation and projection transformation is used to obtain the corrected nameplate region. The connectionist text proposal network (CTPN) is then activated to detect text lines on the corrected nameplate area. Next, a deep-learning method is proposed to study the classification methods of convolutional recurrent neural networks and connectionist time classification for identifying text in each line of text detected by CTPN. Finally, we use Apache Flink to parallelize the above processes, including parallelization preprocessing and bidirectional long short-term memory parallelization in the process of text line detection and text recognition. Experimental results on the collected nameplate show that the proposed imaging processing method has a good recognition performance and that the parallelization method significantly reduces the data processing time cost.

Analysis of English Education Quality Evaluation and Internationalization Integration Based on Deep Learning

English education is one of the most active research directions in the field of natural language processing. With the gradual implementation of deep learning in various fields, more and more industries have begun to use deep learning to carry out more efficient work. In the field of education, it is also urgent to adopt a more intelligent set of algorithms to relieve the pressure of teachers to correct test papers, and also to increase the fairness of non-subjective evaluations in the process of scoring. Teachers conduct teaching evaluation when the concept of teaching evaluation is not clear; there are defects in learning evaluation goals; there are many problems in the relationship between ability evaluation and knowledge evaluation; in the process of English teaching evaluation, the phenomenon of using summative evaluation instead of procedural evaluation is very serious. Therefore, this subject uses deep learning to study the problem of text line positioning and recognition. At the same time, this subject also builds a text scoring network based on RNN and STLM as a quantitative evaluation index for text line detection and recognition algorithms. We will examine students later. Whether problem-based learning theory can be used to promote deep learning among students to determine whether students’ systematic use of PBL in teaching can promote the use of deep learning in college English courses. Finally, comparing the effects of deep learning and shallow learning, it is concluded that the evaluation of deep English teaching can provide students with more learning opportunities, access to more learning-related materials, and questions are more transparent and are free, which is easy. It is speculated that the purpose of this problem is to facilitate the use of deep learning methods to find meaning types.

PageNet: Towards End-to-End Weakly Supervised Page-Level Handwritten Chinese Text Recognition

Handwritten Chinese text recognition (HCTR) has been an active research topic for decades. However, most previous studies solely focus on the recognition of cropped text line images, ignoring the error caused by text line detection in real-world applications. Although some approaches aimed at page-level text recognition have been proposed in recent years, they either are limited to simple layouts or require very detailed annotations including expensive line-level and even character-level bounding boxes. To this end, we propose PageNet for end-to-end weakly supervised page-level HCTR. PageNet detects and recognizes characters and predicts the reading order between them, which is more robust and flexible when dealing with complex layouts including multi-directional and curved text lines. Utilizing the proposed weakly supervised learning framework, PageNet requires only transcripts to be annotated for real data; however, it can still output detection and recognition results at both the character and line levels, avoiding the labor and cost of labeling bounding boxes of characters and text lines. Extensive experiments conducted on five datasets demonstrate the superiority of PageNet over existing weakly supervised and fully supervised page-level methods. These experimental results may spark further research beyond the realms of existing methods based on connectionist temporal classification or attention. The source code is available at https://github.com/shannanyinxiang/PageNet .

Text Line Extraction in Historical Documents Using Mask R-CNN

Text line extraction is an essential preprocessing step in many handwritten document image analysis tasks. It includes detecting text lines in a document image and segmenting the regions of each detected line. Deep learning-based methods are frequently used for text line detection. However, only a limited number of methods tackle the problems of detection and segmentation together. This paper proposes a holistic method that applies Mask R-CNN for text line extraction. A Mask R-CNN model is trained to extract text lines fractions from document patches, which are further merged to form the text lines of an entire page. The presented method was evaluated on the two well-known datasets of historical documents, DIVA-HisDB and ICDAR 2015-HTR, and achieved state-of-the-art results. In addition, we introduce a new challenging dataset of Arabic historical manuscripts, VML-AHTE, where numerous diacritics are present. We show that the presented Mask R-CNN-based method can successfully segment text lines, even in such a challenging scenario.

Persian printed text line detection based on font size

Persian printed text line detection based on font size

Robust text line detection in historical documents: learning and evaluation methods

Text line segmentation is one of the key steps in historical document understanding. It is challenging due to the variety of fonts, contents, writing styles and the quality of documents that have degraded through the years. In this paper, we address the limitations that currently prevent people from building line segmentation models with a high generalization capacity. We present a study conducted using three state-of-the-art systems Doc-UFCN, dhSegment and ARU-Net and show that it is possible to build generic models trained on a wide variety of historical document datasets that can correctly segment diverse unseen pages. This paper also highlights the importance of the annotations used during training: each existing dataset is annotated differently. We present a unification of the annotations and show its positive impact on the final text recognition results. In this end, we present a complete evaluation strategy using standard pixel-level metrics, object-level ones and introducing goal-oriented metrics.

An Unsupervised Learning Approach to Text Line Detection in Complex Illuminated Medieval Manuscripts

This paper outlines a simple and effective clustering and filtering approach for text line detection in challenging illuminated medieval manuscripts from Western Europe. As illuminated medieval manuscripts were copied and illustrated by hand and do not have regular formats, they pose particular difficulties for traditional methods of text line extraction, which have been designed for printed books. This paper introduces an unsupervised learning approach to text line detection in challenging manuscripts based on clustering, using a k-means algorithm with a combination of three salient features that are well-suited for image processing. These are: the gradient in the y-direction, mean values of rows, and grayscale values. The strength of the method lies in its reduced number of features, its computational lightness, low-memory use, transparency at every step of the process, and versatility. It can also be used on a single image, being perpage based, unlike supervised learning approaches which require large training datasets. This stands as an alternative to computationally-heavy algorithms such as neural networks, which have been increasingly used in recent years to solve such task.

SCUT-HCCDoc: A new benchmark dataset of handwritten Chinese text in unconstrained camera-captured documents

In this paper, we introduce a large-scale dataset, called SCUT-HCCDoc, to address challenging detection and recognition problems of handwritten Chinese text (HCT) in the camera-captured documents. Despite extensive studies of optical character recognition (OCR) and offline handwriting recognition for document images, text detection and recognition in the camera-captured documents remains an unsolved problem that is worth for extensive study and investigation. With recent advances in deep learning, researchers have proposed useful architectures for feature learning, detection, and recognition for the scene text. However, the performance of deep learning methods highly depends on the amount and diversity of training data. Previous OCR and offline HCT datasets were built under specific constraints, and most of the recent scene text datasets are for non-handwritten text. Hence, there is a lack of a comprehensive scene handwritten text benchmark. This study focuses on scenes with handwritten Chinese text. We introduce the SCUT-HCCDoc database for HCT detection, recognition and spotting. SCUT-HCCDoc contains 12,253 camera-captured document images with 116,629 text lines and 1,155,801 characters. The diversity of SCUT-HCCDoc can be described at three levels: (1) image-level diversity: image appearance and geometric variances caused by camera-captured settings (such as perspective, background, and resolution) and different applications (such as note-taking, test papers, and homework); (2) text-level diversity: variances of text line length, rotation, etc.; (3) character-level diversity: variances of character categories (up to 6109 classes with additional English letters, and digits), character size, individual writing style, etc. Three kinds of baseline experiments were conducted, where we used several popular text detection methods for text line detection, CTC-based/attention-based methods for text line recognition, and combine text detectors with CTC-based recognizer to achieve end-to-end text spotting. The results indicate the diversity of SCUT-HCCDoc and the challenges of HCT understanding in document images. The dataset is available at https://github.com/HCIILAB/SCUT-HCCDoc_Dataset_Release.

A two-stage method for text line detection in historical documents

This work presents a two-stage text line detection method for historical documents. Each detected text line is represented by its baseline. In a first stage, a deep neural network called ARU-Net labels pixels to belong to one of the three classes: baseline, separator or other. The separator class marks beginning and end of each text line. The ARU-Net is trainable from scratch with manageably few manually annotated example images (less than 50). This is achieved by utilizing data augmentation strategies. The network predictions are used as input for the second stage which performs a bottom-up clustering to build baselines. The developed method is capable of handling complex layouts as well as curved and arbitrarily oriented text lines. It substantially outperforms current state-of-the-art approaches. For example, for the complex track of the cBAD: ICDAR2017 Competition on Baseline Detection the F-value is increased from 0.859 to 0.922. The framework to train and run the ARU-Net is open source.

Improved Chinese Complexly Arranged Scene Text Detection

At present, the existing text detection algorithms in natural scenes have made great progress, but most of them are for text line detection. On the one hand, they have limitation on the effective detection of complexly arranged (such as vertical arrangement, circular arrangement) text lines. On the other hand, the length of text lines varies greatly, which makes text detection very challenging. In this paper, an optimized SSD detection algorithm is proposed for single Chinese text detection. Convolution enhanced module is used to improve the detection of small target text. The introduction of deformable convolution improves the detection of deformable text. A text region connectivity algorithm is proposed to connect complexly arranged single text into readable text lines. In the natural scene Chinese text datasets on CTW and RCTW 17 datasets, mAP of 73.1% and F1 scores of 61.3% were achieved respectively.

A Large Chinese Text Dataset in the Wild

In this paper, we introduce a very large Chinese text dataset in the wild. While optical character recognition (OCR) in document images is well studied and many commercial tools are available, the detection and recognition of text in natural images is still a challenging problem, especially for some more complicated character sets such as Chinese text. Lack of training data has always been a problem, especially for deep learning methods which require massive training data. In this paper, we provide details of a newly created dataset of Chinese text with about 1 million Chinese characters from 3 850 unique ones annotated by experts in over 30 000 street view images. This is a challenging dataset with good diversity containing planar text, raised text, text under poor illumination, distant text, partially occluded text, etc. For each character, the annotation includes its underlying character, bounding box, and six attributes. The attributes indicate the character’s background complexity, appearance, style, etc. Besides the dataset, we give baseline results using state-of-the-art methods for three tasks: character recognition (top-1 accuracy of 80.5%), character detection (AP of 70.9%), and text line detection (AED of 22.1). The dataset, source code, and trained models are publicly available.

Chinese text-line detection from web videos with fully convolutional networks

BackgroundIn recent years, video becomes the dominant resource of information on the Web, where the text within video usually carries significant semanticinformation. Video text extraction and recognition plays an essential role in web multimedia understanding and retrieval for big visual data analytics and applications. To deal with challenging backgrounds and embedding noises, most conventional approaches usually tend to design sophisticated pre-processing and post-progressing steps before and after text detection. In this paper, we present a simple yet powerful pipeline that directly and uniformly detects Chinese text lines for embedded captions from web videos.ResultsIn this Chinese text-line detection system, a fully convolutional network with local context is adopted to localize via an end-to-end learning way. The produced caption predictions are with the word level that could be directly fed into the character classifier. Text-line construction is then performed by heuristic strategies. A variety of experiments are conducted on several real-world web video datasets and demonstrated the effectiveness and efficiency of our proposed method.ConclusionThe proposed system can directly detect the English word and Chinese characters in the caption text-lines without word or character segmentation with the high performance on real-world web video datasets.

Text Line Detection from Rectangle Traffic Panels of Natural Scene

Traffic sign detection and recognition is very important for Intelligent Transportation. Among traffic signs, traffic panel contains rich information. However, due to low resolution and blur in the rectangular traffic panel, it is difficult to extract the character and symbols. In this paper, we propose a coarse-to-fine method to detect the Chinese character on traffic panels from natural scenes. Given a traffic panel Color Quantization is applied to extract candidate regions of Chinese characters. Second, a multi-stage filter based on learning is applied to discard the non-character regions. Third, we aggregate the characters for text lines by Distance Metric Learning method. Experimental results on real traffic images from Baidu Street View demonstrate the effectiveness of the proposed method.

Textline detection in degraded historical document images

This paper presents a textline detection method for degraded historical documents. Our method follows a conventional two-step procedure that the binarization is first performed and then the textlines are extracted from the binary image. In order to address the challenges in historical documents such as document degradation, structure noise, and skews, we develop new methods for the binarization and textline extraction. First, we improve the performance of binarization by detecting the non-text regions and processing only text regions. We also improve the textline detection method by extracting main textblock and compensating the skew angle and writing style. Experimental results show that the proposed method yields the state-of-the-art performance for several datasets.

Improving the Accessibility of Mobile OCR Apps Via Interactive Modalities.

Mobile optical character recognition (OCR) apps have come of age. Many blind individuals use them on a daily basis. The usability of such tools, however, is limited by the requirement that a good picture of the text to be read must be taken, something that is difficult to do without sight. Some mobile OCR apps already implement auto-shot and guidance mechanisms to facilitate this task. In this paper, we describe two experiments with blind participants, who tested these two interactive mechanisms on a customized iPhone implementation. These experiments bring to light a number of interesting aspects of accessing a printed document without sight, and enable a comparative analysis of the available interaction modalities.

Projection–Based Text Line Segmentation with a Variable Threshold

AbstractDocument image segmentation into text lines is one of the stages in unconstrained handwritten document recognition. This paper presents a new algorithm for text line separation in handwriting. The developed algorithm is based on a method using the projection profile. It employs thresholding, but the threshold value is variable. This permits determination of low or overlapping peaks of the graph. The proposed technique is shown to improve the recognition rate relative to traditional methods. The algorithm is robust in text line detection with respect to different text line lengths.