Robust Text Detection Research Articles

Domain adaptive multigranularity proposal network for text detection under extreme traffic scenes

Traffic text detection is an important and meaningful research task as it can provide abundant semantic information for autonomous driving. Although major breakthroughs have been achieved on conventional datasets, existing scene text detectors generally suffer from significant performance degradation in real-life traffic scenes since various extreme scenarios may cause a large domain shift. To realize robust traffic text detection under scene changes, we propose a novel network for cross-domain traffic text detection, which integrates both text detection and domain adaptation into one framework. In the text detection pipeline, we introduce a Multigranularity Text Proposal Network (MG-TPN) to generate fine-grained and coarse-grained text proposals, which could deeply interact during both training and inference stages, benefiting the pipeline in learning more robust text features and generating accurate detection results. To transfer the text detection ability from common scenes to unlabeled extreme traffic scenes, we propose an inter&intradomain adaptation (I2-DA) strategy, which adequately excavates domain-invariant features between the source domain and target domain (interdomain), as well as multiple extreme scenarios of the target domain (intradomain). To the best of our knowledge, this is the first study on cross-domain text detection under extreme traffic scenes. Extensive experiments on the traffic text datasets and standard benchmarks, including SynthText, VISD, ICDAR2013 and ICDAR2015 validate the superiority of our method. The proposed two datasets (CTST and ES-CTST) are available at https://github.com/pummi823/test.

Feature Fusion Pyramid Network for End-to-end Scene Text Detection

How to properly involve text characteristics like multi-scale, arbitrary direction, length aspect ratio, into detection network design has become a hot topic in computer vision. Feature Pyramid Network (FPN) is a typical method to achieve robust text detection, where its low-level and high-level feature map retains spatial structure and global semantic information, respectively. However, its strict hierarchical structure fails to fuse low-level and high-level information to improve distinguish ability of feature map. To address this problem, we propose a novel feature fusion pyramid network for end-to-end scene text detection by fusing multi-modal information. By diving pyramid structure into high-level and low-level layers, channel and spatial attention modules are adopted to enhance high-level and low-level feature representation by encoding channel and spatial -wise context information, respectively. In order to reduce information loss by layer transmission, a special residual network is designed to achieve short-cut between high-level and low-level features, so as to realize multi-modal feature fusion. Experiments show the precision and recall of the propose method on ICDAR2015, ICDAR2017-MLT and MSRA-TD500 datasets reach 88.7%/82.1%, 77.0%/60.3% and 85.3%/74.8%, respectively.

A Deep Learning Framework for Grocery Product Detection and Recognition

Object detection and recognition are the most important and challenging problems in computer vision. The remarkable advancements in deep learning techniques have significantly accelerated the momentum of object detection/recognition in recent years. Meanwhile, text detection/recognition is also a critical task in computer vision and has gotten more attention from many researchers due to its wide range of applications. This work focuses on detecting and recognizing multiple retail products stacked on the shelves and off the shelves in the grocery stores by identifying the label texts. In this paper, we proposed a new framework is composed of three modules: (a) retail product detection, (b) product-text detection, (c) product-text recognition. In the first module, on-the-shelf and off-the-shelf retail products are detected using the YOLOv5 object detection algorithm. In the second module, we improve the performance of the state-of-the-art text detection algorithm by replacing the backbone network with ResNet50 + FPN and by introducing a new post-processing technique, Width Height based Bounding Box Reconstruction, to mitigate the problem of inaccurate text detection. In the final module, we used a state-of-the-art text recognition model to recognize the retail product’s text information. The YOLOv5 algorithm accurately detects both on-the-shelf and off-the-shelf grocery products from the video frames and the static images. The experimental results show that the proposed post-processing approach improves the performance of the existing methods on both regular and irregular text. The robust text detection and text recognition methods greatly support our proposed framework to recognize the on-the-shelf retail products by extracting product information such as product name, brand name, price, and expiring date. The recognized text contexts around the retail products can be used as the identifier to distinguish the product.

A Robust and Effective Text Detector Supervised by Contrastive Learning

Scene text detection is a task that detects the position of text in natural scenes. Due to the different sizes, arbitrary orientations, different colors of texts, as well as low contrast and resolution in the complex background, text detection in natural scene images is very challenging. So far, the detection results for text instances in motion blur, low-resolution images are still not satisfactory. In this paper, in order to solve the above problems, we propose an effective and robust text detection network that combines a state-of-the-art contrastive learning method SimCLR. Before being input to the feature extractor, the data is augmented in different methods, and then we calculate the similarity of the extracted corresponding feature pairs. This can significantly improve the performance of the detector in difficult conditions. We conduct a series of experiments on the public dataset ICDAR2013, ICDAR2015 and MSRA-TD500. On the ICDAR 2015 dataset, our method achieves F-measure of 0.840 and runs at 9.1 FPS at 720p resolution, demonstrating that the proposed method is effective and efficient.

Natural Scene Text Detection and Segmentation Using Phase-Based Regions and Character Retrieval

Multioriented text detection and recognition in natural scene images are still challenges in the document analysis and computer vision communities. In particular, character segmentation plays an important role in the complete end-to-end recognition system performance. In this work, a robust multioriented text detection and segmentation method based on a biological visual system model is proposed. The proposed method exploits the local energy model instead of a common approach based on variations of local image pixel intensities. Features such as lines and edges are obtained by searching for the maximum local energy utilizing the scale-space monogenic signal framework. The candidate text components are extracted from maximally stable extremal regions of the local phase information of the image. The candidate regions are filtered by their phase congruency and classified as text and nontext components by the AdaBoost classifier. Finally, misclassified characters are restored, and all final characters are grouped into words. Experimental results show that the proposed text detection and segmentation method is invariant to scale and rotation changes and robust to perspective distortions, blurring, low resolution, and illumination variations (low contrast, high brightness, shadows, and nonuniform illumination). Besides, the proposed method achieves often a better performance compared with state-of-the-art methods on typical natural scene datasets.

IOS-Net: An inside-to-outside supervision network for scale robust text detection in the wild

Accurately detecting scene text is a challenging task due to perspective distortion, scale variance, varied orientations, uneven illumination. Among them, scale variance has always been a core issue and generally involves two types: various size and diverse aspect ratios of the text regions. In contrast to most existing approaches focusing on addressing one type of scale variance, this paper presents a novel inside-to-outside supervision network (IOS-Net) that can well tackle both two. Specifically, we design a hierarchical supervision module (HSM), which consists of a new inception unit with parallel asymmetric convolution and a skip-layer fusion structure. Inside the HSM, we introduce hierarchical supervision into the new inception unit to effectively capture the texts with diverse aspect ratios. Outside the HSM, we adopt multiple-scale supervision on the stacked HSMs to accurately detect the texts with various sizes. Moreover, a position-sensitive segmentation is used to enhance the representation of difficult text objects and the discrimination of adjacent ones. The proposed method achieves state-of-the-art performance on representative public benchmarks, reaching 86% F-score and 11.5 frames per second (FPS) on the ICDAR 2015 incidental text dataset, 47% F-score and 16.1 FPS on the COCO-Text dataset, 69% F-score and 11.7 FPS on the ICDAR 2013 video text dataset.

Feature Enhancement Network: A Refined Scene Text Detector

In this paper, we propose a refined scene text detector with a novel Feature Enhancement Network (FEN)for Region Proposal and Text Detection Refinement. Retrospectively, both region proposal with only 3 x 3 sliding-window feature and text detection refinement with single scale high level feature are insufficient, especially for smaller scene text. Therefore, we design a new FEN network with task-specific, low and high level semantic features fusion to improve the performance of text detection. Besides, since unitary position-sensitive RoI pooling in general object detection is unreasonable for variable text regions, an adaptively weighted position-sensitive RoI pooling layer is devised for further enhancing the detecting accuracy. To tackle the sample-imbalance problem during the refinement stage,we also propose an effective positives mining strategy for efficiently training our network. Experiments on ICDAR2011 and 2013 robust text detection benchmarks demonstrate that our method can achieve state-of-the-art results, outperforming all reported methods in terms of F-measure.

Multi-oriented text detection and verification in video frames and scene images

In this paper, we bring forth a novel approach of video text detection using Fourier-Laplacian filtering in the frequency domain that includes a verification technique using Hidden Markov Model (HMM). The proposed approach deals with the text region appearing not only in horizontal or vertical directions, but also in any other oblique or curved orientation in the image. Until now only a few methods have been proposed that look into curved text detection in video frames, wherein lies our novelty. In our approach, we first apply Fourier-Laplacian transform on the image followed by an ideal Laplacian-Gaussian filtering. Thereafter K-means clustering is employed to obtain the asserted text areas depending on a maximum difference map. Next, the obtained connected components (CC) are skeletonized to distinguish various text strings. Complex components are disintegrated into simpler ones according to a junction removal algorithm followed by a concatenation performed on possible combination of the disjoint skeletons to obtain the corresponding text area. Finally these text hypotheses are verified using HMM-based text/non-text classification system. False positives are thus eliminated giving us a robust text detection performance. We have tested our framework in multi-oriented text lines in four scripts, namely, English, Chinese, Devanagari and Bengali, in video frames and scene texts. The results obtained show that proposed approach surpasses existing methods on text detection.

Scene text detection via extremal region based double threshold convolutional network classification.

In this paper, we present a robust text detection approach in natural images which is based on region proposal mechanism. A powerful low-level detector named saliency enhanced-MSER extended from the widely-used MSER is proposed by incorporating saliency detection methods, which ensures a high recall rate. Given a natural image, character candidates are extracted from three channels in a perception-based illumination invariant color space by saliency-enhanced MSER algorithm. A discriminative convolutional neural network (CNN) is jointly trained with multi-level information including pixel-level and character-level information as character candidate classifier. Each image patch is classified as strong text, weak text and non-text by double threshold filtering instead of conventional one-step classification, leveraging confident scores obtained via CNN. To further prune non-text regions, we develop a recursive neighborhood search algorithm to track credible texts from weak text set. Finally, characters are grouped into text lines using heuristic features such as spatial location, size, color, and stroke width. We compare our approach with several state-of-the-art methods, and experiments show that our method achieves competitive performance on public datasets ICDAR 2011 and ICDAR 2013.

Rotation and script independent text detection from video frames using sub pixel mapping

Text detection and recognition in video frames is a challenging task due to low contrast and noise from background that hinder the processing. In this paper we have proposed a robust multi-oriented text detection approach in video frames. Proposed method uses sub pixel mapping based super resolution approach to enhance the image. Next, in this enhanced image, Histogram of Oriented Moment feature is extracted from connected components and Support Vector Machine (SVM) classifier is used for multi-oriented text/non-text identification. Finally, Recurrent Neural Network (RNN) based classifier is used for recognition of text characters. We have performed our experiment in ICDAR2013 Robust Reading Competition Karatzas et al. (2013). To evaluate the script independent text extraction performance, we tested our framework in IITR dataset Verma et al. (2016) that contains text of multiple scripts in scene images. We have obtained F-measure of 0.82 which surpasses the current state of art techniques in ICDAR2013 Karatzas et al. (2013) and 0.8 in IITR Dataset Verma et al. (2016) for text detection.

Histograms of Stroke Widths for Multi-script Text Detection and Verification in Road Scenes

Robust text detection and recognition in arbitrarily distributed, unrestricted images is a difficult problem, e.g. when interpreting traffic panels outdoors during autonomous driving. Most previous work in text detection considers only a single script, usually Latin, and it is not able to detect text with multiple scripts. Our contribution combines an established technique -Maximum Stable Extremal Regions-with a histogram of stroke width (HSW) feature and a Support Vector Machine classifier. We combined characters into groups by raycasting and merged aligned groups into lines of text that can also be verified by using the HSW. We evaluated our detection pipeline on our own dataset of road scenes from Autobahn (German Highways), and show how the character classifier stage can be trained with one script and be successfully tested on a different one. While precision and recall match to state of the art solution. A unique characteristic of the HSW feature is that it can learn and detect multiple scripts, which we believe can yield script independence.

Robust text detection via multi-degree of sharpening and blurring

Text detection is an important process for many content-based image analysis tasks. In this paper, we propose an approach to scene text detection via multi-degree of sharpening and blurring. Input image is sharpened and blurred with unsharp masking (USM) and bilateral filter. Then components are extracted with Maximally Stable Extremal Regions (MSER) from origin and processed images. Color, spatial layout and distance features of component are calculated, and features are weighted to construct the text candidates with distance function where the weights of features were trained before. At last, text candidates are estimated with a character classifier and the non-text candidates are eliminated. Experiments show that the proposed approach is robust to complex backgrounds and low image quality.

A robust approach for text detection from natural scene images

This paper presents a robust text detection approach based on color-enhanced contrasting extremal region (CER) and neural networks. Given a color natural scene image, six component-trees are built from its grayscale image, hue and saturation channel images in a perception-based illumination invariant color space, and their inverted images, respectively. From each component-tree, color-enhanced CERs are extracted as character candidates. By using a “divide-and-conquer” strategy, each candidate image patch is labeled reliably by rules as one of five types, namely, Long, Thin, Fill, Square-large and Square-small, and classified as text or non-text by a corresponding neural network, which is trained by an ambiguity-free learning strategy. After pruning unambiguous non-text components, repeating components in each component-tree are pruned further. Remaining components are then grouped into candidate text-lines and verified by another set of neural networks. Finally, results from six component-trees are combined, and a post-processing step is used to recover lost characters. Our proposed method achieves superior performance on both ICDAR-2011 and ICDAR-2013 “Reading Text in Scene Images” test sets.

LED Dot matrix text recognition method in natural scene

In recent years, light-emitting diodes dot-matrix text (LED text) is being widely used for displaying information and announcements. However, there is currently no text detection system that is capable of handling LED text. Unlike general printed text, it is not easy to detect and recognize LED text due to its discontinuity. A character of the LED is generally displayed with a matrix of segments and composed with them to generate the text. Furthermore, it is necessary to detect each character from a line of LED text for creating a robust text detection system. Thus, this paper proposes a method for LED text detection and recognition in natural scene images. To perform this goal of detection and recognition of a character and text, it consists of two main steps with the following steps: the first step, a Canny edge was used to detect character pixels which appear in LED display area from scene images. The center points of edge segments are calculated. These points are merged based on their properties to generate a character candidate. In order to obtain character feature, the spatial information such as a centroid and orientation of the character candidate are used. These values are then analyzed using a k-nearest neighbor approach for classifying the character candidate as a certain alphanumeric. In the second step, the recognized characters are later combined into a text line based on the similarity of their characteristics such as width, height, aspect ratio and color. The post-processing of text line generating is then applied for rectifying the falsely recognized characters. In experiments, our proposed method achieves 68.8% and 47% for detection and recognition rate, respectively. These results show the robustness and effectiveness of the proposed method for detecting and recognizing the LED text in natural scene images that has filled the vacancy that the printed and dense text detection system has not covered.

Robust Text Detection in Natural Scenes Using Text Geometry and Visual Appearance

This paper proposes a new two-phase approach to robust text detection by integrating the visual appearance and the geometric reasoning rules. In the first phase, geometric rules are used to achieve a higher recall rate. Specifically, a robust stroke width transform (RSWT) feature is proposed to better recover the stroke width by additionally considering the cross of two strokes and the continuousness of the letter border. In the second phase, a classification scheme based on visual appearance features is used to reject the false alarms while keeping the recall rate. To learn a better classifier from multiple visual appearance features, a novel classification method called double soft multiple kernel learning (DS-MKL) is proposed. DS-MKL is motivated by a novel kernel margin perspective for multiple kernel learning and can effectively suppress the influence of noisy base kernels. Comprehensive experiments on the benchmark ICDAR2005 competition dataset demonstrate the effectiveness of the proposed two-phase text detection approach over the state-of-the-art approaches by a performance gain up to 4.4% in terms of F-measure.

A robust arbitrary text detection system for natural scene images

Text detection in the real world images captured in unconstrained environment is an important yet challenging computer vision problem due to a great variety of appearances, cluttered background, and character orientations. In this paper, we present a robust system based on the concepts of Mutual Direction Symmetry (MDS), Mutual Magnitude Symmetry (MMS) and Gradient Vector Symmetry (GVS) properties to identify text pixel candidates regardless of any orientations including curves (e.g. circles, arc shaped) from natural scene images. The method works based on the fact that the text patterns in both Sobel and Canny edge maps of the input images exhibit a similar behavior. For each text pixel candidate, the method proposes to explore SIFT features to refine the text pixel candidates, which results in text representatives. Next an ellipse growing process is introduced based on a nearest neighbor criterion to extract the text components. The text is verified and restored based on text direction and spatial study of pixel distribution of components to filter out non-text components. The proposed method is evaluated on three benchmark datasets, namely, ICDAR2005 and ICDAR2011 for horizontal text evaluation, MSRA-TD500 for non-horizontal straight text evaluation and on our own dataset (CUTE80) that consists of 80 images for curved text evaluation to show its effectiveness and superiority over existing methods.

A Robust Multilingual Text Detection Approach Based on Transforms and Wavelet Entropy

In the present work, we describe the study and performance of a robust text detection method in color and regular images. The system employs two important stages: Firstly the combination of wavelet transforms and gabor filter is applied to extract sharpened edges and textural features of a given input image. In the second stage, wavelet entropy is imposed to get an energy value of a resulted gabor image in order to find the high frequency texture elements of a processed one to determine the true text region of an image. Experiments are conducted on standard ICDAR database and with multi-lingual dataset. Experimental results show that the proposed method is able to detect text of different font size, complex background and contrast. Furthermore, the system performance surpasses the existing method in terms of detection accuracy.

Fast and robust text detection in images and video frames

Text in images and video frames carries important information for visual content understanding and retrieval. In this paper, by using multiscale wavelet features, we propose a novel coarse-to-fine algorithm that is able to locate text lines even under complex background. First, in the coarse detection, after the wavelet energy feature is calculated to locate all possible text pixels, a density-based region growing method is developed to connect these pixels into regions which are further separated into candidate text lines by structural information. Secondly, in the fine detection, with four kinds of texture features extracted to represent the texture pattern of a text line, a forward search algorithm is applied to select the most effective features. Finally, an SVM classifier is used to identify true text from the candidates based on the selected features. Experimental results show that this approach can fast and robustly detect text lines under various conditions.

Texture-based approach for text detection in images using support vector machines and continuously adaptive mean shift algorithm

The current paper presents a novel texture-based method for detecting texts in images. A support vector machine (SVM) is used to analyze the textural properties of texts. No external texture feature extraction module is used, but rather the intensities of the raw pixels that make up the textural pattern are fed directly to the SVM, which works well even in high-dimensional spaces. Next, text regions are identified by applying a continuously adaptive mean shift algorithm (CAMSHIFT) to the results of the texture analysis. The combination of CAMSHIFT and SVMs produces both robust and efficient text detection, as time-consuming texture analyses for less relevant pixels are restricted, leaving only a small part of the input image to be texture-analyzed.