Text Detection In Natural Scenes Research Articles

A new DCT-PCM method for license plate number detection in drone images

• A new problem of license plate/text detection in drone images is addressed. • The concept of Phase congruency in DCT domain is introduced for detection. • The combination of DCT and Phase congruency is new. • Deep learning is used in different way for eliminating false positives. • The combination of feature extraction and DCT-Phase is new. License plate number detection in drone images is a complex problem because the images are generally captured at oblique angles and pose several challenges like perspective distortion, non-uniform illumination effect, degradations, blur, occlusion, loss of visibility etc. Unlike, most existing methods that focus on images captured by orthogonal direction (head-on), the proposed work focuses on drone text images. Inspired by the Phase Congruency Model (PCM), which is invariant to non-uniform illuminations, contrast variations, geometric transformation and to some extent to distortion, we explore the combination of DCT and PCM (DCT-PCM) for detecting license plate number text in drone images. Motivated by the strong discriminative power of deep learning models, the proposed method exploits fully connected neural networks for eliminating false positives to achieve better detection results. Furthermore, the proposed work constructs working model that fits for real environment. To evaluate the proposed method, we use our own dataset captured by drones and benchmark license plate datasets, namely, Medialab for experimentation. We also demonstrate the effectiveness of the proposed method on benchmark natural scene text detection datasets, namely, SVT, MSRA-TD-500, ICDAR 2017 MLT and Total-Text.

SEMPANet: A Modified Path Aggregation Network with Squeeze-Excitation for Scene Text Detection.

Recently, various object detection frameworks have been applied to text detection tasks and have achieved good performance in the final detection. With the further expansion of text detection application scenarios, the research value of text detection topics has gradually increased. Text detection in natural scenes is more challenging for horizontal text based on a quadrilateral detection box and for curved text of any shape. Most networks have a good effect on the balancing of target samples in text detection, but it is challenging to deal with small targets and solve extremely unbalanced data. We continued to use PSENet to deal with such problems in this work. On the other hand, we studied the problem that most of the existing scene text detection methods use ResNet and FPN as the backbone of feature extraction, and improved the ResNet and FPN network parts of PSENet to make it more conducive to the combination of feature extraction in the early stage. A SEMPANet framework without an anchor and in one stage is proposed to implement a lightweight model, which is embodied in the training time of about 24 h. Finally, we selected the two most representative datasets for oriented text and curved text to conduct experiments. On ICDAR2015, the improved network’s latest results further verify its effectiveness; it reached 1.01% in F-measure compared with PSENet-1s. On CTW1500, the improved network performed better than the original network on average.

R-YOLO: A Real-Time Text Detector for Natural Scenes with Arbitrary Rotation.

Accurate and efficient text detection in natural scenes is a fundamental yet challenging task in computer vision, especially when dealing with arbitrarily-oriented texts. Most contemporary text detection methods are designed to identify horizontal or approximately horizontal text, which cannot satisfy practical detection requirements for various real-world images such as image streams or videos. To address this lacuna, we propose a novel method called Rotational You Only Look Once (R-YOLO), a robust real-time convolutional neural network (CNN) model to detect arbitrarily-oriented texts in natural image scenes. First, a rotated anchor box with angle information is used as the text bounding box over various orientations. Second, features of various scales are extracted from the input image to determine the probability, confidence, and inclined bounding boxes of the text. Finally, Rotational Distance Intersection over Union Non-Maximum Suppression is used to eliminate redundancy and acquire detection results with the highest accuracy. Experiments on benchmark comparison are conducted upon four popular datasets, i.e., ICDAR2015, ICDAR2013, MSRA-TD500, and ICDAR2017-MLT. The results indicate that the proposed R-YOLO method significantly outperforms state-of-the-art methods in terms of detection efficiency while maintaining high accuracy; for example, the proposed R-YOLO method achieves an F-measure of 82.3% at 62.5 fps with 720 p resolution on the ICDAR2015 dataset.

Deep Learning Based Text Detection and Recognition in Natural Scenes

Deep Learning Based Text Detection and Recognition in Natural Scenes

Natural scene text detection based on YOLO V2 network model

As an important part in the field, the natural scene text detection has been widely applied in visual navigation system, content-based image and video retrieval, instant translation system and so on. In this paper, we introduce several object detection network based on deep learning, and apply the YOLO v2 into natural scene text detection, changing the multi objects detection problems into the two classification problems. The main works in the paper include the following: prepare the datasets; we train the YOLO v2 with the optimum parameters, carry out the regression analysis of the coordinate parameters and categories of bounding boxes, obtain the detection result; according to different detection models, the detection results of different datasets are compared and analyzed, YOLO V2 model detection speed 0.105s/image has certain advantages.

Single shot multi-oriented text detection based on local and non-local features

In order to improve the robustness of text detector on scene text of various scales, a single shot text detector that combines local and non-local features is proposed in this paper. A dilated inception module for local feature extraction and a text self-attention module for non-local feature extraction are presented, and these two kinds of modules are integrated into single shot detector (SSD) of generic object detection so as to perform multi-oriented text detection in natural scene. The proposed modules make a contribution to richer and wider receptive field and enhance feature representation. Furthermore, the performance of our text detector is improved. In addition, compared with previous text detectors based on SSD which classify positive and negative samples depending on default boxes, we exploit pixels as reference for more accurate matching with ground truth which avoids complex anchor design. Furthermore, to evaluate the effectiveness of the proposed method, we carry out several comparative experiments on public standard benchmarks and analyze the experimental results in detail. The experimental results illustrate that the proposed text detector can compete with the state-of-the-art methods.

Necklace: a novel long text detection model

This study proposes a new text detection model and a novel tag generation method for long text detection in natural scenes. The background suppression module is added to the network architecture to integrate feature mapping of each channel, weaken background features, enhance text features, and improve detection accuracy. In the design of the regression frame, the use of elliptical geometric regression, through the formation of long text through a plurality of elliptical connections, better fit the shape of the text line to solve the problem of long text detection. The proposed method has achieved the performance of state-of-art in the popular datasets.

Natural scene text detection based on multiscale connectionist text proposal network

The technique of recognising text in natural scene pictures is widely used in social production. For the existing identification methods, it is difficult to accurately identify in complex environments. The accuracy of the detection determines the efficiency of the identification. A text detection method based on Multiscale Connectionist Text Proposal Network is proposed. The Multiscale-Region Proposal Network regresses and classifies the extracted region to obtain the final candidate region. Taking a large number of commodity image samples as a dataset, the multi-scale joint text proposal network is used to detect and locate the text content area in the image. The experimental results show that the proposed algorithm improves the detection accuracy in complex environments.

Natural Scene Text Detection and Segmentation Using Phase-Based Regions and Character Retrieval

Multioriented text detection and recognition in natural scene images are still challenges in the document analysis and computer vision communities. In particular, character segmentation plays an important role in the complete end-to-end recognition system performance. In this work, a robust multioriented text detection and segmentation method based on a biological visual system model is proposed. The proposed method exploits the local energy model instead of a common approach based on variations of local image pixel intensities. Features such as lines and edges are obtained by searching for the maximum local energy utilizing the scale-space monogenic signal framework. The candidate text components are extracted from maximally stable extremal regions of the local phase information of the image. The candidate regions are filtered by their phase congruency and classified as text and nontext components by the AdaBoost classifier. Finally, misclassified characters are restored, and all final characters are grouped into words. Experimental results show that the proposed text detection and segmentation method is invariant to scale and rotation changes and robust to perspective distortions, blurring, low resolution, and illumination variations (low contrast, high brightness, shadows, and nonuniform illumination). Besides, the proposed method achieves often a better performance compared with state-of-the-art methods on typical natural scene datasets.

Specific category region proposal network for text detection in natural scene

Natural scene text usually carries considerable abstract semantic information, which is closely related to the surrounding environment. Thus, natural scene text detection plays a vital role in image content retrieval and understanding. In this study, the authors propose a novel specific category region proposal network (SCRPN) based on maximally stable extremal regions (MSER) and fully convolutional network (FCN) for natural scene text detection. First, FCN for pixel-level recognition is utilised to obtain the text saliency map and MSER is used to obtain oversegmented regions. Then, the multiple features of oversegmented regions and text saliency map are used for region aggregation. Next, single-linkage clustering method is adopted to cluster the segmentation regions to obtain a hierarchical structure of text region proposals. Finally, for the top-ranking region proposals, SCRPN built an end-to-end pipeline for scene text detection directly. Experiments on street view text and international conference on document analysis and recognition (ICDAR) 2013 have demonstrated the effectiveness of SCRPN for generating the text proposals. SCRPN could work with various two-stage text detection networks; thus, faster region convolutional neural network was used as the text detection framework to evaluate the performance of SCRPN in the ICDAR 2015 and MSRA-TD500 benchmarks. The experimental results confirmed that SCRPN makes text detection more robust in complex scenarios.

Cursive-Text: A Comprehensive Dataset for End-to-End Urdu Text Recognition in Natural Scene Images.

Reading text in natural scene images is an active research area in the fields of computer vision and pattern recognition as text detection, text recognition and script identification are required. In this data article, a comprehensive dataset for Urdu text detection and recognition in natural scene images is presented and analysed. To develop the dataset, more than 2500 natural scene images were captured using a digital camera and a built-in mobile phone camera. Three separate datasets for isolated Urdu character images, cropped word images and end-to-end text spotting were developed. The isolated Urdu character and cropped word images dataset contain a much larger number of samples than existing Arabic natural scene text datasets. The Urdu text spotting dataset contains images with Urdu, English and Sindhi text instances. However, the focus has been given to the Urdu text instances. The ground truths for each image in the isolated character, cropped word or text spotting datasets are provided separately. The proposed datasets can be used to perform Urdu text detection and recognition or end-to-end recognition in natural scenes. These datasets can also be helpful to develop Arabic and Persian natural scene text detection and recognition systems, as Urdu is a derived language of these scripts and has many similar letters. The datasets can also be helpful to develop multi-language translation systems, which can facilitate foreign tourists to read and translate multilingual text in natural scene images. To evaluate the datasets, state-of-the-art machine learning and deep neural networks were used to build the text detection and recognition models, where the best classification accuracies are achieved. To the best of the authors’ knowledge, this is the first dataset proposed for Urdu text detection, recognition or end-to-end text recognition in natural scene images. The aim of this data article is to present a benchmark work in the field of document analysis and recognition.Computer ScienceComputer Vision and Pattern RecognitionTablesFiguresImagesText FilesUsing a digital camera with a 20 megapixels (MP) sensor, an iPhone with a 12 MP back camera and a Samsung mobile with a 16MP back camera.RawAnalyzedEnvironmental factors such as illuminations, blurring and lighting conditions were considered while capturing images. The focus was given to the text within an image.The images in the dataset were obtained from the advertisement banners, sign-boards along the road side and streets, shop name boards, text written on the passing vehicles and walls.The images provided in this dataset were collected in different cities of Sindh, Pakistan.Summarized data are hosted with the article.The datasets and their related files are hosted in a Mendeley public data repository.DOI: https://data.mendeley.com/datasets/k5fz57zd9z/1URL: http://dx.doi.org/10.17632/k5fz57zd9z.1

Natural Scene Text Detection Based On Multi-level Fusion Proposal Network

Natural scene text detection is a challenging issues. In this work, a multi-level features fusion two-stage text detection network was developed to solve the problem of the insufficient use for feature map and the difference between text and common target. In order to obtain deep semantic feature, the network framework of VGG-16 was improved, and different levels of convolution feature map were confused, meanwhile the fusion approach was partially adjusted. Also, vertical proposal network was used to classify and regress bounding box. Results indicated the accuracy and recall rate of this method are 85.4% and 81.0%, respectively, by evaluating the net on ICDAR2013 dataset. The experimental results suggest that the multi-level features fusion method can improve the efficiency of feature map in natural scene text detection.

Real-time Traffic Sign Text Detection Based on Deep Learning

Aiming at the task of traffic sign text detection in natural scenes, a two-stage cascade detection model based on deep learning is proposed. The proposed model first locates the regions of interest (RoI) of text-based traffic sign by applying improved SSD (Single Shot MultiBox Detector) network. Then a rotation-based text detection network is used to detect text strings in the located RoI. Moreover, the lightweight convolutional neural network MobileNetV2 is combined with the deep learning component in two stages, which reduces network parameters and improves detection speed of the model. On the one hand, the proposed approach takes full advantage of the information of traffic signs. By this way the search area of text detection can be reduced, which makes it possible to simplify the text detector. On the other hand, it can obtain more complete text detection results by using rotation-based text detector. The experimental results show that the proposed method can perform well on different data sets, it can not only keep a high accuracy of detection, but also meets the realtime requirements.

A review of methods for text detection in imagery of natural scenes

Text detection in natural scenes plays a key role in recognition and understanding of text in computer vision, while providing the basic possibilities for combining with Natural Language Processing (NLP). The ability of detection for small text and adjacent text is weak although the text detection in natural scenes has evolved from the horizontal text detection initially to the arbitrary shape text detection. Meanwhile, the environment of natural scenes is complex and changeable, which is a huge challenge for text detection. In this paper, we discuss and compare the horizontal and multi-directional detection methods based on target detection, as well as the curved and arbitrary shape detection methods based on the instance segmentation and semantic segmentation. The algorithm ideas are described and their advantages and disadvantages are analysed. Moreover, we found two research routes of curved text and arbitrary shape text detection known as Top-Down and Bottom-Up. According to the evaluations, PMTD [20] performs best in multi-directional text detection, while SPCNet [16] wins in the curve and arbitrary shape text detection.

Arbitrary Shape Natural Scene Text Detection Method Based on Soft Attention Mechanism and Dilated Convolution

Natural scene text detection has attracted much attention in the research field of computer vision, and it has been widely used in many applications, i.e., unmanned driving, robot sensing. Some methods have been proposed for horizontal and oriented text detection, but detecting irregular shapes and highly varying orientated text is still a challenging problem. To tackle this problem, we propose a robust arbitrary shape text detection method called Soft Dilated network (SDnet). The proposed method has two essential steps: (1) feature extraction by backbone; (2) post-processing approach to generate elaborated polygons or boundaries. In particular, the backbone is based on soft attention mechanism and dilated convolution. The soft attention mechanism learns and obtains importance feature from each feature channel, and dilated convolution can effectively aggregate multi-scale contextual information without losing the resolution, and enhance the robust of the network model. The proposed method can accurately detect curve text and discriminate text and non-text areas in an efficient fashion. In addition, Jaccard coefficient is used as loss function to promote the post-processing capability of detecting sparse-arranged and arbitrary shape text. Based on the aforementioned technique, the proposed method an effectively handle the problem of sparse arranged arbitrary natural scene text detection. Experiments were conducted on three benchmark datasets: curved text dataset CTW1500, Total-Text and oriented dataset ICDAR2015, and the results show that when compared with the state-of-the-art text detection methods, the proposed method is more robust and it can find smaller text blocks in the image due to the Loss Function calculation with Jaccard coefficient. Furthermore, we performed multiple sets of ablation experiments, verify the effectiveness of the propose method.

FDTA: Fully Convolutional Scene Text Detection With Text Attention

Text detection is the premise and guarantee of text recognition. Multi-oriented text detection is the current research hotspot. Due to the variability in size, spatial layout, color and the arrangement direction of natural scene text, natural scene text detection is still very challenging. Therefore, this paper proposes a simple and fast multi-oriented text detection method. Our method first optimizes the regression branch by designing a diagonal adjustment factor to make the position regression more accurate, which increases F-score by 0.8. Secondly, we add an attention module to the model, which improves the accuracy of detecting small text regions and increases F-score by 1.2. Then, we introduce DR Loss to solve the problem of positive and negative sample imbalance, which increases F-score by 0.5. Finally, we conduct experimental verification and analysis on the ICDAR2015, MSRA-TD500 and ICDAR2013 datasets. The experimental results demonstrate that this method can significantly improve the precision and recall of scene text detection, and it has achieved competitive results compared with existing advanced methods. On the ICDAR 2015 dataset, the proposed method achieves an F-score of 0.849 at 9.9fps at 720p resolution. On the MSRA-TD500 dataset, the proposed method achieves an F-score of 0.772 at 720p resolution. On the ICDAR 2013 dataset, the proposed method achieves an F-score of 0.887 at 720p resolution.

Natural Scene Chinese Character Text Detection Method Based on Improved CTPN

Text detection in natural scenes, due to differences in size, font, line direction, lighting conditions, text weakness and image background complexity, plays an important role in the research field and remains a challenging and important topic. We have improved the CTPN text detection network and changed the Side-refinement detection box to determine the scaling mechanism. And based on the experiment, change the LSTM network to the GRU neural network. In the dataset of Chinese character text game in natural scene released by Meituan, it reached 0.78 F1-Measure, which reached 0.89 and 0.61 F1-Measure on the ICDAR 2013 and 2015 data sets respectively. Compared to the 0.88 and 0.61 F1-Measure in the CTPN article, there is a big improvement.

Automatic video scene text detection based on saliency edge map

Video scene text contains valuable information for scene understanding, as scene text in video provides important semantic clues for human beings to sense the environment. Text detection in natural scene is challenging due to low resolution/low contrast, cluttered backgrounds and various illumination changes. Therefore, in this paper, a new approach has been proposed to detect video scene text based on saliency edge map, which combines both saliency map and edge features for scene text detection. The saliency map is conducive to detecting the text with cluttered backgrounds whereas the edge map is suitable for detecting the scene text with low resolution and various illumination changes. First of all, we retrieve the saliency map and edge map on the video frame/image, respectively. The saliency map can keep most of saliency regions in the video frame/image which will remove some complicated background. The edge map retrieves the edge feature which is not sensitive to the illumination changes and low resolution/low contrast regions. Then we integrate the edge map and saliency map into saliency edge map (SEM), which preserves the advantages of saliency map and edge maps. Finally, based on Gaussian mixture model (GMM), the SEM can be divided into three kinds of components: bright characters, dark characters and background, and we perform connected component analysis on these three components to get the text regions. Experimental evaluations based on public dataset, such as ICDAR 2003, 2013, MSRA-TD500 and SVT, and news video dataset demonstrate that our method significantly outperforms the other 4 text detection algorithms in terms of recall, precision, F-Score and detection speed, especially when there are challenges such as text with different alignments, character sizes, languages, appearances and uneven illumination.

A CCD based machine vision system for real-time text detection.

Text detection and recognition is a hot topic in computer vision, which is considered to be the further development of the traditional optical character recognition (OCR) technology. With the rapid development of machine vision system and the wide application of deep learning algorithms, text recognition has achieved excellent performance. In contrast, detecting text block from complex natural scenes is still a challenging task. At present, many advanced natural scene text detection algorithms have been proposed, but most of them run slow due to the complexity of the detection pipeline and cannot be applied to industrial scenes. In this paper, we proposed a CCD based machine vision system for real-time text detection in invoice images. In this system, we applied optimizations from several aspects including the optical system, the hardware architecture, and the deep learning algorithm to improve the speed performance of the machine vision system. The experimental data confirms that the optimization methods can significantly improve the running speed of the machine vision system and make it meeting the real-time text detection requirements in industrial scenarios.

Method for Detecting Chinese Texts in Natural Scenes Based on Improved Faster R-CNN

In this study, a natural scene text detection method based on the improved faster region-based convolutional neural network (R-CNN) is proposed. This method extracts image features with the Inception-ResNet architecture, adopts a region proposal network to generate region proposals for the extracted features, merges the fine-tuned features with the region proposals, and finally, uses Fast R-CNN to classify and locate text. The proposed method solves the problems of varying text sizes and the text being obscured in the image. Compared with the original Faster R-CNN, the multilevel Inception-ResNet network model presented in this study can extract deeper text features. The extracted feature map is further sparsely represented by Reduction B, Inception ResNet C and Avg Pool, and then is fused with text regions obtained by the text feature mapping lower layer network to acquire the exact text regions. The text detection method presented in this study is tested on the 2017 dataset of ICDAR2017 Competition on Reading Chinese Text in the Wild (RCTW-17), which contains a large number of distorted, blurry, different scale and size texts. An accuracy of 76.4% is achieved in this platform, thereby proving the efficiency of the proposed method.