Born-digital Images Research Articles

How to handle bi/tri-lingual Indic texts in a single image? A new dataset of natural scene and born-digital images

How to handle bi/tri-lingual Indic texts in a single image? A new dataset of natural scene and born-digital images

ClusTi: Clustering Method for Table Structure Recognition in Scanned Images

OCR (Optical Character Recognition) for scanned paper invoices is very challenging due to the variability of 19 invoice layouts, different information fields, large data tables, and low scanning quality. In this case, table structure recognition is a critical task in which all rows, columns, and cells must be accurately positioned and extracted. Existing methods such as DeepDeSRT only dealt with high-quality born-digital images (e.g., PDF) with low noise and apparent table structure. This paper proposes an efficient method called CluSTi (Clustering method for recognition of the Structure of Tables in invoice scanned Images). The contributions of CluSTi are three-fold. Firstly, it removes heavy noises in the table images using a clustering algorithm. Secondly, it extracts all text boxes using state-of-the-art text recognition. Thirdly, based on the horizontal and vertical clustering algorithm with optimized parameters, CluSTi groups the text boxes into their correct rows and columns, respectively. The method was evaluated on three datasets: i) 397 public scanned images; ii) 193 PDF document images from ICDAR 2013 competition dataset; and iii) 281 PDF document images from ICDAR 2019’s numeric tables. The evaluation results showed that CluSTi achieved an F1-score of 87.5%, 98.5%, and 94.5%, respectively. Our method also outperformed DeepDeSRT with an F1-score of 91.44% on only 34 images from the ICDAR 2013 competition dataset. To the best of our knowledge, CluSTi is the first method to tackle the table structure recognition problem on scanned images.

Multi-scale sequential network for semantic text segmentation and localization

We present a novel method for semantic text document analysis which in addition to localizing text it labels the text in user-defined semantic categories. More precisely, it consists of a fully-convolutional and sequential network that we apply to the particular case of slide analysis to detect title, bullets and standard text. Our contributions are twofold: (1) A multi-scale network consisting of a series of stages that sequentially refine the prediction of text and semantic labels (text, title, bullet); (2) A synthetic database of slide images with text and semantic annotation that is used to train the network with abundant data and wide variability in text appearance, slide layouts, and noise such as compression artifacts.We evaluate our method on a collection of real slide images collected from multiple conferences, and show that it is able to localize text with an accuracy of 95%, and to classify titles and bullets with accuracies of 94% and 85% respectively. In addition, we show that our method is competitive on scene and born-digital image datasets, such as ICDAR 2011, where it achieves an accuracy of 91.1%.

Live detection of text in the natural environment using Convolutional Neural Network

Abstract With the exponential growth in the quantity of born-digital images, the problem of comprehending text from natural scene images has acquired greater significance. This paper proposes a deep-learning based approach, to detect the presence of text in the natural scene images. The proposed approach is built with the capability to distinguish text and non-text images from the live-stream of the smart-phone camera, thereby eliminating the need for capturing the image, to locate the presence of text. A streamlined Convolutional Neural Network (CNN) MobileNet is harnessed for the process of distinguishing text images from non-text images. The proposed approach can be adopted as a filter, to decide whether to permit the image further down the processing pipeline for the text detection task, which in turn leads to the reduction in false-positives and false-negatives by not processing an image which does not have text. It was inferred from the experimental results that the width multiplier value of 0.75 and resolution multiplier of 224 yields the accuracy of 99.31% in classifying the text images from non-text images.

Unsupervised refinement of color and stroke features for text binarization

Color and strokes are the salient features of text regions in an image. In this work, we use both these features as cues, and introduce a novel energy function to formulate the text binarization problem. The minimum of this energy function corresponds to the optimal binarization. We minimize the energy function with an iterative graph cut based algorithm. Our model is robust to variations in foreground and background as we learn Gaussian mixture models for color and strokes in each iteration of the graph cut. We show results on word images from the challenging ICDAR 2003/2011, born-digital image and street view text datasets, as well as full scene images containing text from ICDAR 2013 datasets, and compare our performance with state-of-the-art methods. Our approach shows significant improvements in performance under a variety of performance measures commonly used to assess text binarization schemes. In addition , our method adapts to diverse document images, like text in videos, handwritten text images.

Detecting natural scenes text via auto image partition, two-stage grouping and two-layer classification

Text detection in natural scene images is important and challenging work for image analysis. In this paper, we present a robust system to detect natural scene text according to text region appearances. The framework includes three parts: auto image partition, two-stage grouping and two-layer classification. The first part partitions images into unconstrained sub-images through statistical distribution of sampling points. The designed two-stage grouping method performs grouping in each sub-image in first stage and connects different partitioned image regions in second stage to group connected components (CCs) to text regions. Then a two-layer classification mechanism is designed for classifying candidate text regions. The first layer is to compute the similarity score of region blocks and the second layer is a SVM classifier using HOG features. We add a normalization step to rectify perspective distortion before candidate text region classification which improves the accuracy and robustness of the final output result. The proposed system is evaluated on four types datasets including two ICDAR Robust Reading Competition datasets, a born-digital image dataset, a video image dataset and a perspective distortion image dataset. The experimental results demonstrate that our proposed framework outperforms state-of-the-art localization algorithms and is robust in dealing with multiple background outliers.

Methods for text segmentation from scene images

Methods for text segmentation from scene images

Text Detection in Born-Digital Images Using IT-LBP

Fine text detection plays a crucial role in a text detection algorithm as it is capable of removing the false alarms while keeping the detected text lines in coarse text detection. Good performance of a machine learning-based fine text detection heavily depends on the powerful feature to depict the characteristics of text. In this paper, a novel texture-based descriptor, named IT-LBP, is proposed by considering horizontal, vertical, diagonal and anti-diagonal directions of character strokes to better describe the texture of text. The new feature demonstrates its superiority by comparing with other texture-based features. The new feature is used to train an SVM classifier to further filter out non-text candidates. The ICDAR 2011 born-digital image dataset is used to evaluate and demonstrate the performance of the proposed method. Following the same performance evaluation criteria, the proposed method outperforms the winner algorithm of the ICDAR 2011 Robust Reading Competition Challenge 1.

Research on born-digital image text extraction based on conditional random field

With the number of digital videos and digital images increasing tremendously in e-mails and web pages, text extraction from images becomes important more than ever. Born-digital images are generated directly with the computer and the text in the images is important to help the semantic understanding of the images. Although there are many methods proposed over the past years for text extraction from natural scene images, the text detection and extraction from born-digital images remains a challenge. This paper proposes a novel method to segment the text connected components CCs from a born-digital image. Firstly, binarisation is conducted on the given image to get all candidate text CCs based on wavelet theory. Secondly, classification is conducted on the extracted CCs to label text CCs based on conditional random field CRF - a probabilistic graph model that has been widely used in natural language processing. Experimental results show that the proposed method can effectively extract text from the born-digital images.

Text Localization and Detection Method for Born-digital Images

Multimedia data has increased rapidly in recent years. Textual information present in multimedia contains important information about the image/video content. The proposed method provides very efficient way to extract text from Born-Digital images. Firstly, edges are extracted from a grayscale image. New edge detection technique is introduced in this research, which gives better results for low-contrast web images. Then morphological operators are applied on the image. These operators are used to connect the broken edges of objects present in an image. Each object is classified as text or non-text on the basis of text features such as size, height to width ratio, and binary transitions, using K-Means clustering. Two new features, namely horizontal fluctuation count and vertical fluctuation count, are introduced in the proposed work. Dataset of International Conference on Document Analysis and Recognition 2011 Robust Reading Competition, Challenge 1: “Reading Text in Born-Digital Images (Web and Email)” is used in this research. The proposed method performed best in the above-mentioned competition.

Text localization and detection method for born-digital images

Text localization and detection method for born-digital images

Localizing Text in Scene Images by Boundary Clustering, Stroke Segmentation, and String Fragment Classification

In this paper, we propose a novel framework to extract text regions from scene images with complex backgrounds and multiple text appearances. This framework consists of three main steps: boundary clustering (BC), stroke segmentation, and string fragment classification. In BC, we propose a new bigram-color-uniformity-based method to model both text and attachment surface, and cluster edge pixels based on color pairs and spatial positions into boundary layers. Then, stroke segmentation is performed at each boundary layer by color assignment to extract character candidates. We propose two algorithms to combine the structural analysis of text stroke with color assignment and filter out background interferences. Further, we design a robust string fragment classification based on Gabor-based text features. The features are obtained from feature maps of gradient, stroke distribution, and stroke width. The proposed framework of text localization is evaluated on scene images, born-digital images, broadcast video images, and images of handheld objects captured by blind persons. Experimental results on respective datasets demonstrate that the framework outperforms state-of-the-art localization algorithms.

Key Issues in the Creation, Delivery, and Preservation of Born Digital Images

Key Issues in the Creation, Delivery, and Preservation of Born Digital Images