Maximally Stable Extremal Regions Research Articles

A Novel Framework for Text Recognition in Street View Images

This paper addresses a new text recognition solution, which is mainly used in detection of street view images. This paper employs two different approaches to detect text based regions and recognize corresponding text fields. The first approach utilizes Maximally Stable Extremal Regions (MSER), whereas the second approach relies on Class Specific Extremal Regions (CSER) algorithm. Two separate frameworks, designed with respect to the aforementioned methods, are applied to the street view images so as to extract text based regions. Numerous experiments were performed to evaluate and compare both approaches. Especially results obtained from ERs based approach are quite encouraging and verify the system’s ability to detect text based regions and recognize corresponding text fields.

Deteksi Teks Secara Otomatis Pada Natural Image Berbasis Superpixel Menggunakan Maximally Stable Extremal Regions dan Stroke Width Transform

Text detection in natural image is something to do before performing character recognition. The process of text detection plays an important role in the acquisition of information in an image. This research aims to detect text automatically in natural image based on superpixels with Maximally Stable Extremal Regions (MSER) and Stroke Width Transform (SWT). The superpixel method used is Simple Linear Iterative Clustering (SLIC). The SLIC method is used for segmenting text images into superpixel spaces. Image segmentation to superpixel aims to group pixels into homogeneous regions that capture redundant images. SLIC is a technique that effectively divides images into homogeneous regions (superpixels). Furthermore MSER is used as a feature to locate the text candidate region in a segmented image with superpixel. Then edge detection is done to validate the text area that has been found. Next, the SWT method is used to distinguish both text and non-text image regions. The dataset used is ICDAR 2003. Based on test result, MSER with superpixel is able to detect region of text in natural image. SWT is also able to recover the region which is the candidate of the text in natural image.

Human Reading Knowledge Inspired Text Line Extraction

Text in images contains exact semantic information and the text knowledge can be utilized in many image cognition and understanding applications. The human reading habits can provide the clues of text line structure for text line extraction. In this paper, we propose a novel human reading knowledge inspired text line extraction method based on k-shortest paths global optimization. Firstly, the candidate character extraction is reformulated as Maximal Stable Extremal Region (MSER) algorithm on gray, red, blue, and green channels of the target images, and the extracted MSERs are fed into Convolutional Neural Network (CNN) to remove the noise components. Then, the directed graph is built upon the character component nodes with edges inspired by human reading sense. The directed graph can automatically construct the relationship to eliminate the disorder of candidate text components. The text line paths optimization is inspired by the human reading ability in planning of a text line path sequentially. Therefore, the text line extraction problem can be solved using the k-shortest paths optimization algorithm by taking advantage of the human reading sense structure of the directed graph. It can extract the text lines iteratively to avoid the exhaustive searching and obtain global optimized text line number. The proposed method achieves the f-measure of 0.820 and 0.812 on public ICDAR2011 and ICDAR2013 dataset, respectively. The experimental results demonstrate the effectiveness of the proposed human reading knowledge inspired text line extraction method in comparison with state-of-the-art methods This paper presents one human reading knowledge inspired text line extraction method, which approves that the human reading knowledge can benefit the text line extraction and image text discovery.

Chinese Text Detection and Recognition in Natural Scene Using HOG and SVM

It remains an important, yet challenging problem to detect and recognize text from natural scene images. Since earlier this year a number of methods have been proposed for reading text from scene images. But all of them are adapted for English alphabets, and not suitable for Chinese characters, which present unique challenges including highly versatile fonts, complex background and uneven illumination, a huge number of different characters, unconnected strokes within a character. In this paper we design a method tailored for reading Chinese characters from natural scenes. During the phase of character detection, we employ MSER (Maximally Stable Extremal Region) method to extract candidate characters, and then integrate extracted strokes through mathematical morphology computation. Based on the attributes of Chinese characters, we also lay down heuristic rules that distinguish text and non-text to screen the region of a candidate character. In this method, we describe the features of characters with HOG descriptor and accurately use SVM (Support Vector Machine) according to classification. And the positive region is the region covering the text. At the stage of text recognition, we use KNN (K-Nearest Neighbors) as the classifier. We test this method on 400 natural scene images containing Chinese characters collected from different sources. And the results have successfully validated the efficacy of our approach.

Bag-of-Feature model for sweet and bitter almond classification

The use of computer vision techniques in post-harvest processing of agricultural products has increased considerably in recent years due to their non-destructive and rapid monitoring abilities. Image processing, combined with pattern recognition, has been applied in fruit sorting and classification. In this study, a Bag-of-Feature (BoF) model is used for the classification of 20 sweet and bitter almond varieties. Harris, Harris–Laplace, Hessian, Hessian–Laplace and Maximally Stable Extremal Regions (MSER) keypoint detectors along with a Scale Invariant Feature Transform (SIFT) descriptor are used in the BoF model. The k-means clustering method is applied for building a codebook from keypoint descriptors. The performance of 3 classifiers, which were k-Nearest Neighbour (k-NN), linear and chi-square Support Vector Machine (L-SVM and Chi-SVM, respectively) were compared using classification results in the model. It was observed that the Chi-SVM classifier outperformed the k-NN and L-SVM classifiers. Using the BoF model, it was possible to detect and classify sweet and bitter varieties with high overall accuracy.

A Novel Image Structural Similarity Index Considering Image Content Detectability Using Maximally Stable Extremal Region Descriptor

The image content detectability and image structure preservation are closely related concepts with undeniable role in image quality assessment. However, the most attention of image quality studies has been paid to image structure evaluation, few of them focused on image content detectability. Examining the image structure was firstly introduced and assessed in Structural SIMilarity (SSIM) measure, in which, the definition of image structure is constrained to the intensity covariance between the reference and test images. Indeed, this measure discerns the luminance changes in the pixels of the reference and test images, by employing the low-level statistical features. But this minimal definition of image structure does not cover the issue of image content detectability. In this study, we found that the status of image region smoothness can reflect its structural content. So, we proposed a novel smoothness measure based on the maximally stable extremal regions (MSER) descriptor. Subsequently, we proposed a novel image structural similarity measure, in which the fidelity of image region smoothness is also taken into account. Experimental results on five popular benchmark image databases, include A57, LIVE, CSIQ, TID2008 and TID2013, are provided, which confirm that the proposed approach has a reasonable prediction performance compared to the state-of-the-art image quality metrics.

Research on Infrared and Visible Images Registration Algorithm Based on Graph

In this paper, we proposed a registration algorithm based on the combination of pyramid hierarchical idea and graph theory. It can solve the challenging problem of extracting consistent characteristics and matching the infrared and visible images. First, extracting the maximally stable extremal regions (MSERs) in the determining maximum down-sampling images and each MSER is represented by a polygon. Then constructing the gragh features and the mapping relationship of MSERs between the infrared and visible images are determined by the graph matching method. Next we can construct the initial point set for matching according to the mapping relationship. Finally, using the random sample consensus (RANSAC) algorithm to obtain the optimal parameters and determine the error evaluation parameters. According to the idea of pyramid stratification, the above process is repeated in the high resolution images under the constraint condition of current matching error. The experiment results show that the algorithm can make full use of the visual similarity structures between images, and can achieve a smaller matching error under the premise of ensuring the robustness of the matching.

基于相位对称性和MESR的道路交通标志检测方法研究

本文提出了一种基于相位对称性的交通标志检测方法，对光照条件变化、尺度变化旋转等情况下都具有较好的适应性。该方法首先根据交通标志的颜色特征，对图像进行红蓝化阈值处理，区分交通标志和环境背景；对红蓝化图像做相位对称性计算处理，增强具有对称性特征的交通标志轮廓亮度；然后，通过形态学滤波和MSER(maximally stable extremal regions)特征检测，提取图像中的交通标志ROI候选区域。通过在德国交通标志数据集上的实验测试German Traffic Sign Detection Benchmark (GTSDB)，该方法对交通标志的检测率到达94%，对于光照条件变化、局部遮挡、旋转尺度变化等复杂情况具有更好的适应性。 The high variability of sign appearance in complex road environments has made the detection and classification of road signs a challenging problem in computer vision. In this paper, we propose a novel approach for the detection of traffic signs, which is well-suitable to different light conditions, change of scale and rotation. The detection process has four stages: image color enhancement, phase symmetry computation, morphological filter and maximally stable extremal regions (MSERs) detection phase. We combine the color information of traffic signs using color enhancement with the symmetry of its shape type, and performing phase symmetry computation to highlight the Region of interest (ROI). Finally, the candidate regions of traffic signs are detected by MSER. The proposed system attains a highly accuracy up to 94% on the German Traffic Sign Detection Benchmark (GTSDB). It has better adaptability to complex conditions such as illumination change, rotation scale change and so on.

Cell Detection Using Extremal Regions in a Semisupervised Learning Framework.

This paper discusses an algorithm to build a semisupervised learning framework for detecting cells. The cell candidates are represented as extremal regions drawn from a hierarchical image representation. Training a classifier for cell detection using supervised approaches relies on a large amount of training data, which requires a lot of effort and time. We propose a semisupervised approach to reduce this burden. The set of extremal regions is generated using a maximally stable extremal region (MSER) detector. A subset of nonoverlapping regions with high similarity to the cells of interest is selected. Using the tree built from the MSER detector, we develop a novel differentiable unsupervised loss term that enforces the nonoverlapping constraint with the learned function. Our algorithm requires very few examples of cells with simple dot annotations for training. The supervised and unsupervised losses are embedded in a Bayesian framework for probabilistic learning.

A method for real-time memory efficient implementation of blob detection in large images

In this paper we propose a method for real-time blob detection in large images with low memory cost. The method is suitable for implementation on the specialized parallel hardware such as multi-core platforms, FPGA and ASIC. It uses parallelism to speed-up the blob detection. The input image is divided into blocks of equal sizes to which the maximally stable extremal regions (MSER) blob detector is applied in parallel. We propose the usage of multiresolution analysis for detection of large blobs which are not detected by processing the small blocks. This method can find its place in many applications such as medical imaging, text recognition, as well as video surveillance or wide area motion imagery (WAMI). We explored the possibilities of usage of detected blobs in the feature-based image alignment as well. When large images are processed, our approach is 10 to over 20 times more memory efficient than the state of the art hardware implementation of the MSER.

Shadow Areas Robust Matching Among Image Sequence in Planetary Landing

In this paper, an approach for robust matching shadow areas in autonomous visual navigation and planetary landing is proposed. The approach begins with detecting shadow areas, which are extracted by Maximally Stable Extremal Regions (MSER). Then, an affine normalization algorithm is applied to normalize the areas. Thirdly, a descriptor called Multiple Angles-SIFT (MA-SIFT) that coming from SIFT is proposed, the descriptor can extract more features of an area. Finally, for eliminating the influence of outliers, a method of improved RANSAC based on Skinner Operation Condition is proposed to extract inliers. At last, series of experiments are conducted to test the performance of the approach this paper proposed, the results show that the approach can maintain the matching accuracy at a high level even the differences among the images are obvious with no attitude measurements supplied.

An enhanced text detection technique for the visually impaired to read text

An enhanced text detection technique (ETDT) is proposed, which is expected to aid the visually impaired to overcome their reading challenges. This work enhances the edge-preserving maximally stable extremal regions (eMSER) algorithm using the pyramid histogram of oriented gradients (PHOG). Histogram of oriented gradients (HOG) derived from different pyramid levels is important while detecting maximally stable extremal regions (MSER) in the ETDT approach because it gives more spatial information when compared to HOG information from a single level. To group text, a four-line, text-grouping method is newly designed for this work. Also, a new text feature, Shapeness Score is proposed, which significantly identifies text regions when combined with the other features based on morphology and stroke widths. Using the feature vector of dimension 10, the J48 decision tree and AdaBoost machine learning algorithms identify the text regions in the images. The algorithm yields better results than the existing benchmark algorithms for the ICDAR 2011 born-digital dataset and must be improved with respect to the scene text dataset.

Wildfire smoke detection based on local extremal region segmentation and surveillance

A novel video-based method is proposed for long-distance wildfire smoke detection. Since the long-distance wildfire smoke usually moves slowly and lacks salient features in the video, the detection is still a challenging problem. Unlike many traditional video-based methods that usually rely on the smoke color or motion for initial smoke region segmentation, we use the Maximally Stable Extremal Region (MSER) detection method to extract local extremal regions of the smoke. This makes the initial segmentation of possible smoke region less dependent on the motion and color information. Potential smoke regions are then selected from all the possible regions by using some static visual features of the smoke, helping to eliminate the non-smoke regions as many as possible. Once a potential smoke region is found, we keep tracking it by searching the best-matched extremal regions in the subsequent frames. At the same time, the propagating motions of the potential smoke region are monitored based on a novel cumulated region approach, which can be effectively used to identify the distinctive expanding and rising motions of smoke. This approach can also make the final smoke motion identification insensitive to image shaking. It was proved that the proposed method is able to reliably detect the long-distance wildfire smoke and simultaneously produce very few false alarms in actual applications.

Hierarchical Filtering Strategy for Registration of Remote Sensing Images of Coral Reefs

Registration of remote sensing images of coral reefs is basis for detection and analysis of changes to coral reefs, which is difficult due to inadequate and unstable texture information. Affine invariant features matching (AIFM) method, combining maximally stable extremal region (MSER) detector and scale invariant feature transformation (SIFT) descriptor, followed by optimizing using RANdom SAmple Consensus (RANSAC), still cannot lead to satisfactory results of image registration. To address this problem, we propose a hierarchical filtering strategy for image registration, which is composed of two stages. Filter 1, constructed based on a geometric transformation model determined by the corresponding contours of coral reefs, is employed to remove the wrong matching pairs which obviously dissatisfy spatial distributions of matching features. Based on the filtering results of Filters 1 and 2, with an appropriate threshold value determined by overlapping ratio of the corresponding coral reefs, can lead to further optimized results for image registration. This threshold is used to filter all erroneous matching pairs, while retaining as many correct matching pairs as possible. To test the effect of these filters, we design two experiments to compare the four approaches: AIFM without filtering, AIFM with Filter 1, AIFM with Filter 2, and AIFM with hierarchical filtering ( ${\bf{Filter}}\;{\bf{1}} + {\bf{Filter}}\;{\bf{2}}$ ). The experimental results demonstrate that the approach with hierarchical filtering performs much better than the other approaches.

Blur robust extremal region-based interest points for medical image registration

Various studies on interest point (IP) detection have concluded that maximally stable extremal region (MSER)-based IPs outperform others on repeatability, localization accuracy, robustness, efficiency and covariance to global and local image distortions. Since medical images lack sharp detail, corner IPs are not a suitable choice for them. Instead, MSERs which offer region-based IPs are useful. However, sensitivity of MSERs to image blur and scale makes them less useful practically. In this context, through this paper, following contributions are made—(1) It is proposed to study MSER-based IPs in Intensity Scale Space instead of conventional Scale Space to better understand and mitigate the problem of IP clutter. (2) By modulating the connectivity of previously proposed ER-based IPs (inspired from visual saliency approach), blur and scale sensitivity of region-based IPs is shown to reduce significantly. The newly developed IPs are called ‘blur robust extremal region (BRER)’ IPs. (3) Owing to the global nature of evaluation parameters (like repeatability) for IP detection, the problem of incorrect judgment is highlighted. As a solution to it, three new evaluation parameters called ‘Uniformity Index,’ ‘10 % core distance’ and ‘Informativeness’ are proposed. These indices capture the idea of uniform distribution of IPs over the entire image, IP clutter and the redundancy of registered IP pairs, respectively. Experiments on database of medical images of different modalities and various organs/diseases suggest that proposed BRER IPs are robust to blur and scale. Also, proposed indices of evaluation offer better judgment of quality of image registration.

Object Recognition Based on Maximally Stable Extremal Region and Scale-Invariant Feature Transform

For the defect in describing affine and blur invariable of scale-invariant feature transform (SIFT) at large viewpoint variation, a new object recognition method is proposed in this paper, which used maximally stable extremal region (MSER) detecting MSERs and SIFT describing local feature of these regions. First, a new most stability criterion is adopt to improve the detection effect at irregular shaped regions and under blur conditions; then, the local feature descriptors of MSERs is extracted by the SIFT; and finally, the method proposed is comparing then correct rate of SIFT and the proposed through image recognition with standard test images. Experimental results show that the method proposed can still achieve more than 74% recognition correct rate at different viewpoint, which is better than SIFT.

Could scene context be beneficial for scene text detection?

Scene text detection and scene segmentation are meaningful tasks in the computer vision field. Could the semantic scene segmentation assist scene text detection in any degree? For example, can we expect the probability of a region being text is low if its surrounding segment, i.e., its context, is labeled as sky? In this paper, we have a positive answer by constructing a scene context-based text detection model. In this model, we use texton features and a fully-connected conditional random field (CRF) to estimate pixel-level scene class׳s probability to be considered as image׳s context feature. Meanwhile, maximally stable extremal regions (MSERs) are extracted, integrated and extended as image patches of character candidates. Then, each image patch is fed to a simple two-layer convolutional neural network (CNN) to automatically extract its character feature. The averaged context feature of the corresponding patch is considered as the patch׳s context feature. The character feature and context feature are fused as the input into a support vector machine for text/non-text determination. Finally, as a post-processing, neighboring text regions are grouped hierarchically. The performance evaluation on ICDAR2013 and SVT databases, as well as a preliminary evaluation on a patch-level database, proves that the scene context can improve the performance of scene text detection. Moreover, the comparative study with state-of-the-art methods shows the top-level performance of our method.

Inter-vehicle communication of warning information: an experimental study

In this paper, we show experimentally the feasibility of intervehicle communication of warning information. Warning messages convey significant information that might improve the safety of drivers and passengers. Intervehicle communication can be achieved by the detection of important events through a vision-based detection module, and sharing them between vehicles using a transmission module. In this paper, we developed a testbed that considers both modules in order to detect, recognize and share relevant information, such as traffic signs. To the best of our knowledge, our architecture is the first that combines detection and transmission of messages in the same platform. We detect traffic signs as blobs using the Maximally Stable Extremal Regions (MSER) algorithm, and we recognize them using Random forest classifiers. In the transmission module, we used a simplied broadcasting mechanism that avoids the use of handshaking to establish a communication. In order to assess our system, a set of indoor and outdoor experiments are considered.

Combining SURF and MSER along with Color Features for Image Retrieval System Based on Bag of Visual Words

Content-Based Image Retrieval (CBIR) has received an extensive attention from researchers due to the rapid growing and widespread of image databases. Despite the massive research efforts consumed for CBIR, the completely satisfactory results have not yet been attained. In this article, we offer a new CBIR technique that relies on extracting Speeded Up Robust Features (SURF) and Maximally Stable Extremal Regions (MSER) feature descriptors as well as the color features; color correlograms and Improved Color Coherence Vector (ICCV). These features are joined and used to build a multidimensional feature vector. Bag-of-Visual-Words (BoVW) technique is utilized to quantize the extracted feature vector. Then, a multiclass Support Vector Machine (SVM) is implemented to classify the query images. The performance of the presented retrieval framework is analyzed and scrutinized by comparing it with three alternative approaches. The first one is based on extracting SURF descriptors while the second one is based on extracting SURF descriptors, color correlograms and ICCV. The third approach, on the other hand, is based on extracting MSER, color correlograms and ICCV. All implemented schemes are tested on two benchmark datasets; Corel-1000 and COIL-100 datasets. The empirical results show that our suggested approach has a superior discriminative classification and retrieval performance with respect to other approaches. The proposed method achieves average precisions of 88 and 93% for the Corel-1000 and COIL-100 datasets, respectively. Moreover, the proposed system has shown a substantial advance in the retrieval precision when compared with other existing systems.

Text-Attentional Convolutional Neural Network for Scene Text Detection

Recent deep learning models have demonstrated strong capabilities for classifying text and non-text components in natural images. They extract a high-level feature globally computed from a whole image component (patch), where the cluttered background information may dominate true text features in the deep representation. This leads to less discriminative power and poorer robustness. In this paper, we present a new system for scene text detection by proposing a novel text-attentional convolutional neural network (Text-CNN) that particularly focuses on extracting text-related regions and features from the image components. We develop a new learning mechanism to train the Text-CNN with multi-level and rich supervised information, including text region mask, character label, and binary text/non-text information. The rich supervision information enables the Text-CNN with a strong capability for discriminating ambiguous texts, and also increases its robustness against complicated background components. The training process is formulated as a multi-task learning problem, where low-level supervised information greatly facilitates the main task of text/non-text classification. In addition, a powerful low-level detector called contrast-enhancement maximally stable extremal regions (MSERs) is developed, which extends the widely used MSERs by enhancing intensity contrast between text patterns and background. This allows it to detect highly challenging text patterns, resulting in a higher recall. Our approach achieved promising results on the ICDAR 2013 data set, with an F-measure of 0.82, substantially improving the state-of-the-art results.