Local Image Regions Research Articles

Visual Sentiment Analysis by Attending on Local Image Regions

Visual sentiment analysis, which studies the emotional response of humans on visual stimuli such as images and videos, has been an interesting and challenging problem. It tries to understand the high-level content of visual data. The success of current models can be attributed to the development of robust algorithms from computer vision. Most of the existing models try to solve the problem by proposing either robust features or more complex models. In particular, visual features from the whole image or video are the main proposed inputs. Little attention has been paid to local areas, which we believe is pretty relevant to human's emotional response to the whole image. In this work, we study the impact of local image regions on visual sentiment analysis. Our proposed model utilizes the recent studied attention mechanism to jointly discover the relevant local regions and build a sentiment classifier on top of these local regions. The experimental results suggest that 1) our model is capable of automatically discovering sentimental local regions of given images and 2) it outperforms existing state-of-the-art algorithms to visual sentiment analysis.

Combined Local color curvelet and mesh pattern for image retrieval system

This manuscript presents the content based image retrieval system using new textural features such as colour local curvelet (CLC) based textural descriptor and colour local mesh pattern (CLMP), for the intention of increasing the performance of the image retrieval system. The proposed methods can be able to utilise the distinctive details obtained from spatial coloured textural patterns of various spectral components within the particular local image region. Furthermore, to acquire the benefit of harmonising effect through joint colour texture information, the oppugant colour textural features that obtain the texture patterns of spatial interactions among spectral planes are also integrated in to the creation of CLC and CLMP. Extensive and comparative experiments have been conducted on two benchmark databases, i.e., Corel-1k, MIT VisTex. Retrieval results show that image retrieval using colour local texture features yields better precision and recall than retrieval approaches using either by colour or texture features.

Joint and collaborative representation with local adaptive convolution feature for face recognition with single sample per person

With the aid of a universal facial variation dictionary, sparse representation based classifier (SRC) has been naturally extended for face recognition (FR) with single sample per person (SSPP) and achieved promising performance. However, extracting discriminative facial features and building powerful representation framework for classifying query face images are still the bottlenecks of improving the performance of FR with SSPP. In this paper, by densely sampling and sparsely detecting facial points, we extract complete and robust local regions and learn convolution features adaptive to the local regions and discriminative to the face identity by using convolutional neural networks (CNN). With this powerful facial description and a generic face dataset with common facial variations, a joint and collaborative representation framework, which performs representation for each local region of the query face image while requires all regions of the query face image to have similar representation coefficients, is presented to exploit the distinctiveness and commonality of different local regions. In the proposed joint and collaborative representation with local adaptive convolution feature (JCR-ACF), both discriminative local facial features that are robust to various facial variations and powerful representation dictionaries of facial variations that can overcome the small-sample-size problem are fully exploited. JCR-ACF has been extensively evaluated on several popular databases including AR, CMU Multi-PIE, LFW and the large-scale CASIA-WebFace databases. Experimental results demonstrate the much higher robustness and effectiveness of JCR-ACF to complex facial variations compared to the state-of-the-art methods.

Unsupervised Multi-Class Co-Segmentation via Joint-Cut Over $L_{1}$ -Manifold Hyper-Graph of Discriminative Image Regions

This paper systematically advocates a robust and efficient unsupervised multi-class co-segmentation approach by leveraging underlying subspace manifold propagation to exploit the cross-image coherency. It can combat certain image co-segmentation difficulties due to viewpoint change, partial occlusion, complex background, transient illumination, and cluttering texture patterns. Our key idea is to construct a powerful hyper-graph joint-cut framework, which incorporates mid-level image regions-based intra-image feature representation and L1-manifold graph-based inter-image coherency exploration. For local image region generation, we propose a bi-harmonic distance distribution difference metric to govern the super-pixel clustering in a bottom-up way. It not only affords drastic data reduction but also gives rise to discriminative and structure meaningful feature representation. As for the inter-image coherency, we leverage multi-type features involved L1-graph to detect the underlying local manifold from cross-image regions. As a result, the implicit supervising information could be encoded into the unsupervised hyper-graph joint-cut framework. We conduct extensive experiments and make comprehensive evaluations with other state-of-the-art methods over various benchmarks, including iCoseg, MSRC, and Oxford flower. All the results demonstrate the superiorities of our method in terms of accuracy, robustness, efficiency, and versatility.

Color-Line Regularization for Color Artifact Removal

Existing regularization functions for color image restoration cannot remove color artifact, undesirable appearance of colors, sufficiently, which degrades the quality of color images. In this paper, we propose a new regularization function, named the local color nuclear norm (LCNN), for removing color artifact in color image restoration. LCNN is designed to promote the local color-line property: the color distribution of each local region of a color image exhibits strong linearity. The local color-line property is inherent in clean color images and is violated by color artifact, so that suppressing LCNN is expected to reduce color artifact effectively. In addition, the very nature of LCNN allows us to incorporate it into various color image restoration formulations, where the associated optimization problems can be efficiently solved by proximal splitting techniques. Several illustrative applications of LCNN are presented with comprehensive experimental results.

A two-stage image segmentation via global and local region active contours

Based on popular active contours, this paper proposes a novel two-stage image segmentation method, which incorporates the global and local image region fitting energies. In the first stage, according to the global region active contour, we preliminarily segment the image by globally using the Gaussian distribution, which can rapidly get a coarse segmentation result. Subsequently, by employing a window function, we further segment the image by using the local region active contour, where we use the final active contour of the first stage as the initialization. Compared with the first stage, the local object details are accurately segmented in the second stage, which can be considered as an accurate segmentation result. Due to the suitable initialization from the first stage, the second stage works well in accurately segmenting the image, especially in local details. To regularize the level set function, we introduce a Laplace operator, which efficiently eliminates the expensive re-initialization process of traditional level set methods. Compared with the state-of-the-art methods, experiment results demonstrate the effectiveness and performance of the proposed method with applications to synthetical and real-world images, which usually contain noise, blurry boundaries, and intensity inhomogeneities.

Face Classification by Local Texture Analisys through CBIR and SURF Points

This study presents a robust face recognition system that takes into account both, local texture and points-ofinterest analysis. This system uses the CBIR (Content Based Image Retrieval) technique considering as descriptors the mean, the standard deviation, and the homogeneity of each of the several image windows subjected to analysis; that is, each window acts as a local image region subjected to the face analysis having a face point of interest at its center. In this way, the system retrieves descriptive data of people by analyzing their own texture characteristics on the interior of each face. The system achieves to get a self-organization of the data which a similitud-based order approximation of the face images in a database (DB). With the support of the analysis provided by the points of interest technique SURF, complemented with the CBIR technique, we generated a robust map able to achieve a 100% classification conducted on DB. The results have also been highly successful when conducted under controlled conditions.

A multi-objective approach towards cost effective isolated handwritten Bangla character and digit recognition

Identifying the most informative local regions of a handwritten character image is necessary for a robust handwritten character recognition system. But identifying them from a character image is a difficult task. If this task were to be performed incurring minimum possible cost, it becomes more challenging due to having two independent, apparently contradicting objectives which need to be optimized simultaneously, i.e. maximizing the recognition accuracy and minimizing the associated recognition cost. To address the problem a multi-objective approach is required. In the present task, two popular multi-objective optimization Algorithm (1) a Non-Dominated Sorting Harmony-Search Algorithm (NSHA) and (2) a Non-Dominated Sorting Genetic Algorithm-II (NSGA-II, Deb et al., 2002 [18]) are employed for region sampling separately. The method objectively selects the most informative set of local regions using the framework of Axiomatic Fuzzy Set (AFS) theory, from the sets of pareto-optimal solutions provided by the multi-objective region sampling algorithms. The system has been evaluated on two isolated handwritten Bangla datasets, (1) a dataset of randomly mixed handwritten Bangla Basic and Compound characters and (2) a dataset of handwritten Bangla numerals separately, with SVM based classifier, using a feature set containing convex-hull based features and CG based quad-tree partitioned longest-run based local features extracted from the selected local regions. The results have shown a significant increase in recognition accuracy and decrease in recognition cost for all the datasets. Thus the present system introduces a cost effective approach towards isolated handwritten character recognition systems.

Hyperspectral Local Intrinsic Dimensionality

The intrinsic dimensionality (ID) of multivariate data is a very important concept in spectral unmixing of hyperspectral images. A good estimation of the ID is crucial for a correct retrieval of the number of endmembers (the spectral signatures of macroscopic materials) in the image, for dimensionality reduction or for subspace learning, among others. Recently, some approaches to perform spectral unmixing and superresolution locally have been proposed, which require a local estimation of the number of endmembers to use. However, the role of ID in local regions of hyperspectral images has not been properly addressed. Some important issues when dealing with small regions of hyperspectral data can seriously affect the performance of conventional hyperspectral ID estimators. We show that three factors mainly affect local ID estimation: the number of pixels in the local regions, which has to be high enough for the estimations to be relevant, the number of hyperspectral bands which complicates the estimations if the ambient space has a high dimensionality, and the noise, which can be misinterpreted as a signal when its power is important. Here, we review the hyperspectral ID estimators on the literature for local ID estimation, we show how they behave in a local setting on synthetic and real data sets, and we provide some guidelines to make proper use of these estimators in local approaches.

Center Symmetric Local Multilevel Pattern Based Descriptor and Its Application in Image Matching

This paper presents an effective local image region description method, called CS-LMP (Center Symmetric Local Multilevel Pattern) descriptor, and its application in image matching. The CS-LMP operator has no exponential computations, so the CS-LMP descriptor can encode the differences of the local intensity values using multiply quantization levels without increasing the dimension of the descriptor. Compared with the binary/ternary pattern based descriptors, the CS-LMP descriptor has better descriptive ability and computational efficiency. Extensive image matching experimental results testified the effectiveness of the proposed CS-LMP descriptor compared with other existing state-of-the-art descriptors.

Semi-automatic segmentation of prostate in CT images via coupled feature representation and spatial-constrained transductive lasso.

Conventional learning-based methods for segmenting prostate in CT images ignore the relations among the low-level features by assuming all these features are independent. Also, their feature selection steps usually neglect the image appearance changes in different local regions of CT images. To this end, we present a novel semi-automatic learning-based prostate segmentation method in this article. For segmenting the prostate in a certain treatment image, the radiation oncologist will be first asked to take a few seconds to manually specify the first and last slices of the prostate. Then, prostate is segmented with the following two steps: (i) Estimation of 3D prostate-likelihood map to predict the likelihood of each voxel being prostate by employing the coupled feature representation, and the proposed Spatial-COnstrained Transductive LassO (SCOTO); (ii) Multi-atlases based label fusion to generate the final segmentation result by using the prostate shape information obtained from both planning and previous treatment images. The major contribution of the proposed method mainly includes: (i) incorporating radiation oncologist's manual specification to aid segmentation, (ii) adopting coupled features to relax previous assumption of feature independency for voxel representation, and (iii) developing SCOTO for joint feature selection across different local regions. The experimental result shows that the proposed method outperforms the state-of-the-art methods in a real-world prostate CT dataset, consisting of 24 patients with totally 330 images, all of which were manually delineated by the radiation oncologist for performance evaluation. Moreover, our method is also clinically feasible, since the segmentation performance can be improved by just requiring the radiation oncologist to spend only a few seconds for manual specification of ending slices in the current treatment CT image.

Probabilistic Deformation Models for Challenging Periocular Image Verification

The periocular region as a biometric trait has recently gained considerable traction, especially under challenging scenarios where reliable iris information is not available for human authentication. In this paper, we consider the problem of one-to-one (1 : 1) matching of highly nonideal periocular images captured in-the-wild under unconstrained imaging conditions. Such images exhibit considerable appearance variations, including nonuniform illumination variations, motion and defocus blur, off-axis gaze, and nonstationary pattern deformations. To address these challenges, we propose periocular probabilistic deformation models (PPDMs) that: 1) reduce the image matching problem to matching local image regions and 2) approximate the periocular distortions by local patch level spatial translations whose relationships are modeled by a Gaussian Markov random field. Given a periocular image pair, we determine the distortion-tolerant similarity metric by regularizing local match scores by the maximum a posteriori probability estimate of the relative local deformations between them. Unlike the existing global periocular image matching techniques, by accounting for local image deformations in the periocular matching process, PPDM exhibits greater tolerance to pattern variations. We demonstrate the effectiveness of our model via extensive evaluation on a large number of in-the-wild periocular images. We find that PPDMs outperform many benchmark 1 : 1 image matching techniques (improving verification rates at 0.1% false accept rate by ${\sim }30$ % over previous work and ${\sim }40$ % when compared with the best baseline) in challenging scenarios leading to state-of-the-art verification performance on multiple real-world periocular data sets.

A ROBUST MATCHING METHOD FOR UNMMANED AERIAL VEHICLE IMAGES WITH DIFFERENT VIEWPOINT ANGLES BASED ON REGIONAL COHERENCY

Abstract. One of the main challenges confronting high-resolution remote sensing image matching is how to address the issue of geometric deformation between images, especially when the images are obtained from different viewpoints. In this paper, a robust matching method for Unmanned Aerial Vehicle images of different viewpoint angles based on regional coherency is proposed. The literature on the geometric transform analysis reveals that if transformations between different pixel pairs are different, they can't be expressed by a uniform affine transform. While for the same real scene, if the instantaneous field of view or the target depth changes is small, transformation between pixels in the whole image can be approximated by an affine transform. On the basis of this analysis, a region coherency matching method for Unmanned Aerial Vehicle images is proposed. In the proposed method, the simplified mapping from image view change to scale change and rotation change has been derived. Through this processing, the matching between view change images can be converted into the matching between rotation and scale changed images. In the method, firstly local image regions are detected and view changes between these local regions are mapped to rotation and scale change by performing local region simulation. And then, point feature detection and matching are implemented in the simulated image regions. Finally, a group of Unmanned Aerial Vehicle images are adopted to verify the performance of proposed matching method respectively, and a comparative analysis with other methods demonstrates the effectiveness of the proposed method.

Novel Approach for Child and Adulthood Classification Based on Significant Prominent Binary Patterns of Local Maximum Edge (SPBPLME)

This paper derives a new procedure for age classification of facial image based on the local region of facial image. The local region of facial image is extracted from a Significant Binary Pattern of Local Maximum Edge (SBPLME). The SBPLME is generated by calculating the absolute value of local difference between the average of local 3×3 sub window pixel values and its neighbors instead of the center pixel value. In the case of Local Maximum Edge Binary Pattern (LMEBP) calculating the absolute value of local difference between the center pixel value of local 3×3 sub window and its neighbors. The proposed SBPLME can generate 512 (0 to 511) different patterns. The present paper utilized Prominent LBP (PLBP) on the proposed SBPLME. The PLBP contains the significant patterns of Uniform LBP (ULBP) and Non Uniform LBP (NULBP). Thus the derived Significant PLBP of Local Maximum Edge (SPBPLME) becomes an efficient image classification and analysis, which will have a significant role in many areas. The novelty of the proposed SPBPLME method is, it has shown excellent age classification results by reducing the overall dimension, thus reducing the overall complexity.

Enhanced radiographic imaging of defects in aircraft structure materials with the dehazing method

The aircraft structures are made of aluminium alloys because of its various advantages, including ease of manufacture, high tolerance and ease of maintenance. Corrosions and cracks are often found in high-strength aluminium alloys. The industrial radiographic testing method and digital radiography are two most important tools for detecting different kinds of defects in aluminium structures. However, because of greater sensitivity and dynamic range of phosphor plates in computed radiography than in film, digital radiography can produce clear and high-contrast images, but digital radiography images appear foggy. In this study, a dehazing algorithm is implemented for the digital radiography images of airplane parts to remove fog. The used dehazing algorithm is based on the dark channel prior and it is based on the statistics of outdoor haze-free images. In most of the local regions of the radiography images, some pixels very often have very low intensity in at least one colour (RGB: red, green, blue) channel which are called dark pixels. In hazy radiography images, the intensity of these dark pixels in that channel is mainly contributed by scattering. Therefore, these dark pixels can directly provide an accurate estimation of the haze transmission and combining a haze imaging model and a soft matting interpolation method can be recovered a high-quality haze free in the radiography image and produce a good depth map and the defects. The results show that the fog-removed images have better contrast and the shapes of defects are very clear. In addition, some invisible cracks in the digital images can be seen in the defogged image.

Single image dehazing using the change of detail prior

In this paper, we present a simple but effective image prior, change of detail (CoD) prior, to remove haze from a single input image. The CoD prior is developed based on the multiple scattering phenomenon in the propagation of light. By adopting the CoD prior in an image model, the thickness of the haze can be estimated effectively to recover a high quality, haze-free image. Our method is stable to image local regions containing objects in different depths. In addition, since the CoD prior is mostly sensitive to local details of the image rather than color and intensity, our method is able to handle both color and grayscale images. Our experiments showed that the proposed method achieved better results than several state-of-the-art methods, and it can be implemented very quickly.

Handwritten Bangla character recognition using a soft computing paradigm embedded in two pass approach

The work presented here mainly involves recognition of handwritten Bangla characters using a soft computing paradigm embedded within the framework of previously developed Two pass approach. Bangla script, which has nearly 400 character symbols, is used by a vast majority of population in India and Bangladesh. Considering the large number of character classes, Two pass approach is chosen here with some important extensions. Typically, Two pass approach coarsely classifies an unknown sample pattern in a group of pattern classes in first-pass. In second pass, it makes finer classification of the sample by determining its membership to a class within the group. Two major aspects of extensions made with Two pass approach here are development of an algorithmic approach for grouping of various pattern classes needed for the second pass and a soft computing methodology for optimal selection of local regions of character images for extraction of features, specific to each grouping of pattern classes. To test the performances of the method, benchmark databases of handwritten Bangla characters are also developed. A methodology for reading filled in forms of handwritten Bangla characters is also presented here. The experimental results show significant improvement of recognition rates on handwritten Bangla characters compared to traditionally followed Single pass approach.

Label Embedding: A Frugal Baseline for Text Recognition

The standard approach to recognizing text in images consists in first classifying local image regions into candidate characters and then combining them with high-level word models such as conditional random fields. This paper explores a new paradigm that departs from this bottom-up view. We propose to embed word labels and word images into a common Euclidean space. Given a word image to be recognized, the text recognition problem is cast as one of retrieval: find the closest word label in this space. This common space is learned using the Structured SVM framework by enforcing matching label-image pairs to be closer than non-matching pairs. This method presents several advantages: it does not require ad-hoc or costly pre-/post-processing operations, it can build on top of any state-of-the-art image descriptor (Fisher vectors in our case), it allows for the recognition of never-seen-before words (zero-shot recognition) and the recognition process is simple and efficient, as it amounts to a nearest neighbor search. Experiments are performed on challenging datasets of license plates and scene text. The main conclusion of the paper is that with such a frugal approach it is possible to obtain results which are competitive with standard bottom-up approaches, thus establishing label embedding as an interesting and simple to compute baseline for text recognition.

Shape primitive histogram: low‐level face representation for face recognition

Human face contains abundant shape features. This fact motivates a lot of shape feature-based face detection and three-dimensional (3D) face recognition approaches. However, as far as we know, there is no prior low-level face representation which is purely based on shape feature proposed for conventional 2D (image-based) face recognition. In this study, the authors present a novel low-level shape-based face representation named ‘shape primitives histogram’ (SPH) for face recognition. In this approach, the face images are separated into a number of tiny shape fragments and they reduce these shape fragments to several uniform atomic shape patterns called ‘shape primitives’. Then the face representation is obtained by implementing a histogram statistic of shape primitives in a local image region. To take scale information into consideration, they also produce multi-scale SPHs (MSPHs) by concatenating the SPHs extracted from different scales. Moreover, they experimentally study the influences of each stage of SPH computation on performance, concluding that a small cell with 1/2 overlap and a fine size block with 1/2 overlap are important for good results. Four popular face databases, namely ORL, AR, YaleB and LFW-a, are employed to evaluate SPH and MSPH. Surprisingly, such seemingly naive shape-based face representations outperform the state-of-the-art low-level face representations.

Semantic Concepts Classification on Outdoor Scene Images Based on Region-Based Approach

Outdoor scene analysis is a complex problem for both image processing and pattern recognition domains. There are two methods of segmenting images to look for objects in an image, block-based and region-based. Region-based method can provide some useful information about objects even though segmentation may not be perfect. There are three phases in this system: segmentation, features extraction and classification. The basic idea of this system is to classify local image regions into semantic concept classes such as tree, sky and road etc. In this paper, modified Marker-Controlled Watershed (MCWS) algorithm is proposed. Firstly, the modified (MCWS) algorithm is used to segment input image. And then, texture feature vectors are extracted from segmented regions by Gray-Level Co-occurrence Matrix (GLCM). Finally, classification is performed by 3-layer Artificial Neural Network (ANN). This system is applied on real scene images dataset.