Spatial Pyramid Matching Framework Research Articles

An Enhancement to the Spatial Pyramid Matching for Image Classification and Retrieval

Spatial pyramid matching (SPM) is one of the widely used methods to incorporate spatial information into the image representation. Despite its effectiveness, the traditional SPM is not rotation invariant. A rotation invariant SPM has been proposed in the literature but it has many limitations regarding the effectiveness. In this paper, we investigate how to make SPM robust to rotation by addressing those limitations. In an SPM framework, an image is divided into an increasing number of partitions at different pyramid levels. In this paper, our main focus is on how to partition images in such a way that the resulting structure can deal with image-level rotations. To do that, we investigate three concentric ring partitioning schemes. Apart from image partitioning, another important component of the SPM framework is a weight function. To apportion the contribution of each pyramid level to the final matching between two images, the weight function is needed. In this paper, we propose a new weight function which is suitable for the rotation-invariant SPM structure. Experiments based on image classification and retrieval are performed on five image databases. The detailed result analysis shows that we are successful in enhancing the effectiveness of SPM for image classification and retrieval.

An Improved Image Classification Method Based On Spatial Pyramid Matching Framework

Spatial Pyramid Matching (SPM) and its variants have achieved a lot of success in image classification. However, the process of extraction of SIFT descriptors may cause texture information loss under the SPM framework. In addition, traditional Grey Level Co-occurrence Matrices (GLCM) method does not consider the spatial relationship of raw pixels. In this paper, to solve the issues, an improved GLCM method are proposed, which enhances the performance of original GLCM method for image texture feature representation. Firstly, the original images are subdivided to increasingly fine sub-regions and a weighted vector is generated by GLCM method inside each sub-regions. Secondly, the weighted vector of each sub-regions and the SIFT descriptors are concatenated. Thirdly, the descriptors are encoded by using the technique of sparse coding and the final features are obtained by max pooling. Finally, the features are fed to kernel classifiers to evaluate the effectiveness of our method. Comprehensive experimental results reveal that our method achieves comparable classification accuracy compared to the previous methods.

A New Scene Classification Method Based on Spatial Pyramid Matching Model

Scene classiflcation is an appealing and challenging problem in image processing and machine vision. Recently, Bag-of-visual-words (BOVW) method using pyramid matching scheme has shown remarkable performance for scene classiflcation. But this method deriving from local keypoints does not contain texture features which are rich in scene images. To further improves the classiflcation accuracy, this paper presents a new method combining Rotation Invariant Local Binary Patterns (RILBP) texture features and BOVW model in spatial pyramid matching framework. First, scene image is subdivided at difierent resolutions for constructing a spatial pyramid. Then based on scale invariant feature transform descriptor and K-means clustering, Pyramid Histogram of visual Words (PHOW) is extracted. And RILBP texture feature is extracted using the mean of a 3*3 neighborhood as threshold. Last we construct a composite kernel of spatial pyramid matching. We regard the keypoint features and texture features as two independent feature channels, and combine them to realize scene classiflcation using one-against-rest SVMs with the composite kernel. Experiments results on the three difierent scene datasets show that our method is efiective.

A New Scene Classification Method Based on Local Gabor Features

A new scene classification method is proposed based on the combination of local Gabor features with a spatial pyramid matching model. First, new local Gabor feature descriptors are extracted from dense sampling patches of scene images. These local feature descriptors are embedded into a bag-of-visual-words (BOVW) model, which is combined with a spatial pyramid matching framework. The new local Gabor feature descriptors have sufficient discrimination abilities for dense regions of scene images. Then the efficient feature vectors of scene images can be obtained byK-means clustering method and visual word statistics. Second, in order to decrease classification time and improve accuracy, an improved kernel principal component analysis (KPCA) method is applied to reduce the dimensionality of pyramid histogram of visual words (PHOW). The principal components with the bigger interclass separability are retained in feature vectors, which are used for scene classification by the linear support vector machine (SVM) method. The proposed method is evaluated on three commonly used scene datasets. Experimental results demonstrate the effectiveness of the method.

Stacked Predictive Sparse Decomposition for Classification of Histology Sections

Image-based classification of histology sections, in terms of distinct components (e.g., tumor, stroma, normal), provides a series of indices for histology composition (e.g., the percentage of each distinct components in histology sections), and enables the study of nuclear properties within each component. Furthermore, the study of these indices, constructed from each whole slide image in a large cohort, has the potential to provide predictive models of clinical outcome. For example, correlations can be established between the constructed indices and the patients' survival information at cohort level, which is a fundamental step towards personalized medicine. However, performance of the existing techniques is hindered as a result of large technical variations (e.g., variations of color/textures in tissue images due to non-standard experimental protocols) and biological heterogeneities (e.g., cell type, cell state) that are always present in a large cohort. We propose a system that automatically learns a series of dictionary elements for representing the underlying spatial distribution using stacked predictive sparse decomposition. The learned representation is then fed into the spatial pyramid matching framework with a linear support vector machine classifier. The system has been evaluated for classification of distinct histological components for two cohorts of tumor types. Throughput has been increased by using of graphical processing unit (GPU), and evaluation indicates a superior performance results, compared with previous research.

Classification of Histology Sections via Multispectral Convolutional Sparse Coding.

Image-based classification of histology sections plays an important role in predicting clinical outcomes. However this task is very challenging due to the presence of large technical variations (e.g., fixation, staining) and biological heterogeneities (e.g., cell type, cell state). In the field of biomedical imaging, for the purposes of visualization and/or quantification, different stains are typically used for different targets of interest (e.g., cellular/subcellular events), which generates multi-spectrum data (images) through various types of microscopes and, as a result, provides the possibility of learning biological-component-specific features by exploiting multispectral information. We propose a multispectral feature learning model that automatically learns a set of convolution filter banks from separate spectra to efficiently discover the intrinsic tissue morphometric signatures, based on convolutional sparse coding (CSC). The learned feature representations are then aggregated through the spatial pyramid matching framework (SPM) and finally classified using a linear SVM. The proposed system has been evaluated using two large-scale tumor cohorts, collected from The Cancer Genome Atlas (TCGA). Experimental results show that the proposed model 1) outperforms systems utilizing sparse coding for unsupervised feature learning (e.g., PSD-SPM [5]); 2) is competitive with systems built upon features with biological prior knowledge (e.g., SMLSPM [4]).