Low-level Feature Space Research Articles

Transductive multi-view zero-shot learning.

Most existing zero-shot learning approaches exploit transfer learning via an intermediate semantic representation shared between an annotated auxiliary dataset and a target dataset with different classes and no annotation. A projection from a low-level feature space to the semantic representation space is learned from the auxiliary dataset and applied without adaptation to the target dataset. In this paper we identify two inherent limitations with these approaches. First, due to having disjoint and potentially unrelated classes, the projection functions learned from the auxiliary dataset/domain are biased when applied directly to the target dataset/domain. We call this problem the projection domain shift problem and propose a novel framework, transductive multi-view embedding, to solve it. The second limitation is the prototype sparsity problem which refers to the fact that for each target class, only a single prototype is available for zero-shot learning given a semantic representation. To overcome this problem, a novel heterogeneous multi-view hypergraph label propagation method is formulated for zero-shot learning in the transductive embedding space. It effectively exploits the complementary information offered by different semantic representations and takes advantage of the manifold structures of multiple representation spaces in a coherent manner. We demonstrate through extensive experiments that the proposed approach (1) rectifies the projection shift between the auxiliary and target domains, (2) exploits the complementarity of multiple semantic representations, (3) significantly outperforms existing methods for both zero-shot and N-shot recognition on three image and video benchmark datasets, and (4) enables novel cross-view annotation tasks.

Self-Growing RBF Neural Network Approach for Semantic Image Retrieval

Traditional methods of content-based image retrieval deal with the retrieval of images according to the similari- ty between them and the sample image in some low-level feature space such as color, shape and structure. But the relevant images satisfying user information need tend to have different distribution in the low-level feature space. In this case, the query image needs to be represented as multiple query images corresponding to the scattered relevant images. This paper proposes a new relevance feedback technique for semantic image retrieval which is based on the self-growing radial basis function (SGRBF) neural network. The approach can adaptively construct SGRBF neural network based on the users' feedbacks. Thus, hidden nodes of the SGRBF neural network can represent the distribution of the users' perceptual in the low-level feature space and bridge the semantic gap between low-level feature and high-level concept of the image con- tent. The method is verified on a database of 1000 images and experimental results demonstrate that our method proposed in this paper is an effective method to promote semantic image retrieval performance.

Mining Semantically Consistent Patterns for Cross-View Data

In some real world applications, like information retrieval and data classification, we often are confronted with the situation that the same semantic concept can be expressed using different views with similar information. Thus, how to obtain a certain Semantically Consistent Patterns (SCP) for cross-view data, which embeds the complementary information from different views, is of great importance for those applications. However, the heterogeneity among cross-view representations brings a significant challenge on mining the SCP. In this paper, we propose a general framework to discover the SCP for cross-view data. Specifically, aiming at building a feature-isomorphic space among different views, a novel Isomorphic Relevant Redundant Transformation (IRRT) is first proposed. The IRRT linearly maps multiple heterogeneous low-level feature spaces to a high-dimensional redundant feature-isomorphic one, which we name as mid-level space. Thus, much more complementary information from different views can be captured. Furthermore, to mine the semantic consistency among the isomorphic representations in the mid-level space, we propose a new Correlation-based Joint Feature Learning (CJFL) model to extract a unique high-level semantic subspace shared across the feature-isomorphic data. Consequently, the SCP for cross-view data can be obtained. Comprehensive experiments on three data sets demonstrate the advantages of our framework in classification and retrieval.

Unsupervised Multi-Spectral Satellite Image Segmentation Combining Modified Mean-Shift and a New Minimum Spanning Tree Based Clustering Technique

An unsupervised object based segmentation, combining a modified mean-shift (MS) and a novel minimum spanning tree (MST) based clustering approach of remotely sensed satellite images has been proposed in this correspondence. The image is first pre-processed by a modified version of the standard MS based segmentation which preserves the desirable discontinuities present in the image and guarantees oversegmentation in the output. A nearest neighbor based method for estimating the bandwidth of the kernel density estimator (KDE) and a novel termination condition have been incorporated into the standard MS. Considering the segmented regions as nodes in a low level feature space, an MST is constructed. An unsupervised technique to cluster a given MST has also been devised here. This type of hybrid segmentation technique which clusters the regions instead of image pixels reduces greatly the sensitivity to noise and enhances the overall segmentation performance. The superiority of the proposed method has been experimented on a large set of multi-spectral images and compared with some well-known hybrid segmentation models.

HBIR: Hypercube-Based Image Retrieval

In this paper, we propose a mapping from low level feature space to the semantic space drawn by the users through relevance feedback to enhance the performance of current content based image retrieval (CBIR) systems. The proposed approach makes a rule base for its inference and configures it using the feedbacks gathered from users during the life cycle of the system. Each rule makes a hypercube (HC) in the feature space corresponding to a semantic concept in the semantic space. Both short and long term strategies are taken to improve the accuracy of the system in response to each feedback of the user and gradually bridge the semantic gap. A scoring paradigm is designed to determine the effective rules and suppress the inefficient ones. For improving the response time, an HC merging approach and, for reducing the conflicts, an HC splitting method is designed. Our experiments on a set of 11 000 images from the Corel database show that the proposed approach can better describe the semantic content of images for image retrieval with respect to some existing approaches reported recently in the literature. Moreover, our approach can be better trained and is not saturated in long time, i.e., any feedback improves the precision and recall of the system. Another strength of our method is its ability to address the dynamic nature of the image database such that it can follow the changes occurred instantaneously and permanently by adding and dropping images.

Multilabel classification with meta-level features in a learning-to-rank framework

Effective learning in multi-label classification (MLC) requires an appropriate level of abstraction for representing the relationship between each instance and multiple categories. Current MLC methods have focused on learning-to-map from instances to categories in a relatively low-level feature space, such as individual words. The fine-grained features in such a space may not be sufficiently expressive for learning to rank categories, which is essential in multi-label classification. This paper presents an alternative solution by transforming the conventional representation of instances and categories into meta-level features, and by leveraging successful learning-to-rank retrieval algorithms over this feature space. Controlled experiments on six benchmark datasets using eight evaluation metrics show strong evidence for the effectiveness of the proposed approach, which significantly outperformed other state-of-the-art methods such as Rank-SVM, ML-kNN (Multi-label kNN), IBLR-ML (Instance-based logistic regression for multi-label classification) on most of the datasets. Thorough analyses are also provided for separating the factors responsible for the improved performance.

Hybrid Query Refinement

This paper proposes a hybrid query refinement model for distance-based index structures supporting content-based image retrievals. The framework refines a query by considering both the low-level feature space as well as the high-level semantic interpretations separately. Thus, it successfully handles queries where the gap between the feature components and the semantics is large. It refines the low-level feature space, indexed by the distance based index structure, in multiple iterations by introducing the concept of multipoint query in a metric space. It refines the high-level semantic space by dynamically adjusting the constructs of a framework, called the Markov Model Mediator (MMM), utilized to introduce the semantic relationships in the index structure. A k-nearest neighbor (k-NN) algorithm is designed to handle similarity searches that refine a query in multiple iterations utilizing the proposed hybrid query refinement model. Extensive experiments are performed demonstrating an increased relevance of query results in subsequent iterations while incurring a low computational overhead. Further, an evaluation metric, called the Model_Score, is proposed to compare the performance of different retrieval frameworks in terms of both computation overhead and query result relevance. This metric enables the users to choose the retrieval framework appropriate for their requirements.

FUZZY MODE SIMILARITY MEASURE BASED ON MEASURE OF ILLUMINATION INSTABILITY

Most color-based multimedia data indexing systems are sensitive to changes in the illumination environment. Some color-based systems coming into practical use are claimed to be limited to target videos with stable lighting. In this paper, a fuzzy mode similarity measure is proposed to adaptively calibrate the feature-matching criterion based on the measure of the illumination instability. The purpose of this approach is to retrieve the identical semantic object from the video and to alleviate the impact of lighting changes. An information-theoretic measure is first proposed to automatically measure the illumination instability of the video. A fuzzy model is then proposed to estimate and characterize the impact of illumination changes to the distribution shape of the semantic object in the low-level visual feature space. Experiments are shown to demonstrate how the proposed measure of the illumination instability, which takes the information distribution within an image into account, can reflect the instability more effectively than other simple and straightforward measures. Experiments also show how the retrieval performance can be improved by using the fuzzy mode similarity measure.

Measuring Concept Similarities in Multimedia Ontologies: Analysis and Evaluations

The recent development of large-scale multimedia concept ontologies has provided a new momentum for research in the semantic analysis of multimedia repositories. Different methods for generic concept detection have been extensively studied, but the question of how to exploit the structure of a multimedia ontology and existing inter-concept relations has not received similar attention. In this paper, we present a clustering-based method for modeling semantic concepts on low-level feature spaces and study the evaluation of the quality of such models with entropy-based methods. We cover a variety of methods for assessing the similarity of different concepts in a multimedia ontology. We study three ontologies and apply the proposed techniques in experiments involving the visual and semantic similarities, manual annotation of video, and concept detection. The results show that modeling inter-concept relations can provide a promising resource for many different application areas in semantic multimedia processing.

S-IRAS

Relevance feedback and semantic retrieval have received extensive attention recently in the computer vision community. In this article, we present a semantic image query system with integrated feedback mechanism. Our system has two major components: the low-level feature space and the semantic space. In the low-level feature space, images are described by multidimensional vectors and are clustered based on the similarity of their contents. In the semantic space, the relationship among keywords is captured by a semantic hierarchy built by the aid of WordNet. Based on our system architecture, we propose a novel feedback solution for semantic retrieval called semantic feedback, which allows our system to interact with users directly at the semantic level. The short-term and long-term learning process of the semantic feedback substantially improves the image retrieval and annotation performance of our system. We demonstrate the effectiveness of our approach with experiments using 5,000 images from Corel database. The significant contribution of this article is in the scenario of having a relatively small training data set compared to the testing data set. Some of the previous work in the same direction has chosen a very large training set and a very small testing set. Clearly, the problem that we try to solve is more realistic and challenging.

Exploring statistical correlations for image retrieval

Bridging the cognitive gap in image retrieval has been an active research direction in recent years, of which a key challenge is to get enough training data to learn the mapping functions from low-level feature spaces to high-level semantics. In this paper, image regions are classified into two types: key regions representing the main semantic contents and environmental regions representing the contexts. We attempt to leverage the correlations between types of regions to improve the performance of image retrieval. A Context Expansion approach is explored to take advantages of such correlations by expanding the key regions of the queries using highly correlated environmental regions according to an image thesaurus. The thesaurus serves as both a mapping function between image low-level features and concepts and a store of the statistical correlations between different concepts. It is constructed through a data-driven approach which uses Web data (images, their surrounding textual annotations) as training data source to learn the region concepts and to explore the statistical correlations. Experimental results on a database of 10,000 general-purpose images show the effectiveness of our proposed approach in both improving search precision (i.e. filter irrelevant images) and recall (i.e. retrieval relevant images whose context may be varied). Several major factors which have impact on the performance of our approach are also studied.

Optical associative processor for general linear transformations

A new technique for the realization of general linear transformations using associative memories is described. An optical architecture for its implementation is also presented. A low-level feature space processor using this architecture is proposed. The processor is capable of recognizing and locating objects of various shapes and uses certain linear transformations in the feature space for distortion invariance.