Relevance Feedback In Image Retrieval Research Articles

Robust discriminative extreme learning machine for relevance feedback in image retrieval

Relevance feedback (RF) has long been an important approach for multi-media retrieval because of the semantic gap in image content, where SVM based methods are widely applied to RF of content-based image retrieval. However, RF based on SVM still has some limitations: (1) the high dimension of image features always make the RF time-consuming; (2) the model of SVM is not discriminative, because labels of image features are not sufficiently exploited. To solve above problems, we proposed robust discriminative extreme learning machine (RDELM) in this paper. RDELM involved both robust within-class and between-class scatter matrices to enhance the discrimination capacity of ELM for RF. Furthermore, an angle criterion dimensionality reduction method is utilized to extract the discriminative information for RDELM. Experimental results on four benchmark datasets (Corel-1K, Corel-5K, Corel-10K and MSRC) illustrate that our proposed RF method in this paper achieves better performance than several state-of-the-art methods.

Optimized Feature Subset Selection and Relevance Feedback for Image Retrieval Based on Multiresolution Enhanced Orthogonal Polynomials Model

In this paper, a simple image retrieval method incorporating relevance feedback based on the multiresolution enhanced orthogonal polynomials model is proposed. In the proposed method, the low level image features such as texture, shape and color are extracted from the reordered orthogonal polynomials model coefficients and linearly combined to form a multifeature set. Then the dimensionality of the multifeature set is reduced by utilizing multi objective Genetic Algorithm (GA) and multiclass binary Support Vector Machine (SVM). The obtained optimized multifeature set is used for image retrieval. In order to improve the retrieval accuracy and to bridge the semantic gap, a correlation based k-Nearest Neighbor (k-NN) method for relevance feedback is also proposed. In this method, an appropriate relevance score is computed for each image in the database based on relevant and non relevant set chosen by the user with correlation based k-NN method. The experiments are carried out with Corel and Caltech database images and the retrieval rates are computed. The proposed method with correlation based k-NN for relevance feedback gives an average retrieval rate of 94.67%.

Multi-Query Parallel Field Ranking for image retrieval

Relevance feedback image retrieval is an effective scheme bridging the gap between low-level features and high-level concepts. It is essentially a multi-query ranking problem where the user submitted image and provided positive examples are considered as queries. Most of the existing approaches either merge the multiple queries into a single query or consider them independently, and then the geodesic distances on the image manifold are used to measure the similarities between the query image and the other images in database. In this paper, we propose a novel approach called Multi-Query Parallel Field Ranking (MQPFR) which finds an optimal ranking function whose gradient field is as parallel as possible. In this way, the obtained ranking function varies linearly along the geodesics of the data manifold, and achieves the highest value at the multiple queries simultaneously. Extensive experiments are carried out on a large image database and demonstrate the effectiveness of the proposed approach.

Improving Relevance Feedback in Image Retrieval by Incorporating Unlabelled Images

In content-base image retrieval, relevance feedback (RF) schemes based on support vector machine (SVM) have been widely used to narrow the semantic gap between low-level visual features and high-level human perception. However, the performance of image retrieval with SVM active learning is known to be poor when the training data is insufficient. In this paper, the problem is solved by incorporating the unlabelled images into the learning process. We proposed a semi-supervised active learning algorithm which uses not only labeled training samples but also unlabeled ones to build better models. In relevance feedback, active learning algorithm is often used to reduce the cost of labeling by selecting only the most informative data. In addition, we introduced a semi-supervised approach which employed Nearest-Neighbor technique to label the unlabeled sample with a certain degree of uncertainty in its class information. Using these samples, Fuzzy support vector machine (FSVM) which takes into account the fuzzy nature of some training samples during its training is trained. We compared our method with standard active SVM on a database of 10,000 images, the experiment results show that the efficiency of SVM active learning can be improved by incorporating unlabelled images, and thus improve the overall retrieval performance. DOI: http://dx.doi.org/10.11591/telkomnika.v11i7.2807

Generalized adaptive Bayesian Relevance Feedback for image retrieval in the Orthogonal Polynomials Transform domain

In this paper, we propose a generalized Bayesian Relevance Feedback (RF) algorithm for image retrieval systems with enhanced adaptability to the users' requirements. The adaptability of the algorithm is owing to the different weights that are given to the current and the prior learning. This algorithm is implemented in an image retrieval system which learns in the integer-arithmetic Orthogonal Polynomials Transform (OPT) domain. With the transform's partial coefficients of the image being the features extracted, a mixture of Gaussians is used to represent the image. The image retrieval system is trained on the COIL-100 database. Experimental evidence illustrates the clear benefits of this introduction of adaptability into RF algorithm which can account for both positive and negative feedback. The superiority of the proposed algorithm in terms of increased recall and reduced number of feedback iterations when compared to the already existing Bayesian RF implementations is demonstrated.

Performance Evaluation of Relevance Feedback for Image Retrieval by “Real-World” Multi-Tagged Image Datasets

Anyone who has ever tried to describe a picture in words is aware that it is not an easy task to find a word, a concept, or a category that characterizes it completely. Most images in real life represent more than a concept; therefore, it is natural that images available to users over the Internet (e.g., FLICKR) are associated with multiple tags. By the term ‘tag’, the authors refer to a concept represented in the image. The purpose of this paper is to evaluate the performances of relevance feedback techniques in content-based image retrieval scenarios with multi-tag datasets, as typically performances are assessed on single-tag dataset. Thus, the authors show how relevance feedback mechanisms are able to adapt the search to user’s needs either in the case an image is used as an example for retrieving images each bearing different concepts, or the sample image is used to retrieve images containing the same set of concepts. In this paper, the authors also propose two novel performance measures aimed at comparing the accuracy of retrieval results when an image is used as a prototype for a number of different concepts.

A biased minimax probability machine-based scheme for relevance feedback in image retrieval

In recent years, minimax probability machines (MPMs) have demonstrated excellent performance in a variety of pattern recognition problems. At the same time various machine learning methods have been applied on relevance feedback tasks in content-based image retrieval (CBIR). One of the problems in typical techniques for relevance feedback is that they treat the relevant feedback and irrelevant feedback equally. Since the negative instances largely outnumber the positive instances, the assumption that they are balanced is incorrect as the data are biased. In this paper we study how biased minimax probability machine (BMPM), a variation of MPM, can be applied for relevance feedback in image retrieval tasks. Different from previous methods, this model directly controls the accuracy of classification of the future data to construct biased classifiers. Hence, it provides a rigorous treatment on imbalanced dataset. Mathematical formulation and explanations are provided to demonstrate the advantages. Experiments are conducted to evaluate the performance of our proposed framework, in which encouraging and promising experimental results are obtained.

Variants of evolutionary learning for interactive image retrieval

We systematically compare four variants of evolutionary algorithms (EAs) for relevance feedback (RF) in image retrieval. We focus on the small sample size (SSS) performance, i.e., the requirement to learn the user’s information need based on a small – between 2 and 25 – number of positive and negative training images. Despite this being a fundamental requirement, none of the existing works dealing with EAs for RF in image retrieval addresses it. Common for the four variants is the hierarchical, region-based image similarity model, with region and feature weights as parameters. The difference between the variants is in the objective function of the EA used to adjust the model parameters. The objective functions include: (O-1) precision; (O-2) average rank; (O-3) ratio of within-class (i.e., positive images) and between-class (i.e., positive and negative images) scatter; and (O-4) combination of O-2 and O-3. We note that – unlike O-1 and O-2 – O-3 and O-4 are not used in any of the existing works dealing with EAs for RF. The four variants are evaluated on five test databases, containing 61,895 general-purpose images, in 619 semantic categories. Results of the evaluation reveal that variants with objective functions O-3 and O-4 consistently outperform those with O-1 and O-2. Furthermore, comparison with the representative of the existing RF methods shows that EAs are both effective and efficient approaches for SSS learning in region-based image retrieval.

Asymmetric bagging and random subspace for support vector machines-based relevance feedback in image retrieval

Relevance feedback schemes based on support vector machines (SVM) have been widely used in content-based image retrieval (CBIR). However, the performance of SVM-based relevance feedback is often poor when the number of labeled positive feedback samples is small. This is mainly due to three reasons: 1) an SVM classifier is unstable on a small-sized training set, 2) SVM's optimal hyperplane may be biased when the positive feedback samples are much less than the negative feedback samples, and 3) overfitting happens because the number of feature dimensions is much higher than the size of the training set. In this paper, we develop a mechanism to overcome these problems. To address the first two problems, we propose an asymmetric bagging-based SVM (AB-SVM). For the third problem, we combine the random subspace method and SVM for relevance feedback, which is named random subspace SVM (RS-SVM). Finally, by integrating AB-SVM and RS-SVM, an asymmetric bagging and random subspace SVM (ABRS-SVM) is built to solve these three problems and further improve the relevance feedback performance.

BALAS: Empirical Bayesian learning in the relevance feedback for image retrieval

This paper is on user relevance for image retrieval. We take this problem as a standard two-class pattern classification problem aiming at refining the retrieval precision by learning through the user relevance feedback data. However, we have investigated the problem by noting two important unique characteristics of the problem: small sample collection and asymmetric sample distributions between positive and negative samples. We have developed a novel approach to empirical Bayesian learning to solve for this problem by explicitly exploiting the two unique characteristics, which is the methodology of Bayesian Learning in Asymmetric and Small sample collections, thus called BALAS. In BALAS different learning strategies are used for positive and negative sample collections, respectively, based on the two unique characteristics. By defining the relevancy confidence as the relevant posterior probability, we have developed an integrated ranking scheme in BALAS, which complementarily combines the subjective relevancy confidence and the objective similarity measure to capture the overall retrieval semantics. The experimental evaluations have confirmed the rationale of the proposed ranking scheme, and have also demonstrated that BALAS is superior to an existing relevance feedback method in the current literature in capturing the overall retrieval semantics.

Kernel Vector Approximation Files for Relevance Feedback Retrieval in Large Image Databases

Many data partitioning index methods perform poorly in high dimensional space and do not support relevance feedback retrieval. The vector approximation file (VA-File) approach overcomes some of the difficulties of high dimensional vector spaces, but cannot be applied to relevance feedback retrieval using kernel distances in the data measurement space. This paper introduces a novel KVA-File (kernel VA-File) that extends VA-File to kernel-based retrieval methods. An efficient approach to approximating vectors in an induced feature space is presented with the corresponding upper and lower distance bounds. Thus an effective indexing method is provided for kernel-based relevance feedback image retrieval methods. Experimental results using large image data sets (approximately 100,000 images with 463 dimensions of measurement) validate the efficacy of our method.

Modified hierarchical genetic algorithm for relevance feedback in image retrieval

A modified hierarchical genetic algorithm (mHGA) is proposed for relevance feedback in image retrieval. In the underlying image similarity model, image similarity is expressed as a weighted aggregation of the corresponding region similarities, while each region similarity is expressed as a weighted aggregation of the corresponding feature similarities. Two distinguishing characteristics of the proposed relevance feedback method are: (1) unlike the existing relevance feedback methods, mHGA modifies both aggregation operators and weights, in order to adapt the similarity model to the user; and (2) unlike the ordinary genetic algorithm (GA), mHGA automatically switches between different combinations of the four adaptation targets (region aggregation operator, region weights, feature aggregation operators, and feature weights). The resulting image similarity function is: (1) more general than in case of the existing image similarity models; and (2) mathematically simpler (and thus computationally faster) than corresponding function adapted by ordinary GA. The proposed method is evaluated on five test databases, with around 2,500 images, covering 62 semantic categories. Compared with twelve of the representative image retrieval methods, including four based on relevance feedback, the proposed method brings in average between 6% and 36% increase in the retrieval precision. Results suggest that using mHGA to adapt both aggregation operators and weights is an effective approach to the relevance feedback in image retrieval.

Learning a semantic space from user's relevance feedback for image retrieval

As current methods for content-based retrieval are incapable of capturing the semantics of images, we experiment with using spectral methods to infer a semantic space from user's relevance feedback, so that our system will gradually improve its retrieval performance through accumulated user interactions. In addition to the long-term learning process, we also model the traditional approaches to query refinement using relevance feedback as a short-term learning process. The proposed short- and long-term learning frameworks have been integrated into an image retrieval system. Experimental results on a large collection of images have shown the effectiveness and robustness of our proposed algorithms.