Set Of Relevant Images Research Articles

Annapurna: An automated smartwatch-based eating detection and food journaling system

Maintaining a food journal can allow an individual to monitor eating habits, including unhealthy eating sessions, food items causing severe reactions, or portion size related information. However, manually maintaining a food journal can be burdensome. In this paper, we explore the vision of a pervasive, automated, completely unobtrusive, food journaling system using a commodity smartwatch. We present a prototype system — Annapurna— which is composed of three key components: (a) a smartwatch-based gesture recognizer that can robustly identify eating-specific gestures occurring anywhere, (b) a smartwatch-based image captor that obtains a small set of relevant images (containing views of the food being consumed) with a low energy overhead, and (c) a server-based image filtering engine that removes irrelevant uploaded images. Through lessons learnt from multiple user studies, we refine Annapurna progressively and show that our vision is indeed achievable: Annapurna can identify eating episodes and capture food images (involving a very wide diversity in food content, eating styles and environments) in over 95% of all free-living eating episodes.

Data‐driven image captioning via salient region discovery

In the past few years, automatically generating descriptions for images has attracted a lot of attention in computer vision and natural language processing research. Among the existing approaches, data‐driven methods have been proven to be highly effective. These methods compare the given image against a large set of training images to determine a set of relevant images, then generate a description using the associated captions. In this study, the authors propose to integrate an object‐based semantic image representation into a deep features‐based retrieval framework to select the relevant images. Moreover, they present a novel phrase selection paradigm and a sentence generation model which depends on a joint analysis of salient regions in the input and retrieved images within a clustering framework. The authors demonstrate the effectiveness of their proposed approach on Flickr8K and Flickr30K benchmark datasets and show that their model gives highly competitive results compared with the state‐of‐the‐art models.

On integrating re-ranking and rank list fusion techniques for image retrieval

This paper aims to unify image re-ranking and rank aggregation strategies to enhance the retrieval precision of content-based image retrieval (CBIR) systems. In general, CBIR systems are concerned with the retrieval of a set of relevant images from large repositories in response to a submitted query. The primary objective of CBIR systems is the exact ordering of database images in accordance with the presented query. To this end, we present a novel image re-ranking scheme for reordering the initial search results returned by multiple retrieval models and an efficient rank list fusion scheme to combine these refined retrieval results to achieve better performance. The re-ranking algorithm introduced in this work utilizes distance correlation coefficient to refine the search result generated by a given retrieval model. It involves two-step clustering of the initial retrieval list followed by an adaptive procedure for updating the similarity scores among images based on the created clusters. Similarly, the Particle Swarm Optimization-based similarity score fusion framework presented in this work optimally combines the retrieval results generated by multiple CBIR systems. The proposed approach is evaluated on various retrieval tasks using state-of-the-art low-level and high-level descriptors. Experimental results show that our model can significantly enhance the overall effectiveness of CBIR systems.

Social Event Storyboard Generation from Image Click-A Survey

Traditional websites were driven by human-edited events which lead to huge web search traffic. This paper is a survey conducted for identifying the various event detection methods which are useful for event mining. Moreover this paper also suggests an automatic system to detect events from search log data and generate storyboards where the events are arranged along a timeline. Image search log is treated as a good data resource for event mining, as search logs directly reflects people’s interests. In order to discover events from log data, an approach known as Smooth Nonnegative Matrix Factorization framework (SNMF) is used. Moreover, time factor is considered as an important element for event detection as different events develop at different time. In addition, to provide a visually appealing storyboard, each event is mapped with a set of relevant images arranged along a timeline. These relevant images are automatically generated from image search results by analyzing both local and global image content feature

A Hybrid Approach to Content Based Image Retrieval Using Computational Intelligence Techniques

Objectives: Improvement in the retrieval performance of the system can be brought by fusion of image features with different similarity measures. Performance of the system can be further improved by incorporating user's feedback into the system. Method: An extensive system is proposed based on one of the computational intelligence techniques for the effective retrieval of required images from systems. The proposed technique consists of two main modules, namely: feature vector processing and result enhancement. Feature fusion is performed by application of genetic operators to find the final distance between probe and stored images. Subsequently estimated retrieved images are presented to the user, first k-images will be selected by the user as K-NN query images and are ranked according to their relevance values provided by the user. Optimization of retrieved images is done by iteratively providing user's feedback to the system. Findings: Feature fusion and effective relevance feedback methods can contribute extensive benefits to content based image retrieval. Feature fusion combines different image features in such a way to get a single feature vector for all of them but combination of different image features however is not always beneficial. Semantic gap is reduced by providing user's feedback after the retrieval of images. This is an iterative process and it defines a set of relevant images in the end. Genetic algorithm based search is applied for a series of weighting functions which helps to maximize the fitness function. Applications/Improvements: Multifarious features are introduced where different image features are combined with different similarity measures and the final distance is drawn by employing genetic programming technique.

Learning “initial feature weights” for CBIR using query augmentation

Content-based image retrieval (CBIR) is one of the most active fields of research in image processing and information retrieval. In CBIR, an image is given as query, instead of text, and a set of relevant images is returned as an output. Researchers have generally used multiple image features in conjunction to achieve high CBIR performance. Relevance feedback has also seen widespread adoption as technique to utilise user feedback to further refine search results. In this paper, we propose a technique to ascertain the initial feature weights before the actual query is processed. The weights of the features are determined by augmenting the query image through different transformations of the query image. The proposed method is tested on the VisTex and Outex_TR_00000 texture collections. The performance is measured by average retrieval rate, precision and recall. Our results do not show any degradation on retrieval performance on collections that have relevance classes which are generally uniform. However, on collections that are more heterogeneous, our proposed method leads to better search results.

Learning Query-Specific Distance Functions for Large-Scale Web Image Search

Current Google image search adopt a hybrid search approach in which a text-based query (e.g., “Paris landmarks”) is used to retrieve a set of relevant images, which are then refined by the user (e.g., by re-ranking the retrieved images based on similarity to a selected example). We conjecture that given such hybrid image search engines, learning per-query distance functions over image features can improve the estimation of image similarity. We propose scalable solutions to learning query-specific distance functions by 1) adopting a simple large-margin learning framework, 2) using the query-logs of text-based image search engine to train distance functions used in content-based systems. We evaluate the feasibility and efficacy of our proposed system through comprehensive human evaluation, and compare the results with the state-of-the-art image distance function used by Google image search.

Web Image Retrieval Using Visual Dictionary

In this research, we have proposed semantic based image retrieval system to retrieve set of relevant images for the given query image from the Web. We have used global color space model and Dense SIFT feature extraction technique to generate visual dictionary using proposed quantization algorithm. The images ar e transformed intoset of features. These features are used as inputs in ourproposedQuantization algorithm for generating the code word to form visual dictionary. These codewords are used to represent images s emantically to form visual labels using Bag-of-Features (BoF). The Histogram intersection method is used to measure thedistance between input image and the set of images in the image database to retrieve similar images. The experimental results are evaluated over a collection of1000generic Web images to demonstrate the effectiveness of the proposed system.

Web Image Retrieval Using Visual Dictionary

In this research, we have proposed semantic based image retrieval system to retrieve set of relevant images for the given query image from the Web. We have used global color space model and Dense SIFT feature extraction technique to generate visual dictionary using proposed quantization algorithm. The images are transformed into set of features. These features are used as inputs in our proposed Quantization algorithm for generating the code word to form visual dictionary. These codewords are used to represent images semantically to form visual labels using Bag-of-Features (BoF). The Histogram intersection method is used to measure the distance between input image and the set of images in the image database to retrieve similar images. The experimental results are evaluated over a collection of 1000 generic Web images to demonstrate the effectiveness of the proposed system.

Weighted Local Similarity Pattern as image similarity model incorporated in GA-based relevance feedback mechanism

Weighted Local Similarity Pattern (WLSP)} is proposed as a new image similarity model, which considers two fundamental properties of the human visual system: (1) saliency of regions within an image, and (2) saliency of features within each region. Furthermore, since both region and feature saliencies are context dependent, genetic algorithm (GA)-based relevance feedback mechanism is proposed to automatically infer the (sub-)optimal assignment of the two saliencies, based on the query image and the set of relevant images, provided by the user. None of the existing image similarity models considers both region and feature saliencies in a context-dependent sense, allowing their automatic inference. In addition, this paper is the first to explicitly discuss the implications of the region and feature saliency properties to the design of an image similarity model, in the framework of image retrieval. The proposed method - including the WLSP image similarity model and the GA-based relevance feedback mechanism - is evaluated on five test databases, with around 2,500 images, covering 62 semantic categories. Compared with eleven of the representative image similarity models, including three based on relevance feedback, the proposed model brings in average between 6% and 30% increase in the retrieval precision. Results suggest that considering region and feature saliencies in a context-dependent sense enables the image similarity model to more accurately capture the human similarity perception.

Genetic algorithms for a family of image similarity models incorporated in the relevance feedback mechanism

Abstract We propose two extensions of the Local Similarity Pattern/Weighted Local Similarity Pattern (LSP/WLSP) image similarity models, proposed by Stejic (2002). The objective is to improve the retrieval precision, by increasing the expressive power of the LSP/WLSP models. We formalize LSP and WLSP as special cases of a general similarity model, which defines image similarity as a weighted mean of the corresponding region similarities, and, in turn, each region similarity as a weighted mean of the corresponding feature similarities. In the case of LSP/WLSP models, region and feature weights have discrete non-negative values. The proposed two extensions are: (1) LSP-C+/WLSP-C+ models, with continuous non-negative weight values; and (2) LSP-C±/WLSP-C± models, with continuous positive and negative weight values. Similar to the LSP/WLSP, the proposed four models are incorporated in the relevance feedback mechanism, using genetic algorithm (GA) to infer the (sub-)optimal assignment of region and feature weights, which maximizes the similarity between the query image and the set of relevant images, chosen by the user. Accordingly, for the weight inference, GAs used for the LSP/WLSP models are extended from the integer-coded to the real-coded ones, and, in addition, new chromosome coding, two crossover modifications, six new mutation types, and a weights normalization operator are proposed. The proposed four models are evaluated on five test databases, with around 2500 images, covering 62 semantic categories. Compared with the existing image similarity models, including LSP/WLSP, and other models based on relevance feedback, proposed LSP-C±/WLSP-C± models bring in average over 10% increase in the retrieval precision. The main contributions, not limited to the LSP/WLSP models and the image retrieval, are: (1) the introduction of negative weights into the weighted mean of similarities, which increases the expressive power of the similarity model, and results in a significantly higher retrieval precision; and (2) new real-coded genetic operators suitable for the weight inference in general.