Web Image Search Research Articles

I-Cubid: a nonlinear cubic graph-based approach to visualize and in-depth browse Flickr image results.

The existing image search engines allow web users to explore images from the grids. The traditional interaction is linear and lookup-based. Notably, scanning web search results is horizontal-vertical and cannot support in-depth browsing. This research emphasizes the significance of a multidimensional exploration scheme over traditional grid layouts in visually exploring web image search results. This research aims to antecedent the implications of visualization and related in-depth browsing via a multidimensional cubic graph representation over a search engine result page (SERP). Furthermore, this research uncovers usability issues in the traditional grid and 3-dimensional web image search space. We provide multidimensional cubic visualization and nonlinear in-depth browsing of web image search results. The proposed approach employs textual annotations and descriptions to represent results in cubic graphs that further support in-depth browsing via a search user interface (SUI) design. It allows nonlinear navigation in web image search results and enables exploration, browsing, visualization, previewing/viewing, and accessing images in a nonlinear, interactive, and usable way. The usability tests and detailed statistical significance analysis confirm the efficacy of cubic presentation over grid layouts. The investigation reveals improvement in overall user satisfaction, screen design, information & terminology, and system capability in exploring web image search results.

Bird predation on Roseau cane scale as revealed by a web image search and querying a citizen monitoring database

AbstractAn invasion of Roseau cane scale, Nipponaclerda biwakoensis, native to East Asia, is associated with recent widespread dieback of common reed stands in the Mississippi River Delta, Louisiana. A previous study suggested that, in its native range, the combined mortality from Common Reed Bunting Emberiza schoeniclus and parasitoid wasps may control the abundance of Roseau cane scale. However, there is limited knowledge about which other bird species may prey on Roseau cane scale or the feeding patterns of these birds. Here we used web search and citizen science monitoring data to tabulate bird predation events on Roseau cane scales across sites in Japan and determine which bird species may be the most important predators and when the observed feeding typically occurs. We scanned web entries using both Google image searches and the social networking services including Twitter, Instagram, and Facebook. Additionally, we accessed long‐term field observation records obtained by volunteer monitoring conducted by the members of the Kanagawa Branch of the Wild Bird Society of Japan. By using these citizen science approaches, we detected 16 bird species, mainly Common Reed Bunting but also other migratory and resident species, preying on the Roseau cane scale in reed beds throughout central Japan and most detection occurred during the winter. This study demonstrates that biological information obtained from publicly available data from citizen observers can be useful for exploring ecological patterns of various species including potential predators against invasive species.

Multiple Distance-Based Coding: Toward Scalable Feature Matching for Large-Scale Web Image Search

For scalable feature matching in large-scale web image search, the bag-of-visual-words-based (BOW) approaches generally code local features as visual words to construct an inverted index file to match features efficiently. Both the popular feature coding techniques, i.e., K-means-based vector quantization and scalar quantization, directly quantize features to generate visual words. K-means-based vector quantization requires expensive visual codebook training, whereas scalar quantization leads to the miss of many matches due to the low stability of individual components of feature vectors. To address the above issues, we demonstrate that the corresponding sub-vectors of similar features generally have similar distances to multiple reference points in feature subspace and propose a multiple distance-based feature coding scheme for scalable feature matching. Specifically, based on the distances between the sub-vectors and multiple distinct reference points, we transform each feature to a set of feature codes, where one code is treated as a visual word required to construct the inverted index file whereas the others are embedded into the index file to further verify the feature matching based on the visual words. The proposed coding scheme does not need visual codebook training and shows desirable stability and discriminability. Moreover, in the matching verification, a feature-distance estimation method is proposed to estimate the Euclidean distances between features for an accurate matching verification. Extensive experimental results demonstrate the superiority of the proposed approach in comparison to the other approaches using recent feature quantization methods for large-scale web image search.

THINGS: A database of 1,854 object concepts and more than 26,000 naturalistic object images.

In recent years, the use of a large number of object concepts and naturalistic object images has been growing strongly in cognitive neuroscience research. Classical databases of object concepts are based mostly on a manually curated set of concepts. Further, databases of naturalistic object images typically consist of single images of objects cropped from their background, or a large number of naturalistic images of varying quality, requiring elaborate manual image curation. Here we provide a set of 1,854 diverse object concepts sampled systematically from concrete picturable and nameable nouns in the American English language. Using these object concepts, we conducted a large-scale web image search to compile a database of 26,107 high-quality naturalistic images of those objects, with 12 or more object images per concept and all images cropped to square size. Using crowdsourcing, we provide higher-level category membership for the 27 most common categories and validate them by relating them to representations in a semantic embedding derived from large text corpora. Finally, by feeding images through a deep convolutional neural network, we demonstrate that they exhibit high selectivity for different object concepts, while at the same time preserving variability of different object images within each concept. Together, the THINGS database provides a rich resource of object concepts and object images and offers a tool for both systematic and large-scale naturalistic research in the fields of psychology, neuroscience, and computer science.

Does Diversity Affect User Satisfaction in Image Search

Diversity has been taken into consideration by existing Web image search engines in ranking search results. However, there is no thorough investigation of how diversity affects user satisfaction in image search. In this article, we address the following questions: (1) How do different factors, such as content and visual presentations, affect users’ perception of diversity? (2) How does search result diversity affect user satisfaction with different search intents? To answer those questions, we conduct a set of laboratory user studies to collect users’ perceived diversity annotations and search satisfaction. We find that the existence of nearly duplicated image results has the largest impact on users’ perceived diversity, followed by the similarity in content and visual presentations. Besides these findings, we also investigate the relationship between diversity and satisfaction in image search. Specifically, we find that users’ preference for diversity varies across different search intents. When users want to collect information or save images for further usage (the Locate search tasks), more diversified result lists lead to higher satisfaction levels. The insights may help commercial image search engines to design better result ranking strategies and evaluation metrics.

Retrieval effectiveness of image search engines

Purpose The purpose of this study is to explore the retrieval effectiveness of three image search engines (ISE) – Google Images, Yahoo Image Search and Picsearch in terms of their image retrieval capability. It is an effort to carry out a Cranfield experiment to know how efficient the commercial giants in the image search are and how efficient an image specific search engine is. Design/methodology/approach The keyword search feature of three ISEs – Google images, Yahoo Image Search and Picsearch – was exploited to make search with keyword captions of photos as query terms. Selected top ten images were used to act as a testbed for the study, as images were searched in accordance with features of the test bed. Features to be looked for included size (1200 × 800), format of images (JPEG/JPG) and the rank of the original image retrieved by ISEs under study. To gauge the overall retrieval effectiveness in terms of set standards, only first 50 result hits were checked. Retrieval efficiency of select ISEs were examined with respect to their precision and relative recall. Findings Yahoo Image Search outscores Google Images and Picsearch both in terms of precision and relative recall. Regarding other criteria – image size, image format and image rank in search results, Google Images is ahead of others. Research limitations/implications The study only takes into consideration basic image search feature, i.e. text-based search. Practical implications The study implies that image search engines should focus on relevant descriptions. The study evaluated text-based image retrieval facilities and thereby offers a choice to users to select best among the available ISEs for their use. Originality/value The study provides an insight into the effectiveness of the three ISEs. The study is one of the few studies to gauge retrieval effectiveness of ISEs. Study also produced key findings that are important for all ISE users and researchers and the Web image search industry. Findings of the study will also prove useful for search engine companies to improve their services.

Clustering of near duplicate images using bundled features

Clustering the images is generally based on the image’s visual features. Selection of relevant features is the most essential task. A clustering approach based on the bundled features is presented in this paper. Bundling of affine scale invariant feature transform (ASIFT) feature helps to cluster the near duplicates. When the local features are combined with the ASIFT features, the clustering efficiency is increased. Clustering the results from the web image search engines is very essential to help users narrow their search. We applied our idea of clustering with bundled features over Google Image search results. The results obtained show that the presented approach outperforms compared to the clustering done only with local features.

Web image search revealed large-scale variations in breeding season and nuptial coloration in a mutually ornamented fish, Tribolodon hakonensis

Geographic variations in reproductive traits are important for evolutionary biology, but often difficult to investigate because of the need for a large-scale survey and the ephemeral nature of secondary sexual characteristics. Here, using web image searches (Google Images and Twitter), we revealed large-scale geographic variations (>1500 km) in breeding timing and nuptial coloration in the mutually ornamented fish, Japanese dace (Tribolodon hakonensis). As this fish is easily caught and many anglers upload their photographs on the web, we were able to find a total of 401 high-resolution photographs from all over Japan. Breeding periods were determined from dates of photographs with/without nuptial coloration, which matched with previous studies. We also found that breeding periods might have advanced three weeks over the last 80 years in the Chitose River, potentially because of climate change. Additionally, the pattern of latitudinal cline for breeding timing revealed delayed timing in higher latitudes, although regional variations were also high. Finally, we quantified the patterns of nuptial coloration for this mutually ornamented fish, and confirmed that over 80% of individuals showed typical colorations, but others showed rare phenotypes that were never previously described, including an intermediate coloration between T. hakonensis and a potentially hybridizing species, T. brandtii. Our web-based method for estimating breeding phenology could be adopted for organisms with temporal sexual characteristics, such as plants (e.g., flowering time) and some fishes. Web image analyses are still preliminarily with many limitations, but could be promising for investigating variations in visible traits.

Optimized web image search using meta annotation re-ranking technique

Optimized web image search using meta annotation re-ranking technique

Effects of relevance criteria and subjective factors on web image searching behaviour

Searching for images is an everyday activity. Nevertheless, even a highly skilled searcher often struggles to find what they are looking for. This article studies the factors that affect users’ online web image search behaviour, investigating (1) the use of criteria in making image relevance judgements and (2) the effect of familiarity, difficulty and satisfaction. The study includes 48 users who performed four online image search tasks using Google Images. Simulated work scenarios, questionnaires and screen capture recordings were used to collect data of their image search behaviour. The results show in judging image relevance, users may apply similar criterion, however, the importance of these criteria depends on the type of image search. Similarly, ratings of users’ perception on subjective aspects of performing image search shows they were task dependent. Users’ perception on subjective aspects of performing image search did not always correspond with their actual search behaviour. Correlation analysis shows that subjective factors cannot be definitively measured by using only one component of search behaviour. Future work includes further analysis on the effects of topic familiarity and satisfaction.

Web Image Search Re-Ranking With Click-Based Similarity and Typicality.

In image search re-ranking, besides the well-known semantic gap, intent gap, which is the gap between the representation of users' query/demand and the real intent of the users, is becoming a major problem restricting the development of image retrieval. To reduce human effects, in this paper, we use image click-through data, which can be viewed as the implicit feedback from users, to help overcome the intention gap, and further improve the image search performance. Generally, the hypothesis-visually similar images should be close in a ranking list-and the strategy-images with higher relevance should be ranked higher than others-are widely accepted. To obtain satisfying search results, thus, image similarity and the level of relevance typicality are determinate factors correspondingly. However, when measuring image similarity and typicality, conventional re-ranking approaches only consider visual information and initial ranks of images, while overlooking the influence of click-through data. This paper presents a novel re-ranking approach, named spectral clustering re-ranking with click-based similarity and typicality. First, to learn an appropriate similarity measurement, we propose click-based multi-feature similarity learning algorithm, which conducts metric learning based on click-based triplets selection, and integrates multiple features into a unified similarity space via multiple kernel learning. Then, based on the learnt click-based image similarity measure, we conduct spectral clustering to group visually and semantically similar images into same clusters, and get the final re-rank list by calculating click-based clusters typicality and within-clusters click-based image typicality in descending order. Our experiments conducted on two real-world query-image data sets with diverse representative queries show that our proposed re-ranking approach can significantly improve initial search results, and outperform several existing re-ranking approaches.

Investigating the precision of Web image search engines for popular and less popular entities

Image search is the second most frequently used search service on the Web. However, there are very few studies investigating any aspect of it. In this study, we investigate the precision of Web image search engines of Google and Bing for popular and less popular entities using text-based queries. Furthermore, we investigate four additional aspects of Web image search engines that have not been studied before. We used 60 different queries in total from three different domains for popular and less popular categories. We examined the relevancy of the top 100 images for each query. Our results indicate that image search is a solved problem for popular entities. They deliver 97% precision on the average for popular entities. However, precision values are much lower for less popular entities. For the top 100 results, average precision is 48% for Google and 33% for Bing. The most important problem seems to be the worst cases in which the precision can be less than 10%. The results show that significant improvement is needed to better identify relevant images for less popular entities. One of the main issues is the association problem. When a Web page has query words and multiple images, both Google and Bing are having difficulty determining the relevant images.

Relevance Preserving Projection and Ranking for Web Image Search Reranking With Hierarchical Topic Awareness

Relevance Preserving Projection and Ranking for Web Image Search Reranking With Hierarchical Topic Awareness

The Word Outside and the Pictures in Our Heads: Contingent Framing Effects of Labels on Health Policy Preferences by Political Ideology

ABSTRACTThis study uses data from systematic Web image search results and two randomized survey experiments to analyze how frames commonly used in public debates about health issues, operationalized here as alternative word choices, influence public support for health policy reforms. In Study 1, analyses of Bing (N = 1,719), Google (N = 1,872), and Yahoo Images (N = 1,657) search results suggest that the images returned from the search query “sugar-sweetened beverage” are more likely to evoke health-related concepts than images returned from a search query about “soda.” In contrast, “soda” search queries were more likely to incorporate brand-related concepts than “sugar-sweetened beverage” search queries. In Study 2, participants (N = 206) in a controlled Web experiment rated their support for policies to reduce consumption of these drinks. As expected, strong liberals had more support for policies designed to reduce the consumption of these drinks when the policies referenced “soda” compared to “sugar-sweetened beverage.” To the contrary, items describing these drinks as “soda” produced lower policy support than items describing them as “sugar-sweetened beverage” among strong conservatives. In Study 3, participants (N = 1,000) in a national telephone survey experiment rated their support for a similar set of policies. Results conceptually replicated the previous Web-based experiment, such that strong liberals reported greater support for a penny-per-ounce taxation when labeled “soda” versus “sugar-sweetened beverages.” In both Studies 2 and 3, more respondents referred to brand-related concepts in response to questions about “sugar-sweetened beverages” compared to “soda.” We conclude with a discussion of theoretical and methodological implications for studying framing effects of labels.

An Identification of the Image Retrieval Domain from the Perspective of Library and Information Science with Author Co-citation and Author Bibliographic Coupling Analyses

As the improvement of digital technologies increases the use of images from various fields, the domain of image retrieval has evolved and become a growing topic of research in the Library and Information Science field. The purpose of this study is to identify the knowledge structure of the image retrieval domain by using the author co-citation analysis and author bibliographic coupling as analytical tools in order to understand the domain’s past and present. The data set for this study is 245 articles with 8,031 cited articles in the field of image retrieval from 1998 to 2013, from the Web of Science citation database. According to the results of author co-citation analysis for the past of the image retrieval domain, our findings demonstrate that the intellectual structure of image retrieval in the LIS field consists of predominantly user-oriented approaches, but also includes some areas influenced by the CBIR area. More specifically, the user-oriented approach contains six specific areas which include image needs, information seeking, image needs and search behavior, image indexing and access, indexing of image collection, and web image search. On the other hand, for CBIR approaches, it contains feature-based image indexing, shape-based indexing, and IR & CBIR. The recent trends of image retrieval based on the results from author bibliographic coupling analysis show that the domain is expanding to emerging areas of medical images, multimedia, ontology- and tag-based indexing which thus reflects a new paradigm of information environment.

PWIS: Personalized Web Image Search using One-Click Method

Personalized Web Image search is the one searching for the particular images of User intention on the Web. For searching images, a user might provide query terms like keyword, image file, or click on few image file, and therefore the system can determine the images similar to the query. The similarity used for search criteria could be Meta tags, color distribution in images, region/shapes attributes, etc. Web-scale image search engines namely Google and Bing searches for images are relying on the surrounding text features. It is highly cumbersome and complicated for the web-scale based image search engines to interpret users search intention only by querying of keywords. This leads to the incorporation of noise and high ambiguity in the search results which are extremely unfit in the context of the users. It's also a necessary mandate for using visual information for solving the problem of ambiguity in the text-based image retrieval scenario. In the case of Google search, search text box will auto complete while user is typing similar added keywords. This method will differ from user intention while searching. So to avoid this kind of faults, it is important to use visual information in order to solve the uncertainty in text-based image retrieval. To retrieve exact matching, and acquire user‟s intention we can allow them text query with extended or related images as a suggestion. We have proposed an innovative Web image search approach. It only needs the user to click on one query image with minimal effort and images from a pool fetched by text-based search are re-ranked based on both visual and textual contents.

Malicious URL Detection Algorithm based on BM Pattern Matching

In the virus world of Internet, it is a challenging and urgent problem that how can we ensure the safety of search engines. A security subsystem of the search engine based on the research of content-based image search engine system V2.0 is developed. A malicious URL (Uniform Resource Locator) detection method based on BM (Boyer-Moore) pattern matching is proposed. The main research contents and results are as follows: Many malicious URLs could be downloaded by web image search, which may cause unnecessary loses to the users. So the malicious URL detection algorithm based on BM pattern matching is proposed. This method is to let the URL source code match the virus characteristics in the database to confirm whether the URL is safe or not. Web image search detects 203 malicious URLs based on this method. By kaspersky scanning, we confirmed 189 URLs to be malicious URLs, and the error rate is 6.9%, and the accurate rate is 93.1%. The experimental results show that the malicious URL detection algorithm provides secure URLs for web image search engine..

Automatic generation of visual story for fairy tales with digital narrative

Pictures can realize the impressions of texts for readers, especially for fairy tales. If we can present the fairy stories in the form of visual pictures, children will not only be more willing to concentrate their attention on but also easily perceive the underlying messages. This work aims to present the fairy tales by transforming the texts to the visual form of pictures. To achieve such goal, we develop a system, VizStory, which consists of three steps. First, we investigate the narrative structure of the story to segment the whole story. Second, we select representative keywords for each segment. Third, through Web image search, we find suitable pictures to compose the visualization. Experimental results with a set of human study show VizStory can present the narrative of stories with 75% accuracy.

Survey on Web-Scale Image Search and Re-Ranking With Semantic Signatures.

Web-scale image search results can be improved by using image re-ranking technique. Many commercial search engines such as Google, Yahoo and Bing have been adopted this strategy. Firstly user is given a query keyword; text-based image retrieval is done. Then user is asked to select a query image from pool with minimum effort just by one click and images from a pool extracted by text-based information are re-ranked based on visual resemblance with the query image. There are some challenges in this method. Visual feature vectors need to be short to achieve high matching efficiency. But some of famous visual features are large in size. Another challenge is that the resemblance of visual features may not well associate with images’ high-level semantic meanings. For overcoming this problem, in this paper, a new technique is proposed for web-scale image re-ranking. As an alternative to manually defining an entire concept glossary, different semantic spaces for different query keywords can be found offline independently and automatically. Semantic signatures of the images can be acquired by projecting their visual features into their related semantic spaces and these semantic signatures can be computed using Hashing techniques. At the online stage, these compacted semantic signatures of images are to be compared to re-rank images. It significantly improves the efficiency and accuracy of web-image search and re-ranking.

Relevance Preserving Projection and Ranking for Web Image Search Reranking.

An image search reranking (ISR) technique aims at refining text-based search results by mining images' visual content. Feature extraction and ranking function design are two key steps in ISR. Inspired by the idea of hypersphere in one-class classification, this paper proposes a feature extraction algorithm named hypersphere-based relevance preserving projection (HRPP) and a ranking function called hypersphere-based rank (H-Rank). Specifically, an HRPP is a spectral embedding algorithm to transform an original high-dimensional feature space into an intrinsically low-dimensional hypersphere space by preserving the manifold structure and a relevance relationship among the images. An H-Rank is a simple but effective ranking algorithm to sort the images by their distances to the hypersphere center. Moreover, to capture the user's intent with minimum human interaction, a reversed k-nearest neighbor (KNN) algorithm is proposed, which harvests enough pseudorelevant images by requiring that the user gives only one click on the initially searched images. The HRPP method with reversed KNN is named one-click-based HRPP (OC-HRPP). Finally, an OC-HRPP algorithm and the H-Rank algorithm form a new ISR method, H-reranking. Extensive experimental results on three large real-world data sets show that the proposed algorithms are effective. Moreover, the fact that only one relevant image is required to be labeled makes it has a strong practical significance.