Design and Implementation of Image Search Algorithm

Purpose The purpose of this study is to explore the retrieval effectiveness of three image search engines (ISE) – Google Images, Yahoo Image Search and Picsearch in terms of their image retrieval capability. It is an effort to carry out a Cranfield experiment to know how efficient the commercial giants in the image search are and how efficient an image specific search engine is. Design/methodology/approach The keyword search feature of three ISEs – Google images, Yahoo Image Search and Picsearch – was exploited to make search with keyword captions of photos as query terms. Selected top ten images were used to act as a testbed for the study, as images were searched in accordance with features of the test bed. Features to be looked for included size (1200 × 800), format of images (JPEG/JPG) and the rank of the original image retrieved by ISEs under study. To gauge the overall retrieval effectiveness in terms of set standards, only first 50 result hits were checked. Retrieval efficiency of select ISEs were examined with respect to their precision and relative recall. Findings Yahoo Image Search outscores Google Images and Picsearch both in terms of precision and relative recall. Regarding other criteria – image size, image format and image rank in search results, Google Images is ahead of others. Research limitations/implications The study only takes into consideration basic image search feature, i.e. text-based search. Practical implications The study implies that image search engines should focus on relevant descriptions. The study evaluated text-based image retrieval facilities and thereby offers a choice to users to select best among the available ISEs for their use. Originality/value The study provides an insight into the effectiveness of the three ISEs. The study is one of the few studies to gauge retrieval effectiveness of ISEs. Study also produced key findings that are important for all ISE users and researchers and the Web image search industry. Findings of the study will also prove useful for search engine companies to improve their services.

Content-based image search is an urgent problem as the next generation of search engines. This paper firstly analyzes and discusses its main features and key techniques, and then presents a designs method of image search engine based on image content. Next, we give a detailed elaboration about the main function modules and introduce the testing process. By using data filtering technology and ELFHash algorithms, the search time-consume is less than 1 second. The testing results finally prove the overall performance of our image search engine is excellent and achieves the desired design requirements.

With the explosive growth of the Internet, Web search technology marked by keywords has acquired a great success in the tremendous information retrieval. A search engine can gather new pages for its universal ways. Content-based image search is an urgent problem as the next generation of search engines. This paper firstly analyzes and discusses its main features and key techniques, and then presents a designs method of image search engine based on image content. Next, we give a detailed elaboration about the main function modules and introduce the testing process. By using data filtering technology and ELF Hash algorithms, the search time-consume is less than 1 s. Using data-filtering technology and the ELF Hash algorithm, our search engine reaches a good effect. The testing results finally prove the overall performance of our image search engine is excellent and achieves the desired design requirements. After being modified, our search engine could be applied to the intelligent wearable devices.

This study examines the differences between Web image and textual queries, and attempts to develop an analytic model to investigate their implications for Web image retrieval systems. A large number of Web queries from image and textual search engines were analyzed and compared based on their factual characteristics, query types, and search interests. A feasible analytic model employing the concepts of uniqueness and refinement was adapted to categorize query types and analyze the characteristics of failed queries. Useful results include the findings that image requests may have higher specificity and contain more refined queries (especially among failed queries), and that the queries were refined more by interpretive attributes than by reactive and perceptual attributes. Current text retrieval technology is not capable of fulfilling such complex image requests. It is suggested that there is a need to increase the number of appropriate annotations for Web images and to utilize more advanced retrieval techniques for more effective Web image searching. Few previous large‐scale studies have investigated visual information retrieval using image search engines. Thus, this study provides results that might enhance our understanding of Web image searching behavior and suggests implications for the improvement of current Web image search engines.

Image search engines show results differently from general Web search engines in three key ways: (1) most Web-based image search engines adopt the two-dimensional result placement instead of the linear result list; (2) image searches show snapshots instead of snippets (query-dependent abstracts of landing pages) on search engine result pages (SERPs); and (3) pagination is usually not (explicitly) supported on image search SERPs, and users can view results without having to click on the next page'' button. Compared with the extensive study of user behavior in general Web search scenarios, there exists no thorough investigation how the different interaction mechanism of image search engines affects users' examination behavior. To shed light on this research question, we conducted an eye-tracking study to investigate users' examination behavior in image searches. We focus on the impacts of factors in examination including position, visual saliency, edge density, the existence of textual information, and human faces in result images. Three interesting findings indicate users' behavior biases: (1) instead of the traditional Golden Triangle'' phenomena in the user examination patterns of general Web search, we observe a middle-position bias, (2) besides the position factor, the content of image results (e.g., visual saliency) affects examination behavior, and (3) some popular behavior assumptions in general Web search (e.g., examination hypothesis) do not hold in image search scenarios. We predict users' examination behavior with different impact factors. Results show that combining position and visual content features can improve prediction in image searches.

Web-based image search engines differ from Web search engines greatly. The intents or goals behind human interactions with image search engines are different. In image search, users mainly search images instead of Web pages or online services. It is essential to know why people search for images because user satisfaction may vary as intent varies. Furthermore, image search engines show results differently. For example, grid-based placement is used in image search instead of the linear result list, so that users can browse result list both vertically and horizontally. Different user intents and system UIs lead to different user behavior. Thus, it is hard to apply standard user behavior models developed for general Web search to image search. To better understand user intent and behavior in image search scenarios, we plan to conduct the lab-based user study, field study and commercial search log analysis. We then propose user behavior models based on the observation from data analysis to improve the performance of Web image search engines.

Modern image search engines such as Google, Yahoo!, Microsoft Live image search are all text meta word based. To search for images, the users type in a text query and the search engines rank the result images almost sorely based on the text meta-words. The abundant visual information in the images themselves is largely neglected. Recently, we have observed several new features released in the aforementioned image search engines, especially Microsoft Live image search, which are clearly based on the analysis of the visual content. We summarize some of these features, give insights about how they are designed, and motivate new content analysis based features for text based image search engines.

Image search plays an important role in our daily life. Given a query, the image search engine is to retrieve images related to it. However, different queries have different search difficulty levels. For some queries, they are easy to be retrieved (the search engine can return very good search results). While for others, they are difficult (the search results are very unsatisfactory). Thus, it is desirable to identify those “difficult” queries in order to handle them properly. Query difficulty prediction (QDP) is an attempt to predict the quality of the search result for a query over a given collection. QDP problem has been investigated for many years in text document retrieval, and its importance has been recognized in the information retrieval (IR) community. However, little effort has been conducted on the image query difficulty prediction problem for image search. Compared with QDP in document retrieval, QDP in image search is more challenging due to the noise of textual features and the well-known semantic gap of visual features. This paper aims to investigate the QDP problem in Web image search. A novel method is proposed to automatically predict the quality of image search results for an arbitrary query. This model is built based on a set of valuable features that are designed by exploring the visual characteristic of images in the search results. The experiments on two real image search datasets demonstrate the effectiveness of the proposed query difficulty prediction method. Two applications, including optimal image search engine selection and search results merging, are presented to show the promising applicability of QDP.

The availability of large-scale images from the Internet has made the research on image search attract a lot of attention. Text-based image search engines, for example, Google/Microsoft Bing/Yahoo! image search engines using the surrounding text, have been developed and widely used. However, they suffer from an inability to search image content. In this article, we present an interactive image search system, image search by color map, which can be applied to, but not limited to, enhance text-based image search. This system enables users to indicate how the colors are spatially distributed in the desired images, by scribbling a few color strokes, or dragging an image and highlighting a few regions of interest in an intuitive way. In contrast to the conventional sketch-based image retrieval techniques, our system searches images based on colors rather than shapes, and we, technically, propose a simple but effective scheme to mine the latent search intention from the user’s input, and exploit the dominant color filter strategy to make our system more efficient. We integrate our system to existing Web image search engines to demonstrate its superior performance over text-based image search. The user study shows that our system can indeed help users conveniently find desired images.

In most major search engines, the interface for image search is the same as traditional Web search: a keyword query followed by a paginated, ranked list of results. Although many image search innovations have appeared in both the literature and on the Web, few have seen widespread use in practice. In this work, we explore the differences between image and general Web search to better support users’ needs. First, we describe some unique characteristics of image search derived through informal interviews with researchers, designers, and managers responsible for building and deploying a major Web search engine. Then, we present results from a large scale analysis of image and Web search logs showing the differences in user behaviour. Grounded in these observations, we present design recommendations for an image search engine supportive of the unique experience of image search. We iterate on a number of designs, and describe a functional prototype that we built.

A major bottleneck in content-based image retrieval (CBIR) systems or search engines is the large gap between low-level image features used to index images and high-level semantic contents of images. One solution to this bottleneck is to apply relevance feedback to refine the query or similarity measures in image search process. In this paper, we first address the key issues involved in relevance feedback of CBIR systems and present a brief overview of a set of commonly used relevance feedback algorithms. Almost all of the previously proposed methods fall well into such framework. We present a framework of relevance feedback and semantic learning in CBIR. In this framework, low-level features and keyword annotations are integrated in image retrieval and in feedback processes to improve the retrieval performance. We have also extended framework to a content-based web image search engine in which hosting web pages are used to collect relevant annotations for images and users' feedback logs are used to refine annotations. A prototype system has developed to evaluate our proposed schemes, and our experimental results indicated that our approach outperforms traditional CBIR system and relevance feedback approaches.

Existing Web image search engines index images by textual descriptions including filename, image caption, surrounding text, etc. However, the textual description available on the Web could be ambiguous or inaccurate in describing the actual image content and some images irrelevant to user's query are also returned by text-based search engines. In this paper, we propose to integrate the existing text-based image search engine with visual features, in order to improve the performance of pure text-based Web image search. The proposed algorithm is named SIEVE. Practical fusion methods are proposed to integrate SIEVE with contemporary text-based search engines. In our approach, text-based image search results for a given query are obtained first. Then, SIEVE is used to filter out those images which are semantically irrelevant to the query. Experimental results show that the image retrieval performance using SIEVE improves over Google image search significantly.

In our society, generations of systemic biases have led to some professions being more common among certain genders and races. This bias is also reflected in image search on stock image repositories and search engines, e.g., a query like “male Asian administrative assistant” may produce limited results. The pursuit of a utopian world demands providing content users with an opportunity to present any profession with diverse racial and gender characteristics. The limited choice of existing content for certain combinations of profession, race, and gender presents a challenge to content providers. Current research dealing with bias in search mostly focuses on re-ranking algorithms. However, these methods cannot create new content or change the overall distribution of protected attributes in photos. To remedy these problems, we propose a new task of high-fidelity image generation conditioning on multiple attributes from imbalanced datasets. Our proposed task poses new sets of challenges for the state-of-the-art Generative Adversarial Networks (GANs). In this paper, we also propose a new training framework to better address the challenges. We evaluate our framework rigorously on a real-world dataset and perform user studies that show our model is preferable to the alternatives.

Different types of search engines are used to search image and text contents. Two types of image search methods are available in the Internet. They are query keyword based model and content based image retrieval models. Text query strings are used in the textual image retrieval model. Content based image retrieval (CBIR) model uses the visual information of the images. Image search methods use the text annotation and image visual features. Google image search and Bing image search engines are used to fetch images from the web. Image query string is used to search image on Internet. One click query image selection method is used to submit user intention for image retrieval. Content based image re-ranking is performed with visual and textual similarity metrics. Adaptive Weight Schema is used for the similarity analysis. Feature weight learning algorithm is applied to estimate feature weights for the images and its category. Query is expanded with keyword and visual information. Rank boost framework algorithm is enhanced to rank images with photographic quality. Content similarity and visual quality factors are used for the re-ranking process. In this paper, we propose an image indexing and retrieval using speech annotations based on a predefined structured syntax. To improve the retrieval effectively, N-best lists for index generation is used .so, a query expansion technique is explored to enhance the query terms. All this process is automatic, without extra effort from the user. This is critically important in web-based image search engine for any commercial, where the user interface has to be extremely simple.

The year 2020 brought a big concern for the global community because of COVID-19, which affected every sector of society, and tourism is no exception. Researchers across the globe are publishing their studies related to different dimensions of tourism in the context of COVID-19, and images have formed an essential component of their research. In tourism, images related to COVID-19 can open new dimensions for scholars. The main aim of the research is to measure the retrieval effectiveness of three image search engines (ISEs), that is, Bing Images, Google Images and Yahoo Image Search, concerning images related to COVID-19 and tourism. The study attempts to identify the capability of the ISEs to retrieve the desired and actual images related to COVID-19 and tourism. The PubMed Central (PMC) Database was consulted to retrieve the desired images and develop a testbed. The advanced search feature of PMC Database was explored by typing the search terms ‘COVID-19’ and ‘Tourism’ using ‘AND’ operator to make the search more comprehensive. Both the terms were searched against the ‘Figure/Table’ caption to retrieve papers carrying images related to COVID-19 and tourism. Queries were executed across the select ISEs, that is, Bing Images, Google Images and Yahoo Image Search. Retrieved images were individually analysed against the original image from the articles to determine the Precision, Relative Recall, F -Measure and Fallout Ratio. The format of the images in JPG/JPEG, besides checking the original image rank in the retrieved lot, was also ascertained. Bing Images scores more in terms of Mean Precision, followed by Google Images and Yahoo Image Search. For the Relative Recall measure, Google Images scores high, followed by Bing Images and Yahoo Image Search, respectively. Regarding F -Measure and Fallout Ratio, Bing Images outperforms Google Images and Yahoo Image Search. In retrieving the sought format of JPG/JPEG, Google Images performs best, followed by Yahoo Image Search and Bing Images. Google Images produces the original image at the first rank on more than one occasion. In contrast, Bing Images retrieves the original image at the first rank in two instances. Yahoo Images performs poorly over this metric as it does not retrieve any original image at the first rank on any other instance. The study cannot be generalised as the scope is only limited to the images indexed by PMC . Furthermore, the retrieval effectiveness of only three ISEs is measured. The study is the first to measure the retrieval effectiveness of ISEs in retrieving images related to the COVID-19 pandemic and tourism. The study can be extended across other image-indexing databases pertinent to tourism studies, and the retrieval effectiveness of other ISEs can also be considered.