Image Search Engines Research Articles

MTKeras: An Automated Metamorphic Testing Platform

This paper presents an automated, domain-independent, metamorphic testing platform called MTKeras. In this paper, we report on an investigation demonstrating the effectiveness and usability of MTKeras through five case studies in the four domains of image classification, sentiment analysis, search engines and database management systems. We also report on the effectiveness of combining metamorphic relation (input) patterns in individual metamorphic relations, enhancing the failure-finding abilities of the individual relations. The results of our experiments support combining patterns, and the use of MTKeras. The research reported in this paper shows the applicability of metamorphic relation patterns, and introduces a practical tool for the research community.

Text-based multi-dimensional medical images retrieval according to the features-usage correlation.

Emerging medical imaging applications in healthcare, the number and volume of medical images is growing dramatically. Information needs of users in such circumstances, either for clinical or research activities, make the role of powerful medical image search engines more significant. In this paper, a text-based multi-dimensional medical image indexing technique is proposed in which correlation of the features-usages (according to the user’s queries) is considered to provide an off-the content indexing while taking users’ interestingness into account. Assuming that each medical image has some extracted features (e.g., based on the DICOM standard), correlations of the features are discovered by performing data mining techniques (i.e., quantitative association pattern discovery), on the history of users’ queries as a data set. Then, based on the pairwise correlation of the features of medical images (a.k.a. Affinity), set of the all features is fragmented into subsets (using method like the vertical fragmentation of the tables in distribution of relational DBs). After that, each of these subsets of the features turn into a hierarchy of the features (by applying a hierarchical clustering algorithm on that subset), subsequently all of these distinct hierarchies together make a multi-dimensional structure of the features of medical images, which is in fact the proposed text-based (feature-based) multi-dimensional index structure. Constructing and using such text-based multi-dimensional index structure via its specific required operations, medical image retrieval process would be improved in the underlying medical image search engine. Generally, an indexing technique is to provide a logical representation of documents in order to optimize the retrieval process. The proposed indexing technique is designed such that can improve retrieval of medical images in a medical image search engine in terms of its effectiveness and efficiency. Considering correlation of the features of the image would semantically improve precision (effectiveness) of the retrieval process, while traversing them through the hierarchy in one dimension would try to optimize (i.e., minimize) the resources to have a better efficiency. The proposed text-based multi-dimensional indexing technique is implemented using the open source search engine Lucene, and compared with the built-in indexing technique available in the Lucene search engine, and also with the Terrier platform (available for the benchmarking of information retrieval systems) and other the most related indexing techniques. Evaluation results of memory usage and time complexity analysis, beside the experimental evaluations efficiency and effectiveness measures show that the proposed multi-dimensional indexing technique significantly improves both efficiency and effectiveness for a medical image search engine.Graphical abstract

Performance evaluation of web search engines in image retrieval: An experimental study

This experimental study used a checklist to evaluate the performance of seven search engines consisting of four Image General Search Engines (IGSEs) (namely, Google, Yahoo DuckDuckGo and Bing), and three Image Specialized Search Engines (ISSEs) (namely, Flicker, PicSearch, and GettyImages) in image retrieval. The findings indicated that the recall average of Image General Search Engines and Image Specialized Search Engines was found to be 76.32% and 24/51% with the precision average of 82/08% and 32/21%, respectively. As the results showed, Yahoo, Google and DuckDuckGo ranked at the top in image retrieval with no significant difference. However, a remarkable superiority with almost 50% difference was observed between the general and specialized image search engines. It was also found that an intense competition existed between Google, Yahoo and DuckDuckGo in image retrieval. The overall results can provide valuable insights for new search engine designers and users in choosing the appropriate search engines for image retrieval. Moreover, the results obtained through the applied equations could be used in assessing and evaluating other search tools, including search engines.

A New Content-Based Image Retrieval System Using Deep Visual Features

Content-based image retrieval (CBIR) is a technique for images retrieval based on their visual features, i.e. induced by their pixels. The images are, classically, described by the image feature vectors. Those vectors reflect the texture, color or a combination of them. The accuracy of the CBIR system is highly influenced by the (i) definition of the image feature vector describing the image, (ii) indexing and (iii) retrieval process. In this paper, we propose a new CBIR system entitled ISE (Image Search Engine). Our ISE system defines the optimum combination of color and texture features as an image feature vector, including the Particle Swarm Optimization (PSO) algorithm and employing an Interactive Genetic Approach (GA) for the indexing process. The performance analysis shows that our suggested PCM (Proposed Combination Method) upgrades the average precision metric from 66.6% to 89.30% for the “Food” category color histogram, from 77.7% to 100% concerning CCVs (Color Coherence Vectors) for the “Flower” category and from 58% to 87.65% regarding the DCD (Dominant Color Descriptor) for the “Building” category using the Corel dataset. Besides, our ISE system showcases an average precision of 98.23%, which is significantly higher than other CBIR systems presented in related works.

Deep learning based search engine for biomedical images using convolutional neural networks.

The development of efficient search engine queries for biomedical images, especially in case of query-mismatch is still defined as an ill-posed problem. Vector-space model is found to be useful for handling the query-mismatch issue. However, vector-space model does not consider the relational details among the keywords and biomedical image search space is not evaluated. Therefore, in this paper, we have proposed a deep learning based fusion vector-space based model. The proposed model enhances the biomedical image query similarity matching approach by fusing the vector space model and convolutional neural networks. Deep learning model is defined by converting the vector-space model to a classification model. Finally, deep learning model is trained to implement the search engine for biomedical images. Extensive experiments reveal that the proposed model achieves significant improvement over the existing models.

Diamond: multi-dimensional indexing technique for medical images retrieval using vertical fragmentation approach

Over the last decade, a huge number of medical visual data are widely used for diagnose, treatment, and follow-up. Retrieving needed medical image(s) from a huge number of images is one of the most widely used features in medical information systems, especially in medical image search engines. Indexing as part of search engines (or information retrieval systems), increases the speed (efficiency) of search and the information retrieval process. In this paper, a multidimensional indexing technique for medical images is presented that can improve effectiveness and efficiency of medical image search engines. The structure of the proposed multi-dimensional indexing technique and its main operations (i.e., creation, insertion, deletion and search) is designed and evaluated. In order to create this multidimensional index, the "vertical fragmentation" approach (which is usually applied for distributed database design) is used to determine the each of dimensions; roughly speaking, dimensions are different aspects of the medical images for a/some information need (e.g., image type and format, type of disorder, etc.). Accordingly, data structure of the proposed multidimensional indexing technique (which is named "Diamond") is formed by using the vertical fragmentation of medical image attributes (to differentiate the dimensions), and then using agglomerative hierarchical clustering to build up the hierarchy in each dimension. The proposed multi-dimensional indexing technique is implemented using the open-source search engine Lucene and compared with the built-in indexing technique available in the Lucene search engine, and also with the Terrier Platform (available for the benchmarking of information retrieval systems). In this evaluation, efficiency and effectiveness measures of the proposed multidimensional indexing technique (Diamond) are evaluated experimentally, beside the analysis of the designed data structure and its operations. For the experimental evaluation data set, images from Tabriz Behbood Hospital and a subset of TCIA images were used. Experimental evaluation results show that Diamond, the proposed multidimensional indexing technique improves both efficiency and effectiveness for a medical image search engine.

A Classification of Arab Ethnicity Based on Face Image Using Deep Learning Approach

Human face and facial features gain a lot of attention from researchers and are considered as one of the most popular topics recently. Features and information extracted from a person are known as soft biometric, they have been used to improve the recognition performance and enhance the search engine for face images, which can be further applied in various fields such as law enforcement, surveillance videos, advertisement, and social media profiling. By observing relevant studies in the field, we noted a lack of mention of the Arab world and an absence of Arab dataset as well. Therefore, our aim in this paper is to create an Arab dataset with proper labeling of Arab sub-ethnic groups, then classify these labels using deep learning approaches. Arab image dataset that was created consists of three labels: Gulf Cooperation Council countries (GCC), the Levant, and Egyptian. Two types of learning were used to solve the problem. The first type is supervised deep learning (classification); a Convolutional Neural Network (CNN) pre-trained model has been used as CNN models achieved state of art results in computer vision classification problems. The second type is unsupervised deep learning (deep clustering). The aim of using unsupervised learning is to explore the ability of such models in classifying ethnicities. To our knowledge, this is the first time deep clustering is used for ethnicity classification problems. For this, three methods were chosen. The best result of training a pre-trained CNN on the full Arab dataset then evaluating on a different dataset was 56.97%, and 52.12% when Arab dataset labels were balanced. The methods of deep clustering were applied on different datasets, showed an ACC from 32% to 59%, and NMI and ARI result from zero to 0.2714 and 0.2543 respectively.

빅데이터 기반 다중언어 문화이미지프레임망의 해외취업과정 운영에의 활용성 일고 - 수도권 S대학의 해운항만물류전문인력 양성사업을 중심으로 -

This research is an experiment to apply the Cultural-Image-Frame-Network (CIFN) based on a Big-data framework to an overseas employment program. The CIFN has been made with a purpose to enhance the language education efficiency by comparing visual cultural elements which are extracted from the big data based language corpuses and image search engines. With the recent crisis of Coronavirus, many overseas employment programs which the Korean government which has been pushing ahead with a purpose to promote youth employment is in crisis, especially, with the shortage of on-line language education contents. In this paper, we suggest the CIFN as a tool to help overseas employment trainees to broaden and deepen their understanding about the culture of each target country, as well as improve their command of language.

Deep Root Memory Optimized Indexing Methodology in Image Search Engines

Deep Root Memory Optimized Indexing Methodology in Image Search Engines

STUDYING 'LIVE' CROSS-PLATFORM CIRCULATION OF IMAGES WITH A COMPUTER VISION API: AN EXPERIMENT BASED ON A SPORTS MEDIA EVENTd'Andréa, Carlos Frederico de Brito (Federal University of Minas Gerais, Brazil) <carlosfbd@gmail.com>

Considering the importance of cross-platform circulation of web contents for digital methods-oriented research, in this study we aim to expand the types of digital objects used as ‘traffic tags’ by focusing on static images as traces of online associations. We pursue this goal through a methodological experiment with Google Cloud Vision API, a computing framework for visual content analysis. Its Web Detection module pairs typical computer vision operations with Google’s search mechanism, partly performing as a more specialized batch reverse image search engine. This feature thus allows to potentially retrieve images’ spread across the web. We discuss the implications of this non-verbal methodological approach by tracking the 'live' cross-platform circulation of images shared on Twitter in the context of 2018 FIFA World Cup Final Draw ceremony, held on December 2017. Following a novel methodological protocol, we ran several iterations of Vision API processing, thus generating a time series of URLs pointing to pages in which images matching the ones being processed were found. The study analyses in depth the circulation of four popular images about the broadcasted media event, observing their ‘live’ spreading dynamics as well as the computer vision API performance. Among the findings, we point out the adoption of images as 'traffic tags' for cross-platform analysis as a promising approach to study web circulation beyond language barriers and mainstream platforms. Also, we find relevant data to discuss the specificities of the API’s algorithms and its opacity as inherent issues of the digital methods approach.

Learning Semantic Text Features for Web Text-Aided Image Classification

The good generalization performance of conventional pattern classifiers often relies on the size of training data labeled by costly human labor. These days, publicly available web resources grow explosively, and this allows us to easily obtain abundant and cheap web data. Yet, web data are usually not as cooperative as human labeled data. In this paper, we explore the use of web text data to aid image classification. Without requiring the previous collection of auxiliary data from the web, we directly retrieve the web text information with the aid of the powerful reverse image search engine. We develop a novel textual modeling method named semantic matching neural network (SMNN) that is capable of learning semantic features from the associated text of web images. The SMNN text features have improved reliability and applicability, compared to the text features obtained from other methods. The SMNN text features and convolutional neural network (CNN) visual features are merged into a shared representation, which learns to capture the correlations between the two modalities. Experimental results on benchmark UIUC-Sports, Scene-15, Caltech-256, and Pascal VOC-2012 data sets show that the visual and text modalities of data from different sources are remarkably complementary and the fusion of them achieves substantial performance improvement.

이미지 검색 엔진 기반 색채 리서치 도구 : 고바야시 컬러 이미지 스케일과 비교

기존의 색채 배색에 관한 연구들은 대부분 색의 조화라는 측면에 집중되어 있고 문화, 시간, 세대, 제품이라는 요인에 따른 선호 배색의 차이를 간과하고 있다. 본 연구에서는 이러한 문제들을 해결하기 위하여, 이미지 검색 엔진이 제공하는 시간, 지역, 언어 등 다양한 검색 옵션 및 해시태그, 그리고 결과물을 내부의 메커니즘에 따라 순위를 결정하는 기능을 활용, 배색 추출을 하는 모델을 정의하고 이를 구현하였다. 모델의 성능을 평가하기 위하여, 설문을 통하여 고바야시 컬러 이미지 스케일의 배색과 비교하였으며, 평가결과 고바야시의 배색에 상당한 성능을 보였으며 이 모델을 활용한 문화권별 색채 비교의 가능성도 제시하였다. 본 연구에서 제안하는 배색 추출 모델은 기존의 배색 연구에서의 일회적이고, 시간, 문화, 분야, 개인 등 여러 요인을 고려하지 못한 한계점을 극복하려고 하였다.

What is it to be a pharmacist? Reflecting upon symbolism

What is it to be a pharmacist? Reflecting upon symbolism

VISE

We present VISE, or Vehicle Image Search Engine, to support the fast search of similar vehicles from low-resolution traffic camera images. VISE can be used to trace and locate vehicles for applications such as police investigations when high-resolution footage is not available. Our system consists of three components: an interactive user-interface for querying and browsing identified vehicles; a scalable search engine for fast similarity search on millions of visual objects; and an image processing pipeline that extracts feature vectors of objects from video frames. We use transfer learning technique to integrate state-of-the-art Convolutional Neural Networks with two different refinement methods to achieve high retrieval accuracy. We also use an efficient high-dimensional nearest neighbor search index to enable fast retrieval speed. In the demo, our system will offer users an interactive experience exploring a large database of traffic camera images that is growing in real time at 200K frames per day.

Deep multiple classifier fusion for traffic scene recognition

The recognition of the traffic scene in still images is an important yet difficult task in an intelligent transportation systems. The main difficulty lies in how to improve the image processing algorithms for different traffic participants and the various layouts of roads while discriminating the different traffic scenes. In this paper, we attempt to solve the traffic scene recognition problem with three distinct contributions. First, we propose a deep multi-classifier fusion method in the setting of granular computing. Specifically, the deep multi-classifier fusion method which involves local deep-learned feature extraction at one end that is connected to the other end for classification through a multi-classifier fusion approach. At the local deep-learned feature extraction end, the operation of convolution to extract feature maps from the local patches of an image is essentially a form of information granulation, whereas the fusion of classifiers at the classification end is essentially a form of organization. The second contribution is the creation of new traffic scene data set, named the “WZ-traffic”. The WZ-traffic data set consists of 6035 labeled images, which belong to 20 categories collected from both an image search engine as well as from personal photographs. Third, we make extensive comparisons with state-of-the-art methods on the WZ-traffic and FM2 data sets. The experiment results demonstrate that our method dramatically improves traffic scene recognition and brings potential benefits to many other real-world applications.

Image Search Engine and Individual Profile Building

Look technique utilized for the content gives semantically significant outcome, however isn't a similar with regards to the scan strategy utilized for pictures. Interactive media information is being distributed on the Web at an extraordinary rate. Likewise, in this time of innovation, it is conceivable to get data about any person from web. It has turned out to be fundamental to perform picture hunt of a person to recover the comparative pictures from Web. It is even conceivable to get any kind of data about any superstar from Wikipedia and different locales. This project aims at building the Image Search Engine for recovering the pictures just as structure the profile of a person, from World Wide Web. This is finished via preparing set of pictures of an individual and after that the web crawler creeps over the connections for getting the pertinent pictures. These recovered pictures coordinate with the name entered by the client. A similar outcome is utilized to get the data and manufacture the profile of a similar individual by slithering over the connections.

Does Diversity Affect User Satisfaction in Image Search

Diversity has been taken into consideration by existing Web image search engines in ranking search results. However, there is no thorough investigation of how diversity affects user satisfaction in image search. In this article, we address the following questions: (1) How do different factors, such as content and visual presentations, affect users’ perception of diversity? (2) How does search result diversity affect user satisfaction with different search intents? To answer those questions, we conduct a set of laboratory user studies to collect users’ perceived diversity annotations and search satisfaction. We find that the existence of nearly duplicated image results has the largest impact on users’ perceived diversity, followed by the similarity in content and visual presentations. Besides these findings, we also investigate the relationship between diversity and satisfaction in image search. Specifically, we find that users’ preference for diversity varies across different search intents. When users want to collect information or save images for further usage (the Locate search tasks), more diversified result lists lead to higher satisfaction levels. The insights may help commercial image search engines to design better result ranking strategies and evaluation metrics.

Image Search Engine and Individual Profile Building

Image Search Engine and Individual Profile Building

Retrieval effectiveness of image search engines

Purpose The purpose of this study is to explore the retrieval effectiveness of three image search engines (ISE) – Google Images, Yahoo Image Search and Picsearch in terms of their image retrieval capability. It is an effort to carry out a Cranfield experiment to know how efficient the commercial giants in the image search are and how efficient an image specific search engine is. Design/methodology/approach The keyword search feature of three ISEs – Google images, Yahoo Image Search and Picsearch – was exploited to make search with keyword captions of photos as query terms. Selected top ten images were used to act as a testbed for the study, as images were searched in accordance with features of the test bed. Features to be looked for included size (1200 × 800), format of images (JPEG/JPG) and the rank of the original image retrieved by ISEs under study. To gauge the overall retrieval effectiveness in terms of set standards, only first 50 result hits were checked. Retrieval efficiency of select ISEs were examined with respect to their precision and relative recall. Findings Yahoo Image Search outscores Google Images and Picsearch both in terms of precision and relative recall. Regarding other criteria – image size, image format and image rank in search results, Google Images is ahead of others. Research limitations/implications The study only takes into consideration basic image search feature, i.e. text-based search. Practical implications The study implies that image search engines should focus on relevant descriptions. The study evaluated text-based image retrieval facilities and thereby offers a choice to users to select best among the available ISEs for their use. Originality/value The study provides an insight into the effectiveness of the three ISEs. The study is one of the few studies to gauge retrieval effectiveness of ISEs. Study also produced key findings that are important for all ISE users and researchers and the Web image search industry. Findings of the study will also prove useful for search engine companies to improve their services.

On Annotation Methodologies for Image Search Evaluation

Image search engines differ significantly from general web search engines in the way of presenting search results. The difference leads to different interaction and examination behavior patterns, and therefore requires changes in evaluation methodologies. However, evaluation of image search still utilizes the methods for general web search. In particular, offline metrics are calculated based on coarse-fine topical relevance judgments with the assumption that users examine results in a sequential manner. In this article, we investigate annotation methods via crowdsourcing for image search evaluation based on a lab-based user study. Using user satisfaction as the golden standard, we make several interesting findings. First, instead of item-based annotation, annotating relevance in a row-based way is more efficient without hurting performance. Second, besides topical relevance, image quality plays a crucial role when evaluating the image search results, and the importance of image quality changes with search intent. Third, compared to traditional four-level scales, the fine-grain annotation method outperforms significantly. To our best knowledge, our work is the first to systematically study how diverse factors in data annotation impact image search evaluation. Our results suggest different strategies for exploiting the crowdsourcing to get data annotated under different conditions.