User's Search Intention Research Articles

The Contemporary Art of Image Search: Iterative User Intent Expansion via Vision-Language Model

Image search is an essential and user-friendly method to explore vast galleries of digital images. However, existing image search methods heavily rely on proximity measurements like tag matching or image similarity, requiring precise user inputs for satisfactory results. To meet the growing demand for a contemporary image search engine that enables accurate comprehension of users' search intentions, we introduce an innovative user intent expansion framework. Our framework leverages visual-language models to parse and compose multi-modal user inputs to provide more accurate and satisfying results. It comprises two-stage processes: 1) a parsing stage that incorporates a language parsing module with large language models to enhance the comprehension of textual inputs, along with a visual parsing module that integrates an interactive segmentation module to swiftly identify detailed visual elements within images; and 2) a logic composition stage that combines multiple user search intents into a unified logic expression for more sophisticated operations in complex searching scenarios. Moreover, the intent expansion framework enables users to perform flexible contextualized interactions with the search results to further specify or adjust their detailed search intents iteratively. We implemented the framework into an image search system for NFT (non-fungible token) search and conducted a user study to evaluate its usability and novel properties. The results indicate that the proposed framework significantly improves users' image search experience. Particularly the parsing and contextualized interactions prove useful in allowing users to express their search intents more accurately and engage in a more enjoyable iterative search experience.

A survey on cross-media search based on user intention understanding in social networks

With the increasing popularity of online social networks, more and more people are posting information, updating their statuses, and searching for topics there. Massive cross-media big data has been gathered by online social networks, with high dynamics, context-sparsity, and cross-media semantic gaps. In addition, it can be challenging to understand users' search intentions in the setting of social networks. The above problems have brought severe challenges and obstacles to cross-media searches, which also attracted more and more attention on social networks. As a relatively new research topic and interest, the concept, methodology, and overall research idea of cross-media search based on user search intention understanding are not evident in the literature. The research also lacks a unified paradigm and relatively complete research ideas on social networks. To solve these problems, we reviewed more than 100 references based on our preliminary exploration and research experience in this field from the whole process involved. We also detailed methodology, datasets, evaluation indicators, experiment evaluation, and research trends and analyzed the challenges. These works will help beginners quickly establish research ideas and processes in this field, and enable them to focus on algorithm design without paying too much attention to datasets, evaluation metrics, and research frameworks. We believe this review will attract more researchers to focus on social network cross-media search based on user search intention understanding and benefit their work.

Query Sense Discovery Approach to Realize the User's Search Intent

The main goal of information retrieval is getting the most relevant documents to a user’s query. So, a search engine must not only understand the meaning of each keyword in the query but also their relative senses in the context of the query. Discovering the query meaning is a comprehensive and evolutionary process; the precise meaning of the query is established as developing the association between concepts. The meaning determination process is modeled by a dynamic system operating in the semantic space of WordNet. To capture the meaning of a user query, the original query is reformulating into candidate queries by combining the concepts and their synonyms. A semantic score characterizing the overall meaning of such queries is calculated, the one with the highest score was used to perform the search. The results confirm that the proposed "Query Sense Discovery" approach provides a significant improvement in several performance measures.

Fast Local Spatial Verification for Feature-Agnostic Large-Scale Image Retrieval.

Images from social media can reflect diverse viewpoints, heated arguments, and expressions of creativity, adding new complexity to retrieval tasks. Researchers working on Content-Based Image Retrieval (CBIR) have traditionally tuned their algorithms to match filtered results with user search intent. However, we are now bombarded with composite images of unknown origin, authenticity, and even meaning. With such uncertainty, users may not have an initial idea of what the search query results should look like. For instance, hidden people, spliced objects, and subtly altered scenes can be difficult for a user to detect initially in a meme image, but may contribute significantly to its composition. It is pertinent to design systems that retrieve images with these nuanced relationships in addition to providing more traditional results, such as duplicates and near-duplicates - and to do so with enough efficiency at large scale. We propose a new approach for spatial verification that aims at modeling object-level regions using image keypoints retrieved from an image index, which is then used to accurately weight small contributing objects within the results, without the need for costly object detection steps. We call this method the Objects in Scene to Objects in Scene (OS2OS) score, and it is optimized for fast matrix operations, which can run quickly on either CPUs or GPUs. It performs comparably to state-of-the-art methods on classic CBIR problems (Oxford 5K, Paris 6K, and Google-Landmarks), and outperforms them in emerging retrieval tasks such as image composite matching in the NIST MFC2018 dataset and meme-style imagery from Reddit.

A Visual Saliency-Based Approach for Content-Based Image Retrieval

During the past two decades an enormous amount of visual information has been generated; as a result, content-based image retrieval (CBIR) has received considerable attention. In CBIR the image is used as a query to find the most similar images. One of the biggest challenges in CBIR system is to fill up the “semantic gap,” which is the gap between low-level visual features and the high-level semantic concepts of an image. In this paper, the authors have proposed a saliency-based CBIR system that utilizes the semantic information of image and users search intention. In the proposed model, firstly a significant region is identified with the help of method structured matrix decomposition (SMD) using high-level priors that highlight the prominent area of the image. After that, a two-dimensional principal component analysis (2DPCA) is used as a feature, which is compact and effectively used for fast recognition. Experiment results are validated on different image dataset having an extensive collection of semantic classifications.

When Are Search Completion Suggestions Problematic?

Problematic web search query completion suggestions-perceived as biased, offensive, or in some other way harmful-can reinforce existing stereotypes and misbeliefs, and even nudge users towards undesirable patterns of behavior. Locating such suggestions is difficult, not only due to the long-tailed nature of web search, but also due to differences in how people assess potential harms. Grounding our study in web search query logs, we explore when system-provided suggestions might be perceived as problematic through a series of crowd-experiments where we systematically manipulate: the search query fragments provided by users, possible user search intents, and the list of query completion suggestions. To examine why query suggestions might be perceived as problematic, we contrast them to an inventory of known types of problematic suggestions. We report our observations around differences in the prevalence of a) suggestions that are problematic on their own versus b) suggestions that are problematic for the query fragment provided by a user, for both common informational needs and in the presence of web search voids-topics searched by few to no users. Our experiments surface a rich array of scenarios where suggestions are considered problematic, including due to the context in which they were surfaced. Compounded by the elusive nature of many such scenarios, the prevalence of suggestions perceived as problematic only for certain user inputs, raises concerns about blind spots due to data annotation practices that may lead to some types of problematic suggestions being overlooked.

R2DCLT: retrieving relevant documents using cosine similarity and LDA in text mining

The availability of digital documents over web has increased exponentially and hence there is a need for effective methods to retrieve and organise. Since data is dispersed globally and unorganised, a number of algorithms have been proposed based on relevance calculations. However, it is found that there is a gap between user's search intention and retrieved results. In this paper, we propose a framework for retrieving relevant documents using cosine similarity (CS) and LDA in text mining (R2DCLT). The uniqueness of this approach is that LDA is applied for the documents and extracted patterns like unigram, bigram and trigram. Documents are ranked based on the CS score. Experiments are conducted on Reuters Corpus volume and custom news dataset. It is observed that R2DCLT outperforms pattern taxonomy and relevance feature discovery models by providing high quality relevant documents with improved response time and dynamically updated document set.

Learning Click-Based Deep Structure-Preserving Embeddings with Visual Attention

One fundamental problem in image search is to learn the ranking functions (i.e., the similarity between query and image). Recent progress on this topic has evolved through two paradigms: the text-based model and image ranker learning. The former relies on image surrounding texts, making the similarity sensitive to the quality of textual descriptions. The latter may suffer from the robustness problem when human-labeled query-image pairs cannot represent user search intent precisely. We demonstrate in this article that the preceding two limitations can be well mitigated by learning a cross-view embedding that leverages click data. Specifically, a novel click-based Deep Structure-Preserving Embeddings with visual Attention (DSPEA) model is presented, which consists of two components: deep convolutional neural networks followed by image embedding layers for learning visual embedding, and a deep neural networks for generating query semantic embedding. Meanwhile, visual attention is incorporated at the top of the convolutional neural network to reflect the relevant regions of the image to the query. Furthermore, considering the high dimension of the query space, a new click-based representation on a query set is proposed for alleviating this sparsity problem. The whole network is end-to-end trained by optimizing a large margin objective that combines cross-view ranking constraints with in-view neighborhood structure preservation constraints. On a large-scale click-based image dataset with 11.7 million queries and 1 million images, our model is shown to be powerful for keyword-based image search with superior performance over several state-of-the-art methods and achieves, to date, the best reported NDCG@25 of 52.21%.

SHOAL

E-commerce taxonomy plays an essential role in online retail business. Existing taxonomy of e-commerce platforms organizes items into an ontology structure. However, the ontology-driven approach is subject to costly manual maintenance and often does not capture user's search intention, particularly when user searches by her personalized needs rather than a universal definition of the items. Observing that search queries can effectively express user's intention, we present a novel large-Scale Hierarchical taxOnomy via grAph based query coaLition ( SHOAL ) to bridge the gap between item taxonomy and user search intention. SHOAL organizes hundreds of millions of items into a hierarchical topic structure . Each topic that consists of a cluster of items denotes a conceptual shopping scenario, and is tagged with easy-to-interpret descriptions extracted from search queries. Furthermore, SHOAL establishes correlation between categories of ontology-driven taxonomy, and offers opportunities for explainable recommendation. The feedback from domain experts shows that SHOAL achieves a precision of 98% in terms of placing items into the right topics, and the result of an online A/B test demonstrates that SHOAL boosts the Click Through Rate (CTR) by 5%. SHOAL has been deployed in Alibaba and supports millions of searches for online shopping per day.

Semantic-Aware Searching Over Encrypted Data for Cloud Computing

With the increasing adoption of cloud computing, a growing number of users outsource their datasets to cloud. To preserve privacy, the datasets are usually encrypted before outsourcing. However, the common practice of encryption makes the effective utilization of the data difficult. For example, it is difficult to search the given keywords in encrypted datasets. Many schemes are proposed to make encrypted data searchable based on keywords. However, keyword-based search schemes ignore the semantic representation information of users’ retrieval, and cannot completely meet with users search intention. Therefore, how to design a content-based search scheme and make semantic search more effective and context-aware is a difficult challenge. In this paper, we propose ECSED, a novel semantic search scheme based on the concept hierarchy and the semantic relationship between concepts in the encrypted datasets. ECSED uses two cloud servers. One is used to store the outsourced datasets and return the ranked results to data users. The other one is used to compute the similarity scores between the documents and the query and send the scores to the first server. To further improve the search efficiency, we utilize a tree-based index structure to organize all the document index vectors. We employ the multi-keyword ranked search over encrypted cloud data as our basic frame to propose two secure schemes. The experiment results based on the real world datasets show that the scheme is more efficient than previous schemes. We also prove that our schemes are secure under the known ciphertext model and the known background model.

Privacy Protection in Interactive Content Based Image Retrieval

Privacy protection in Content Based Image Retrieval (CBIR) is a new research topic in cyber security and privacy. The state-of-art CBIR systems usually adopt interactive mechanism, namely relevance feedback, to enhance the retrieval precision. How to protect the user's privacy in such Relevance Feedback based CBIR (RF-CBIR) is a challenge problem. In this paper, we investigate this problem and propose a new Private Relevance Feedback CBIR (PRF-CBIR) scheme. PRF-CBIR can leverage the performance gain of relevance feedback and preserve the user's search intention at the same time. The new PRF-CBIR consists of three stages: 1) private query; 2) private feedback; 3) local retrieval. Private query performs the initial query with a privacy controllable feature vector; private feedback constructs the feedback image set by introducing confusing classes following the K -anonymity principle; local retrieval finally re-ranks the images in the user side. Privacy analysis shows that PRF-CBIR fulfills the privacy requirements. The experiments carried out on the real-world image collection confirm the effectiveness of the proposed PRF-CBIR scheme.

Multimodal analysis of user behavior and browsed content under different image search intents

The motivation or intent of a search for content may vary between users and use-cases. Knowledge and understanding of these underlying objectives may therefore be important in order to return appropriate search results, and studies of user search intent are emerging in information retrieval to understand why a user is searching for a particular type of content. In the context of image search, our work targets automatic recognition of users’ intent in an early stage of a search session. We have designed seven different search scenarios under the intent conditions of finding items, re-finding items and entertainment. Moreover, we have collected facial expressions, physiological responses, eye gaze and implicit user interactions from 51 participants who performed seven different search tasks on a custom-built image retrieval platform, and we have analyzed the users’ spontaneous and explicit reactions under different intent conditions. Finally, we trained different machine learning models to predict users’ search intent from the visual content of the visited images, the user interactions and the spontaneous responses. Our experimental results show that after fusing the visual and user interaction features, our system achieved the F-1 score of 0.722 for classifying three classes in a user-independent cross-validation. Eye gaze and implicit user interactions, including mouse movements and keystrokes are the most informative features for intent recognition. In summary, the most promising results are obtained by modalities that can be captured unobtrusively and online, and the results therefore demonstrate the potential of including intent-based methods in multimedia retrieval platforms.

Constructing target-aware results for keyword search on knowledge graphs

Existing work of processing keyword searches on graph data focuses on efficiency of result generation. However, being oblivious to user search intention, a query result may contain multiple instances of user search target, and multiple query results may contain information for the same instance of user search target. With the misalignment between query results and search targets, a ranking function is unable to effectively rank the instances of search targets. In this paper we propose the concept of target-aware query results driven by inferred user search intention. We leverage the Information Theory and develop a general probability model to infer search targets by analyzing return specifiers, modifiers, relatedness relationships, and query keywords' information gain. Then we propose two important properties for a target-aware result: atomicity and intactness. We develop techniques to efficiently generate target-aware results. Extensive experimental evaluation shows the effectiveness and efficiency of our approach.

SOR: An optimized semantic ontology retrieval algorithm for heterogeneous multimedia big data

Semantic information can express the search intentions of users, and this approach has become an important tool in the field of information retrieval. To support semantic-based multimedia retrieval in big data environment, this paper presents an optimized algorithm called semantic ontology retrieval (SOR), which uses big data processing tools to store and retrieve ontologies from heterogeneous multimedia data. First, the background of semantic extraction and ontology representation for multimedia big data are addressed. Second, the methodology of SOR, including the model definition and retrieval algorithm, is proposed. Third, for parallel processing SOR in distributed nodes, a MapReduce-based retrieval framework is presented. Finally, to achieve high retrieval precision and good user experience, a user feedback scheme is designed. The experimental results illustrate that SOR is suitable for semantic-based retrieval for heterogeneous multimedia big data.

Web Image Re-ranking using Semantic Signature and Visual Query Expansion

Web mining is a important aspect for users to get accurate and relevant data as per user need and satisfaction. Web image reranking is an effective way to improve the results of web based image searching. A major challenge in web image reranking is that the similarities of visual features are not correlated with images and semantic meanings which can not interpret users search intention. In proposed approach, By providing keyword expansion to textual query and visual query expansion to image query the results retrieved are promising as compared to existing system. Feature extraction, Hashing, Euclidean distance are used to make semantic signature more compact and to further enhance efficiency.

PWIS: Personalized Web Image Search using One-Click Method

Personalized Web Image search is the one searching for the particular images of User intention on the Web. For searching images, a user might provide query terms like keyword, image file, or click on few image file, and therefore the system can determine the images similar to the query. The similarity used for search criteria could be Meta tags, color distribution in images, region/shapes attributes, etc. Web-scale image search engines namely Google and Bing searches for images are relying on the surrounding text features. It is highly cumbersome and complicated for the web-scale based image search engines to interpret users search intention only by querying of keywords. This leads to the incorporation of noise and high ambiguity in the search results which are extremely unfit in the context of the users. It's also a necessary mandate for using visual information for solving the problem of ambiguity in the text-based image retrieval scenario. In the case of Google search, search text box will auto complete while user is typing similar added keywords. This method will differ from user intention while searching. So to avoid this kind of faults, it is important to use visual information in order to solve the uncertainty in text-based image retrieval. To retrieve exact matching, and acquire user‟s intention we can allow them text query with extended or related images as a suggestion. We have proposed an innovative Web image search approach. It only needs the user to click on one query image with minimal effort and images from a pool fetched by text-based search are re-ranked based on both visual and textual contents.

A method of searching for related literature on protein structure analysis by considering a user's intention.

BackgroundIn recent years, with advances in techniques for protein structure analysis, the knowledge about protein structure and function has been published in a vast number of articles. A method to search for specific publications from such a large pool of articles is needed. In this paper, we propose a method to search for related articles on protein structure analysis by using an article itself as a query.ResultsEach article is represented as a set of concepts in the proposed method. Then, by using similarities among concepts formulated from databases such as Gene Ontology, similarities between articles are evaluated. In this framework, the desired search results vary depending on the user's search intention because a variety of information is included in a single article. Therefore, the proposed method provides not only one input article (primary article) but also additional articles related to it as an input query to determine the search intention of the user, based on the relationship between two query articles. In other words, based on the concepts contained in the input article and additional articles, we actualize a relevant literature search that considers user intention by varying the degree of attention given to each concept and modifying the concept hierarchy graph.ConclusionsWe performed an experiment to retrieve relevant papers from articles on protein structure analysis registered in the Protein Data Bank by using three query datasets. The experimental results yielded search results with better accuracy than when user intention was not considered, confirming the effectiveness of the proposed method.

Effective keyword query structuring using NER for XML retrieval

Purpose – The purpose of this paper is to propose and evaluate XKQSS, a query structuring method that relegates the task of generating structured queries from a user to a search engine while retaining the simple keyword search query interface. A more effective way for searching XML database is to use structured queries. However, using query languages to express queries prove to be difficult for most users since this requires learning a query language and knowledge of the underlying data schema. On the other hand, the success of Web search engines has made many users to be familiar with keyword search and, therefore, they prefer to use a keyword search query interface to search XML data. Design/methodology/approach – Existing query structuring approaches require users to provide structural hints in their input keyword queries even though their interface is keyword base. Other problems with existing systems include their inability to put keyword query ambiguities into consideration during query structuring and how to select the best generated structure query that best represents a given keyword query. To address these problems, this study allows users to submit a schema independent keyword query, use named entity recognition (NER) to categorize query keywords to resolve query ambiguities and compute semantic information for a node from its data content. Algorithms were proposed that find user search intentions and convert the intentions into a set of ranked structured queries. Findings – Experiments with Sigmod and IMDB datasets were conducted to evaluate the effectiveness of the method. The experimental result shows that the XKQSS is about 20 per cent more effective than XReal in terms of return nodes identification, a state-of-art systems for XML retrieval. Originality/value – Existing systems do not take keyword query ambiguities into account. XKSS consists of two guidelines based on NER that help to resolve these ambiguities before converting the submitted query. It also include a ranking function computes a score for each generated query by using both semantic information and data statistic, as opposed to data statistic only approach used by the existing approaches.

G-Bean: an ontology-graph based web tool for biomedical literature retrieval.

BackgroundCurrently, most people use NCBI's PubMed to search the MEDLINE database, an important bibliographical information source for life science and biomedical information. However, PubMed has some drawbacks that make it difficult to find relevant publications pertaining to users' individual intentions, especially for non-expert users. To ameliorate the disadvantages of PubMed, we developed G-Bean, a graph based biomedical search engine, to search biomedical articles in MEDLINE database more efficiently.MethodsG-Bean addresses PubMed's limitations with three innovations: (1) Parallel document index creation: a multithreaded index creation strategy is employed to generate the document index for G-Bean in parallel; (2) Ontology-graph based query expansion: an ontology graph is constructed by merging four major UMLS (Version 2013AA) vocabularies, MeSH, SNOMEDCT, CSP and AOD, to cover all concepts in National Library of Medicine (NLM) database; a Personalized PageRank algorithm is used to compute concept relevance in this ontology graph and the Term Frequency - Inverse Document Frequency (TF-IDF) weighting scheme is used to re-rank the concepts. The top 500 ranked concepts are selected for expanding the initial query to retrieve more accurate and relevant information; (3) Retrieval and re-ranking of documents based on user's search intention: after the user selects any article from the existing search results, G-Bean analyzes user's selections to determine his/her true search intention and then uses more relevant and more specific terms to retrieve additional related articles. The new articles are presented to the user in the order of their relevance to the already selected articles.ResultsPerformance evaluation with 106 OHSUMED benchmark queries shows that G-Bean returns more relevant results than PubMed does when using these queries to search the MEDLINE database. PubMed could not even return any search result for some OHSUMED queries because it failed to form the appropriate Boolean query statement automatically from the natural language query strings. G-Bean is available at http://bioinformatics.clemson.edu/G-Bean/index.php.ConclusionsG-Bean addresses PubMed's limitations with ontology-graph based query expansion, automatic document indexing, and user search intention discovery. It shows significant advantages in finding relevant articles from the MEDLINE database to meet the information need of the user.

A Visual Analytics Approach Using the Exploration of Multidimensional Feature Spaces for Content-Based Medical Image Retrieval.

Content-based image retrieval (CBIR) is a search technique based on the similarity of visual features and has demonstrated potential benefits for medical diagnosis, education, and research. However, clinical adoption of CBIR is partially hindered by the difference between the computed image similarity and the user's search intent, the semantic gap, with the end result that relevant images with outlier features may not be retrieved. Furthermore, most CBIR algorithms do not provide intuitive explanations as to why the retrieved images were considered similar to the query (e.g., which subset of features were similar), hence, it is difficult for users to verify if relevant images, with a small subset of outlier features, were missed. Users, therefore, resort to examining irrelevant images and there are limited opportunities to discover these "missed" images. In this paper, we propose a new approach to medical CBIR by enabling a guided visual exploration of the search space through a tool, called visual analytics for medical image retrieval (VAMIR). The visual analytics approach facilitates interactive exploration of the entire dataset using the query image as a point-of-reference. We conducted a user study and several case studies to demonstrate the capabilities of VAMIR in the retrieval of computed tomography images and multimodality positron emission tomography and computed tomography images.