An Exploratory Study on a Reinforcement Learning Prototype for Multimodal Image Retrieval Using a Conversational Search Interface

Abhishek Kaushik,Pankaj Velavan,Billy Jacob

doi:10.3390/knowledge2010007

Abhishek Kaushik, Pankaj Velavan + Show 1 more

Open Access

https://doi.org/10.3390/knowledge2010007

Copy DOI

Journal: Knowledge	Publication Date: Feb 28, 2022
Citations: 5	License type: CC BY 4.0

Affiliation: University College Dublin, Dublin Business School

Abstract

In the realm of information, conversational search is a relatively new trend. In this study, we have developed, implemented, and evaluated a multiview conversational image search system to investigate user search behaviour. We have also explored the potential for reinforcement learning to learn from user search behaviour and support the user in the complex information seeking process. A conversational image search system may mimic a natural language discussion with a user via text or speech, and then assist the user in locating the required picture via a dialogue-based search. We modified and improved a dual-view search interface that displays discussions on one side and photos on the other. Based on the states, incentives, and dialogues in the initial run, we developed a reinforcement learning model and a customized search algorithm in the back end that predicts which reply and images would be provided to the user among a restricted set of fixed responses. Usability of the system was validated using methodologies such as Chatbot Usability Questionnaire, System Usability Scale, and User Experience Questionnaire, and the values were tabulated. The result of this usability experiment proved that most of the users found the system to be very usable and helpful for their image search.

Highlights

Web search has become an inevitable activity in the day-to-day lives of people
Exploratory Research Question: “How can reinforcement learning be used for improving the search experience of the user?”
We evaluated this image search interface on three standard different usability metrics invigorated by the evaluation framework developed by Kaushik et al [2]

Summary

Introduction

Web search has become an inevitable activity in the day-to-day lives of people. Various pioneers in the field of search engines, comprising of Google, Bing, DuckDuckGo, and more, have made revolutionary improvements in the search process. Image search is one key aspect of the web search results which helps users gain a better picture of what they are looking for. With the plethora of information available across the internet, it is a strenuous task to provide relevant information to the end user. Another challenge is the user’s shortage of knowledge of the topic about which they are querying. To overcome the above-mentioned challenges, an alternative model for search interaction is gaining momentum. In this model, the user communicates with an agent that seeks to help their search activities. Conversational search is a very engaging method of image retrieval as it simulates the way in which people converse with each other and find the required data [2–5]

Methods

Results

Discussion

Conclusion