The Browsing Issue in Multimodal Information Retrieval: A Navigation Tool Over a Multiple Media Search Result Space

Abstract

In the field of Multimodal Information Retrieval, one of the issue to tackle is how to effectively browse the search result space. In addressing this issue, it is particularly important to take into consideration that, especially nowadays, data is highly semantically interlinked. In this scenario, we present a tool to navigate and visualize the results produced by the evaluation of a query over a set of multiple media objects. The search result space can be represented via a graph-based data model where (i) multiple media objects are represented as nodes with multiple modalities of information associated with them, and (ii) media objects can be connected via different kinds of relationships. Our idea is to give to the user the possibility to navigate the space of the results of a query, constituted by multiple media objects, as s/he was exploring a graph of connected entities. As a preliminary work, in this paper we only deal with textual information for building similarity relationships among media objects and part-of relationships in the case of media objects belonging to a same (multimedia) document. This way, we show how a user can navigate and visualize the result space following different links connecting media objects. We illustrate our navigation and visualization tool with different examples.