Multimodal query-level fusion for efficient multimedia information retrieval

Abstract

Managing a large volume of multimedia data containing various modalities such as visual, audio, and text reveals the necessity for efficient methods for modeling, processing, storing, and retrieving complex data. In this paper, we propose a fusion-based approach at the query level to improve query retrieval performance of multimedia data. We discuss various flexible query types including the combination of content as well as concept-based queries that provide users with the ability to efficiently perform multimodal querying. We have carried out a number of experiments on a video database to show the efficiency of our approach for various types of queries. Our experimental results show that our query-level fusion approach presents a notable improvement in retrieval performance especially for the concept-based queries.