Pattern graph-based image retrieval system combining semantic and visual features

Abstract

In the literature, several image retrieval approaches that allow mapping between low-level features and high-level semantics have been proposed. Among these one can cite object recognition, ontologies, and relevance feedback. However, their main limitations concern their high dependence on reliable external resources (existing ontologies, learning sets, etc.) and lack of capacity to combine semantic and visual information and provide relevant results. This paper proposes a system aiming to improve image retrieval results. The proposed system is based on a pattern graph combining semantic and visual features. The idea is (1) to automatically build a modular ontology based on a learning step from textual corpus and terminological resource, (2) to organize visual features in a graph-based model where the combined module and graph represent a unique component called “pattern,” and (3) to build a pattern graph. To this end our system has been implemented. The obtained experimental results show that the pattern graph that we propose enables an improvement of retrieval task.