A browsing approach to online photo search results

Abstract

Image sharing Web sites are constantly growing in size, containing up to several billion photographs. With such large databases, effective tools are needed in order to browse these sites. Search using keywords, tags and albums provide a basic level of filtering but is not sufficient to handle the thousands of results that are typically returned. Consequently, there is a need to go beyond image metadata, taking into account the image content itself to perform integrated browsing based on visual characteristics as well as keywords. While various approaches to visualisation and browsing of image databases have been introduced, these are too slow to be applied to online retrieval results, since the complete image files would need to be downloaded for feature extraction and exploitation. In this paper, we present a method that addresses this problem whilst being entirely client-side. We construct an interactive image database navigation application that uses the Huffman table in JPEG files as features, and projects image thumbnails onto a 2-dimensional visualisation space based on principal component analysis derived from the Huffman entries. Images are dynamically placed into a grid structure and organised in a tree-like hierarchy for visual browsing. Since we utilise information only from the JPEG header, we significantly reduce the required bandwidth and thus allow for interactive browsing of photo search results.