Combining text retrieval and content-based image retrieval for searching a large-scale medical image database in an integrated RIS/PACS environment

Abstract

Medical imaging modalities generate huge amount of medical images daily, and there are urgent demands to search large-scale image databases in an RIS-integrated PACS environment to support medical research and diagnosis by using image visual content to find visually similar images. However, most of current content-based image retrieval (CBIR) systems require distance computations to perform query by image content. Distance computations can be time consuming when image database grows large, and thus limits the usability of such systems. Furthermore, there is still a semantic gap between the low-level visual features automatically extracted and the high-level concepts that users normally search for. To address these problems, we propose a novel framework that combines text retrieval and CBIR techniques in order to support searching large-scale medical image database while integrated RIS/PACS is in place. A prototype system for CBIR has been implemented, which can query similar medical images both by their visual content and relevant semantic descriptions (symptoms and/or possible diagnosis). It also can be used as a decision support tool for radiology diagnosis and a learning tool for education.