Reducing the semantic gap in content-based image retrieval in mammography with relevance feedback and inclusion of expert knowledge

Abstract

We investigate the use of relevance feedback (RFb) and the inclusion of expert knowledge to reduce the semantic gap in content-based image retrieval (CBIR) of mammograms. Tests were conducted with radiologists, in which their judgment of the relevance of the retrieved images was used with techniques of query-point movement to incorporate RFb. The measures of similarity of images used for CBIR were based upon textural characteristics and the distribution of density of fibroglandular tissue in the breast. The features used include statistics of the gray-level histogram, texture features based upon the gray-level co-occurrence matrix, moment-based features, measures computed in the Radon domain, and granulometric measures. Queries for CBIR with RFb were executed by three radiologists. The performance of CBIR was measured in terms of precision of retrieval and a measure of relevance-weighted precision (RWP) of retrieval. The results indicate improvement due to RFb of up to 62% in precision and 39% in RWP. The gain in performance of CBIR with RFb depended upon the BI-RADS breast density index of the query mammographic image, with greater improvement in cases of mammograms with higher density.