A Model for Semantic Medical Image Retrieval Applied in a Medical Social Network

Abstract

We present in this article a multimodal research model for the retrieval of medical images based on the extracted multimedia information from a radiological collaborative social network. However, opinions shared on a medical image in a medical social network constitute a textual description that requires in most of the time cleaning using a medical thesaurus. In addition, we describe the textual description and medical image in a TF-IDF weight vector using an approach of « bag-of-words ». We use latent semantic analysis to establish relationships between textual and visual terms from the shared opinions on the medical image. Multimodal modeling will search for medical information through multimodal queries. Our model is evaluated on the basis ImageCLEFmed’2015 for which we have the ground-truth. We have carried many experiments with different descriptors and many combinations of modalities. Analysis of the results shows that the model is based on two methods can increase the performance of a research system based on only one modality, either visual or textual.