Text- and content-based biomedical image modality classification

Abstract

Image modality classification is an important task toward achieving high performance in biomedical image and article retrieval. Imaging modality captures information about its appearance and use. Examples include X-ray, MRI, Histopathology, Ultrasound, etc. Modality classification reduces the search space in image retrieval. We have developed and evaluated several modality classification methods using visual and textual features extracted from images and text data, such as figure captions, article citations, and MeSH®. Our hierarchical classification method using multimodal (mixed textual and visual) and several class-specific features achieved the highest classification accuracy of 63.2%. The performance was among the best in ImageCLEF2012 evaluation.