Disease-Oriented Image Embedding With Pseudo-Scanner Standardization for Content-Based Image Retrieval on 3D Brain MRI

Hayato Arai,Kenichi Oishi,Yusuke Chayama,Hitoshi Iyatomi,Yuto Onga,Kumpei Ikuta

doi:10.1109/access.2021.3129105

Abstract

To build a robust and practical content-based image retrieval (CBIR) system that is applicable to a clinical brain MRI database, we propose a new framework -- Disease-oriented image embedding with pseudo-scanner standardization (DI-PSS) -- that consists of two core techniques, data harmonization and a dimension reduction algorithm. Our DI-PSS uses skull stripping and CycleGAN-based image transformations that map to a standard brain followed by transformation into a brain image taken with a given reference scanner. Then, our 3D convolutioinal autoencoders (3D-CAE) with deep metric learning acquires a low-dimensional embedding that better reflects the characteristics of the disease. The effectiveness of our proposed framework was tested on the T1-weighted MRIs selected from the Alzheimer's Disease Neuroimaging Initiative and the Parkinson's Progression Markers Initiative. We confirmed that our PSS greatly reduced the variability of low-dimensional embeddings caused by different scanner and datasets. Compared with the baseline condition, our PSS reduced the variability in the distance from Alzheimer's disease (AD) to clinically normal (CN) and Parkinson disease (PD) cases by 15.8-22.6% and 18.0-29.9%, respectively. These properties allow DI-PSS to generate lower dimensional representations that are more amenable to disease classification. In AD and CN classification experiments based on spectral clustering, PSS improved the average accuracy and macro-F1 by 6.2% and 10.7%, respectively. Given the potential of the DI-PSS for harmonizing images scanned by MRI scanners that were not used to scan the training data, we expect that the DI-PSS is suitable for application to a large number of legacy MRIs scanned in heterogeneous environments.

Highlights

In the new era of Open Science [1], data sharing has become increasingly crucial for efficient and fair development of science and industry
We propose a novel framework called disease-oriented image embedding with pseudo-scanner standardization (DI-PSS), to obtain a low-dimensional embedding of magnetic resonance (MR) images for practical content-based image retrieval (CBIR) implementation
DISEASE-ORIENTED IMAGE EMBEDDING WITH PSEUDO-SCANNER STANDARDIZATION (DI-PSS) The aim of this study is to obtain a low-dimensional embedding of brain MR imaging (MRI) that is independent of the MRI scanner and individual characteristics but dependent on the pathological features of the brain, to realize a practical CBIR system for brain MRI

Summary

Introduction

In the new era of Open Science [1], data sharing has become increasingly crucial for efficient and fair development of science and industry. In the field of medical image science, various datasets have been released and used for the development of new methods and benchmarks. There have been attempts to create publicly open databases consisting of medical images, demographic data, and clinical information, such as ADNI, AIBL, PPMI, 4RTN, PING, ABCD and UK BioBank. Big data, consisting of large amounts of brain magnetic resonance (MR) images and corresponding medical records, could provide new evidence for the diagnosis and treatment of various diseases. Text-based searching is widely used for the retrieval of brain MR images. Since this approach requires skills and experience during retrieval and data registration, there is a strong demand from the field to realize content-based image retrieval (CBIR) [2]–[4]

Objectives

Methods

Results

Conclusion