Overcoming the challenges of multi-modal medical image sharing: A novel data distillation strategy via contrastive learning

Abstract

Listen

Translate article

Multi-modal medical images serve as a critical basis for disease diagnosis. However, sharing multi-modal medical data between hospitals is a significant challenge due to privacy protection concerns as well as the substantial costs associated with data transfer and storage. These hurdles are further compounded by two primary issues: (1) the complexity involved in sharing data features, which arises from the intricate process of multi-modal image registration; (2) the difficulty and high cost of acquiring accurate data labels. To address these issues, we propose a novel multi-modal medical dataset distillation method based on contrastive learning. We consider each modality as a unique approach to data augmentation for the corresponding imaging region, offering a unique perspective on the same anatomical region, which is then fused to collectively contribute to a comprehensive global representation of the medical image. Validation results on two datasets indicate our method is capable of reducing the original dataset to just 5% of its original size while retaining classification performance comparable to that of the full dataset, and notably, this efficiency is achieved using only 30% of the total labels, which significantly lowers the cost and effort associated with dataset labeling.