Abstract 1937: DIME-CT: Self-supervised learning for medical image analysis using patch-based embeddings

Abstract

Abstract Whether tracking patient progress for clinical decision making, or investigating novel therapies, automated analysis of Computed Tomography (CT) imaging data is essential for the future of digital radiomics. In digital radiomics, as in all medical imaging, well annotated data is scarce, whereas unlabelled images are relatively plentiful. In other fields of image processing and medical imaging, the application of self-supervised learning (SSL) to large quantities of unlabelled data has resulted in great strides forward for fast, scalable, and interpretable image analysis. In this work we present a new approach applying SSL to CT imaging data which allows for: 1) Improved performance on image classification tasks, based on 2) dramatically reduced quantity of annotated CT imaging data, whilst also 3) enabling easy exploration and interpretation of the image regions. We applied a selection of self-supervised approaches (BYOL, DINO, SimCLR, & inpainting) to CT imaging data. Because the high dimensionality of CT data prevents us from using them directly to out-of the box SSL models, we adopt a 3D patching approach to reduce the dimensionality of the neural net input, and process each patch independently. We train our self-supervised models on public datasets (DeepLesion, NSCLC), and we specialize these models for tumor classification tasks that we evaluate on AstraZeneca sponsored clinical trials. The specialization is done in two different ways: 1) we use the pre-trained SSL model as an encoder that transforms the images of the clinical study into embeddings, on which we apply supervised classification models; 2) we use transfer learning to fine-tune supervised classification models that take the patches directly as inputs. We find that self-supervised pre-training significantly improves the accuracy on tumor classification tasks compared against a supervised learning baseline. Additionally, using the SSL embeddings we build an interactive map of CT imaging data enabling quick and intuitive inspection of the relevant regions. Our findings show that SSL constitutes an important tool for medical imaging analysis. SSL results in models that generalize better, and enable improved downstream interpretability and predictions. Furthermore, well trained SSL models can be re-applied to multiple indications because they are pre-trained on broad and diverse CT imaging data. Citation Format: Leon Fedden, Zhenning Zhang, Khan Baykaner, Qin Li, Lucas Bordeaux. DIME-CT: Self-supervised learning for medical image analysis using patch-based embeddings [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2022; 2022 Apr 8-13. Philadelphia (PA): AACR; Cancer Res 2022;82(12_Suppl):Abstract nr 1937.