Multimodal Deep Learning for Prognosis Prediction in Renal Cancer.

Stefan Schulz,Ann-Christin Woerl,Stephanie Strobl,Axel Haferkamp,Florian Jungmann,Philipp Stenzel,Christina Glasner,Sebastian Foersch,Aurélie Fernandez,Peter Mildenberger,Wilfried Roth,Daniel-Christoph Wagner

doi:10.3389/fonc.2021.788740

Abstract

BackgroundClear-cell renal cell carcinoma (ccRCC) is common and associated with substantial mortality. TNM stage and histopathological grading have been the sole determinants of a patient’s prognosis for decades and there are few prognostic biomarkers used in clinical routine. Management of ccRCC involves multiple disciplines such as urology, radiology, oncology, and pathology and each of these specialties generates highly complex medical data. Here, artificial intelligence (AI) could prove extremely powerful to extract meaningful information to benefit patients.ObjectiveIn the study, we developed and evaluated a multimodal deep learning model (MMDLM) for prognosis prediction in ccRCC.Design, Setting, and ParticipantsTwo mixed cohorts of non-metastatic and metastatic ccRCC patients were used: (1) The Cancer Genome Atlas cohort including 230 patients and (2) the Mainz cohort including 18 patients with ccRCC. For each of these patients, we trained the MMDLM on multiscale histopathological images, CT/MRI scans, and genomic data from whole exome sequencing.Outcome Measurements and Statistical AnalysisOutcome measurements included Harrell’s concordance index (C-index) and also various performance parameters for predicting the 5-year survival status (5YSS). Different visualization techniques were used to make our model more transparent.ResultsThe MMDLM showed great performance in predicting the prognosis of ccRCC patients with a mean C-index of 0.7791 and a mean accuracy of 83.43%. Training on a combination of data from different sources yielded significantly better results compared to when only one source was used. Furthermore, the MMDLM’s prediction was an independent prognostic factor outperforming other clinical parameters.InterpretationMultimodal deep learning can contribute to prognosis prediction in ccRCC and potentially help to improve the clinical management of this disease.Patient SummaryAn AI-based computer program can analyze various medical data (microscopic images, CT/MRI scans, and genomic data) simultaneously and thereby predict the survival time of patients with renal cancer.

Highlights

Clear-cell renal cell carcinoma is the most common type of kidney cancer and more than 175,000 patients die from this entity each year (1)
We wanted to investigate how the uni- and multimodal deep learning models performed, when compared to known prognostic factors in renal cell carcinoma
Since the Concordance index (C-index) cannot be applied to an individual patient and might prove difficult to be translated into clinical decision making, we investigated the possibility to predict the 5-year survival status (5YSS) using a MMDLM and binary classification

Summary

Introduction

Clear-cell renal cell carcinoma (ccRCC) is the most common type of kidney cancer and more than 175,000 patients die from this entity each year (1). In contrast to other tumor types, there is no clearly defined set of biomarkers used in clinical routine This might be partly because ccRCC development seems to be driven by a multitude of interacting metabolic pathways and regulated by complex epigenetic programs (2). Clinical management of ccRCC usually involves various specialties including urology, radiology, oncology, pathology, and others. This results in a vast amount of medical data on each patient, such as CT/MRI scans, histopathological images, and other clinical information. Heng et al for example developed a score, which consists of various clinical parameters such as Karnofsky performance status, hemoglobin, corrected calcium, and others With this strategy, they were able to achieve an overall C-index of 0.73 in the prognosis prediction of 645 metastatic RCC (4). Artificial intelligence (AI) could prove extremely powerful to extract meaningful information to benefit patients

Methods

Results

Conclusion