From Self-supervised Learning to Transfer Learning with Musculoskeletal Radiographs

Abstract

Abstract Ewing sarcomas are malignant neoplasm entities typically found in children and adolescents. Early detection is crucial for therapy and prognosis. Due to the low incidence the general experience as well as according data is limited. Novel support tools for diagnosis, such as deep learning models for image interpretation, are required. While acquiring sufficient data is a common obstacle in medicine, several techniques to tackle small data sets have emerged. The general necessity of large data sets in addition to a rare disease lead to the question whether transfer learning can solve the issue of limited data and subsequently support tasks such as distinguishing Ewing sarcoma from its main differential diagnosis (acute osteomyelitis) in paediatric radiographs. 42,608 unstructured radiographs from our musculoskeletal tumour centre were retrieved from the PACS. The images were clustered with a DeepCluster, a self-supervised algorithm. 1000 clusters were used for the upstream task (pretraining). Following, the pretrained classification network was applied for the downstream task of differentiating Ewing sarcoma and acute osteomyelitis. An untrained network achieved an accuracy of 81.5%/54.2%, while an ImageNet-pretrained network resulted in 89.6%/70.8% for validation and testing, respectively. Our transfer learning approach surpassed the best result by 4.4%/17.3% percentage points. Transfer learning demonstrated to be a powerful technique to support image interpretation tasks. Even for small data sets, the impact can be significant. However, transfer learning is not a final solution to small data sets. To achieve clinically relevant results, a structured and systematic data acquisition is of paramount importance.