Standardization in Quantitative Imaging: A Multicenter Comparison of Radiomic Features from Different Software Packages on Digital Reference Objects and Patient Data Sets

Michael F Mcnitt‐Gray ,Muhammad Wahi-Anwar ,Mahmoud A Abdalah ,Sarthak Pati ,Hao Yang ,Wenzhi Lv ,Liwu Lin ,Binsheng Zhao ,Spyridon Bakas ,Sandy Napel ,Arman Rahmim ,Mark Muzi ,Sarah A Mattonen ,Akshay Jaggi ,Ella F Jones ,Apurva Virkud ,Lubomir M Hadjiiski ,Anwell Nguyen ,Larry A Pierce ,Heang Ping Chan ,Yoganand Balagurunathan ,Megan E Daly ,Aimilia Gastounioti ,Despina Kontos ,Dmitry B Goldgof ,Nastaran Emaminejad ,Paul E Kinahan ,Jayashree Kalpathy–Cramer ,Keyvan Farahani

doi:10.18383/j.tom.2019.00031

Abstract

Radiomic features are being increasingly studied for clinical applications. We aimed to assess the agreement among radiomic features when computed by several groups by using different software packages under very tightly controlled conditions, which included standardized feature definitions and common image data sets. Ten sites (9 from the NCI's Quantitative Imaging Network] positron emission tomography–computed tomography working group plus one site from outside that group) participated in this project. Nine common quantitative imaging features were selected for comparison including features that describe morphology, intensity, shape, and texture. The common image data sets were: three 3D digital reference objects (DROs) and 10 patient image scans from the Lung Image Database Consortium data set using a specific lesion in each scan. Each object (DRO or lesion) was accompanied by an already-defined volume of interest, from which the features were calculated. Feature values for each object (DRO or lesion) were reported. The coefficient of variation (CV), expressed as a percentage, was calculated across software packages for each feature on each object. Thirteen sets of results were obtained for the DROs and patient data sets. Five of the 9 features showed excellent agreement with CV < 1%; 1 feature had moderate agreement (CV < 10%), and 3 features had larger variations (CV ≥ 10%) even after attempts at harmonization of feature calculations. This work highlights the value of feature definition standardization as well as the need to further clarify definitions for some features.

Highlights

Radiomics is described as the high-throughput extraction of large amounts of image features from radiographic images [1,2,3,4]
The results from the 3 investigations are described
It should be noted that the agreement is still relatively modest, which indicates there are still outstanding issues to resolve to obtain the levels of agreement seen in the features with the excellent levels of agreement

Summary

Introduction

Radiomics is described as the high-throughput extraction of large amounts of image features from radiographic images [1,2,3,4]. Radiomic features provide quantitative descriptions of objects (tissues, suspected pathology, and anatomic regions) contained within the image data. These mathematical descriptors provide ways to characterize the size, shape, texture, intensity, margin, and other aspects of the imaging features of these objects, with. There has been an effort to standardize feature definitions as described by the Image Biomarker Standardization Initiative (IBSI), which has published a reference manual [13] This reference manual has described a large number of features in detail and has even introduced conventions that provide unique codes to identify each of these features

Objectives

Methods

Results

Discussion

Conclusion