Cancerclass: AnRPackage for Development and Validation of Diagnostic Tests from High-Dimensional Molecular Data

Jan Budczies,Silvia Drab-Esfahani,Carsten Denkert,Albrecht Stenzinger,Christian Von Törne,Manfred Dietel,Daniel Kosztyla

doi:10.18637/jss.v059.i01

Abstract

Progress in molecular high-throughput techniques has led to the opportunity of a comprehensive monitoring of biomolecules in medical samples. In the era of personalized medicine, these data form the basis for the development of diagnostic, prognostic and predictive tests for cancer. Because of the high number of features that are measured simultaneously in a relatively low number of samples, supervised learning approaches are sensitive to overfitting and performance overestimation. Bioinformatic methods were developed to cope with these problems including control of accuracy and precision. However, there is demand for easy-to-use software that integrates methods for classifier construction, performance assessment and development of diagnostic tests. To contribute to filling of this gap, we developed a comprehensive R package for the development and validation of diagnostic tests from high-dimensional molecular data. An important focus of the package is a careful validation of the classification results. To this end, we implemented an extended version of the multiple random validation protocol, a validation method that was introduced before. The package includes methods for continuous prediction scores. This is important in a clinical setting, because scores can be converted to probabilities and help to distinguish between clear-cut and borderline classification results. The functionality of the package is illustrated by the analysis of two cancer microarray data sets.

Highlights

Progress in molecular high-throughput techniques has led to the opportunity of simultaneous monitoring of hundreds or thousands of biomolecules in medical samples, e.g. using microarrays
Using a protocol similar to [1] we investigate the dependence of classification accuracy on the number of features (Fig. 1):
The confidence interval of the overall classification rate is estimated from 200 random splits in training and test sets

Summary

Introduction

Progress in molecular high-throughput techniques has led to the opportunity of simultaneous monitoring of hundreds or thousands of biomolecules in medical samples, e.g. using microarrays. Because of the high dimensionality of the data and connected to the multiple testing problem, the development of molecular tests is sensitive to model overfitting and performance overestimation. Bioinformatic methods have been developed to cope with these problems, e.g. the multiple random validation protocol that was presented in [1]. Cancerclass integrates methods for development and validation of diagnostic tests from high-dimensional molecular data. Simple classifiers were shown to have a good performance on high-dimensional data compared to more sophisticated methods [2]. The accuracy of the predictor can be evaluated using training and test set validation, leave-one-out cross-validation or in a multiple random validation protocol. The functionality of cancerclass is illustrated using two sets of cancer gene expression data. Gene expression data of breast cancer with good and poor prognosis [4, 5] are obtained from the ExperimentData package cancerdata

Multiple random validation protocol

Number of genes

Predictor construction and validation

Findings

NODM DM