On Selection of Training and Test Sets for the Development of Predictive QSAR models

Abstract

AbstractThe development of predictive QSAR models depends not only on the statistical method but also on the algorithm used for the selection of training and test sets. Here, we describe the validation of QSAR models for three data sets with different sizes (n=35, 56 and 87) based on random division, sorted biological activity data and K‐means clusters for the factor scores of the original variable matrix along with/without biological activity values. When the training and test sets were generated by random division or by the activity‐range algorithm, predictive models were not obtained in most of the cases. In case of random division of the data sets into training and test sets, there is no correlation between internal and external validation statistics. However, good external validation statistics were obtained when training and test sets were selected based on K‐means clusters of factor scores of the descriptor space along with/without the biological activity values. So, the selection of training and test sets should be based on the proximity of the representative points of the test set to representative points of the training set in the multidimensional descriptor space. The concept of closeness is based on the general assumption underlying all QSAR theories: similar compounds have similar activities. Thus, if one wishes to validate a QSAR model, the points of the test set must be close to the points of the training set in the multidimensional descriptor space. Based on the results of several methods for the division of the training and test sets, we propose that K‐means‐cluster‐based division of training and prediction sets can be used as a reliable method of division of data set into training and test sets for developing predictive QSAR models.