Definition and classification of evaluation units for CASP10

Abstract

For the 10th experiment on Critical Assessment of the techniques of protein Structure Prediction (CASP), the prediction target proteins were broken into independent evaluation units (EUs), which were then classified into template-based modeling (TBM) or free modeling (FM) categories. We describe here how the EUs were defined and classified, what issues arose in the process, and how we resolved them. EUs are frequently not the whole target proteins but the constituting structural domains. However, the assessors from CASP7 on combined more than one domain into 1 EU for some targets, which implied that the assessment also included evaluation of the prediction of the relative position and orientation of these domains. In CASP10, we followed and expanded this notion by defining multidomain EUs for a number of targets. These included 3 EUs, each made of two domains of familiar fold but arranged in a novel manner and for which the focus of evaluation was the interdomain arrangement. An EU was classified to the TBM category if a template could be found by sequence similarity searches and to FM if a structural template could not be found by structural similarity searches. The EUs that did not fall cleanly in either of these cases were classified case-by-case, often including consideration of the overall quality and characteristics of the predictions.