The performance of diabetes risk prediction models in new populations: the role of ethnicity of the development cohort.

Abstract

It is believed that diabetes risk scores need to be ethnic specific. However, this prerequisite has not been tested. We examined the performance of several risk models, developed in various populations, in a Europid and a South Asian population. The performance of 14 published risk prediction models were tested in two prospective studies: the Australian Diabetes, Obesity and Lifestyle (AusDiab) study and the Mauritius non-communicable diseases survey. Eight models were developed in Europid populations; the remainder in various non-Europid populations. Model performance was assessed using area under the receiver operating characteristic curves (discrimination), Hosmer-Lemeshow tests (goodness-of-fit) and Brier scores (accuracy). In both AusDiab and Mauritius, discrimination was highest for a model developed in a mixed population (non-Hispanic white and African American) and lowest for a model developed in a Europid population. Discrimination for all scores was higher in AusDiab than in Mauritius. For almost all models, goodness-of-fit was poor irrespective of the ethnicity of the development cohort, and accuracy was higher in AusDiab compared to Mauritius. Our results suggest that similarity of ethnicity or similarity of diabetes risk may not be the best way of identifying models that will perform well in another population. Differences in study methodology likely account for much of the difference in the performance. Thus, identifying models which use measurements that are clearly described and easily reproducible for both research and clinical settings may be more important.