Which of the favorite optimal cut-off determination methods is preferable for the ordinal response data? A simulation study

Abstract

Abstract The aim of this study is to investigate the performance of the optimal cut-off methods, which are generally used for the diagnostic tests with the continuous response, for the tests with the ordinal response. Diagnostic accuracy studies examine the ability of a diagnostic test to discriminate between the patients with and without the condition. For diagnostic tests with a continuous response, it is important in practice to calculate the optimal cut-off point that can differentiate patients and healthy individuals. There are many methods proposed in the literature to obtain the optimal cut-point value for continuous test results. The Youden index, the point closest-to-(0, 1) corner in the ROC plane approach, the concordance probability, and the minimum P-value approach are commonly used methods to determine optimal-cut-point. But the researches examining the performance of these methods in the setting of the ordinal response tests are lacking in the literature. So, we compared the mentioned optimal cut-off methods for the ordinal response data by the way of simulation design by considering the sample size and the balance of groups as simulation conditions. The sample sizes of the diseased and non-diseased group were set (50, 50), (100, 100), and (200, 200) for balanced design and (50, 100), (50, 150) and (50, 200) for unbalanced design. For each scenario, 1000 repeats were generated. The differences between the estimated and the true cut-off points (biases) were calculated. All these methods overestimated the true cut-off point, but the median biases of the methods were varying. For the unbalanced design, the same result was relevant but for the balanced design, the minimum P-value approach had a median bias as 0 while others have 1.