How big is a big effect size? A nonparametric measure of effect size when modeling cytokines

Abstract

Abstract Cytokines play a major role in driving immune responses in cancer and other diseases. There is a critical need to understand how they interact and regulate one another. In this study, we examine new approaches to understand meaningful networks between cytokines and their relevance to human disease. Effect size (ES) is a way of assessing the difference between two groups with respect to a continuous measure such as cytokines. Its use, relative to p-values, gives easier understanding to whether a difference is actually clinically important or not. Results of a study can be statistically significant but still be too small to be of any practical value, or statistically insignificant but still be too large to not be of any practical value; especially when multiple comparisons adjustments are applied. As opposed to p-values, ES is independent of sample size, and thus has recently attracted the attention of researchers. When modeling cytokines, we recommend the use of the A measure of stochastic superiority A = [#Y1 > Y2) + 0.5#(Y1=Y2)]/n1n2, due to Delaney and Vargha (2002), which is appropriate for the nonparametric Wilcoxon Test. The ES of interest from this measure is A/1 – A which gives the odds that an individual in one group will score higher than an individual in the other group. Our simulation indicated that odds= 1.253, 1.763, and 2.500 are equivalent to Cohen’s d = 0.2 (small), 0.5 (medium), and 0.8 (large). We applied this measure on a panel of 26 cytokines identified in 29 subjects (15 with Metastasis and 14 without Metastasis) with gastric cancer, and revealed 11 cytokines with large ES, five of which with non-significant Sidak adjusted p-values: (from largest to smallest) G-CSF, IL-6, IL-8, MCP-1, IL-10, TNF-a, GM-CSF, MIP-1a, IL-1b, MIP-1b, and VEGF.