Assessing group change under conditions of anonymity and overlapping samples.

Abstract

A commonly used research design in the social sciences involves the matching of observations over 2 time periods (i.e., Time 1 --> Time 2) to assess group change. Because coupled observations are usually correlated, a paired- or dependent-samples t test is generally recommended in such applications to determine if there has been a statistically significant change in mean scores across time. Consequently, it is typically believed that unless information for matching respondents' observations is available, researchers have no choice but to treat the observations as if they were independent. To demonstrate alternative statistical approaches for employing the paired samples ttest when information for matching respondents' observations is unavailable and to illustrate the applicability of these alternatives to longitudinal designs in which respondents at Time 1 are partially replaced by new respondents at Time 2. Theoretical arguments and examples are employed to achieve the specified objectives. Performing an independent-samples ttest when a paired-samples t test is more appropriate will lead to a loss of statistical power and, thus, increase the likelihood of a Type II statistical error. The statistical approaches that are demonstrated allow researchers to account for pair wise dependency across observations and, therefore, to obtain a fairer test of group change in means.