Replication in Observational Couples Research: A Commentary

Abstract

We read with great interest both the Generalizability of Gottman and Colleagues' Affective Process Models of Couples' Relationship Outcomes of Kim, Capaldi, and Crosby (2007, Volume 69) and the response of Coan and Gottman (2007, Volume 69). In this brief commentary, we review the philosophy of replication and put the article of Kim et al. in context, relating it to both replications in general and the replication of Gottman, Coan, Carrere, and Swanson (1998) specifically. REPLICATION hallmark of science is reproducibility of results. We can only have confidence that a true phenomenon, relationship, or model has been established if it is reproducible. Replications thus serve to (a) establish the robustness of a purported phenomenon, relationship, or model; (b) establish the generalizability of findings; and (c) self-correct (through disconfirmation) or limit the application of nonreproducible results (e.g., Lamal, 1991; Rosenthal, 1991). Replications vary on two dimensions: population sampled and methods used (Tsang & Kwan, 1999). Somewhat different terms are used across disciplines to describe the varying forms of replication; Table 1 displays the four basic types. These types represent how the replication is conducted. Rosenthal (1991) also emphasizes two other factors that affect the value of a replication: who is conducting it (the original investigator/team or someone else), and their respective capabilities, and when it is conducted. latter two factors are not issues in the present case, as the who comprises two highly esteemed research teams and the when does not differ substantially. Replications, by their very nature, come with built-in hypotheses (i.e., those of the original investigators). Literal/exact replications (i.e., those in the same/same quadrant), because they duplicate all facets of the original study, do not have original hypotheses (Amir & Sharon, 1991). Replications that use a different population may have specific hypotheses in doing so but typically will simply be interested in whether the findings from the original population generalize to another population. Similarly, substantive differences in method typically will be accompanied by specific hypotheses, whereas nonsubstantive differences typically will not (Amir & Sharon). Finally, as with any scientific manipulation, the more things change from the original study, the harder it is to make inferences about disparate results. Replications in which the methods are the same as the original study are the easiest to interpret; those in which both the populations and the methods differ (i.e., generalization and extension studies) are the most difficult to interpret because of multiple differences between the original and replication studies. KIM ET AL. REPLICATION STUDY Kim et al. (2007) were clear about the intent of the study: The purpose of the current study was to examine whether the affective clusters and sequences found to be differentially predictive of relationship status and satisfaction in [Gottman et al., 1998] would also be predictive of such outcomes for a community-based sample of 85 married or cohabiting couples with differing characteristics from the original sample. Thus, the Kim et al. study used a different population and different methods (i.e., an expanded protocol observing the top issues of both partners, although using the same measure of affect, the Specific Affect Coding System [SPAFF]). Populations Kim et al. (2007) followed up a sample of boys recruited from fourth-grade classrooms in schools from higher crime (and lower socioeconomic status) communities of Eugene, OR. Although their outcomes (e.g., high prevalence of arrest) establish the validity of labeling these boys risk, it should be emphasized that (a) neighborhood schools, not the boys themselves, were chosen for risk and (b) neighborhoods were selected if they were at elevated risk within a universityoriented small city; Eugene is neither the South Bronx nor Appalachia. …