Statement Validity Assessment Research Articles

Reliability of criteria‐based content analysis of child witness statements: Response to Tully

Tully (1998) suggests that the quality of statements from child witnesses is likely to affect the reliability of criteria‐based content analysis (CBCA) ratings. Recent studies suggest that this may be true. He also suggests that giving raters more latitude in their ratings, rather than less, might improve both reliability and validity. This is an interesting idea that is worth testing. Finally, Tully argues that CBCA and statement validity assessment (SVA), of which it is a part, are protocols rather than psychometric instruments. Therefore, CBCA should not be subject to the rigours of testing for reliability and validity. With this I must disagree.

Reliability of criteria‐based content analysis of child witness statements: Cohen's kappa doesn't matter

The recent attempt by Horowitz, Lamb, Esplin, Boychuk, Orit & Reiter‐Lavery (1997) to establish reliabilities of evaluators applying criteria‐based content analysis (CBCA) leads to misleading conclusions about the reliability of CBCA as a protocol. The history and procedure of statement validity assessment (SVA) of which CBCA is a component, has not established CBCA as the equivalent of a psychometric type instrument or method with any particular level of instrinsic reliability. There are many reasons why certainty as to whether given criteria are agreed to fit a slice of narrative may be high or low. The adequacy of that particular slice of narrative for such criterion fitting is as important as the evaluators agreeing they mean the same thing and are trying to do the same thing on one or more occasions. The outcome reliabilities statistically computed are a composite of both. Professional evaluators do not need to rely on the results of forcing themselves to decide without adequate certainty whether any or all criteria are ‘present’ or ‘absent’. Their reliability when they make a decision with adequate certainty is what matters. Horowitz et al.'s recommendations that certain criteria should be further defined ‘more clearly’ or dropped is particularly erroneous. This would lead to the loss of important if infrequently found instances of presence or absence of criteria which are telling because of what is known about how children recall sexual victimization, and not because a statistical manual sets an empirical threshold.

Can criteria-based content analysis distinguish between true and false statements of African-American speakers?

The Criteria-Based Content Analysis (CBCA) technique is the core component of the Statement Validity Assessment (SVA), an interviewing process that attempts to judge allegations of child sexual abuse for truthfulness. As the nucleus of the SVA, the CBCA has been promoted as a way to detect truth by analyzing the verbal content of such an allegation. However, all conclusions of the CBCA's usefulness as an indicator of truth have been based on European and/or White perspectives, and this is the first study to address the implications of probable differences of verbal content meaning across ethnic groups. The results failed to support the hypotheses that Black speakers' truthful statements contained fewer criteria than Whites' statements, or that CBCA training would only enhance truth detection of White speakers. First, in contrast to earlier work, the present study found that, despite the fact that true and false statements contained criteria to differing degrees, the total number of CBCA criteria in an adult's statement was uncorrelated with truthfulness for statements made by Blacks or by Whites. A discriminant analysis showed that CBCA verbal content criteria did, in fact, have different significance for Whites than for Blacks. Certain CBCA criteria were stronger predictors of truth for one ethnic group than for the other. It is suggested that more research is needed before the CBCA is relied upon in criminal investigations.