Criteria-Based Content Analysis Research Articles

The Willingness of Children to Lie and the Assessment of Credibility in an Ecologically Relevant Laboratory Setting

A series of 4 experiments were conducted to (a) determine the willingness of children to lie in a realistic setting, (b) compare judgments of credibility from both lay evaluations and Criterion-Based Content Analysis (CBCA), and (c) examine the effects of expert testimony regarding Statement Validity Assessment on mock jurors who were asked to make evaluations of the children's statements. In Experiment 1, 81% of children who witnessed a research assistant steal a textbook made accusations against the thief (truthful), 69% of children who did not witness the theft accused the research assistant of the theft following prompting by significant others, and 56% of the children who witnessed a significant other steal the textbook incorrectly accused the research assistant following a request from their significant other. Using the statements obtained from the children in Experiment 1, Experiments 2 and 3 found that classification accuracy of lay evaluators was significantly poorer than expert application of CBCA, which resulted in 89% classification accuracy. Finally, brief exposure to CBCA expert testimony appeared to have no benefits on mock jurors' assessments of credibility of the children's statements in Experiment 4. Implications for North American legal systems are discussed.

The Dynamics of Interviews Involving Plausible and Implausible Allegations of Child Sexual Abuse

Interviews of 12 children describing sexual abuse incidents that were deemed unlikely to have happened were matched with 12 interviews involving descriptions of events that appeared likely to have happened. Each interviewer utterance and each child response was categorized into several types. For each of the child's responses, coders also tabulated the number of words, informative details, and Criterion-Based Content Analysis (CBCA) criteria present. The distribution of the different interviewer's utterance types and child's response types was similar in the 2 groups, as was the number of words and details provided in the average response by the child. However, differences were evident in the children's responses to the specific types of interviewer utterance. The children provided more words, details, and contents that qualified as CBCA criteria in response to open-ended utterances than to focused utterances in plausible statements, but those effects were not apparent in implausible statements.

What is forensic psychology, anyway?

The issue of how “forensic psychology” is defined has taken on a new urgency in the context of an application to have forensic psychology designated a “specialty” by the American Psychological Association. To provide a historical perspective, I briefly review early attempts to apply psychological concepts to legal issues, beginning with the McNaughten trial in England in 1843. I then review current conceptualizations of forensic psychology, which have either a broad focus on all psychology–law interactions or focus more narrowly on clinical applications to the legal system. Potential advantages and disadvantages of each conceptualization are briefly discussed. After touching upon the major differences in the “cultures” of psychology and of law, I discuss three law-related areas of contemporary controversy: use of clinically based evidence in the courtroom, “recovered” memories of child abuse, and the use of the criteria-based content analysis technique to evaluate children's claims of sexual abuse. Issues concerning the education and training of psychology–law scholars and practitioners are briefly surveyed. I conclude that a broad conceptualization of forensic psychology is important in terms of relevant ethical standards, but that a more narrow distinction that differentiates between clinicians and researchers or legal scholars also is useful.

Reliability of criteria‐based content analysis of child witness statements: Cohen's kappa doesn't matter

The recent attempt by Horowitz, Lamb, Esplin, Boychuk, Orit & Reiter‐Lavery (1997) to establish reliabilities of evaluators applying criteria‐based content analysis (CBCA) leads to misleading conclusions about the reliability of CBCA as a protocol. The history and procedure of statement validity assessment (SVA) of which CBCA is a component, has not established CBCA as the equivalent of a psychometric type instrument or method with any particular level of instrinsic reliability. There are many reasons why certainty as to whether given criteria are agreed to fit a slice of narrative may be high or low. The adequacy of that particular slice of narrative for such criterion fitting is as important as the evaluators agreeing they mean the same thing and are trying to do the same thing on one or more occasions. The outcome reliabilities statistically computed are a composite of both. Professional evaluators do not need to rely on the results of forcing themselves to decide without adequate certainty whether any or all criteria are ‘present’ or ‘absent’. Their reliability when they make a decision with adequate certainty is what matters. Horowitz et al.'s recommendations that certain criteria should be further defined ‘more clearly’ or dropped is particularly erroneous. This would lead to the loss of important if infrequently found instances of presence or absence of criteria which are telling because of what is known about how children recall sexual victimization, and not because a statistical manual sets an empirical threshold.

Reliability of criteria‐based content analysis of child witness statements: Response to Tully

Tully (1998) suggests that the quality of statements from child witnesses is likely to affect the reliability of criteria‐based content analysis (CBCA) ratings. Recent studies suggest that this may be true. He also suggests that giving raters more latitude in their ratings, rather than less, might improve both reliability and validity. This is an interesting idea that is worth testing. Finally, Tully argues that CBCA and statement validity assessment (SVA), of which it is a part, are protocols rather than psychometric instruments. Therefore, CBCA should not be subject to the rigours of testing for reliability and validity. With this I must disagree.

Can criteria-based content analysis distinguish between true and false statements of African-American speakers?

The Criteria-Based Content Analysis (CBCA) technique is the core component of the Statement Validity Assessment (SVA), an interviewing process that attempts to judge allegations of child sexual abuse for truthfulness. As the nucleus of the SVA, the CBCA has been promoted as a way to detect truth by analyzing the verbal content of such an allegation. However, all conclusions of the CBCA's usefulness as an indicator of truth have been based on European and/or White perspectives, and this is the first study to address the implications of probable differences of verbal content meaning across ethnic groups. The results failed to support the hypotheses that Black speakers' truthful statements contained fewer criteria than Whites' statements, or that CBCA training would only enhance truth detection of White speakers. First, in contrast to earlier work, the present study found that, despite the fact that true and false statements contained criteria to differing degrees, the total number of CBCA criteria in an adult's statement was uncorrelated with truthfulness for statements made by Blacks or by Whites. A discriminant analysis showed that CBCA verbal content criteria did, in fact, have different significance for Whites than for Blacks. Certain CBCA criteria were stronger predictors of truth for one ethnic group than for the other. It is suggested that more research is needed before the CBCA is relied upon in criminal investigations.

The applicability of Criteria-Based Content Analysis to Written Statements Given by Members of an Ethnic Minority

The applicability of Criteria-Based Content Analysis to Written Statements Given by Members of an Ethnic Minority

The less travelled road to truth: verbal cues in deception detection in accounts of fabricated and self-experienced events

In evaluating the truthfulness of children's allegations of (sexual) abuse, German forensic experts have focused on qualitative aspects of the content of a witness's statement. Within the overall credibility assessment of a witness's statement, known as statement validity analysis (SVA), they have developed a technique referred to as criterion-based content analysis (CBCA), which utilizes content criteria that supposedly are indicative of the truthfulness of a statement. While first validation studies of CBCA criteria have been undertaken, a theoretical basis of why and under what circumstances deceptive and truthful accounts should differ with respect to these criteria has been wanting. The reality monitoring (RM) approach is proposed as a theoretical basis for discriminating between fabricated and self-experienced events. The present experiment links forensic CBCA credibility criteria to the reality monitoring approach and tests the relative validity of CBCA and RM criteria in discriminating between fabricated and self-experienced video recorded accounts of adult participants. Transcripts rated for the presence of CBCA and RM criteria by trained experts could be classified in an above-chance fashion. On the basis of a factor analysis of CBCA and RM criteria, commonalities and differences between the two approaches are noted. © 1997 John Wiley & Sons, Ltd.

The less travelled road to truth: verbal cues in deception detection in accounts of fabricated and self-experienced events

In evaluating the truthfulness of children's allegations of (sexual) abuse, German forensic experts have focused on qualitative aspects of the content of a witness's statement. Within the overall credibility assessment of a witness's statement, known as statement validity analysis (SVA), they have developed a technique referred to as criterion-based content analysis (CBCA), which utilizes content criteria that supposedly are indicative of the truthfulness of a statement. While first validation studies of CBCA criteria have been undertaken, a theoretical basis of why and under what circumstances deceptive and truthful accounts should differ with respect to these criteria has been wanting. The reality monitoring (RM) approach is proposed as a theoretical basis for discriminating between fabricated and self-experienced events. The present experiment links forensic CBCA credibility criteria to the reality monitoring approach and tests the relative validity of CBCA and RM criteria in discriminating between fabricated and self-experienced video recorded accounts of adult participants. Transcripts rated for the presence of CBCA and RM criteria by trained experts could be classified in an above-chance fashion. On the basis of a factor analysis of CBCA and RM criteria, commonalities and differences between the two approaches are noted. © 1997 John Wiley & Sons, Ltd.

The relationships among interviewer utterance type, CBCA scores and the richness of children's responses

Objectives. The goal of the study was to determine whether the criterion‐based content analysis (CBCA) indicators of credibility were more likely to be elicited by open‐ended interview prompts than by more directive prompts.Methods. Coders independently applied a revised CBCA coding scheme while others rated interviewer utterance types and the length and richness of children's responses in transcripts of 20 forensic interviews of alleged victims of child sexual abuse.Results. There were high correlations between the number of CBCA criteria identified and both the length and richness of the children's utterances. Open‐ended invitations were especially useful in eliciting responses that contained CBCA criteria, as expected.Conclusions. Open‐ended invitations thus appear to elicit both more numerous details and details that are believed to suggest credibility. Implications for the application of CBCA codes to credibility assessment are discussed.

Assessing the credibility of children's allegations of sexual abuse: A survey of recent research

Alarmed by the increasing numbers of alleged incidents of child sexual abuse, forensic psychologists have attempted to learn whether credible and implausible allegations can be discriminated reliably. Most attention has focused on components of Statement Validity Analysis (SVA), particularly Criterion-Based Content Analysis (CBCA). Recent studies have shown that CBCA scores indeed distinguish plausible from implausible accounts, although the precision is still too poor to permit forensic application. Sensitivity may be enhanced by further improvements in the quality of the investigative interviews on which evaluations of credibility are based, and may be additionally enhanced by development and application of techniques like the Validity Checklist, a little studied component of SVA.

The usefulness of the criteria-based content analysis technique in distinguishing between truthful and fabricated allegations: A critical review.

The Criteria-Based Content Analysis (CBCA) technique was developed to distinguish children's truthful from fabricated allegations. Research results suggest some use for the procedure, but many important theoretical and empirical issues remain unresolved, including wide differences in the apparent us

The usefulness of the criteria-based content analysis technique in distinguishing between truthful and fabricated allegations: A critical review.

The usefulness of the criteria-based content analysis technique in distinguishing between truthful and fabricated allegations: A critical review.

Children's Testimony in the Netherlands

The authors first provide a brief overview of Dutch agencies that deal with child abuse and neglect, the approach to child abuse, and the position of children as witnesses in the Netherlands. Next, they present Dutch data on child abuse and neglect and child sexual abuse. They then go on to review studies on statement validity analysis. Finally, they report the results of a study in which they compared the presence of content criteria in the statements of alleged sexual abuse victims in four age groups. The comparison was made using criteria-based content analysis, one of the components of the statement validity analysis procedure.

The cognitive interview and the assessment of the credibility of adults' statements.

The authors examined whether a cognitive interview would negatively affect the accuracy of a criteria-based content analysis (CBCA) in distinguishing truthful and fabricated statements. Adults (N = 59 ) gave a truthful or a fabricated account of a blood-donation episode. They were interviewed with either a cognitive or a structured interview. The number of correct, incorrect, and confabulated details were counted from written transcripts of the accounts. The cognitive interview produced significantly more correct and confabulated details than did the structured interview. The transcripts were further evaluated according to a subset of CBCA criteria plus 6 additional criteria. The content characteristics reliably discriminated truthful and fabricated accounts. No interaction was found between truthfulness and the type of interview, indicating that the cognitive interview did not impair the potential of CBCA in distinguishing truthful and fabricated statements.

Commentary on criteria-based content analysis

Criteria-Based Content Analysis (CBCA) may be viewed as an attempt to construct a standardized procedure for determining the evidentiary value of a testimonial statement. Its theoretical basis is the “Undeutsch Hypothesis” that asserts that truthful accounts differ “significantly and noticeably” from false accounts and that these differences can be defined in terms of explicitly stated criteria. In his interesting and authoritative account of the development of Statement Reality Analysis-of which CBCA forms a part-Professor Udo Undeutsch asserts that: “The scientific assessment of the truthfulness of a testimonial statement rests on two pillars: the interview and the analysis of the obtained statement. Together they form one methodological whole.” (1989, p. 116). Relevant to the first pillar, Lamb, Sternberg, & Esplin (1994) have summarized much recent research on the interviewing of children and, basing themselves on that research and their own clinical experience, have presented a thoughtful set of recommendations for obtaining accurate information from the child utilizing forensically defensible interviewing techniques. These recommendations would receive further support from the extensive review of the research literature on the suggestibility of children provided by Ceci and Bruck (1993a) and by their more recent account of the policy implications of this research as it effects the conduct of interviews and the court testimony of experts related to such interviews (Ceci & Bruck, 1993b). Thus, there is growing acceptance in the field that although children-even pre-school children-can provide informative and credible information regarding abusive situations there is need for careful, systematic, non-suggestive, and well-documented interviews conducted by thoroughly trained and sensitive investigators. Such acceptance would support the first of Undeutsch’s two methodological pillars. The second pillar is the analysis of the obtained statement, which brings us to a consideration of CBCA. As Undeutsch indicated in his 1989 article, the analysis of statements for their credibility is for all of us a part of everyday life. What was unique and innovative for the forensic psychologists practicing in the European courts in the 1950s and

Child sexual abuse allegations: Reliability of criteria-based content analysis.

Statement Validity Analysis (SVA), an assessment system for the credibility of children's allegations of sexual abuse, has recently undergone quantification and empirical testing. Twenty-three videotapes of investigative interviews of confirmed child sexual abuse victims were obtained for an interrater reliability analysis. Each tape was rated by two of four trained raters according to the criteria-based content analysis (CBCA) procedure of the SVA method. The average proportion agreement (.75) was equivalent to the value(.72) reported by Steller (1989), but the chance-corrected reliabilities of the CBCA criteria varied from moderate to low (Maxwell's RE coefficient of agreement range =−.22 to 1.00). The criteria need more explicit, behaviorally anchored definitions, and rater training needs to focus on the identified problem criteria. Potential limitations of interpretation include the small restricted sample, the use of uncontrolled community interviews, and the rating of videotapes rather than transcripts of the interviews.

The effect of training in criteria-based content analysis on the ability to detect deception in adults.

The Criteria-Based Content Analysis (CBCA) technique has been proposed as a way to differentiate truth from falsehood in interviews of children. We studied adults, utilizing 14 of the 19 CBCA criteria. In a 2×2 design, 114 students estimated the truthfulness of the statements of 12 adults; 6 were true and 6 described an invented traumatic personal experience. Subjects viewed a videotape or read a written transcript; half were trained in CBCA and half were not. Trained subjects who saw videotapes performed significantly better than chance and were significantly more accurate than each of the other 3 groups. For the trained subjects, 10 of the 14 CBCA criteria yielded significant differences in the predicted direction between evaluations of truthful and invented statements.