Criteria-Based Content Analysis Research Articles

Criteria-Based Content Analysis: An empirical test of its underlying processes

Criteria-Based Content Analysis (CBCA) is a tool to assess the veracity of written statements, and is used as evidence in criminal courts in several countries in the world. CBCA scores are expected to be higher for truth tellers than for liars. The underlying assumption of CBCA is that (i) lying is cognitively more difficult than truth telling, and (ii) that liars are more concerned with the impression they make on others than truth tellers. However, these assumptions have not been tested to date. In the present experiment 80 participants (undergraduate students) lied or told the truth about an event. Afterwards, they completed a questionnaire measuring “cognitive load” and “tendency to control speech”. The interviews were transcribed and coded by trained CBCA raters. In agreement with CBCA assumptions, (i) truth tellers obtained higher scores than liars, (ii) liars experienced more cognitive load than truth tellers, and (iii) liars tried harder to control their speech. However, cognitive load and speech control were not correlated with CBCA scores in the predicted way.

How does the legal system respond when children with learning difficulties are victimized?

Objective To understand how the Swedish legal system perceives and handles mentally handicapped children who may have been victimized. Method Twenty-two judicial districts in Sweden provided complete files on 39 District Court cases (including the Appeals Court files on 17 of these cases) involving children with learning difficulties or other handicaps as alleged victims of abuse, threat and neglect. The children (25 girls and 14 boys) averaged 11.8 years of age when first allegedly victimized. Sexual abuse was the most frequently alleged crime (33 cases). Court transcripts, court files and expert assessments of the alleged victims’ handicaps and their possible consequences were examined to elucidate the ways in which courts evaluated the credibility of the alleged victims. Results The children's reports of their victimization were expected to have the characteristics emphasized by proponents of Statement Reality Analysis (SRA) and Criterion Based Content Analysis (CBCA) in order to be deemed credible. Expert reports were seldom available or adequate. Because many reports were poorly written or prepared by experts who lacked the necessary skills, courts were left to rely on their own assumptions and knowledge when evaluating children's capacities and credibility. Conclusions Children with learning difficulties or other handicaps were expected to provide the same sort of reports as other children. To minimize the risk that judgments may be based on inaccurate assumptions courts need to require more thorough assessments of children's limitations and their implications. Assessments by competent mental health professionals could inform and strengthen legal decision-making. A standardized procedure that included psycho-diagnostic instruments would allow courts to understand better the abilities, capacities, and behavior of specific handicapped children.

Statement validity assessment: Inter‐rater reliability of criteria‐based content analysis in the mock‐crime paradigm

Purpose. The inter‐rater reliability of criteria‐based content analysis (CBCA) – the main component of statement validity assessment (SVA) – was investigated in a mock‐crime study. The study also addressed the issue of the adequacy of diverse statistical indices of reliability. Furthermore, CBCA's effectiveness in distinguishing between true and false statements was analysed.Methods. Three raters were trained in CBCA. Subsequently, they analysed transcripts of 102 statements referring to a simulated theft of money. Some of the statements were based on experience and some were confabulated. The raters used 4‐point scales, respectively, to judge the degree to which 18 of the 19 CBCA criteria were fulfilled in each statement.Results. The analysis of rater judgment distributions revealed that, with judgments of individual raters varying only slightly across transcripts, the weighted kappa coefficient, the product‐moment correlation, and the intra‐class correlation were inadequate indices of reliability. The Finn‐coefficient and percentage agreement, which were calculated as indices independent of rater judgment distributions, were sufficiently high with respect to 17 of the 18 assessed criteria. CBCA differentiated significantly between truthful and fabricated accounts.Conclusions. The inter‐rater reliability of CBCA achieved in the present study was satisfactory both, if considered absolutely, and as compared with other empirical findings. This suggests that CBCA can be utilized in the mock‐crime paradigm with a sufficient degree of reliability.

The detection of deception with the reality monitoring approach: a review of the empirical evidence

One of the verbal approaches to the detection of deceit is based on research on human memory that tries to identify the characteristics that differentiate between internal and external memories (reality monitoring). This approach has attempted to extrapolate the contributions of reality monitoring (RM) research to the deception area. In this paper, we have attempted to review all available studies conducted in several countries in order to yield some general conclusions concerning the discriminative power of this approach. Regarding individual criteria, the empirical results are not very encouraging: few criteria discriminate significantly across studies, and there are several variables that moderate their effect. Some of the contradictory findings may have emerged because of differences in the operationalizations and procedures used across individual studies. However, more promising results have been reported in recent studies, and the approach as a whole appears to discriminate above chance level, reaching accuracy rates that are similar to those of criteria-based content analysis (CBCA). Some suggestions for future research are made.

Criteria-Based Content Analysis: A Qualitative Review of the First 37 Studies.

Statement Validity Assessment (SVA) is used to assess the veracity of child witnesses’ testimony in trials for sexual offences. The author reviewed the available SVA research. Issues addressed include the accuracy of Criteria-Based Content Analysis (CBCA; part of SVA), interrater agreement between CBCA coders, frequency of occurrence of CBCA criteria in statements, the correlations between CBCA scores and (i) interviewer’s style and (ii) interviewee’s age and social and verbal skills, and issues regarding the Validity Checklist (another part of SVA). Implications for the use of SVA assessments in criminal courts are discussed. It is argued that SVA evaluations are not accurate enough to be admitted as expert scientific evidence in criminal courts but might be useful in police investigations.

The effects of training professional groups and lay persons to use criteria‐based content analysis to detect deception

AbstractThis experiment was designed to assess, for the first time, the effects of training police officers, social workers and students in Criteria‐Based Content Analysis (CBCA) in an attempt to increase lie detection accuracy. A within‐subjects design was implemented. Participants rated the truthfulness of a maximum of four statements before training in CBCA and rated the truthfulness of a different set of four statements after training. The raters were only exposed to the written transcripts of the communicators. Two thirds of the statements utilized were truthful and one third were based on fabrications. Before training, there were no significant differences in detection accuracy between the police officers (66% accuracy), the social workers (72% accuracy) and the students (56% accuracy). After training, the social workers were 77% accurate and significantly more accurate than the police officers (55%) and the students (61%). However, none of the three groups of raters significantly improved their lie detection accuracy after training, in fact, the police officers performed significantly poorer. Overall, police officers were significantly more confident than social workers and lay persons regardless of accuracy. Further, participants were most confident when labelling a statement truthful regardless of whether or not this was the correct decision. Copyright © 2004 John Wiley & Sons, Ltd.

Assessing children's statements: the impact of a repeated experience on CBCA and RM ratings

AbstractThis study examined the extent to which the Criteria‐based Content Analysis (CBCA) technique and the Reality Monitoring (RM) technique were affected by the number of times children had experienced or imagined an event. Children (age 10–13, N =87 ) participated in an experiment where half the sample experienced a health examination (either one or four times), and the other half imagined (either one or four times) that they took part in a health examination. One week after the final occasion, the children were interviewed. The results showed that RM was sensitive to both the authenticity of the statements (increased presence of the criteria for real events) and whether the event had been repeatedly experienced/imagined (increased presence of the criteria for the repeated actions). The CBCA did not successfully distinguish the real from the imagined. Copyright © 2004 John Wiley & Sons, Ltd.

Telling a good story: Fantasy proneness and the quality of fabricated memories

In two studies, we examined whether fantasy prone people are superior storytellers. In study 1, participants high or low on fantasy proneness ( N=25) were instructed to fabricate a memory about an aversive childhood event. Independent judges rated stories of high fantasy prones as more emotional, more plausible, and richer in Criteria Based Content Analysis (CBCA) elements than those of low fantasy prone people. In study 2, high and low fantasy prone participants ( N=38) wrote down a true and a fabricated story about a negative event. Although the stories of the two groups did not differ in terms of emotionality or plausibility ratings, both true and fabricated stories of high fantasy prones were rated as being richer in CBCA elements than those of low fantasy prone controls. Taken together, the two studies show that fantasy proneness affects CBCA ratings. Forensic experts employing this tool would be well advised to take into account the potential confounding influence that fantasy proneness might have on CBCA evaluations.

Detecting deception in children: event familiarity affects criterion-based content analysis ratings.

Statement Validity Assessment (SVA) is a comprehensive credibility assessment system, with the Criterion-Based Content Analysis (CBCA) as a core component. Worldwide, the CBCA is reported to be the most widely used veracity assessment instrument. We tested and confirmed the hypothesis that CBCA scores are affected by event familiarity; descriptions of familiar events are more likely to be judged true than are descriptions of unfamiliar events. CBCA scores were applied to transcripts of 114 children who recalled a routine medical procedure (control) or a traumatic medical procedure that they had experienced one time (relatively unfamiliar) or multiple times (relatively familiar). CBCA scores were higher for children in the relatively familiar than the relatively unfamiliar condition, and CBCA scores were significantly correlated with age. Results raise serious questions regarding the forensic suitability of the CBCA for assessing the veracity of children's accounts.

Detecting Deceit Via Analyses of Verbal and Nonverbal Behavior in Children and Adults

This experiment examined children's and undergraduates' verbal and nonverbal deceptive behavior, and the extent to which their truths and lies could be correctly classified by paying attention to these responses. Participants (N= 196) aged 5–6, 10–11, and 14–15, as well as university undergraduates, participated in an erasing the blackboard event, and told the truth or lied about the event afterwards. Nonverbal and verbal responses were coded, the latter with Criteria-Based Content Analysis and Reality Monitoring. Although children and undergraduates demonstrated different behaviors (for example, the children obtained lower CBCA scores and made more movements), actual cues to deceit were remarkably similar across different age groups (for example, both 5–6-year-olds and undergraduates obtained lower CBCA scores and made fewer movements while lying). A combination of verbal and nonverbal lie detection methods resulted in more correct classifications of liars and truth tellers than the verbal and nonverbal lie detection methods individually, with the combined method obtaining hit rates as high as 88%.

Detecting Deceit Via Analyses of Verbal and Nonverbal Behavior in Children and Adults

This experiment examined children's and undergraduates' verbal and nonverbal deceptive behavior, and the extent to which their truths and lies could be correctly classified by paying attention to these responses. Participants (N= 196) aged 5–6, 10–11, and 14–15, as well as university undergraduates, participated in an erasing the blackboard event, and told the truth or lied about the event afterwards. Nonverbal and verbal responses were coded, the latter with Criteria-Based Content Analysis and Reality Monitoring. Although children and undergraduates demonstrated different behaviors (for example, the children obtained lower CBCA scores and made more movements), actual cues to deceit were remarkably similar across different age groups (for example, both 5–6-year-olds and undergraduates obtained lower CBCA scores and made fewer movements while lying). A combination of verbal and nonverbal lie detection methods resulted in more correct classifications of liars and truth tellers than the verbal and nonverbal lie detection methods individually, with the combined method obtaining hit rates as high as 88%.

Let me inform you how to tell a convincing story: CBCA and reality monitoring scores as a function of age, coaching, and deception.

The first aim of this experiment was to examine whether being informed about a method of detecting deception called Criteria-Based Content Analysis (CBCA) would increase participants' CBCA scores when deceptive so that they might then be classified as truthful. The second aim was to investigate whether Reality Monitoring could be used as an alternative tool for verbal lie detection. The third aim was to examine whether participants' social skills (social anxiety, self monitoring and social adroitness) affected their CBCA scores. Participants (aged 6-8, 11-12, 14-15, and undergraduates) participated in a rubbing the blackboard event. In a subsequent interview they told the truth or lied about the event, after they were or were not taught some CBCA criteria. Truth tellers obtained higher CBCA scores than liars, and those who were informed about CBCA obtained higher scores than those who were not, except for the 6-8-year-olds. CBCA scores were also significantly correlated with social skills. Finally, Reality Monitoring was a useful alternative to CBCA for distinguishing between liars and truth tellers.

Age differences in Criteria-Based Content Analysis scores in typical child sexual abuse interviews

Criteria-Based Content Analysis (CBCA) by Steller and Koehnken [ Psychological methods in criminal investigation and evidence (1989) 217. New York: Springer.] is used to help determine the veracity of allegations of child sexual abuse by reference to criteria theorized to be present more often in true statements. Studies generally show that older children meet more criteria, indicating potential problems in applying CBCA equally well across the age range. The present study examined 104 actual child sexual abuse interviews with children age 2–14 years ( M=6.5). Two trained raters, blind to the children's ages, evaluated each interview for the original 19 CBCA criteria. Data were coded using various methods, some of which gave children credit for reporting criteria spontaneously or when prompted by the interviewer. For all coding methods, age was significantly correlated with 12 or more criteria, and with total CBCA scores. The youngest children never reported seven criteria, even when interviewer prompting was taken into account. Therefore, many of the criteria may not be useful in typical interviews with children across a wide age range, because they either are rarely present or they are highly correlated with age.

A case study of child sexual false allegation

Purpose: The objectives of the case study reported in this article were twofold. The first objective was to follow the path by which a naive suggestion made in the course of a mother-child conversation was transformed into an allegation of severe sexual abuse. The second objective was to analyze the child’s interview scientifically and explore the limitations of scientific tools for detecting implausible allegations. Methods: Independent case facts were collected and analyzed to determine whether the event described by the child was likely to have happened. The credibility of the child’s account was assessed using Criterion-Based Content Analysis and the information provided in both the “implausible” and “corrected” statements was compared to quantify the fabricated details in the implausible statement. Results: The event described by the child was “very unlikely to have happened” but the credibility assessment failed to detect its implausibility. Comparison of the two statements revealed that the child did fabricate central details but incorporated them into a description of an event she really experienced, and most of the information provided was truthful. Conclusions: The pressure to conform to suggestions can be irresistible, inducing some children to make false allegations of severe sexual abuse. Scientific tools designed for credibility assessment are limited and may fail to detect implausible statements especially when they incorporate information about genuinely experienced events.

Stereotypical Verbal and Nonverbal Responses While Deceiving Others

This experiment examined the impact of public self-consciousness and acting ability on processes and stereotypical responses during deception. Seventy-three nursing students were videotaped while (a) telling the truth and (b) lying about a theft they had observed. A variety of cues were coded, including criteria-based content analysis (CBCA) and reality monitoring (RM), in which high scores indicated honest responding. Compared with truth tellers, liars waited longer before giving an answer, spoke faster, made more speech hesitations, showed fewer illustrators, and showed lower CBCA and RM scores. Public self-consciousness was positively correlated with trying to control behavior but negatively correlated with RM scores. Ability to act was negatively correlated with RM scores, with showing stereotypical deceptive facial behavior (gaze aversion and smiling), with having to think hard while lying, and with being nervous while lying.

Content credibility of accounts derived from live and video presentations

Purpose. The present experiment examined the success rate of well‐trained raters in judging the truthfulness of witness statements. Statements from children aged 7‐8 years, children aged 10‐11 years and adults were rated for the presence of 15 of the 19 original Criteria‐Based Content Analysis (CBCA) criteria (Steller & Köhnken, 1989).Method. This experiment comprised a 3 (age of participant) × 3 (nature of account) design. One‐third of the participants took part in a photography session, one‐third watched a video recording of a peer taking part in a photography session, and one‐third were party to a verbal description regarding the photography session but neither participated in it themselves nor saw a video recording. The first two groups then gave truthful accounts of their experiences in a subsequent interview. However, the third group was asked, by the experimenter, to lie during the interview and to try to convince the interviewers that they also had taken part in a photography session.Results. The CBCA criteria discriminated between truthful accounts based on actual involvement and fabricated accounts. However, the criteria did not significantly discriminate between those truthful accounts based on watching a video recording and fabricated accounts. Interestingly, there was a significant difference between the truthful accounts based on participation and the truthful accounts based on observation. Further, CBCA was equally effective in discriminating between participative truthful accounts and fabricated accounts for all three age groups.Conclusions. CBCA was found to be a very useful tool in discriminating between truthful accounts based on direct experience of an event and fabricated accounts (overall hit rate, 70%). Also, CBCA was found to discriminate between truthful acounts based on actual experience and truthful accounts based on watching a video. This latter finding has implications for the real world when children and adults use information from television programmes and videos on which to base false allegations. Finally, CBCA was found to be equally effective for use with accounts from adults as well as children.

Assessment of child witness statements using criteria-based content analysis (CBCA): The effects of age, verbal ability, and interviewer's emotional style

The goal of the study was to determine what effects age, verbal ability (assessed with WISC-R Vocabulary), and interviewer's emotional style would have on the occurrence of Criteria-Based Content Analysis (CBCA) criteria. Children (N = 68) from three different age-groups (7–8, 10–11, and 13–14 years) made both a true and a false statement concerning a mildly traumatic event. The correct classification rate was 66%. Age and verbal ability were found to increase the occurrence of some of the CBCA criteria irrespective of the truthfulness of the statements. Also, different criteria differentiated between true and false statements in different age groups. Interviewer behaviour also affected the occurrence of the criteria. It is concluded that the CBCA should not be used in court proceedings in its present form.

Criteria Based Content Analysis: The Less Scientific Road to Truth

Criteria Based Content Analysis (CBCA) is a forensic tool that aims atdistinguishing true from false statements made by crime victims. As such,CBCA and similar techniques are regularly employed by psychologicalexpert-witnesses in court. However, it could be argued that the diagnosticaccuracy of CBCA is moderate, and too low for application in the forensiccontext. The current article discusses problems that arise if conclusionsabout credibility of testimonies based on CBCA are presented in court. Afundamental, yet so far hardly addressed flaw is that CBCA suffers from aninherent truth bias. That is, the underlying notion that credibility isexamined by means of searching for signs supporting truthfulness isscientifically less valid than the approach in which credibility isestablished by focusing on alternative explanations and signs of deceit.

Detecting Deceit via Analysis of Verbal and Nonverbal Behavior

We examined the hypotheses that (1) a systematic analysis of nonverbal behavior could be useful in the detection of deceit and (2) that lie detection would be most accurate if both verbal and nonverbal indicators of deception are taken into account. Seventy-three nursing students participated in a study about "telling lies" and either told the truth or lied about a film they had just seen. The interviews were videotaped and audiotaped, and the nonverbal behavior (NVB) and speech content of the liars and truth tellers were analyzed, the latter with the Criteria-Based Content Analysis technique (CBCA) and the Reality Monitoring technique (RM). Results revealed several nonverbal and verbal indicators of deception. On the basis of nonverbal behavior alone, 78% of the lies and truths could be correctly classi- fied. An even higher percentage could be correctly classified when all three detec- tion techniques (i.e., NVB, CBCA, RM) were taken into account.

Interviewer Questions and Content Analysis of Children's Statements of Sexual Abuse

Effects of forensic interview techniques on the production of free-narrative and Criteria-Based Content Analysis (CBCA) criteria were assessed in police interviews with 48 children (ages 3 to 16) who alleged they had been sexually abused. These allegations were later categorized as confirmed (n = 35) or highly doubtful (n = 13) based on information obtained independent of the statements. Two raters independently coded all interviewer utterances and children's responses, and four other raters evaluated the transcripts for the presence of CBCA content criteria. As predicted, open questions yielded more free narrative and CBCA criteria than other types of questions. Confirmed statements of abuse contained more CBCA criteria than highly doubtful statements, and statements made by older children contained more CBCA criteria than those by younger children. The results support the use of open questions for eliciting free narrative and the use of CBCA to assess the validity of children's allegations of sexual abuse.