Criteria-Based Content Analysis Criteria Research Articles

성폭력 피해 진술의 신빙성 평가를 위한 준거기반 내용분석(CBCA)의 준거 재정립 연구

Criteria-Based Content Analysis (CBCA) can effectively discriminate the veracity of child statement in sexual abuse cases. Therefore, CBCA is widely used to evaluate the credibility in cases of suspected child sexual abuse. However, the validity of an CBCA procedure is limited by its low inter-rater reliability. In order to identify the cause of low inter-rater reliability and solve this problem, this study proposed a method for re-establishing the definition of CBCA criteria and the evaluation principle. Hence, 13 domestic papers using CBCA and CBCA manuals of the police and prosecutors were reviewed. Accordingly, problems such as names of CBCA critera, ambiguous operational definitions, and lack of clear evaluation principle were identified. Based on this result, we proposed a specific operational definition and evaluation principle along with a unified name. Trained for 10hours evaluators analyzed transcripts of 84 sexual assault victim statements using reestablished CBCA criteria. As a result, it was found that the inter-rater reliability (ICC) for each criterion was .697-.949, which was satisfactory. Finally, the practical application of the redefined concept in this study and the future development direction were discussed.

Criteria‐Based Content Analysis in Child Sexual Abuse Cases: A Cross‐Cultural Perspective

Statement validity assessment (SVA) was created to evaluate children's accounts in alleged sexual abuse cases and it is the most popular instrument currently used to assess the veracity of their verbal declarations. The core of this method lies in the application of criteria‐based content analysis (CBCA) to the content of the provided account in an attempt to determine its veracity. Despite the popularity and importance of this instrument in legal settings in a variety of countries, there is little evidence that the cultural and ethnic background of the alleged child victims is taken into account. The current paper proposes a theoretical overview of the culturally sensitive aspects of cognition, memory, motivation and self‐identity as reflected in each of the two general categories of the CBCA criteria (i.e. cognitive and motivational). Our investigation revealed that several criteria are likely to be sensitive to cultural differences, therefore further direct empirical proof is needed to improve validation of SVA use in culturally diverse child sexual abuse cases. In the interim, we would encourage the use of culturally sensitive practices and policies among practitioners using CBCA.‘There is little evidence that the cultural and ethnic background of the alleged child victims is taken into account’Key Practitioner Messages The cultural or ethnic background of children influences their accounts; thus, it should be expected that certain CBCA criteria can be affected by these factors. Professionals using this method are recommended to take a more ‘culturally sensitive’ approach, which involves learning more about cultural differences and monitoring their own expectations and biases. More empirical research is needed in order to validate which CBCA criteria are suitable for use in a variety of cultures and cross culturally.

Optimizing CBCA and RM research: recommendations for analyzing and reporting data on content cues to deception

ABSTRACT For more than a century, verbal content cues to deception have been investigated to assess the credibility of statements in judicial contexts. Among the many cues investigated, Criteria-based Content Analysis (CBCA) and criteria based on the reality monitoring (RM) approach have been most prominent. However, research with these cues used as ‘tools’ has not fully exploited their potential. We critically discuss statistical approaches used in past research and recommend a series of 12 principles or guidelines researchers should follow to design, analyze and report future studies on detecting deception with verbal content cues. To illustrate some of these points, we present analyses from two separate studies: A quasi-experiment in a field setting conducted with adults with intellectual disabilities who truthfully or deceptively described a negative autobiographical event to an interviewer, and a large-scale simulation study where adults wrote an account of either an experienced or an invented significant life event. Accounts in both studies were rated with CBCA and RM criteria, as well as by ‘naive’ raters. The guidelines should help to increase the quality and transparency of research in this area.

Verbal Baselining: Within-Subject Consistency of CBCA Scores across Different Truthful and Fabricated Accounts

Statement Validity Assessment (SVA) proposes that baseline statements on different events can serve as a within-subject measure of a witness’ individual verbal capabilities when evaluating scores from Criteria-based Content Analysis (CBCA). This assumes that CBCA scores will generally be consistent across two accounts by the same witness. We present a first pilot study on this assumption. In two sessions, we asked 29 participants to produce one experience-based and one fabricated baseline account as well as one experience-based and one fabricated target account (each on different events), resulting in a total of 116 accounts. We hypothesized at least moderate correlations between target and baseline indicating a consistency across both experience-based and fabricated CBCA scores, and that fabricated CBCA scores would be more consistent because truth-telling has to consider random event characteristics, whereas lies must be constructed completely by the individual witness. Results showed that differences in correlations between experience-based CBCA scores and between fabricated CBCA scores took the predicted direction (cexperience-based = .44 versus cfabricated =.61) but this difference was not statistically significant. As predicted, a subgroup of event-related CBCA criteria were significantly less consistent than CBCA total scores, but only in experience-based accounts. The discussion considers methodological issues regarding the usage of total CBCA scores and whether to measure consistency with correlation coefficients. It is concluded that more studies are needed with larger samples.

"Validity of content-based techniques to distinguish true and fabricated statements: A meta-analysis": Correction to Oberlader et al. (2016).

Reports an error in "Validity of content-based techniques to distinguish true and fabricated statements: A meta-analysis" by Verena A. Oberlader, Christoph Naefgen, Judith Koppehele-Gossel, Laura Quinten, Rainer Banse and Alexander F. Schmidt (Law and Human Behavior, 2016[Aug], Vol 40[4], 440-457). During an update of this meta-analysis it became apparent that one study was erroneously entered twice. The reduced data of k = 55 studies was reanalyzed after excluding the unpublished study by Scheinberger (1993). The corrected overall effect size changed at the second decimal: d = 1.01 (95% CI [0.77, 1.25], Q = 409.73, p < .001, I² = 92.21) and g = 0.98 (95% CI [0.75, 1.22], Q = 395.49, p < .001, I² = 91.71%), k = 55, N = 3,399. This small numerical deviation is negligible and does not change the interpretation of the results. Similarly, results for categorial moderators changed only numerically but not in terms of their statistical significance or direction (see revised Table 4). In the original meta-analysis based on k = 56 studies, unpublished studies had a larger effect size than published studies. Based on k = 55 studies, this difference vanished. Results for continuous moderators also changed only numerically: Q-tests with mixed-effects models still revealed that year of publication (Q = 0.06, p = .807, k = 55) as well as gender ratio in the sample (Q = 1.28, p =.259, k = 43) had no statistically significant influence on effect size. In sum, based on the numerically corrected values our implications for practical advices and boundary conditions for the use of content-based techniques in credibility assessment are still valid. The online version of this article has been corrected. (The following abstract of the original article appeared in record 2016-21973-001.) Within the scope of judicial decisions, approaches to distinguish between true and fabricated statements have been of particular importance since ancient times. Although methods focusing on "prototypical" deceptive behavior (e.g., psychophysiological phenomena, nonverbal cues) have largely been rejected with regard to validity, content-based techniques constitute a promising approach and are well established within the applied forensic context. The basic idea of this approach is that experience-based and nonexperience-based statements differ in their content-related quality. In order to test the validity of the most prominent content-based techniques, criteria-based content analysis (CBCA) and reality monitoring (RM), we conducted a comprehensive meta-analysis on English- and German-language studies. Based on a variety of decision criteria, 55 studies were included revealing an overall effect size of g = 0.98 (95% confidence interval [0.75, 1.22], Q = 395.49, p < .001, I² = 91.71%, N = 3,399). There was no significant difference in the effectiveness of CBCA and RM. Additionally, we investigated a number of moderator variables, such as characteristics of participants, statements, and judgment procedures, as well as general study characteristics. Results showed that the application of all CBCA criteria outperformed any incomplete CBCA criteria set. Furthermore, statement classification based on discriminant functions revealed higher discrimination rates than decisions based on sum scores. All results are discussed in terms of their significance for future research (e.g., developing standardized decision rules) and practical application (e.g., user training, applying complete criteria set). (PsycINFO Database Record (c) 2019 APA, all rights reserved).

The Strategic Meaning of CBCA Criteria From the Perspective of Deceivers.

In 2014, Volbert and Steller introduced a revised model of Criteria-Based Content Analysis (CBCA) that grouped a modified set of content criteria in closer reference to their assumed latent processes, resulting in three dimensions of memory-related, script-deviant and strategy-based criteria. In this model, it is assumed that deceivers try to integrate memory-related criteria—but will not be as good as truth tellers in achieving this—whereas out of strategic considerations they will avoid the expression of the other criteria. The aim of the current study was to test this assumption. A vignette was presented via an online-questionnaire to inquire how participants (n = 135) rate the strategic value of CBCA criteria on a five-point scale. One-sample t-tests showed that participants attribute positive strategic value to most memory-related criteria and negative value to the remaining criteria, except for the criteria self-deprecation and pardoning the perpetrator. Overall, our results corroborated the model's suitability in distinguishing different groups of criteria—some which liars are inclined to integrate and others which liars intend to avoid—and in this way provide useful hints for forensic practitioners in appraising the criteria' diagnostic value.

Can credibility criteria be assessed reliably? A meta-analysis of criteria-based content analysis.

This meta-analysis synthesizes research on interrater reliability of Criteria-Based Content Analysis (CBCA). CBCA is an important component of Statement Validity Assessment (SVA), a forensic procedure used in many countries to evaluate whether statements (e.g., of sexual abuse) are based on experienced or fabricated events. CBCA contains 19 verbal content criteria, which are frequently adapted for research on detecting deception. A total of k = 82 hypothesis tests revealed acceptable interrater reliabilities for most CBCA criteria, as measured with various indices (except Cohen's kappa). However, results were largely heterogeneous, necessitating moderator analyses. Blocking analyses and meta-regression analyses on Pearson's r resulted in significant moderators for research paradigm, intensity of rater training, type of rating scale used, and the frequency of occurrence (base rates) for some CBCA criteria. The use of CBCA summary scores is discouraged. Implications for research vs. field settings, for future research and for forensic practice in the United States and Europe are discussed. (PsycINFO Database Record

Validity of content-based techniques to distinguish true and fabricated statements: A meta-analysis.

Within the scope of judicial decisions, approaches to distinguish between true and fabricated statements have been of particular importance since ancient times. Although methods focusing on "prototypical" deceptive behavior (e.g., psychophysiological phenomena, nonverbal cues) have largely been rejected with regard to validity, content-based techniques constitute a promising approach and are well established within the applied forensic context. The basic idea of this approach is that experience-based and nonexperience-based statements differ in their content-related quality. In order to test the validity of the most prominent content-based techniques, criteria-based content analysis (CBCA) and reality monitoring (RM), we conducted a comprehensive meta-analysis on English- and German-language studies. Based on a variety of decision criteria, 56 studies were included revealing an overall effect size of g = 1.03 (95% confidence interval [0.78, 1.27], Q = 420.06, p < .001, I2 = 92.48%, N = 3,429). There was no significant difference in the effectiveness of CBCA and RM. Additionally, we investigated a number of moderator variables, such as characteristics of participants, statements, and judgment procedures, as well as general study characteristics. Results showed that the application of all CBCA criteria outperformed any incomplete CBCA criteria set. Furthermore, statement classification based on discriminant functions revealed higher discrimination rates than decisions based on sum scores. Finally, unpublished studies showed higher effect sizes than studies published in peer-reviewed journals. All results are discussed in terms of their significance for future research (e.g., developing standardized decision rules) and practical application (e.g., user training, applying complete criteria set). (PsycINFO Database Record

Inter-rater Reliability of Criteria-Based Content Analysis of Children's Statements of Abuse.

The evaluation of children's statements of sexual abuse cases in forensic cases is critically important and must and reliable. Criteria-based content analysis (CBCA) is the main component of the statement validity assessment (SVA), which is the most frequently used approach in this setting. This study investigated the inter-rater reliability (IRR) of CBCA in a forensic context. Three independent raters evaluated the transcripts of 95 statements of sexual abuse. IRR was calculated for each criterion, total score, and overall evaluation. The IRR was variable for the criteria, with several being unsatisfactory. But high IRR was found for the total CBCA scores (Kendall's W=0.84) and for overall evaluation (Kendall's W=0.65). Despite some shortcomings, SVA remains a robust method to be used in the comprehensive evaluation of children's statements of sexual abuse in the forensic setting. However, the low IRR of some CBCA criteria could justify some technical improvements.

Detecting truth in suspect interviews: the effect of use of evidence (early and gradual) and time delay on Criteria-Based Content Analysis, Reality Monitoring and inconsistency within suspect statements

The strategic use of evidence in interviews with suspects has been shown to increase the ability of interviewers to accurately and consistently distinguish truthful from deceptive accounts. The present study considers the effect of early and gradual revelation of evidence by the interviewer, and the effect of shorter and longer delay on the verbal quality of truth-teller and liar statements within a mock crime paradigm. It was hypothesised that gradual disclosure of evidence (1) in terms of inconsistencies (a) within statements and (b) between statements and such evidence and (2) of the criteria of Criteria-Based Content Analysis (CBCA) and of Reality Monitoring (RM) would emphasise differences in the verbal quality of truth-teller and liar statements. Forty-two high school students took part in the study. The use of statement-evidence and within-statement inconsistency appears to be a robust cue to deception across interview style and delay. This indicates that gradual disclosure in interviews may increase interviewer accuracy in veracity decisions by eliciting statement inconsistencies. However, gradual revelation and delay affected the ability of CBCA and RM criteria to distinguish the veracity of suspect statements.

Оценка достоверности свидетельских показаний несовершеннолетних

The article presents a review of the English language publications concerning the history and the current state of differential psychological assessment of validity of testimonies produced by child and adolescent victims of crimes. The topicality of the problem in Russia is high due to the tendency of Russian specialists to use methodical means and instruments developed abroad in this sphere for forensic assessments of witness testimony veracity. A system of Statement Validity Analysis (SVA) by means of Criteria-Based Content Analysis (CBCA) and Validity Checklist is described. The results of laboratory and field studies of validity of CBCA criteria on the basis of child and adult witnesses are discussed. The data display a good differentiating capacity of the method, however, a high level of error probability. The researchers recommend implementation of SVA in the criminal investigation process, but not in the forensic assessment. New perspective developments in the field of methods for differentiation of witness statements based on the real experience and fictional are noted. The conclusion is drawn that empirical studies and a special work for adaptation and development of new approaches should precede their implementation into Russian criminal investigation and forensic assessment practice

Validity of Criteria-Based Content Analysis (CBCA) at trial in free-narrative interviews

Objective The reliability of child witness testimony in sexual abuse cases is often controversial, and few tools are available. Criteria-Based Content Analysis (CBCA) is a widely used instrument for evaluating psychological credibility in cases of suspected child sexual abuse. Only few studies have evaluated CBCA scores in children suspected of being sexually abused. We designed this study to investigate the reliability of CBCA in discriminating allegations of child sexual abuse during court hearings, by comparing CBCA results with the court's final, unappealable sentence. We then investigated whether CBCA scores correlated with age, and whether some criteria were better than others in distinguishing cases of confirmed and unconfirmed abuse. Methods From a pool of 487 child sexual abuse cases, confirmed and unconfirmed cases were selected using various criteria including child IQ ≥ 70, agreement between the final trial outcome and the opinion of 3 experts, presence of at least 1 independent validating informative component in cases of confirmed abuse, and absence of suggestive questions during the child's testimonies. This screening yielded a study sample of 60 confirmed and 49 unconfirmed cases. The 14 item version of CBCA was applied to child witness testimony by 2 expert raters. Results Of the 14 criteria tested, 12 achieved satisfactory inter-rater agreement (Maxwell's Random Error). Analyses of covariance, with case group (confirmed vs. unconfirmed) and gender as independent variables and age as a covariate, showed no main effect of gender. Analyses of the interaction showed that the simple effects of abuse were significant in both sex. Nine CBCA criteria were satisfied more often among confirmed than unconfirmed cases; seven criteria increased with age. Conclusion CBCA scores distinguish between confirmed and unconfirmed cases. The criteria that distinguish best between the 2 groups are Quantity of Details, Interactions, and Subjective Experience. CBCA scores correlate positively with age, and independently from abuse; all the criteria test except 2 (Unusual Details and Misunderstood Details) increase with age. The agreement rate could be increased by merging criteria Unusual and Superfluous details that achieve a low inter-rater agreement when investigated separately. Practice implication Given its ability to distinguish between confirmed and unconfirmed cases of suspected child abuse, the CBCA could be a useful tool for expert opinion. Because our strict selection criteria make it difficult to generalize our results, further studies should investigate whether the CBCA is equally useful in the cases we excluded from our study (for example mental retardation).

Offenders’ uncoerced false confessions: A new application of statement analysis?

Purpose. Deception detection research has mainly studied denials and distortions given by students. This study examined true and false confessions as told by offenders. It was hypothesized that the statement analytic techniques Criteria‐Based Content Analysis (CBCA) and Reality Monitoring (RM) would discriminate truths and lies.Methods. Truthful and deceptive confessions to crime were given by 30 offenders (both women and men) in a within‐subject design. The participants were in prison at the time of data collection, and told the truth about a crime they had committed and been sentenced for. In addition, they made up a lie about a different crime after a few minutes of preparation. The transcribed statements were scored for CBCA and RM criteria.Results. Results showed that neither total CBCA nor total RM scores differentiated between lies and truths. Some individual CBCA criteria, however, showed differences: more self‐deprecations and doubts about own testimony in the told lies, and more unexpected complications in the truths.Conclusions. The results are discussed in relation to statement analysis of offenders’ accounts, individual CBCA criteria, as well as the development of criminal experience and familiarity with the event and setting. Implications for triers of fact and suggestions for future research are considered.

Careful calculation or a leap of faith? A field study of the translation of CBCA ratings to final credibility judgements

This field experiment investigated the influence of Criteria-Based Content Analysis (CBCA) ratings on ultimate decision accuracy regarding the credibility of children's statements of sexual abuse. Following a selection procedure, based on case facts independent of statement quality, 21 truthful accounts and 10 fabricated accounts of 6- to 17-year olds were analysed. Two experts rated the presence of the CBCA criteria and made overall credibility judgements for each statement. Rater one achieved an overall hit rate of 84% (95% for truthful statements and 60% for fabricated statements) and rater two a hit rate of 81% (81% for both truthful and fabricated statements) but the raters did not always agree. The CBCA criteria appeared more often in the truthful statements compared to the fabricated statements. Additional factors that influenced raters' credibility judgements, besides CBCA scores, are discussed.

Statement validity assessment: Inter‐rater reliability of criteria‐based content analysis in the mock‐crime paradigm

Purpose. The inter‐rater reliability of criteria‐based content analysis (CBCA) – the main component of statement validity assessment (SVA) – was investigated in a mock‐crime study. The study also addressed the issue of the adequacy of diverse statistical indices of reliability. Furthermore, CBCA's effectiveness in distinguishing between true and false statements was analysed.Methods. Three raters were trained in CBCA. Subsequently, they analysed transcripts of 102 statements referring to a simulated theft of money. Some of the statements were based on experience and some were confabulated. The raters used 4‐point scales, respectively, to judge the degree to which 18 of the 19 CBCA criteria were fulfilled in each statement.Results. The analysis of rater judgment distributions revealed that, with judgments of individual raters varying only slightly across transcripts, the weighted kappa coefficient, the product‐moment correlation, and the intra‐class correlation were inadequate indices of reliability. The Finn‐coefficient and percentage agreement, which were calculated as indices independent of rater judgment distributions, were sufficiently high with respect to 17 of the 18 assessed criteria. CBCA differentiated significantly between truthful and fabricated accounts.Conclusions. The inter‐rater reliability of CBCA achieved in the present study was satisfactory both, if considered absolutely, and as compared with other empirical findings. This suggests that CBCA can be utilized in the mock‐crime paradigm with a sufficient degree of reliability.

Children report suggested events even when interviewed in a non‐suggestive manner: what are its implications for credibility assessment?

AbstractAn important question in the legal context is how suggested statements about fictitious events develop over a course of various interviews. Sixty‐seven first‐graders were interviewed four times about one real and one fictitious event, applying various suggestive techniques. A fifth, non‐suggestive, interview was conducted by blind experts. Over the course of the interviews there was a considerable increase in assents to the fictitious events. Moreover, few significant differences in criteria‐based content analysis‐criteria were revealed between true and fictitious statements in the fifth interview. Accordingly, experts had difficulties in discriminating between true and suggested accounts. Furthermore, several children were convinced of the truth of their reports, even after having been partially debriefed in a sixth interview. Implications for statement validity assessment are discussed. Copyright © 2004 John Wiley &amp; Sons, Ltd.

Age differences in Criteria-Based Content Analysis scores in typical child sexual abuse interviews

Criteria-Based Content Analysis (CBCA) by Steller and Koehnken [ Psychological methods in criminal investigation and evidence (1989) 217. New York: Springer.] is used to help determine the veracity of allegations of child sexual abuse by reference to criteria theorized to be present more often in true statements. Studies generally show that older children meet more criteria, indicating potential problems in applying CBCA equally well across the age range. The present study examined 104 actual child sexual abuse interviews with children age 2–14 years ( M=6.5). Two trained raters, blind to the children's ages, evaluated each interview for the original 19 CBCA criteria. Data were coded using various methods, some of which gave children credit for reporting criteria spontaneously or when prompted by the interviewer. For all coding methods, age was significantly correlated with 12 or more criteria, and with total CBCA scores. The youngest children never reported seven criteria, even when interviewer prompting was taken into account. Therefore, many of the criteria may not be useful in typical interviews with children across a wide age range, because they either are rarely present or they are highly correlated with age.

Content credibility of accounts derived from live and video presentations

Purpose. The present experiment examined the success rate of well‐trained raters in judging the truthfulness of witness statements. Statements from children aged 7‐8 years, children aged 10‐11 years and adults were rated for the presence of 15 of the 19 original Criteria‐Based Content Analysis (CBCA) criteria (Steller &amp; Köhnken, 1989).Method. This experiment comprised a 3 (age of participant) × 3 (nature of account) design. One‐third of the participants took part in a photography session, one‐third watched a video recording of a peer taking part in a photography session, and one‐third were party to a verbal description regarding the photography session but neither participated in it themselves nor saw a video recording. The first two groups then gave truthful accounts of their experiences in a subsequent interview. However, the third group was asked, by the experimenter, to lie during the interview and to try to convince the interviewers that they also had taken part in a photography session.Results. The CBCA criteria discriminated between truthful accounts based on actual involvement and fabricated accounts. However, the criteria did not significantly discriminate between those truthful accounts based on watching a video recording and fabricated accounts. Interestingly, there was a significant difference between the truthful accounts based on participation and the truthful accounts based on observation. Further, CBCA was equally effective in discriminating between participative truthful accounts and fabricated accounts for all three age groups.Conclusions. CBCA was found to be a very useful tool in discriminating between truthful accounts based on direct experience of an event and fabricated accounts (overall hit rate, 70%). Also, CBCA was found to discriminate between truthful acounts based on actual experience and truthful accounts based on watching a video. This latter finding has implications for the real world when children and adults use information from television programmes and videos on which to base false allegations. Finally, CBCA was found to be equally effective for use with accounts from adults as well as children.

Assessment of child witness statements using criteria-based content analysis (CBCA): The effects of age, verbal ability, and interviewer's emotional style

The goal of the study was to determine what effects age, verbal ability (assessed with WISC-R Vocabulary), and interviewer's emotional style would have on the occurrence of Criteria-Based Content Analysis (CBCA) criteria. Children (N = 68) from three different age-groups (7–8, 10–11, and 13–14 years) made both a true and a false statement concerning a mildly traumatic event. The correct classification rate was 66%. Age and verbal ability were found to increase the occurrence of some of the CBCA criteria irrespective of the truthfulness of the statements. Also, different criteria differentiated between true and false statements in different age groups. Interviewer behaviour also affected the occurrence of the criteria. It is concluded that the CBCA should not be used in court proceedings in its present form.

Interviewer Questions and Content Analysis of Children's Statements of Sexual Abuse

Effects of forensic interview techniques on the production of free-narrative and Criteria-Based Content Analysis (CBCA) criteria were assessed in police interviews with 48 children (ages 3 to 16) who alleged they had been sexually abused. These allegations were later categorized as confirmed (n = 35) or highly doubtful (n = 13) based on information obtained independent of the statements. Two raters independently coded all interviewer utterances and children's responses, and four other raters evaluated the transcripts for the presence of CBCA content criteria. As predicted, open questions yielded more free narrative and CBCA criteria than other types of questions. Confirmed statements of abuse contained more CBCA criteria than highly doubtful statements, and statements made by older children contained more CBCA criteria than those by younger children. The results support the use of open questions for eliciting free narrative and the use of CBCA to assess the validity of children's allegations of sexual abuse.