Criteria-Based Content Analysis Research Articles

Use of statement validity analysis in minors alleging sexual assault: A systematic review.

This systematic review aims to report on the use of Statement Validity Analysis (SVA) with minors involved in criminal justice proceedings. We conducted a literature search of six bibliographic databases up to March 2024. Additional searches were performed using citation tracing strategies. Nineteen studies published between 1991 and 2023 were retained. Most were published between 1991 and 2000, mainly in the USA. A scientific gap was observed for 10 years before studies resumed between 2011 and 2022. These 19 studies involved 2931 children; most were girls (n = 2080; 71%). The mean age was 9.4 years (SD = 2.40; min = 2; max = 17.5). Most studies did not mention the nature of the relationship between the child and the alleged perpetrator of sexual violence, three studies involved intra-family violence and six studies involved victims of intra- and extra-family violence. Nearly 75% of the interviewers were trained with SVA methods. Most were mental health professionals (52.6%) or police officers (15.8%). No study used the SVA as a whole, 10 studies used 19 criteria of the Criteria-Based Content Analysis (CBCA), and no study used the Validity Check List (VCL). Most studies performed SVA on interview transcripts (n = 8), and two studies performed their analysis on both verbatims and video. The conclusion of our literature review highlights the methodological weaknesses of these studies and encourages more research about the use of SVA in the judicial field to reduce the risk of misleading the judiciary.

A criteria-based classification model using augmentation and contrastive learning for analyzing imbalanced statement data

Criteria Based Content Analysis (CBCA) is a forensic tool that analyzes victim statements. It involves the categorization of victims' statements into 19 distinct criteria classifications, playing a crucial role in evaluating the authenticity of testimonies by discerning whether they are rooted in genuine experiences or fabricated accounts. The exclusion of subjective opinions becomes imperative to assess statements through this forensic tool objectively. This study proposes developing an objective classification model for CBCA-based statement analysis using natural language processing techniques. Nevertheless, achieving optimal classification performance proves challenging due to imbalances in data distribution among the various criterion classifications. To enhance the accuracy and reliability of the classification model, this research employs data augmentation techniques and dual contrastive learning methods for fine-tuning the RoBERTa language model. Furthermore, model-based optimization techniques are also applied to identify augmented hyper-parameters and maximize the model's classification performance. The study's findings, including an 8.5% improvement in macro F1 score compared to human classification results, a 24% improvement in macro F1 score, and a 13% improvement in accuracy compared to previous human classification results, suggest that the proposed model is highly effective in reducing the influence of human subjectivity in statement analysis. The proposed model has significant implications for legal proceedings and criminal investigations, as it can provide a more objective and reliable method for evaluating the credibility of victim statements. Reducing human subjectivity in the statement analysis process can increase the accuracy of verdicts and help ensure that justice is served.

준거기반내용분석(CBCA)의 변별력

Criteria-Based Content Analysis (CBCA) is part of Statement Validity Analysis (SVA), a process that checks for the presence of 19 criteria that are known to be frequently found in the statements of truth-tellers. This study examined how well CBCA scores predict trial outcomes (guilty or not guilty) and which conventions are associated with higher conviction rates in order to validate the analysis of adult sexual assault victim statements. The results showed that the CBCA total score was not a reliable predictor of guilt or innocence. Next, we examined the truth discriminatory power of CBCA subcategories: logical structure and superfluous details were significantly higher in the guilty group and self-deprecation was significantly higher in the innocent group. The logical structure of the statement was found to be of significant value in assessing credibility, while the other cues were found to be of varying value depending on the characteristics and psychological state of the individual interviewee and the nature of the case, and thus need to be judged in a case-specific manner. These findings suggest that CBCA is a useful tool for analyzing adult sexual assault victim statements, but it is only valuable when used as part of an SVA and cannot be used as a stand-alone authenticity tool.

Proposing immersive virtual reality scenarios for validating verbal content analysis methods in adult samples.

Verbal content analyses to differentiate truthful and fabricated statements, such as the Criteria-Based Content Analysis (CBCA), are used in lie detection research as well as in practice to assess the credibility of statements in criminal court proceedings. Meta-analyses demonstrate validity of verbal content analyses above chance, but the traditional research paradigms usually lack either ecological or internal validity. The authors discuss the usage of immersive virtual reality scenarios to solve this dilemma, as both types of validity can be increased by this approach. In this integrative review of existing literature on the current use of virtual scenarios in forensic and victimology research, the authors extract strengths and limitations for possible VR studies in the context of verbal content analysis. Furthermore, novel ethical challenges involved are summarized and implications for future studies proposed. Overall, we argue in favor of using virtual reality scenarios to validate methods for verbal content analysis, but also urge to consider ethical limitations regarding unwanted short- and long-term aftereffects.

Pathways to truth: using criteria-based content analysis in missing person cases

PurposeWhen investigating missing persons cases, witness and suspect statements are often the only available evidence to investigators. As such, forensic linguistics can be useful to police when seeking information from individuals during investigative interviews. The presumption of innocence must be maintained in an investigation, and therefore this study aims to use a method focused on veracity detection, rather than the majority of studies using deception detection approaches.Design/methodology/approachThe current research uses criteria-based content analysis (CBCA), a method that has been applied to a variety of criminal cases. Real-world statements of individuals convicted or found innocent of their involvement in missing persons cases were used in the analyses. Additionally, behaviour sequence analysis (BSA) was used to map language patterns within individuals’ statements.FindingsResults indicated that two individual markers occurred at a high frequency across all four groups (the guilty and innocent statements of both case types); however, differences were noted in the sequences based on an individual’s ability to provide experiential details.Research limitations/implicationsThe current research contributes to the growing literature that aims to test CBCA in adult samples across crimes that do not pertain to sexual abuse, in addition to aiding researchers and practitioners to understand better the linguistic differences that occur in missing persons cases.Originality/valueTo the best of the authors’ knowledge, this type of research, using CBCA to assist with missing person's cases with a temporal method (BSA), has never before been tested. BSA has previously been used in forensic linguistics, and shown support for the method. The current research builds on this in terms of missing persons cases.

Verbal Lie Detection: Its Past, Present and Future.

This article provides an overview of verbal lie detection research. This type of research began in the 1970s with examining the relationship between deception and specific words. We briefly review this initial research. In the late 1980s, Criteria-Based Content Analysis (CBCA) emerged, a veracity assessment tool containing a list of verbal criteria. This was followed by Reality Monitoring (RM) and Scientific Content Analysis (SCAN), two other veracity assessment tools that contain lists of verbal criteria. We discuss their contents, theoretical rationales, and ability to identify truths and lies. We also discuss similarities and differences between CBCA, RM, and SCAN. In the mid 2000s, 'Interviewing to deception' emerged, with the goal of developing specific interview protocols aimed at enhancing or eliciting verbal veracity cues. We outline the four most widely researched interview protocols to date: the Strategic Use of Evidence (SUE), Verifiability Approach (VA), Cognitive Credibility Assessment (CCA), and Reality Interviewing (RI). We briefly discuss the working of these protocols, their theoretical rationales and empirical support, as well as the similarities and differences between them. We conclude this article with elaborating on how neuroscientists can inform and improve verbal lie detection.

성폭력 피해 진술의 신빙성 평가를 위한 준거기반 내용분석(CBCA)의 준거 재정립 연구

Criteria-Based Content Analysis (CBCA) can effectively discriminate the veracity of child statement in sexual abuse cases. Therefore, CBCA is widely used to evaluate the credibility in cases of suspected child sexual abuse. However, the validity of an CBCA procedure is limited by its low inter-rater reliability. In order to identify the cause of low inter-rater reliability and solve this problem, this study proposed a method for re-establishing the definition of CBCA criteria and the evaluation principle. Hence, 13 domestic papers using CBCA and CBCA manuals of the police and prosecutors were reviewed. Accordingly, problems such as names of CBCA critera, ambiguous operational definitions, and lack of clear evaluation principle were identified. Based on this result, we proposed a specific operational definition and evaluation principle along with a unified name. Trained for 10hours evaluators analyzed transcripts of 84 sexual assault victim statements using reestablished CBCA criteria. As a result, it was found that the inter-rater reliability (ICC) for each criterion was .697-.949, which was satisfactory. Finally, the practical application of the redefined concept in this study and the future development direction were discussed.

Fake Online Reviews: A Unified Detection Model Using Deception Theories

Online reviews influence consumers’ purchasing decisions. However, identifying fake reviews online automatically remains a complex problem, and current detection approaches are inefficient in preventing the spread of fake reviews. The literature on fake reviews detection lacks a comprehensive and interpretable theory-based model with high performance, which enables us to understand the phenomenon from a psychological perspective and analyze reviews based on user-generated content as well as consumer behavior. In this research, we synthesized ten well-founded deception theories from psychology, namely leakage theory, four-factor theory, interpersonal deception theory, self-presentational theory, reality monitoring theory, criteria-based content analysis, scientific content analysis, verifiability approach, truth-default theory, and information manipulation theory, and selected nine relevant constructs to develop a unified model for detecting fake online reviews. These constructs include specificity, quantity, non-immediacy, affect, uncertainty, informality, consistency, source credibility, and deviation in behavior. We characterized the selected constructs using verbal and non-verbal features to validate the proposed model empirically. Subsequently, we extracted features from the Yelp datasets and used them to train four machine learning algorithms, specifically Logistic Regression, Naïve Bayes, Decision Tree, and Random Forest. We demonstrated that quantity, non-immediacy, affect, informality, consistency, source credibility, and deviation in behavior are essential constructs for detecting fake reviews. To our surprise, we discovered that non-verbal features are more important than verbal features and that combining features from both types improves the prediction performance. Our theory-based model outperformed most of the state-of-the-art fake review detection models and yielded high interpretability and low complexity.

Criteria‐Based Content Analysis in Child Sexual Abuse Cases: A Cross‐Cultural Perspective

Statement validity assessment (SVA) was created to evaluate children's accounts in alleged sexual abuse cases and it is the most popular instrument currently used to assess the veracity of their verbal declarations. The core of this method lies in the application of criteria‐based content analysis (CBCA) to the content of the provided account in an attempt to determine its veracity. Despite the popularity and importance of this instrument in legal settings in a variety of countries, there is little evidence that the cultural and ethnic background of the alleged child victims is taken into account. The current paper proposes a theoretical overview of the culturally sensitive aspects of cognition, memory, motivation and self‐identity as reflected in each of the two general categories of the CBCA criteria (i.e. cognitive and motivational). Our investigation revealed that several criteria are likely to be sensitive to cultural differences, therefore further direct empirical proof is needed to improve validation of SVA use in culturally diverse child sexual abuse cases. In the interim, we would encourage the use of culturally sensitive practices and policies among practitioners using CBCA.‘There is little evidence that the cultural and ethnic background of the alleged child victims is taken into account’Key Practitioner Messages The cultural or ethnic background of children influences their accounts; thus, it should be expected that certain CBCA criteria can be affected by these factors. Professionals using this method are recommended to take a more ‘culturally sensitive’ approach, which involves learning more about cultural differences and monitoring their own expectations and biases. More empirical research is needed in order to validate which CBCA criteria are suitable for use in a variety of cultures and cross culturally.

Verbal content analysis for the study of Verisimilitude (V1) in the System for Analysis of Validity in Evaluation (SAVE)

In the SAVE Metaprotocol (López, et al, 2018), Verisimilitude (V1) refers to that which has the appearance of truth, possibility of being believed, and with "absence of subjective incredibility".
 Corresponding to this section is everything related to the analysis of the content of the verbal message and which gives internal coherence to the subject's statement. This phase is obtained directly through the interview with the subject, which we record in order to be able to carry out the content analysis later. It can be using a criterial, structural, or linguistic analysis, or simply employing a study of the communicative pattern.
 Some examples of content analysis are Statement Validity Analysis (SVA) with its Criteria-Based Content Analysis or CBCA and its validity list. The CBCA is composed of 19 criteria used to assess the content of the statements and which to date has been admitted as court evidence in cases of possible child sexual offense. It is administered together with a validity checklist and its value as evidence depends on the scope of application (Köhnken, Manzanero & Scott 2015).
 The instrument called Reality Monitoring (RM), like the previous one is a tool used to assess the credibility of the testimony and that focuses on the semantic aspects that differentiate a true statement from a false or not very credible one (Raye & Johnson, 1980). Although in this aspect, we should not make the mistake of confusing credibility with veracity, since when assessing a testimony the truth as such does not exist, and depends on the functioning of our memory, which is not like a tape recorder and is not static, but is constantly exposed to changes and distortions. According to this model, the origin of our memories comes from two basic sources: External Memories, generated from the perception of real situations; and Internal Memories, the product of our imagination. The unconscious reasoning process that our mind follows to decide which information comes from an external source and which forms an internal one is called Reality Monitoring. Four qualitative attributes allow us to differentiate between these two types of memories:
 

Forensic linguistics. Reality Monitoring and Criteria Based Content Analysis (CBCA): comparison between methods

Forensic linguistics is about the analysis of written, oral or non-verbal language in the legal field. For this purpose it employs tools finalised to the quantification and qualification of information. Among those we find Reality Monitoring, that discriminates between internal and external memories; Criteria Based Content Analysis, which uses nineteen criteria to verify the authenticity of a speech; Statement Validity Analysis; Scientific Content Analysis; Step Wise Interview. The comparison between these disciplines (and an overview on those) is preceded by important observations about cognitive and mnemonic processes and the role of the interviewer in the collection of testimony.

How Information on a Motive to Lie Influences CBCA-Based Ratings and Veracity Judgments.

We investigated how information on a motive to lie impacts on the perceived content quality of a statement and its subsequent veracity rating. In an online study, 300 participants rated a statement about an alleged sexual harassment on a scale based on Criteria-based Content Analysis (CBCA) and judged its veracity. In a 3 × 3 between-subjects design, we varied prior information (motive to lie, no motive to lie, and no information on a motive), and presented three different statement versions of varying content quality (high, medium, and low). In addition to anticipating main effects of both independent variables (motive information and statement version), we predicted that the impact of motive information on both ratings would be highest for medium quality statements, because their assessment is especially ambiguous (interaction effect). Contrary to our hypotheses, results showed that participants were unaffected by motive information and accurately reproduced the manipulated quality differences between statement versions in their CBCA-based judgments. In line with the expected interaction effect, veracity ratings decreased in the motive-to-lie group compared to controls, but only when the medium- and the low-quality statements were rated (truth ratings dropped from approximately 80 to 50%). Veracity ratings in both the no-motive-to-lie group and controls did not differ across statement versions (≥82% truth ratings). In sum, information on a motive to lie thus encouraged participants to consider content quality in their veracity judgments by being critical only of statements of medium and low quality. Otherwise, participants judged statements to be true irrespective of content quality.

Optimizing CBCA and RM research: recommendations for analyzing and reporting data on content cues to deception

ABSTRACT For more than a century, verbal content cues to deception have been investigated to assess the credibility of statements in judicial contexts. Among the many cues investigated, Criteria-based Content Analysis (CBCA) and criteria based on the reality monitoring (RM) approach have been most prominent. However, research with these cues used as ‘tools’ has not fully exploited their potential. We critically discuss statistical approaches used in past research and recommend a series of 12 principles or guidelines researchers should follow to design, analyze and report future studies on detecting deception with verbal content cues. To illustrate some of these points, we present analyses from two separate studies: A quasi-experiment in a field setting conducted with adults with intellectual disabilities who truthfully or deceptively described a negative autobiographical event to an interviewer, and a large-scale simulation study where adults wrote an account of either an experienced or an invented significant life event. Accounts in both studies were rated with CBCA and RM criteria, as well as by ‘naive’ raters. The guidelines should help to increase the quality and transparency of research in this area.

Séquelles post-traumatiques à long terme rencontrées chez les enfants et adolescents victimes de violence sexuelle : implications pour l’expertise de crédibilité

We present the state-of-the-arts linked to long-term posttraumatic functioning of sexually abused children and adolescents, and especially their risk of disruption of body image, impairment of identity construction, dysfunction of memory and posttraumatic dissociation. The judicial expert who is in charge of an expertise of credibility with the alleged victim has to consider the effects of defence mechanisms and coping strategies on emotional memory, tending to distort facts in order to preserve psychic survival. Thus, current discourse analysis procedures, such as CBCA (Criteria-based content analysis), have to be interpreted with caution, taking into account the whole clinical context.

Verbal Baselining: Within-Subject Consistency of CBCA Scores across Different Truthful and Fabricated Accounts

Statement Validity Assessment (SVA) proposes that baseline statements on different events can serve as a within-subject measure of a witness’ individual verbal capabilities when evaluating scores from Criteria-based Content Analysis (CBCA). This assumes that CBCA scores will generally be consistent across two accounts by the same witness. We present a first pilot study on this assumption. In two sessions, we asked 29 participants to produce one experience-based and one fabricated baseline account as well as one experience-based and one fabricated target account (each on different events), resulting in a total of 116 accounts. We hypothesized at least moderate correlations between target and baseline indicating a consistency across both experience-based and fabricated CBCA scores, and that fabricated CBCA scores would be more consistent because truth-telling has to consider random event characteristics, whereas lies must be constructed completely by the individual witness. Results showed that differences in correlations between experience-based CBCA scores and between fabricated CBCA scores took the predicted direction (cexperience-based = .44 versus cfabricated =.61) but this difference was not statistically significant. As predicted, a subgroup of event-related CBCA criteria were significantly less consistent than CBCA total scores, but only in experience-based accounts. The discussion considers methodological issues regarding the usage of total CBCA scores and whether to measure consistency with correlation coefficients. It is concluded that more studies are needed with larger samples.

"Validity of content-based techniques to distinguish true and fabricated statements: A meta-analysis": Correction to Oberlader et al. (2016).

Reports an error in "Validity of content-based techniques to distinguish true and fabricated statements: A meta-analysis" by Verena A. Oberlader, Christoph Naefgen, Judith Koppehele-Gossel, Laura Quinten, Rainer Banse and Alexander F. Schmidt (Law and Human Behavior, 2016[Aug], Vol 40[4], 440-457). During an update of this meta-analysis it became apparent that one study was erroneously entered twice. The reduced data of k = 55 studies was reanalyzed after excluding the unpublished study by Scheinberger (1993). The corrected overall effect size changed at the second decimal: d = 1.01 (95% CI [0.77, 1.25], Q = 409.73, p < .001, I² = 92.21) and g = 0.98 (95% CI [0.75, 1.22], Q = 395.49, p < .001, I² = 91.71%), k = 55, N = 3,399. This small numerical deviation is negligible and does not change the interpretation of the results. Similarly, results for categorial moderators changed only numerically but not in terms of their statistical significance or direction (see revised Table 4). In the original meta-analysis based on k = 56 studies, unpublished studies had a larger effect size than published studies. Based on k = 55 studies, this difference vanished. Results for continuous moderators also changed only numerically: Q-tests with mixed-effects models still revealed that year of publication (Q = 0.06, p = .807, k = 55) as well as gender ratio in the sample (Q = 1.28, p =.259, k = 43) had no statistically significant influence on effect size. In sum, based on the numerically corrected values our implications for practical advices and boundary conditions for the use of content-based techniques in credibility assessment are still valid. The online version of this article has been corrected. (The following abstract of the original article appeared in record 2016-21973-001.) Within the scope of judicial decisions, approaches to distinguish between true and fabricated statements have been of particular importance since ancient times. Although methods focusing on "prototypical" deceptive behavior (e.g., psychophysiological phenomena, nonverbal cues) have largely been rejected with regard to validity, content-based techniques constitute a promising approach and are well established within the applied forensic context. The basic idea of this approach is that experience-based and nonexperience-based statements differ in their content-related quality. In order to test the validity of the most prominent content-based techniques, criteria-based content analysis (CBCA) and reality monitoring (RM), we conducted a comprehensive meta-analysis on English- and German-language studies. Based on a variety of decision criteria, 55 studies were included revealing an overall effect size of g = 0.98 (95% confidence interval [0.75, 1.22], Q = 395.49, p < .001, I² = 91.71%, N = 3,399). There was no significant difference in the effectiveness of CBCA and RM. Additionally, we investigated a number of moderator variables, such as characteristics of participants, statements, and judgment procedures, as well as general study characteristics. Results showed that the application of all CBCA criteria outperformed any incomplete CBCA criteria set. Furthermore, statement classification based on discriminant functions revealed higher discrimination rates than decisions based on sum scores. All results are discussed in terms of their significance for future research (e.g., developing standardized decision rules) and practical application (e.g., user training, applying complete criteria set). (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Criteria-based Content Analysis in True and Simulated Victims with Intellectual Disability

The aims of the present study were to analyse people’s natural ability to discriminate between true and false statements provided by people with intellectual disability (IQTRUE = 62.00, SD = 10.07; IQFALSE = 58.41, SD = 8.42), and the differentiating characteristics of such people’s statements using criteria-based content analysis (CBCA). Thirty-three people assessed 16 true statements and 13 false statements using their normal abilities. Two other evaluators trained in CBCA evaluated the same statements. The natural evaluators differentiated between true and false statements with somewhat above-chance accuracy, even though error rate was high (38.19%). That lay participants could not effectively discriminate between false and true statements demonstrates that such assessments cannot be considered useful in a forensic context. The CBCA technique did discriminate at a better level than intuitive judgements. However, of the 19 criteria, only one significantly discriminated. More procedures specifically adapted to the abilities of people with intellectual disabilities are thus d. The presence of structured production, quantity of details, characteristics details and unexpected complications increased the probability that a statement would be considered true by non-expert evaluators. The classification made by the non-expert evaluators was independent of the participants’ IQ. A big data analysis is performed in search for better classification quality.

Detecting deception: does the cognitive interview impair discrimination with CBCA criteria in elderly witnesses?

ABSTRACTThe accuracy of an eyewitness statement depends on the interviewing method used. Therefore, empirically tested interviewing techniques have been developed, in particular for vulnerable witnesses. Sometimes witnesses may not tell the truth based on personally experienced events but give testimony about events they have only heard about, thus lying about the event. In this experiment, 80 witnesses over 60 years old either saw a film or heard an audiotaped summary. They were interviewed either with a standard interview (SI) or an Enhanced Cognitive Interview (ECI). The ECI led to an increase in the amount of information gained without jeopardizing its accuracy compared to the SI. Trained coders evaluated transcripts of the interviews with Criteria-Based Content Analysis and judged the truthfulness of each account. Some differences in CBCA criteria were in the expected direction, others opposite to expectation, thus prohibiting the use of a summary score. Accounts from the SI group (80%) were judged correctly significantly more often than accounts from the ECI group (40%). Only 15% of the lies in the ECI group were judged correctly vs. 80% in the SI group. Findings are discussed regarding the importance of testing interview techniques with respect to their potential to detect deception.

Identifying Applicant Faking in Job Interviews

Abstract. Applicants’ use of faking tactics could threaten the validity of employment interviews. We examined criterion-based content analysis (CBCA), an approach used in legal contexts, as a potential indicator of interviewee faking. We also examined the moderating role of storytelling in the faking-CBCA relationship. We conducted one experimental study, with 100 interviewees receiving instructions to respond honestly versus to exaggerate/invent responses, and one mock interview study, with self-reported faking from 111 interviewees. Responses were recorded, transcribed, and coded for CBCA and storytelling. Faking was associated with CBCA when interviewees freely engaged in faking tactics, an overall CBCA indicator was used, and interviewees’ responses contained story features. Additional analyses highlight that CBCA-based assessments of faking/honesty could reach up to 63.4% accuracy.

The Strategic Meaning of CBCA Criteria From the Perspective of Deceivers.

In 2014, Volbert and Steller introduced a revised model of Criteria-Based Content Analysis (CBCA) that grouped a modified set of content criteria in closer reference to their assumed latent processes, resulting in three dimensions of memory-related, script-deviant and strategy-based criteria. In this model, it is assumed that deceivers try to integrate memory-related criteria—but will not be as good as truth tellers in achieving this—whereas out of strategic considerations they will avoid the expression of the other criteria. The aim of the current study was to test this assumption. A vignette was presented via an online-questionnaire to inquire how participants (n = 135) rate the strategic value of CBCA criteria on a five-point scale. One-sample t-tests showed that participants attribute positive strategic value to most memory-related criteria and negative value to the remaining criteria, except for the criteria self-deprecation and pardoning the perpetrator. Overall, our results corroborated the model's suitability in distinguishing different groups of criteria—some which liars are inclined to integrate and others which liars intend to avoid—and in this way provide useful hints for forensic practitioners in appraising the criteria' diagnostic value.