Veracity Assessment Research Articles

Explainable Evidence-Based Veracity Assessment of Textual Claim

Explainable Evidence-Based Veracity Assessment of Textual Claim

Self and other-perceived deception detection abilities are highly correlated but unassociated with objective detection ability: Examining the detection consensus effect

Subjective lying rates are often strongly and positively correlated. Called the deception consensus effect, people who lie often tend to believe others lie often, too. The present paper evaluated how this cognitive bias also extends to deception detection. Two studies (Study 1: N = 180 students; Study 2: N = 250 people from the general public) had participants make 10 veracity judgments based on videotaped interviews, and also indicate subjective detection abilities (self and other). Subjective, perceived detection abilities were significantly linked, supporting a detection consensus effect, yet they were unassociated with objective detection accuracy. More overconfident detectors—those whose subjective detection accuracy was greater than their objective detection accuracy—reported telling more white and big lies, cheated more on a behavioral task, and were more ideologically conservative than less overconfident detectors. This evidence supports and extends contextual models of deception (e.g., the COLD model), highlighting possible (a)symmetries in subjective and objective veracity assessments.

(Don’t) believe me, I’m telling the truth! Speech disfluency and eye contact as cues to veracity, intention, and truth judgement

ABSTRACT There is no consensus in the literature about the role of speech disfluencies and eye contact as cues to deception. The current study uses a picture-description game (the treasure hunting paradigm) to collect speech and eye contact data of speakers and veracity assessments of listeners engaged in a socially meaningful interaction. The paradigm was implemented so that not only statement veracity (i.e. true or false) could be manipulated, but also speaker’s intention (i.e. wanting or not wanting to be believed) and listener decision (i.e. believing or not believing the speaker). Multivariate Pattern Analysis showed that disfluency patterns could predict intention and decision above chance, but not veracity. Eye contact could predict veracity, intention, and decision above the chance level. Future research should focus more on qualitative features (i.e. when and how), while accounting for speaker intention.

Detecting the veracity of children's experiences through drawings.

Research is lacking regarding adults' ability to determine whether children's drawings are based on an experience or not. Drawings are useful in professional settings to alleviate linguistic demands, facilitate memory, and have been used as evidence. Determining the accuracy of veracity assessments of children's drawings would inform professionals regarding their use as evidence of experiences. Twenty-eight children (14 younger, Mage = 7.53 years, SDage = 1.19; 14 older, Mage = 11.67 years, SD = 1.27) produced drawings of two events: one staged experienced, and one narrative-based not experienced event. Fifty (Study 1, Mage = 23.72 years, SDage = 9.70) and 63 (Study 2, Mage = 25.92, SDage = 12.79) adults indicated whether each drawing was based on experience and their confidence in each assessment. In Study 2, additional drawing quality assessments were collected. Results indicated that adults were more accurate at distinguishing experienced than not experienced drawings for older artists. An inverse relationship was observed between confidence and accuracy-participants were more confident when they were inaccurate, especially for younger artists. Drawing quality improved with age and for drawings of experienced events. Adults tended to rate drawings of higher quality as resulting from experience leading to the highest accuracy for drawings from older artists that were based on experience. Overall, results suggest that there may be some features of drawings that allow for above chance levels of accuracy (up to 75%). However, rates are not high enough across assessments (M = 53.93%, range: 39%-75%) to reliably use them as indicators of experience.

Veritas AI: The ChatGPT Polygraph

Aims: The objective of Veritas AI is to revolutionize the domain of lie detection through the deployment of a cutting-edge algorithm within the realms of computational linguistics and artificial intelligence. Study Design: Veritas AI is conceptualized as a groundbreaking framework that integrates advanced syntactic and semantic analysis, leveraging generative pre-trained transformers to identify linguistic cues indicative of deception. Place and Duration of Study: The research underpinning Veritas AI’s algorithm was meticulously executed at the Abacus CSE Lab over a period from December 2022 to March 2024, ensuring a robust empirical foundation for the system’s validation and optimization. Methodology: Employing a deep learning neural network at its core, Veritas AI is trained on a diverse dataset comprising both truthful and deceptive dialogues. This training is complemented by multimodal biometric interrogation techniques and sophisticated natural language processing algorithms. Results: The empirical results underscore Veritas AI’s unparalleled accuracy in discerning truth, marked by its ability to provide real-time adaptive feedback and maintain robust performance across various communication scenarios. Conclusion: In conclusion, Veritas AI stands as a testament to the symbiotic potential of human ingenuity and machine learning. Its precision-engineered algorithm, underpinned by empirical validation, heralds a transformative leap in the field of automated veracity assessment, setting a new benchmark for truth analysis in the digital age.

Generative AI Are More Truth-Biased Than Humans: A Replication and Extension of Core Truth-Default Theory Principles

Humans often display a truth-bias—the perception that others are honest independent of message veracity—but does this phenomenon extend to generative artificial intelligence (AI)? We had humans and large language models make nearly 1,000 veracity judgments across different prompts. Human detection accuracies were near chance (50%–53%) with notable truth-biases (59%–64%); AI had a substantially greater truth-bias than humans (67%–99%). GPT-4 was also truth-default, not suspecting deception when veracity assessments were unprompted. Together, people and AI judge most information to be true.

Examining the role of speaker familiarity and statement practice on deception detection

Detecting deception is a ubiquitous, but difficult component of daily interactions. While prior work has shown that people are poor lie detectors, research has also shown that increased familiarity with the statement sender impacts accuracy. The current study examined how increased familiarity with a statement sender, as well as the type of statement provided, influenced detection accuracy. Participants judged truthful and deceptive statements from different speakers that varied in how familiar they were to the participant (pre-experimental familiarity, experimental familiarity, no familiarity). The statements that were evaluated varied in veracity, statement type (descriptions or denials), and whether the statements had been practiced. Participants believed they were more accurate in their veracity assessments for the pre-experimentally familiar speakers compared to the other speaker types. While participants were more accurate for pre-experimentally familiar speakers compared to strangers, there was no difference in accuracy between judgments for the pre- and experimentally familiar speakers. Participants were also more likely to believe statements that had been practiced, regardless of the statements’ actual veracity or their degree of familiarity with the speaker.

Realtime user ratings as a strategy for combatting misinformation: an experimental study

Because fact-checking takes time, verdicts are usually reached after a message has gone viral and interventions can have only limited effect. A new approach recently proposed in scholarship and piloted on online platforms is to harness the wisdom of the crowd by enabling recipients of an online message to attach veracity assessments to it. The intention is to allow poor initial crowd reception to temper belief in and further spread of misinformation. We study this approach by letting 4000 subjects in 80 experimental bipartisan communities sequentially rate the veracity of informational messages. We find that in well-mixed communities, the public display of earlier veracity ratings indeed enhances the correct classification of true and false messages by subsequent users. However, crowd intelligence backfires when false information is sequentially rated in ideologically segregated communities. This happens because early raters’ ideological bias, which is aligned with a message, influences later raters’ assessments away from the truth. These results suggest that network segregation poses an important problem for community misinformation detection systems that must be accounted for in the design of such systems.

Verbal Lie Detection: Its Past, Present and Future.

This article provides an overview of verbal lie detection research. This type of research began in the 1970s with examining the relationship between deception and specific words. We briefly review this initial research. In the late 1980s, Criteria-Based Content Analysis (CBCA) emerged, a veracity assessment tool containing a list of verbal criteria. This was followed by Reality Monitoring (RM) and Scientific Content Analysis (SCAN), two other veracity assessment tools that contain lists of verbal criteria. We discuss their contents, theoretical rationales, and ability to identify truths and lies. We also discuss similarities and differences between CBCA, RM, and SCAN. In the mid 2000s, 'Interviewing to deception' emerged, with the goal of developing specific interview protocols aimed at enhancing or eliciting verbal veracity cues. We outline the four most widely researched interview protocols to date: the Strategic Use of Evidence (SUE), Verifiability Approach (VA), Cognitive Credibility Assessment (CCA), and Reality Interviewing (RI). We briefly discuss the working of these protocols, their theoretical rationales and empirical support, as well as the similarities and differences between them. We conclude this article with elaborating on how neuroscientists can inform and improve verbal lie detection.

Actions speak louder than words: The Devil's Advocate questioning protocol in opinions about protester actions

SummaryThe Devil's Advocate protocol has been developed to assist making veracity assessments when someone discusses their opinion. The present experiment focused on protester actions rather than controversial issues and also included an adapted version of the Verifiability Approach. Participants told the truth or lied about protester actions and the participants' answers to the eliciting opinion and Devil's Advocate questions were compared. The Devil's Advocate approach predicts the difference in answers (residue scores) to be more pronounced in truth tellers than in lie tellers in terms of quantity of the answers (number of words, details and arguments) and quality of the answers (plausibility, immediacy and clarity). The hypothesis was supported but only in terms of quality: Truth tellers' answers sounded more plausible and immediate and somewhat clearer than lie tellers' answers. Truth tellers also reported more digital verifiable sources than lie tellers.

Cognitive Processes and Personality Traits Underlying Four Phenotypes of Susceptibility to (Mis)Information.

Misinformation on social media poses a serious threat to democracy, sociopolitical stability, and mental health. Thus, it is crucial to investigate the nature of cognitive mechanisms and personality traits that contribute to the assessment of news items' veracity, failures in the discernment of their truthfulness, and behavioral engagement with the news, especially if one wants to devise any intervention to stop the spread of misinformation in social media. The current research aimed to develop and test a 4-fold taxonomy classifying people into four distinct phenotypes of susceptibility to (mis)information. In doing so, it aimed to establish differences in cognitive and psychological profiles between these phenotypes. The investigated cognitive processes included sensitivity to feedback, belief updating, and cognitive judgment bias. Psychological traits of interest included the Big Five model, grandiose narcissism, anxiety, and dispositional optimism. The participants completed online surveys that consisted of a new scale designed to classify people into one of four phenotypes of susceptibility to (mis)information, advanced cognitive tests, and reliable psychological instruments. The four identified phenotypes, Doubters, Knowers, Duffers, and Consumers, showed that believing in misinformation does not imply denying the truth. In contrast, the numerically largest phenotypes encompassed individuals who were either susceptible (Consumers) or resistant (Doubters), in terms of veracity judgment and behavioral engagement, to any news, regardless of its truthfulness. Significantly less frequent were the phenotypes characterized by excellent and poor discernment of the news' truthfulness (the Knowers and the Duffers, respectively). The phenotypes significantly differed in sensitivity to positive and negative feedback, cognitive judgment bias, extraversion, conscientiousness, agreeableness, emotional stability, grandiose narcissism, anxiety, and dispositional optimism. The obtained results constitute a basis for a new and holistic approach in understanding susceptibility to (mis)information as a psycho-cognitive phenotype.

Power and lie detection.

Social power undermines focus on others and increases reliance on stereotype-consistent information. Thus, power may enhance focus on stereotypical cues to deception, thereby decreasing lie detection accuracy. In three studies, we tested whether having power affects lie detection accuracy. Participants (overall N = 502) were asked to identify truthful and lying candidates (N = 12) during mock job interviews. Study 1 was a field experiment involving employees who held managerial and non-managerial positions (N = 88). In the following laboratory experiments, we manipulated power and asked participants to imagine themselves as managers (Study 2, N = 214) or provided them with control over resources and the ability to reward others (Study 3, N = 200). In Studies 2 and 3, we additionally manipulated the method of lie detection (direct vs. indirect). In contrast to the original hypotheses, we found that power led to increased veracity assessment accuracy. Having power over others enhances the accuracy of one's veracity assessment, although this increase is small and limited to lie detection (Study 1) or direct judgments (Studies 2 & 3). Together, power affects the processing of social information and what aspects of this information are taken into account.

Check the Report and Comments: The Veracity Assessment of Unfamiliar News on Social Media

Given growing concerns regarding the spread of medical misinformation, the current research set out to assess the message effects of social media news on reader veracity assessments. A 2 (news report with hedging vs. without hedging) by 3 (uncivil vs. civil vs. no comments) between-subject experiment on Facebook users was conducted (valid N = 824). Results reveal that news hedging was more predictive of perceived credibility, news sharing, and fact-checking tendencies than was comment incivility. Hedged reporting was also found to elevate perceived news credibility, which in turn predicted a greater likelihood of news sharing. Moreover, perceived credibility increased fact-checking tendency only when the news was reported with hedged messages. These findings indicate that when readers encounter an unfamiliar health news issue, the content of news played a more important role in veracity assessment than the style of reader comments.

Veracity assessment by single and multi-source identification algorithms during the crisis

Social networks have become a popular communication tool for information sharing. Twitter offers access to data and provides a significant opportunity to analyze data. During pandemics, Twitter becomes a big source for the dispersal of unverified information. In social media, it is difficult to find the sources of rumors. To tackle this problem the authors have developed a hybrid rumor centrality algorithm for rumor source detection in social networks. The authors propose an S-RSI algorithm for identifying a single rumor centre and an M-RSI algorithm for identifying the propagations of multiple rumor centres in the thread of conversation. The proposed rumor centrality algorithm efficiently predicts the rumor disseminating possibilities in a conversation tree with the aid of graph theoretical approach. The authors have evaluated the performance of the algorithms on the PHEME dataset containing seven real-time event conversational trees based on the tweet messages. The results show that the proposed is best suitable in finding the rumor source centre with a high probability in social media during a crisis.

Veracity Assessment of Multimedia Facebook Posts for Infodemic Symptom Detection using Bi-modal Unsupervised Machine Learning Approach

Abstract: Ascertaining the truthfulness and trustworthiness of information posted on social media has been challenging with the proliferation of unsubstantiated, misleading, and inciting news, with different intents by purveyors. Unlike the traditional media with some level of regulations, user-generated posts on social networks does not pass through censorships in order to establish the truism of news items hence the need to be cautious of posted information on the networks. The lingering issue of recent suspension of Twitter microblogging site by the Nigerian government and the consequent decision to regulate social network operations in the country similarly centers on the subject of social media dependability for legitimate social engagements by millions of savvy Nigerian users. Whereas existing models in literature have proposed state-of-the-arts, this study seeks to improve on obtainable studies with a bi-modal machine learning methodology that indicate symptoms of infodemic social media posts. Using a multimedia facebook corpus, an unsupervised natural language processor, Inception v3 model, coupled with a hierarchical clustering network, is deployed for the duo of image and text sentiment analytics. Experimental result uniquely identified infodemic tendencies in facebook text-corpus and efficiently differentiates image-corpus into respective clusters through the Euclidian distance metrics. The most infodemic post returned a -0.9719 compound score while the most positive post returns 0.9488. Veracity assessment of polarized opinions expressed in negative clusters reveals that provocative, derogatory, obnoxious, etc. indicate propensity for infodemic tendencies. Keywords: Fake news. Facebook. Social media. Sentiment Analysis. Infodemic

Predicting veracity in online hotel reviews using types of reported speech, speaker identity and quotation marks

Many studies of deception detection have supported using direct reported speech as a linguistic indicator of veracity (Vrij 2015). However, indirect reported speech and narrative reports of speech (Leech and Short 1981[2007]; Chafe 1994) have not received such attention in veracity assessment research. The current study investigates whether these types of reported speech (along with speaker identities and the use/non-use of quotation marks) act as indicators of veracity in a corpus of truthful and deceptive negative online hotel reviews developed by Ott, Choi, Cardie and Hancock (2011). Logistic regression modelling found that type of reported speech and speaker identity were significant, with deceptive reviews more often using indirect reported speech and truthful reviews featuring speaker identities other than the reviewer or hotel staff. This article contributes to our understanding of deception detection by considering theoretically meaningful classes of indicators (Burgoon 2018) and connecting previous approaches from multiple fields and perspectives.

Combining Verbal Veracity Assessment Techniques to Distinguish Truth Tellers from Lie Tellers

Cognitive Credibility Assessment (CCA) is a verbal lie detection tool consisting of several interview techniques. These techniques have been examined separately but never together. Reflecting the dynamic nature of CCA we combined several of the techniques (free recall followed by a model statement, followed by a reverse order instruction, and followed by a sketch instruction). We examined the new information provided after each stage of the interview and also compared the information provided in the initial recall with the information provided after the entire interview. A total of 47 truth tellers and 47 lie tellers went on a mission. Truth tellers were asked to report their mission truthfully, whereas lie tellers were requested to lie about several aspects of the mission. We measured the total units of information (total details) provided in the interview and the number of complications reported. The results indicate that the pre-registered hypothesis (Hypothesis 1) was supported for complications. Truth tellers reported more complications than lie tellers in each stage of the interview and the difference was more pronounced after the entire interview than after the free recall. As a conclusion, CCA was an effective lie detection method when complications were taken into account.

Tailoring heuristics and timing AI interventions for supporting news veracity assessments

The detection of false and misleading news has become a top priority to researchers and practitioners. Despite the large number of efforts in this area, many questions remain unanswered about the ideal design of interventions, so that they effectively inform news consumers. In this work, we seek to fill part of this gap by exploring two important elements of tools’ design: the timing of news veracity interventions and the format of the presented interventions. Specifically, in two sequential studies, using data collected from news consumers through Amazon Mechanical Turk (AMT), we study whether there are differences in their ability to correctly identify fake news under two conditions: when the intervention targets novel news situations and when the intervention is tailored to specific heuristics. We find that in novel news situations users are more receptive to the advice of the AI, and further, under this condition tailored advice is more effective than generic one. We link our findings to prior literature on confirmation bias and we provide insights for news providers and AI tool designers to help mitigate the negative consequences of misinformation.

An examination of the Self‐Administered Interview as a verbal veracity assessment tool

SummaryThe self‐administered interview (SAI) is a written eyewitness recall tool that elicits more information from cooperative witnesses than written free recall (WFR) formats. To date, SAI research has examined the accounts of cooperative people providing honest reports. In the current experiment, truthful and fabricating participants (N = 128) either completed a WFR or a SAI after witnessing a crime (initial account). After a 1‐week delay, participants were interviewed verbally (subsequent interview). Truth tellers reported significantly more detail than liars in both the initial account and subsequent interview, and participants who completed the SAI reported more detail than those completing the WFR. Truth tellers repeated and omitted more information in the subsequent interview than liars; however, there was no significant difference in the number of reminiscent details reported. Although the SAI is effective in eliciting information as an initial eyewitness reporting tool, no benefits for the detection of deception were demonstrated.

Lie detection accuracy and beliefs about cues to deception in adult children of alcoholics

When one or both parents misuse alcohol, it can lead to the development of particular and varied traits in their children. The present study tested whether adult children of alcoholics (ACoAs) who participated in therapy had better veracity assessment skills and more reliable beliefs about cues to deception than the control group of non-ACoAs. The results revealed that individuals who grew up in a family with alcohol misuse problems detected truth – but not lies – significantly better than the control group. The groups did not differ in accuracy of their beliefs about cues to deception. It is possible that the ACoAs’ higher truth detection accuracy is to some extent attributable to their participation in therapy, which increased their level of trust in others.