Overclaiming Technique Research Articles

A New Approach to Desirable Responding: Multidimensional Item Response Model of Overclaiming Data.

A variety of approaches have been presented for assessing desirable responding in self-report measures. Among them, the overclaiming technique asks respondents to rate their familiarity with a large set of real and nonexistent items (foils). The application of signal detection formulas to the endorsement rates of real items and foils yields indices of (a) knowledge accuracy and (b) knowledge bias. This overclaiming technique reflects both cognitive ability and personality. Here, we develop an alternative measurement model based on multidimensional item response theory (MIRT). We report three studies demonstrating this new model's capacity to analyze overclaiming data. First, a simulation study illustrates that MIRT and signal detection theory yield comparable indices of accuracy and bias-although MIRT provides important additional information. Two empirical examples-one based on mathematical terms and one based on Chinese idioms-are then elaborated. Together, they demonstrate the utility of this new approach for group comparisons and item selection. The implications of this research are illustrated and discussed.

METODA IDENTIFIKACE NADHODNOCOVANI VLASTNICH ZNALOSTI VE VYZKUMU PISA: PREHLEDOVA STUDIE

The overclaiming technique (OCT) is frequently used in research. It is based on analysis of reported familiarity with existing and non-existing items. The inclusion of the OCT in PISA is important for educational research. The aim of this article is to thematically categorize studies using PISA 2012 OCT data, which has not been done before. We identified five thematic categories: mathematics education, questionnaire surveys responding, dishonest behavior, (over)confidence, and inequalities between students. In future, the technique can be used in the context of school misbehavior research.

Can Overclaiming Technique Improve Self-Assessment Tools for Digital Competence? The Case of DigCompSat

Digital competence is crucial for living, working and participating in current societies. Despite its huge importance, objective measurement tools for it are scarce due to its developmental difficulty. Self-assessment of digital competence seems a promising proxy of objective tests, and it additionally offers the possibility for surveying otherwise unmeasurable constructs such as attitudes and beliefs. However, self-assessment tools are burdened with validity problems, most notably response biases such as overly positive descriptions, overclaiming or careless and insufficient effort responding. In this paper, we investigate how these problems can be mitigated by using the overclaiming technique, a technique that identifies and corrects the bias variance in self-assessments. Our main result was that the use of the overclaiming technique can lead to higher reliability and validity of digital competence self-assessment tools, especially for short scales. Moreover, it allows for correcting additional spurious variance in comparison with careless responding indexes, which allows the use of both these techniques in parallel to increase the quality of data. Our results are important in providing advances in enhanced information on digital competence that can result in better lifelong learning decisions when used at the individual level and in better policy-making decisions when used at the aggregate level.

Identifying the accuracy of and exaggeration in self-reports of ICT knowledge among different groups of students: The use of the overclaiming technique

An inaccurate self-assessment might hinder the effectiveness of the self-management of one's learning, for example, people underestimating their knowledge might not utilize their capabilities. Inaccuracies in self-reported knowledge also hinder the comparability of the data across different respondents. The overclaiming technique (OCT) allows the identification of accuracy and exaggeration in self-reported knowledge and enhances the comparability of the data. Respondents are asked to evaluate their degree of familiarity with a series of items including both existing concepts from a specific field of knowledge and non-existing words. The technique has been used in various knowledge domains (e.g. mathematics) but no previous study has used the OCT in the domain of ICT knowledge on a large-scale sample. This study fills this gap. We created a set of existing ICT concepts and non-existing words. Using the set, we studied the differences in the accuracy and exaggeration of the ICT knowledge self-reports of Czech students at upper secondary schools, which have different emphases on ICT skills development (N = 1,871). ICT students report that they are more familiar than students from other school types with not only existing concepts but also non-existing words. Also, males claim more familiarity with both existing and non-existing concepts. Regards external validity, the OCT adjusted familiarity index corresponds more closely to the actual knowledge, as measured by an ICT achievement test, (r = 0.53) than the unadjusted familiarity index (r = 0.37). We show that the adjustment of student self-reports using the OCT offers promising results.

Liar! Liar! (when stakes are higher): Understanding how the overclaiming technique can be used to measure faking in personnel selection.

Overclaiming questionnaires (OCQs), which capture overclaiming behavior, or exaggerating one's knowledge about a given topic, have been proposed as potentially indicative of faking behaviors that plague self-report assessments in job application settings. The empirical evidence on the efficacy of OCQs in this respect is inconsistent, however. We draw from expectancy theory to reconcile these inconsistencies and identify the conditions under which overclaiming behavior will be most indicative of faking. We propose that the assessment context must be tied to an outcome with high valence, and that the content of the OCQ must match the perceived knowledge requirements of the target job, such that overclaiming knowledge of that content will be instrumental to receiving a job offer. We test these propositions through three studies. First, in a sample of 519 applicants to firefighter positions, we demonstrate that overclaiming on a job-relevant OCQ is positively associated with other indicators of faking and self-presentation. Next, we demonstrate through a repeated-measures experiment (N = 252) that participants in a simulated personnel selection setting overclaim more knowledge on a job-relevant OCQ than on a job-irrelevant OCQ, compared with when they are instructed to respond honestly. Finally, in a novel repeated-measures personnel selection paradigm (N = 259), we observed more overclaiming during a selection assessment compared with a research assessment, and we observed that this job-application overclaiming behavior predicted deviant behavior following selection. Altogether, the results show that overclaiming behavior is most indicative of faking in job application assessments when an OCQ contains job-relevant (rather than job-irrelevant) content. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Self-enhancement and cross-cultural adjustment: overclaiming in a global mobility context

PurposeThis study aimed to test Early and Ang’s (2003) proposition that self-enhancement hinders successful cross-cultural adjustment. The literature on self-enhancement is reviewed, and the overclaiming technique as an unobtrusive measure of self-enhancement is introduced for use in global mobility contexts.Design/methodology/approachUsing the overclaiming technique, an international-cultural overclaiming test was developed. Expatriates in Singapore stated their familiarity with international-cultural knowledge items, with some of them being foil items, and rated their cross-cultural (general, interaction, work) adjustment. Supervisors rated the expatriates on their work adjustment and performance.FindingsOverclaiming was not related to self-rated cross-cultural adjustment. However, overclaiming was negatively related to supervisor rated work adjustment and performance. Additionally, the results showed that international-cultural knowledge accuracy was positively related to self-rated general adjustment and to supervisor rated work adjustment and performance.Research limitations/implicationsThe small sample size with a majority of expatriate teachers from international schools in the sample makes it necessary for the results to be replicated with larger and more varied expatriate samples.Practical implicationsWhile further validation is needed, this research indicates that the overclaiming technique could be a valuable tool for assessing self-enhancement in candidates for expatriate positions in order to gauge potential cross-cultural (mal)adjustment, as perceived by others.Originality/valueThis study was (likely) the first study that has applied the overclaiming technique in a global mobility context. An international-cultural knowledge overclaiming test is provided to academic researchers for future use.

Supplemental Material for Liar! Liar! (When Stakes Are Higher): Understanding How the Overclaiming Technique Can Be Used to Measure Faking in Personnel Selection

Supplemental Material for Liar! Liar! (When Stakes Are Higher): Understanding How the Overclaiming Technique Can Be Used to Measure Faking in Personnel Selection

True virtue, self-presentation, or both?: A behavioral test of impression management and overclaiming.

Measures of self-favoring response biases such as impression management (IM) scales or the overclaiming technique (OCT) have been developed to detect distorted self-reports. However, the validity of these approaches has been questioned. In the present study (N = 461), we further examined both IM (assessed by the respective subscale of the Balanced Inventory of Desirable Responding [BIDR]) and the OCT as measures of self-favoring response biases and their associations with honesty-humility. Specifically, we tested the 2 competing accounts of IM as a measure of true virtues or a measure of overly positive self-presentation against each other. Using 2 behavioral paradigms, we corroborated recent findings that higher IM scores are associated with higher trait honesty as well as with more honest behavior. At the same time, individuals with high IM scores also presented themselves overly favorably, as IM accounted for discrepancies between hypothetical and actual (incentivized) behavior in a dictator game. Thereby, the present results provide direct evidence for the notion that the IM subscale of the BIDR confounds trait honesty with response bias and thus measures both. In contrast, overclaiming was unrelated to trait honesty, all observed behaviors, and self-presentation, indicating that the OCT is not a valid measure of self-favoring response bias. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Enhancing the Cross-Cultural Comparability of Self-Reports Using the Overclaiming Technique: An Analysis of Accuracy and Exaggeration in 64 Cultures

The overclaiming technique (OCT) is a novel way of measuring how socially desirable responding influences survey responses. It has the potential to enhance the cross-cultural comparability of survey data. It allows the identification of respondents’ knowledge accuracy and exaggeration by comparing their assessments of familiarity with existing and nonexisting concepts in a particular field of knowledge. We aim to compare the response patterns of countries and world regions based on their OCT accuracy and exaggeration-index values and validate these OCT scores using external variables. We also introduce a general model for the categorization of respondents based on their OCT indices values. We use the Programme for International Student Assessment (PISA) 2012 data from 64 countries ( N = 275,904). We found considerable differences in response patterns across world regions: high accuracy values in East Asia, low accuracy values in Southern and Central America, high exaggeration values in Southern Europe, and low exaggeration values in Western Europe. Furthermore, we show that familiarity with math concepts changes substantially after adjustment using the OCT. The correlation between unadjusted math familiarity and math test scores is weak and nonsignificant (.13) whereas after adjustment the correlation becomes strong and significant (.66). Concerning other indicators such as gross domestic product (GDP), public expenditure in education, and Corruption Perceptions Index (CPI), before adjustment the correlation is negative and even significant for CPI (–.11, –.22, and –.45) whereas after adjustment the correlation becomes, though nonsignificant, positive (.05, .19, and .07). We also discuss the OCT in the context of other methods indicative of culturally preferred scale usage.

Controlling for Response Bias in Self-Ratings of Personality: A Comparison of Impression Management Scales and the Overclaiming Technique

Self-serving response distortions pose a threat to the validity of personality scales. A common approach to deal with this issue is to rely on impression management (IM) scales. More recently, the overclaiming technique (OCT) has been proposed as an alternative and arguably superior measure of such biases. In this study (N = 162), we tested these approaches in the context of self- and other-ratings using the HEXACO personality inventory. To the extent that the OCT and IM scales can be considered valid measures of response distortions, they are expected to account for inflated self-ratings in particular for those personality dimensions that are prone to socially desirable responding. However, the results show that neither the OCT nor IM account for overly favorable self-ratings. The validity of IM as a measure of response biases was further scrutinized by a substantial correlation with other-rated honesty-humility. As such, this study questions the use of both the OCT and IM to assess self-serving response distortions.

Teasing Apart Overclaiming, Overconfidence, and Socially Desirable Responding.

Contamination with positivity bias is a potential problem in virtually all areas of psychological assessment. To determine the impact of positivity bias, one common approach is to embed special indicators within one's assessment battery. Such tools range from social desirability scales to overconfidence measures to the so-called overclaiming technique. Despite the large literature on these different approaches and underlying theoretical notions, little is known about the overall nomological network-in particular, the degree to which these constructs overlap. To this end, a broad spectrum of positivity bias detection tools was administered in low-stakes settings ( N = 798) along with measures of the Big Five, grandiose narcissism, and cognitive ability. Exploratory factor analyses revealed six first-order and two second-order factors. Overclaiming was not loaded by any of the six first-order factors and overconfidence was not explained by either of the two second-order factors. All other measures were confounded with personality and/or cognitive ability. Based on our findings, overclaiming is the most distinct potential indicator of positivity bias and independent of known personality measures.

Feeling of knowing and over-claiming in students from secondary school to university

The feeling of knowing (FOK) is a component of meta-memory that helps people decide if they know or do not know a specific piece of information. By using over-claiming technique as a convenient method for measuring FOK, this study aims to analyse if it varies systematically across different academic levels. It is worth to analyse this issue because FOK may be influenced by the different curricular content learned in each course. Results show that FOK of compulsory secondary school students (aged 15–16, n=506) is significantly lower than that of students of upper secondary education (aged 16 to 17, n=469), which in turn, is also significantly lower than that of the undergraduates (aged 18 and more, n=968). Also, the research has found that FOK is significantly different depending on the specialization of the students in university. Finally, over claiming technique is shown to be a new and simple way to evaluate FOK, a construct that can contribute to a better understanding of individual differences in academic learning.

Using Over-Claiming Technique to probe social desirability ratings of personality items: A validity examination

During personality inventory construction, researchers often exclude items that correlate strongly with a social desirability measure. The validity of this practice, however, may vary depending on the choice of social desirability measure and this concern has not been thoroughly evaluated. To redress this methodological gap, the current research examines the validity of using the Over-Claiming Technique (OCT; Paulhus, Harms, Bruce, & Lysy, 2003) for probing item social desirability. Results indicate that item desirability values gauged with the OCT showed weak evidence for validity; they correlated negatively, rather than positively, with judges’ direct ratings of personality item desirability. Therefore, a recommendation is developed against the usage of the OCT in gauging item desirability. Possible reasons for the weak validity evidence and future research directions are discussed.

Application of the Overclaiming Technique to Scholastic Assessment

The overclaiming technique is a novel assessment procedure that uses signal detection analysis to generate indices of knowledge accuracy (OC-accuracy) and self-enhancement (OC-bias). The technique has previously shown robustness over varied knowledge domains as well as low reactivity across administration contexts. Here we compared the OC-accuracy index with multiple choice (MC) and short answer (SA) tests in assessing knowledge of introductory psychology topics in a sample of 108 undergraduates. Results indicated that OC-accuracy was (a) comparable to MC and SA in predicting overall course grades and (b) superior to SA tests in reliability achieved per unit administration time. By including the OC-bias index, the overclaiming method also adds a unique element to scholastic testing, namely, a measure of knowledge self-enhancement. The latter index was a negative predictor of overall course grade, suggesting a narcissistic self-destructiveness. Because the self-enhancement index adds no extra administration time to the knowledge measure, the overclaiming approach provides a more rich and efficient information source compared with traditional methods of scholastic assessment.

Overclaiming as a measure of faking

Researchers have recently asserted that popular measures of response distortion (i.e., socially desirable responding scales) lack construct validity (i.e., measure traits rather than test faking) and that applicant faking on personality tests remains a serious concern ( Griffith & Peterson, 2008; Holden, 2008). Thus, although researchers and human resource (HR) selection specialists have been attempting to find measures which readily capture individual differences in faking that increase personality test validity, to date such attempts have rarely, if ever succeeded. The current study, however, finds that the overclaiming technique captures individual differences in faking and subsequently increases personality test score validity via suppressing unwanted error variance in personality test scores. Implications of this research on the overclaiming technique for improving HR selection decisions are illustrated and discussed.

Examining mental health literacy and its correlates using the overclaiming technique

The present study examined mental health literacy among the British general public using the overclaiming technique, where participants rate for familiarity a series of items, some of which are foils resembling real items. In total, 344 Britons rated for familiarity 30 mental health disorders, of which six were foils, and also completed measures of the Big Five personality factors, self-rated intelligence, self-reported knowledge of psychiatry, psychiatric skepticism, and their personal details. Using signal detection analysis, responses on the overclaiming measure were used to compute an accuracy index (accuracy in distinguishing real mental health disorders from foils) and a response bias (the tendency to response affirmatively to both real disorders and foils). Preliminary analysis showed that there were no significant sex differences on either the accuracy or response bias indices. Further analyses showed that greater accuracy was significantly predicted by higher self-rated intelligence, greater knowledge of psychiatry, and lower psychiatric skepticism. These results are discussed in terms of the benefits of using the overclaiming technique and signal detection analysis in the study of mental health literacy.

The recognition of mental health disorders and its association with psychiatric scepticism, knowledge of psychiatry, and the Big Five personality factors: an investigation using the overclaiming technique

The present study examined the general public's ability to recognise mental health disorders and this ability's association with psychiatric scepticism, knowledge of psychiatry, and the Big Five personality factors. A total of 477 members of the British general public completed an overclaiming scale, in which they were asked to rate the degree to which they believed 20 mental health disorders (of which five were foils designed to resemble real disorders) were real or fake. Participants also completed a novel scale measuring psychiatric scepticism, a single-item measure of knowledge of psychiatry, and a measure of the Big Five personality factors. Results showed that participants were significantly more likely to rate foils as fake disorders than real disorders. In addition, the difference between real and foil ratings was significantly predicted by knowledge of psychiatry, psychiatric scepticism, and the Big Five personality factors of agreeableness and openness to experience. These results are discussed in relation to the overclaiming technique as a novel method to study mental health literacy.

Measuring cognitive ability with the overclaiming technique

The overclaiming technique requires respondents to rate their familiarity with a list of general knowledge items (persons, places, things). Because 20% of the items are foils (i.e., do not exist), the response pattern can be analyzed with signal detection methods to yield the accuracy and bias scores for each respondent. In Study 1, the accuracy index showed strong associations with two standard IQ tests ( β=.50–.59). Study 2 replicated the validity of the accuracy index with IQ and showed coherent associations with school grades, peer ratings, and self-ratings. Study 3 ensured that the validity of the accuracy index is invulnerable to a warning about foils. Study 4 showed that validity is maintained even when the ratings are completed at home. The discussion focuses on the advantages of this self-report approach over standard ability tests, namely, minimal stress, lack of time limit, or need for supervision, and flexibility of content.

The over-claiming technique: measuring self-enhancement independent of ability.

Over-claiming is a concrete operalization of self-enhancement based on respondents' ratings of their knowledge of various persons, events, products, and so on. Because 20% of the items are nonexistent, responses can be analyzed with signal detection formulas to index both response bias (over-claiming) and accuracy (knowledge). Study 1 demonstrated convergence of over-claiming with alternative measures of self-enhancement but independence from cognitive ability. In Studies 2-3, the validity of the over-claiming index held even when respondents were (a) warned about the foils or (b) asked to fake good. Study 3 also showed the utility of the over-claiming index for diagnosing faking. In Study 4, the over-claiming technique was applied to the debate over the adaptive value of positive illusions.