Honest Condition Research Articles

A Step Forward in Identifying Socially Desirable Respondents: An Integrated Machine Learning Model Considering T‐Scores, Response Time, Kinematic Indicators, and Eye Movements

Context: In high‐stakes assessments, such as court cases or managerial evaluations, decision‐makers heavily rely on psychological testing. These assessments often play a crucial role in determining important decisions that affect a person’s life and have a significant impact on society.Problem Statement: Research indicates that many psychological assessments are compromised by respondents’ deliberate distortions and inaccurate self‐presentations. Among these sources of bias, socially desirable responding (SDR) describes the tendency to provide overly positive self‐descriptions. This positive response bias can invalidate test results and lead to inaccurate assessments.Objectives: The present study is aimed at investigating the utility of mouse‐ and eye‐tracking technologies for detecting SDR in psychological assessments. By integrating these technologies, the study sought to develop more effective methods for identifying when respondents are presenting themselves in a favorable light.Methods: Eighty‐five participants completed the Lie (L) and Correction (K) scales of the Minnesota Multiphasic Personality Inventory‐2 (MMPI‐2) twice: once answering honestly and once presenting themselves in a favorable light, with the order of conditions balanced. Repeated measures univariate analyses were conducted on L and K scale T‐scores, as well as on mouse‐ and eye‐tracking features, to compare the honest and instructed SDR conditions. Additionally, machine learning models were developed to integrate T‐scores, kinematic indicators, and eye movements for predicting SDR.Results: The results showed that participants in the SDR condition recorded significantly higher T‐scores, longer response times, wider mouse trajectories, and avoided looking at the answers they intended to fake, compared to participants in the honest condition. Machine learning algorithms predicted SDR with 70%–78% accuracy.Conclusion: New assessment strategies using mouse‐ and eye‐tracking can help practitioners identify whether data is genuine or fabricated, potentially enhancing decision‐making accuracy.Implications: Combining self‐report measures with implicit data can improve SDR detection, particularly in managerial, organizational, and forensic contexts where precise assessments are crucial.

Are traditional interviews more prone to effects of impression management than structured interviews?

AbstractResearch on whether interviewees can improve their interview ratings through impression management (IM) relative to an honest condition has focused on highly structured interviews whereas traditional interviews have received little attention. Thus, this study aimed to determine how prone traditional compared to highly structured interviews are to effects of IM. Therefore, we conducted simulated selection interviews using a 2 × 2 within‐subjects design. All participants went through a condition in which they were asked to present themselves as honestly as possible and a condition in which they were instructed to act like an applicant. Additionally, each interview contained eight traditional and eight structured questions. The differences in the usage of self‐reported honest and deceptive IM between the honest and applicant conditions were comparable for both interview types. Furthermore, interview ratings were better in the applicant condition compared to the honest condition, and importantly, this improvement was larger for the traditional interview part compared to the structured interview part. Even though the larger performance improvement was not reflected in self‐reported honest and deceptive IM, our results suggest that it is easier for applicants to intentionally improve their performance ratings in traditional interviews. Additionally, performance improvements correlated positively with applicants' ability to identify criteria.

Detecting Symptom Overreporting – Equivalence of the Dutch and German Self-Report Symptom Inventory

Abstract. The Self-Report Symptom Inventory (SRSI) intends to measure symptom overreporting. To assess the Dutch and German SRSI equivalence, both versions were split into two half-forms. Forty bilingual participants were randomly allocated to two groups that completed the first half in German and the second half in Dutch or vice versa. Each group completed the SRSI honestly and then under feigning instructions. For both conditions, the Dutch and German SRSI did not statistically significantly differ within and across the two groups. For most comparisons, the Bayes factor was ≥ 3, indicating moderate evidence favoring the equivalence of language versions and half-forms. Genuine and pseudosymptoms endorsement was significantly higher in the feigning than in the honest condition (both Zs = 5.44, r rb = 1.00). The SRSI standard cut score correctly identified honest responding and detected 80% of feigned responses. Our results align with Giger and Merten’s (2019) German and French SRSI equivalence study.

The impact of fabrication on recognition memory: An experimental study

We examined whether fabrication affects memory using a new paradigm combining the Deese/Roediger-McDermott paradigm (DRM) and the Amsterdam Short Term Memory task (ASTM). Participants were assigned to either a forced fabrication or honest condition, and encoded emotionally-negative and neutral wordlists by reading words out loud. The wordlists contained words that were associatively related to each other (e.g., symphony, sound, piano, radio, sing, orchestra), and converged to a non-presented critical word (e.g., music). Next, all participants had to indicate two words that were previously read out loud. However, participants in the forced fabrication condition had to fabricate that they read two additional words out loud, which were associated to the same DRM list. Participants in the forced fabrication condition formed false memories for their self-generated fabrications. Although fabrication did not increase spontaneous false memory rates for non-presented critical words, our results did highlight the perils of forcing people to fabricate.

Use of mouse-tracking software to detect faking-good behavior on personality questionnaires: an explorative study

The aim of the present study was to explore whether kinematic indicators could improve the detection of subjects demonstrating faking-good behaviour when responding to personality questionnaires. One hundred and twenty volunteers were randomly assigned to one of four experimental groups (honest unspeeded, faking-good unspeeded, honest speeded, and faking-good speeded). Participants were asked to respond to the MMPI-2 underreporting scales (L, K, S) and the PPI-R Virtuous Responding (VR) scale using a computer mouse. The collected data included T-point scores on the L, K, S, and VR scales; response times on these scales; and several temporal and spatial mouse parameters. These data were used to investigate the presence of significant differences between the two manipulated variables (honest vs. faking-good; speeded vs. unspeeded). The results demonstrated that T-scores were significantly higher in the faking-good condition relative to the honest condition; however, faking-good and honest respondents showed no statistically significant differences between the speeded and unspeeded conditions. Concerning temporal and spatial kinematic parameters, we observed mixed results for different scales and further investigations are required. The most consistent finding, albeit with small observed effects, regards the L scale, in which faking-good respondents took longer to respond to stimuli and outlined wider mouse trajectories to arrive at the given response.

The Relationship Between Faking and Response Latencies

Abstract. The purpose of this meta-analysis was to analyze the relationship between faking and response latencies (RL). Research studies included in online databases, as well as papers identified in previous reviews, were considered for selection. Inclusion criteria for the studies were (a) to have an experimental faking condition, (b) to measure RL using a computer, and (c) to provide data for calculating the d Cohen effect sizes. Overall effects were significant in the case of honest versus fake good condition ( d = 0.20, Z = 3.05, p < .05), and in the case of honest versus fake bad condition ( d = 0.39, Z = 2.21, p < .05). Subgroup analyses indicated moderator effects of item type, with larger effects computed on RL of positively keyed items, as compared with RL of negatively keyed items.

Detecting Cheating when Testing Vision: Variability in Acuity Measures Reveals Misrepresentation.

In certain scenarios, it is advantageous to misrepresent one's ability and "cheat" on vision tests. Our findings suggest that increased variability when testing visual acuity holds promise as a novel means to help detect this cheating and may generalize to other subjective tests of visual function. People who cheat on vision tests generally do so to make their vision appear better than it actually is (e.g., for occupational or driving purposes). However, there are particular settings in which it is advantageous for their vision to appear to be worse than is the case (e.g., to qualify for benefits available to people with low vision). Therefore, a method to help detect cheating in these scenarios is desirable. The aim of this study was to investigate whether the intentional underrepresentation of vision could be detected when testing visual acuity. We tested the visual acuity of 13 participants with simulated vision impairment using the Berkeley Rudimentary Vision Test. Participants were tested in an honest condition when providing their best effort and in a cheating condition when attempting to make their visual acuity appear to be markedly worse. We also tested visual acuity of 17 participants with a wide range of vision impairments. Participants were successfully able to "cheat" on the tests; however, their responses were significantly more variable when cheating (P < .001). Although the variability in visual acuity was larger in individuals with actual vision impairment compared with those providing honest answers with simulated impairment (P < .01), their responses remained significantly less variable than those for individuals in the cheating condition (P = .01). The variability in the estimations of vision provides a promising novel means of detecting the intentional underrepresentation of vision and could help to minimize the chance of successfully cheating on tests of vision.

Tell Me Sweet Little Lies: How Does Faking in Interviews Affect Interview Scores and Interview Validity?

Interviews are a prevalent technique for selection and admission purposes. However, interviews are also viewed as potentially fakeable, raising the question of whether interviewees’ faking behavior impairs the quality of selection decisions. To address these concerns, our study examined whether interviewees can actually improve their interview score by faking and the role that interviewee ability factors play in interview faking. We also explored the effect of faking on criterion-related validity with regard to successfully predicting interviewees’ task and contextual performance. We conducted simulated interviews in an honest and an applicant instruction condition using a within-subjects design. In line with our hypotheses, interviewees were able to improve their interview scores when asked to respond as an applicant. The size of the improvement of these interview scores correlated with interviewees’ cognitive ability and their ability to identify the targeted interview dimensions. Concerning the effects of faking on criterion-related validity, we found that academic performance was better predicted in the applicant instruction condition whereas contextual performance was better predicted in the honest condition. Thus, it appears that claims that “faking impairs criterion-related validity” are too simplified and that we have to consider the kind of criterion predicted.

Are observer ratings of applicants’ personality also faked? Yes, but less than self‐reports

AbstractAlthough self‐report personality tests are a comparatively cheap and easy‐to‐administer personnel selection tool, researchers have criticized them for not predicting enough criterion‐related variance. Researchers have suggested using observer‐ratings of personality (e.g., as part of a reference check from a supervisor) because observer‐ratings have been reported to be more predictive. However, it is theoretically and empirically unclear whether supervisors also engage in faking (the intentional distortion of responses). Study 1 explored faking among managers who were first asked to imagine that a subordinate had to leave his/her job for private reasons and then to rate the personality of the subordinate. A week later, managers rated their subordinates honestly. A repeated‐measures MANOVA indicated that managers did fake. Study 2 (among supervisors of working students) replicated the above findings but also showed that there is less faking in supervisor‐ratings than in self‐ratings. Furthermore, we found no evidence that the validity of personality scales for predicting academic performance depends on self‐ versus observer‐ratings or on an applicant versus an honest condition. These two studies thus show that practitioners should not equate personality ratings obtained from observers in a selection context with honest personality ratings.

Exploring Children's Induced Defensiveness to the Tell Me a Story Test (TEMAS)

ABSTRACTThis study examined the effect of inducing a defensive attitude (fake good) in children on their responses to the Tell Me a Story test (TEMAS; Costantino, Malgady, & Rogler, 1988). Different story elicitation procedures were employed to explore their efficacy for inducing a defensive response style in children. Eighty-four Italian children aged 8 to 10 years old completed the TEMAS and the L scale of the Behavioral Assessment Scale for Children–2 (BASC–2; Reynolds & Kamphaus, 2004), which were used to detect the adoption of a defensive attitude. Children completed the tests under 1 of 3 conditions: (a) honest condition, (b) fake good with verbal instructions, and (c) fake good with verbal instructions and visual stimuli illustrating what was verbally explained. Using simultaneous verbal instructions and visual stimuli facilitated children's understanding of the researcher's instructions compared to using verbal stimuli alone. Results showed that children instructed to fake good used less time to tell stories and tended to modify the simpler and more face valid aspects of their stories. Still, 29 out of 47 children failed to fake good on the BASC–2 L scale. More complex TEMAS variables showed no significant difference between story elicitation conditions.

Cross‐Cultural Differences in Applicant Faking on Personality Tests: A 43‐Nation Study

In a globalised world, more and more organisations have to select from pools of applicants from different cultures, often by using personality tests. If applicants from different cultures were to differ in the amount of faking on personality tests, this could threaten their validity: Applicants who engage in faking will have an advantage, and will put those who do not fake at a disadvantage. This is the first study to systematically examine and explain cross‐cultural differences in actual faking behavior. In N = 3,678 employees from 43 countries, a scenario‐based repeated measures design (faking vs. honest condition) was applied. Results showed that faking differed significantly across countries, and that it was systematically related to countries’ cultural characteristics (e.g. GLOBE's uncertainty avoidance, future orientation, humane orientation, and in‐group collectivism), but in an unexpected way. The study discusses these findings and their implications for research and practitioners.

Faking Fast and Slow: Within-Person Response Time Latencies for Measuring Faking in Personnel Testing

Item response time (RT) latencies offer a potentially promising approach for measuring faking in personnel testing, but have been studied almost exclusively as either long or short RTs relative to group norms. As such, the ability to reliably assess faking RTs at the individual level remains a challenge. To address this issue, the present study set out to examine the usefulness of a within-person difference score index (DSI) method for measuring faking, in which “control question” (baseline) RTs were compared to “target question” RTs, within single test administrations. Two hundred six participants were randomly selected to simulated faking or honest testing conditions, and were administered two types of integrity test items (overt and personality), whereby group classification (faking/honest) served as the main dependent variable. Faking condition RTs were longer than honest condition RTs for both item types (overt: d = .43; personality: d = .47), and overt item RTs were slightly shorter than personality item RTs in both testing conditions (honest: d = .34; faking: d = .41). Finally, using a sample cut score, the DSI correctly classified an average of 26 % more cases of faking, and 53 % less false positives, compared to the traditional normative method. The results suggest that the DSI can be an advantageous method for identifying faking in personnel testing scenarios. This is the one of the first studies to propose a practical method for identifying individual-level faking RTs within single test administrations.

Faking in Job Applicants' Responses in Personality Tests: Evidence from A Eye-Tracking Study of Job Desirability

One of the important concerns in personnel selection process has been job applicants' faking behaviors in Personality tests(or measurement or evaluation). Although many studies have been done on faking behaviors in personality tests, no consensus has yet reached regarding the response processes of faking behaviors in personality tests. At present, researchers have proposed three mutually incompatible social–desirability–based response process models of faking in personality tests: the Self–Schema Model, the Sematic–Exercise Model and the Adopted–Schema Model. In the Adopted–Schema Model, test items are classified as social desirable and social undesirable, but items unrelated to social desirability are neglected. Besides, more and more researchers are inclined to consider faking in personnel selection as a job desirable behavior instead of a social desirable one. Therefore, this study tried to explore job applicants' faking response in personality tests from the perspective of job desirability with the help of eye–tracking techniques. First, fifty participants rated the job desirability of 44 items in Big Five Inventory(BFI-44). Based on the rating scores, BFI-44 items were classified into three categories: job desirable items, job undesirable items and items unrelated to job desirability. Second, in a within–subject simulation experiment design, another fifty participants completed the BFI-44 in two conditions – honest vs. faking in an eye–tracking laboratory. To eliminate order effect, these 50 participants were randomly assigned to the two groups. The first group went through the honest condition and then the faking condition. The second group followed the reversed order. The participants were instructed to complete six items from Mensa IQ test as a filler task between the two sessions. Item responses, response latencies, and eye movement index were recorded using Tobii 120. The results showed that test scores on all of the five dimensions of the Big Five Inventory under the faking condition were significantly higher than the scores under the honest condition. The response latencies on items in the categories of job desirable and job undesirable were significantly shorter in the faking condition than the response latencies in both categories in the honest condition. The number of eye fixations was significantly lower on the question stems than on the extreme options of the categories of items on job desirable and job undesirable in the faking condition, when compared with the honest condition. In the category of items unrelated to job desirability, the number of eye fixations was significantly more in the faking condition than in the honest condition. The same pattern of eye fixations was found on the options in the middle. In the faking condition, the participants' eyes fixed on extreme options(i.e., strongly disagree and strongly agree) more directly after reading the questions. These findings support the idea that faking leads to semantic–exercise interpretations on job desirable and job undesirable items, as well as self–schema interpretations on items unrelated to job desirability. The response process in the faking condition seemed to be simpler than the response process in the honest condition, when answering the items in the categories of job desirability and job undesirability. Based on the findings, the job–desirability–based Mixed–Exercise Model has been proposed, in an attempt to explain faking response in personality tests.

Spanish Adaptation of the Memory Characteristics Questionnaire (MCQ).

The Memory Characteristics Questionnaire (MCQ) was developed by Johnson, Foley, Suengas, and Raye (1988) to assess the characteristics of memories of external and internal origin, postulated in the source monitoring model (Johnson, Hashtroudi, & Lindsay, 1993). The MCQ was translated into Spanish using a back-translation method. Psychometric properties of the translated MCQ were tested using responses collected from an experimental study simulating a forensic context. Ten police officers and 8 psychologists individually interviewed 240 university students who completed the MCQ after reporting what they had seen in a film. Half of the participants were asked to tell the truth, while the other half were asked to lie. The results have shown adequate psychometric properties of the Spanish MCQ items for the total sample and across experimental conditions. Cronbach's alpha value was .79 for the total sample, .78 for the honest condition, and .76 for the lie condition. Validity evidence of dimensionality supports that the factor structure of Spanish MCQ was equivalent to that proposed by the authors of the original version. Also, a two-factor ANOVA (video clip x condition) was performed to analyze experimental data. Neither interaction effects, F(236) = 1.189; p = .277, nor main effects were found to be significant between those asked to tell the truth and those asked to lie. These results demonstrate that the Spanish MCQ has adequate psychometric properties.

Deliberate Faking on Personality and Emotional Intelligence Measures

This study examined the extent the Big Five personality traits and emotional intelligence can be faked. Using a student sample, the equivalence of measurement and theoretical structure of models in a faking and honest condition was tested. Comparisons of the models for the honest and faking groups showed the data fit better in the faking condition. These results suggest that faking does change the rank orders of high scoring participants. The personality dimensions most affected by faking were emotional stability and conscientiousness within the Big Five and the general mood and stress management dimensions of Bar-On's Emotional Quotient Inventory-Short Form (1997) measure of emotional intelligence.

Forced-Choice Personality Tests:A Measure of Personality and Cognitive Ability?

This study examined the effects of item format (single-stimulus vs. forced-choice) and response motivation (honest vs. applicant) on scores for personality scales measuring Conscientiousness and Openness to Experience. Consistent with the hypotheses, cognitive ability was related to forced-choice personality scores in the applicant condition but not in the honest condition. Cognitive ability was unrelated to single-stimulus personality scores in both the applicant and honest conditions. The results suggest that controlling for cognitive ability can reduce the incremental predictive validity of forced-choice personality scales in applicant settings. Findings are discussed in terms of the importance of considering how item format influences the construct and criterion-related validity of personality tests used to make selection decisions.

Validity of the SDS-17 measure of social desirability in the American context

In Germany, Stober et al. (1999, 2001) presented evidence for the validity of the SDS-17, a new measure of social desirability bias. In the current investigation, three experiments ( n = 800) assessed the SDS-17’s validity in the US environment. In all conditions SDS-17 scores correlated highly with Marlowe–Crowne scores. In Study 1, a group administration of a paper and pencil booklet, SDS-17 scores of 327 college students were higher under Fake Good than Standard conditions, and both were higher than scores in the Honest condition. Study 2, an online survey of a demographically diverse adult sample ( n = 257), showed that the increase in SDS-17 scores under Fake Good conditions occurs also in a Web survey and that SDS-17 scores were unrelated to one’s demographic profile. Study 3, a group administration to 216 college students, revealed again that scores under Fake Good were higher than those under Standard administration and that SDS-17 scores correlated more highly with the Impression Management than with the Self-Deception subscales of the BIDR. The SDS-17 appeared valid for the US environment as a measure of socially desirable responding. The evidence, however, encourages its further assessment as an index of social desirability bias per se.

Effects of Response Instructions on Faking a Situational Judgment Test

A situational judgment test (SJT) and a Big 5 personality test were administered to 203 participants under instructions to respond honestly and to fake good using a within-subjects design. Participants indicated both the best and worst response (i.e., Knowledge) and the most likely and least likely response (i.e., Behavioral Tendency) to each situation. Faking effect size for the SJT Behavioral Tendency response format was (d=.34) when participants responded first under honest instructions and (d=.15) when they responded first under faking instructions. Those for the Big 5 dimensions ranged from d=.26 to d=1.0. For the Knowledge response format results were inconsistent. Honest condition Knowledge SJT scores were more highly correlated with cognitive ability (r=.56) than were Behavioral Tendency SJT scores (r=.38). Implications for researchers and practitioners are discussed.

Youth deception: Malingering traumatic stress

To explore how youths malinger symptoms of post-traumatic stress disorder (PTSD), college students completed the Trauma Symptom Inventory (TSI) under standard instructions (honest condition). Then, after learning the symptomatology of traumatic stress, they completed the TSI a second time attempting to fake symptoms of PTSD (deception condition). Motivation level was manipulated: 100 students were given course credits, allowing them to avoid writing a research paper, while 50 students were paid and given the incentive of bonus money for successfully faking PTSD symptoms. Cutoff values were applied to the validity scores to identify students who were malingering by exaggerating or by over-endorsing symptoms. Overall, a majority of participants (57%) responded in a manner that fabricated symptoms of PTSD in the deception condition. However, many of the fabricators (45%) did not pass the validity scales and were identified as malingerers. In addition, most of the successful malingers (66%) also over-endorsed the symptoms of dysphoria and reduced internal resources. This pattern of responding suggests that the malingering youths did not selectively endorse only symptoms of traumatic stress. Hence, some youths are capable of using minimal, publicly available information to fake symptoms of PTSD, regardless of motivation.

Assessing Personality With a Structured Employment Interview: Construct-Related Validity and Susceptibility to Response Inflation.

The authors evaluated the extent to which a personality-based structured interview was susceptible to response inflation. Interview questions were developed to measure facets of agreeableness, conscientiousness, and emotional stability. Interviewers administered mock interviews to participants instructed to respond honestly or like a job applicant. Interviewees completed scales of the same 3 facets from the NEO Personality Inventory, under the same honest and applicant-like instructions. Interviewers also evaluated interviewee personality with the NEO. Multitrait-multimethod analysis and confirmatory factor analysis provided some evidence for the construct-related validity of the personality interviews. As for response inflation, analyses revealed that the scores from the applicant-like condition were significantly more elevated (relative to honest condition scores) for self-report personality ratings than for interviewer personality ratings. In addition, instructions to respond like an applicant appeared to have a detrimental effect on the structure of the self-report and interview ratings, but not interviewer NEO ratings.