Evaluative Priming Research Articles

An Improved Scoring Algorithm for Indirect Evaluation Measurement With the Evaluative Priming Task

People tend to categorize the valence of a target stimulus more quickly and accurately if the target appears after a prime stimulus of the same (than of the opposite) valence. The evaluative priming task (EPT) utilizes this priming effect for indirect evaluation measurement but suffers from low reliability. The present research compared the reliability and validity of 2,160 EPT scoring algorithms across 12 datasets. In contrast to current norms to delete trials with error responses and to rely solely on differences in response latency between different task conditions, superior performance was found when incorporating latency and accuracy data, and when basing the score on differences in mean ranking of each condition’s trials by performance (a scale-invariant non-parametric dominance score named G score). We recommend seven new scoring algorithms that, in comparison to current scoring norms, increase internal consistency by a mean of 41% and correlations with other measures by 17%.

The effect of practice on automatic evaluation: A registered replication

A basic idea in cognitive science is that practicing a response can lead to the automatic activation of the response. Fazio, Sanbonmatsu, Powell, and Kardes (1986) tested that idea on the automatic activation of attitudes. In the experiment that Fazio et al. conducted, participants (N = 18) repeatedly categorized eight nouns as good/bad and eight nouns (the control words) as having one syllable or more. The measure of automatic activation of attitudes was evaluative priming: Participants categorized target adjectives as good/bad faster if their valence matched the valence of a prime noun that appeared before them. This priming effect was stronger for repeatedly evaluated words than for control words. Many have cited this article as evidence that practice automatizes evaluation, but the published research that followed focused on the evaluative priming effect, providing only one incidental and unsuccessful replication for the effect of practice on automatic evaluation. In light of the importance of this finding on the one hand, and the lack of a solid evidential basis for it on the other hand, we conducted three experiments that tested the effect of practice on evaluative priming effect. We attempted to directly replicate the original procedure in Experiments 1a and 1b (N = 108, 102, respectively), with Experiment 1b fixing an unintended prime-target contingency in Experiment 1a. Experiment 2 (N = 172) provided a conceptual replication with modified procedures. Practice in evaluation increased priming only in Experiment 1a. The inconsistent results prevent strong conclusions that practicing an evaluative response automates it, necessitating further research.

How do explicit and implicit evaluations shift? A preregistered meta-analysis of the effects of co-occurrence and relational information.

Based on 660 effect sizes obtained from 23,255 adult participants across 51 reports of experimental studies, this meta-analysis investigates whether and when explicit (self-reported) and implicit (indirectly revealed) evaluations reflect relational information (how stimuli are related to each other) over and above co-occurrence information (the fact that stimuli have been paired with each other). Using a mixed-effects metaregression, relational information was found to dominate over contradictory co-occurrence information in shifting both explicit (mean Hedges' g = 0.97, 95% CI [0.89, 1.05], 95% PI [0.24, 1.70]) and implicit evaluations (g = 0.27, 95% CI [0.19, 0.35], 95% PI [-0.46, 1.00]). However, considerable heterogeneity in relational effects on implicit evaluation made moderator analyses necessary. Implicit evaluations were particularly sensitive to relational information (a) in between-participant (rather than within-participant) designs; when (b) co-occurrence information was held constant (rather than manipulated); (c) targets were novel (rather than known); implicit evaluations were measured (d) first (rather than last) and (e) using an affect misattribution procedure (rather than an Implicit Association Test or evaluative priming task); and (f) relational and co-occurrence information were presented in temporal proximity (rather than far apart in time). Overall, the present findings suggest that both implicit and explicit evaluations emerge from a combination of co-occurrence information and relational information, with relational information usually playing the dominant role. Critically, variability in these effects highlights a need to refocus attention from existence proof demonstrations toward theoretical and empirical work on the determinants and boundary conditions of the influences of co-occurrence and relational information on explicit and implicit evaluations. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

A Comparative Investigation of the Predictive Validity of Four Indirect Measures of Bias and Prejudice.

Although measures of implicit associations are influential in the prejudice literature, comparative tests of the predictive power of these measures are lacking. A large-scale (N > 100,000) analysis of four commonly used measures-the Implicit Association Test (IAT), Single-Category IAT (SC-IAT), evaluative priming task (EPT), and sorting paired features task (SPF)-across 10 intergroup domains and 250 outcomes found clear evidence for the superiority of the SC-IAT in predictive and incremental predictive validity. Follow-up analyses suggested that the SC-IAT benefited from an exclusive focus on associations toward stigmatized group members, as associations toward non-stigmatized group members diluted the predictive strength of relative measures like the IAT, SPF, and EPT. These results highlight how conclusions about predictive validity can vary drastically depending on the measure selected and reveal novel insights about the value of different measures when focusing on predictive than convergent validity.

Involuntary evaluation of others' emotional expressions depends on the expresser's group membership. Further evidence for the social message account from the extrinsic affective Simon task.

The social message account (SMA) hypothesizes that the evaluation of emotional facial expressions depends on the ethnicity of the expressers. For example, according to SMA, a happy face of a member of a prejudiced ethnicity is immediately interpreted as potentially malevolent. Evidence for this approach was found initially in evaluative priming (EP) and approach-avoidance tasks (AA) by showing an emotion × ethnicity interaction on positivity scores (EP) and approach scores (AA), respectively. Recently, attempts to replicate the EP results failed. Due to the inconclusive EP results, it was important to examine the influence of ethnicity on processing of emotional expression with another task testing involuntary evaluations. The extrinsic affective Simon task was used with stimuli varying on emotion (happy vs. fear) and ethnicity (White-Caucasian vs. Middle-Eastern men). This task was chosen because in contrast to EP (where faces are presented as task-irrelevant primes) faces are task-relevant. Experiment 1 yielded an emotion × ethnicity interaction with regard to positivity scores that fit SMA predictions. The results are also important in challenging a recent theoretical alternative to SMA, namely the processing conflict account. A generalization of the emotion × ethnicity pattern to learned arbitrary in- and out-groups (Experiment 2) failed, suggesting that involuntary processing of (task-irrelevant) group status depends on perceptual features.

Perceiving a Danger Within: Black Americans Associate Black Men With Physical Threat

Recent work suggests that good/bad out-group favoritism of Blacks for Whites may reflect positive associations with White rather than negative associations with Black. The Dual Implicit Process Model suggests that Blacks may come to associate their own group with threat, even absent a concurrent Black-negative association. This work tests this idea among Black Americans. Three studies tested this possibility using mouse-tracking (Study 1) and evaluative priming tasks (Studies 2 and 3) to assess how quickly participants make judgments involving Black versus White male faces and names. All studies found that that Black Americans hold automatic Black-threat associations absent automatic Black-negative associations. This supports the Dual Implicit Process Model’s threat versus negativity distinction within the realm of anti-Black bias and supplements recent work by showing that the presence of out-group favoritism on one dimension (i.e., threat) can occur even in the absence of out-group favoritism on a seemingly related dimension (i.e., negativity).

How Emotion Relates to Language and Cognition, Seen Through the Lens of Evaluative Priming Paradigms.

Affect and emotion are essential aspects of human life. These states or feelings signal personally relevant things or situations and color our memories and thoughts. Within the area of affective or emotion processing, evaluation–the assessment of the valence associated with a stimulus or event (i.e., its positivity or negativity)–is considered a fundamental process, representing an early and crucial stage in constructivist emotion theories. Valence evaluation is assumed to occur automatically when encountering a stimulus. But does this really apply always, even if we simply see a word? And if so, what exactly is processed or activated in memory? One approach to investigating this evaluative process uses behavioral priming paradigms and, first and foremost, the evaluative priming paradigm and its variants. In the present review, we delineate the insights gained from this paradigm about the relation of affect and emotion to cognition and language. Specifically, we reviewed the empirical evidence base with regard to this issue as well as the proposed theoretical models of valence evaluation, specifically with regard to the nature of the representations activated via such paradigms. It will become clear that affect and emotion are foremost (and, perhaps, even exclusively) triggered by evaluative priming paradigms in the sense that semantic affective knowledge is activated. This knowledge should be modeled as being active in working memory rather than in long-term memory as was assumed in former models. The emerging evidence concerning the processing of more specific emotion aspects gives rise to the assumption that the activation of these semantic aspects is related to their social importance. In that sense, the fast and (conditionally) automatic activation of valence and other emotion aspects in evaluative priming paradigms reveals something about affect and emotion: Valence and specific emotion aspects are so important for our daily life that encountering almost any stimulus entails the automatic activation of the associated valence and other emotion aspects in memory, when the context requires it.

Implicit Bias Reflects the Company That Words Keep.

In everyday language, concepts appear alongside (i.e., collocate with) related concepts. Societal biases often emerge in these collocations; e.g., female (vs. male) names collocate with art- (vs. science-) related concepts, and African American (vs. White American) names collocate with negative (vs. positive) concepts. It is unknown whether such collocations merely reflect societal biases or contribute to them. Concepts that are themselves neutral in valence but nevertheless collocate with valenced concepts provide a unique opportunity to address this question. For example, when asked, most people evaluate the concept “cause” as neutral, but “cause” is frequently followed by negative concepts (e.g., death, pain, and trouble). We use such semantically prosodic concepts to test the influence of collocation on the emergence of implicit bias: do neutral concepts that frequently collocate with valenced concepts have corresponding implicit bias? In evaluative priming tasks, participants evaluated positive/negative nouns (Study 1) or pictures (Study 2) after seeing verb primes that were (a) strongly valenced (e.g., hate and comfort), (b) neutral in valence but collocated with valenced concepts in corpora (e.g., ease and gain), or (c) neutral in valence and not collocated with valenced concepts in corpora (e.g., reply and describe). Throughout, neutral primes with positive (negative) collocates facilitated the evaluation of positive (negative) targets much like strongly valenced primes, whereas neutral primes without valenced collocates did not. That neutral concepts with valenced collocates parallel the influence of valenced concepts suggests that their collocations in natural language may be sufficient for fostering implicit bias. Societal implications of the causal embedding hypothesis are discussed.

"I Do What I Like": 8- to 10-Year-Old Children's Physical Activity Behavior Is Already Interrelated With Their Automatic Affective Processes.

The majority of middle-age children do not meet current physical activity guidelines. There is growing evidence that adults' physical activity is partially influenced by automatic affective processes, which are derived from affective experiences with physical activity. However, little is known about whether these processes are interrelated with children's physical activity level. A prospective design was used to examine whether automatic affective processes assessed by an evaluative priming procedure predict physical activity of children. Physical activity of 48 children (8.71 ± 0.71years; 65% girls) was measured for 1 week with activity trackers. In a linear regression model, automatic affective processes (β = 0.36) significantly predicted physical activity, accounting for 11.02% of variance. These results indicate that physical-activity-related automatic affective processes are associated with children's physical activity, as has previously been found in adults. This study emphasizes the importance of fostering positive affective experiences associated with physical activity during childhood.

“Some hate it, others love it”: Formation of automatic and reflective affective processes toward exercising in fitness centers and mountain biking

Based on a dual process approach, this study was conducted to test whether automatic affective processes toward exercise are not only interrelated with the amount of exercise but also distinctive for different types of exercise. In a quasi-experimental setting, N = 60 participants (30.87 years ± 7.53; 33% female) completed an Evaluative Priming task to assess automatic affective processes toward exercising in fitness centers, mountain biking and exercise in general and completed a questionnaire about their exercise behavior and their reflective affective processes toward each type of exercise. The results showed that the automatic affective processes toward exercising in fitness centers differed significantly among the three groups (d = 0.74). Those who regularly exercise in fitness centers (n = 21) provided the most positive automatic affective processes, followed by mountain bikers (n = 16) and those engaging in little or no exercise (n = 23). Automatic affective processes toward mountain biking were nonsignificant between groups (p = 0.30; d = 0.42). All reflective affective processes assessed via questionnaire resulted in significant differences between the three groups, always in favor of those who often perform the respective exercise. Our results show that automatic affective processes toward exercising in fitness centers are distinctive for the preference for this type of exercise. Furthermore, they underline the importance of choosing an adequate measurement technique to assess automatic affective processes toward exercise, especially if these processes should be indicative for the preference of different types of exercise.

Race, Gender, and the U.S. Presidency: A Comparison of Implicit and Explicit Biases in the Electorate.

Recent U.S. elections have witnessed the Democrats nominating both black and female presidential candidates, as well as a black and female vice president. The increasing diversity of the U.S. political elite heightens the importance of understanding the psychological factors influencing voter support for, or opposition to, candidates of different races and genders. In this study, we investigated the relative strength of the implicit biases for and against hypothetical presidential candidates that varied by gender and race, using an evaluative priming paradigm on a broadly representative sample of U.S. citizens (n = 1076). Our main research question is: Do measures of implicit racial and gender biases predict political attitudes and voting better than measures of explicit prejudice? We find that measures of implicit bias are less strongly associated with political attitudes and voting than are explicit measures of sexist attitudes and modern racism. Moreover, once demographic characteristics and explicit prejudice are controlled statistically, measures of implicit bias provide little incremental predictive validity. Overall, explicit prejudice has a far stronger association with political preferences than does implicit bias.

Me as good and me as bad: Priming the self triggers positive and negative implicit evaluations.

Although theories of personality and human behavior have long assumed that the self is affectively complex, widely used indirect measures of implicit self-evaluations have largely focused on the robustness and cultural universality of the self's positivity. Such indirect measures assess evaluations on a single continuum, ranging from positive to negative. Thus, they focus on the self's relative positivity and are inherently incapable of assessing whether the self is associated with good and bad. Using the well-established evaluative priming task, the present work tested the hypothesis that positive implicit self-evaluations coexist with an inkling of negative implicit self-evaluations. Studies 1 and 2 empirically demonstrated that priming the self facilitated the classification of both positive and negative targets (bivalent-priming). In contrast, replicating classic findings, priming a personally significant, liked object facilitated the classification of positive targets and inhibited the classification of negative targets (univalent-priming). Study 3 showed that the bivalent-priming triggered by self-primes cannot be explained by alternative accounts (e.g., arousal, vigilance). Meta-analyses of all studies attests to the robustness and reproducibility of self-primes triggering both positive and negative implicit evaluations. Moreover, tests estimating heterogeneity in the strength of implicit self-evaluations indicated that individual differences in nonclinical, healthy individuals may be limited, possibly reflecting measurement limitations, the nature of implicit self-evaluations, or both. Overall, the present work shines a spotlight on a previously undocumented effect: Despite the self's robust net positivity, the self reliably triggers negative implicit self-evaluations. Implications for the conceptualization, assessment, and consequences of implicit self-evaluations are discussed. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

"I Know What I Like" - Indecisiveness Is Unrelated to Behavioral Indicators of Evaluation Difficulties.

Indecisiveness, the subjective inability to make satisfying decisions, is an individual difference trait that may impede effective actions. Mechanisms underlying indecisiveness are largely unknown. In four studies, we tested the prediction that indicators of evaluation difficulty were associated with indecisiveness in simple evaluations. Across studies, indecisiveness was measured via self-report while evaluation difficulties were derived behaviorally from three indicators: difficulty distinguishing between similar evaluation objects (i.e., standard deviation of evaluation ratings), evaluation duration (reaction times), and implicit evaluations (evaluative priming effect) using familiar everyday objects. Study 1 (N = 151) was based on attractiveness evaluations of portraits. Studies 2a (N = 201) and 2b (N = 211) used chocolate as evaluation objects and manipulated to what extent the evaluations were equivalent to a decision. In Study 3 (N = 80) evaluations were measured implicitly through evaluative priming using food pictures. Contrary to our predictions, indecisiveness showed no reliable association to any indicator of evaluation difficulty, regardless of type of evaluation object, equivalence of evaluation and decision, and whether evaluation difficulty was based on explicit or implicit evaluations. All null findings were supported by Bayes factors. These counterintuitive results are a first step toward investigating evaluation processes as potential mechanisms underlying indecisiveness, showing that for both explicit and implicit measurements, indecisiveness is not characterized by difficulties when evaluating familiar everyday objects.

Self–other moral bias: Evidence from implicit measures and the Word-Embedding Association Test

Self–other bias can engender disagreement, misunderstanding, and conflict in real-world interactions. Moral perspectives are powerful in shaping thoughts and behaviors. Whether there is a self–other moral bias is poorly understood. We leveraged the Implicit Association Test (Study 1), the evaluative priming task (Study 2), and the Word-Embedding Association Test (Study 3) to demonstrate the self–other moral bias. We found that 1) people held a significantly stronger association between self and deontology (other and utilitarianism) than other and deontology (self and utilitarianism); and 2) the association between the self and other words and the deontological and utilitarian words was further identified in large bodies of English-language text from the Internet. An awareness of the self–other moral bias may facilitate a better understanding of others' attitudes and behaviors.

Facial Stereotype Bias Is Mitigated by Training

People automatically infer others’ personality (e.g., trustworthiness) based on facial appearance, and such facial stereotype biases predict real-world consequences across political, legal, and business domains. The present research tested whether these biases can be mitigated through counterstereotype training aimed at reconfiguring the associations between specific facial appearances and social traits. Across six studies and a replication, a behavioral counterstereotype training consistently reduced or eliminated facial stereotype biases for White male faces in the context of economic trust games, hiring decisions, and even automatic evaluations assessed via evaluative priming. Together, the results demonstrate a fundamental malleability in facial stereotyping related to trustworthiness, with a minimal training able to mitigate the tendency to activate and apply long-held, highly automatized facial stereotypes. These findings suggest that face impressions are more flexible than typically appreciated, and they provide a potential inroad toward combating our ingrained biases based on facial appearance.

The Challenge of Diagnostic Inferences From Implicit Measures: The Case of Non-Evaluative Influences in the Evaluative Priming Paradigm

Implicit measures are diagnostic tools to assess attitudes and evaluations that people cannot or may not want to report. Diagnostic inferences from such tools are subject to asymmetries. We argue that (causal) conditional probabilities p(AM+|A+) of implicitly measured attitudes AM+ given the causal influence of existing attitudes A+ is typically higher than the reverse (diagnostic) conditional probability p(A+|AM+), due to non-evaluative influences on implicit measures. We substantiate this argument with evidence for non-evaluative influences on evaluative priming—specifically, similarity effects reflecting the higher similarity of positive than negative prime-target pairs; integrativity effects based on primes and targets’ potential to form meaningful semantic compounds; and congruity proportion effects that originate in individuals’ decisional strategies. We also cursorily discuss non-evaluative influences in the Implicit Association Test (IAT). These influences not only have implications for the evaluative priming paradigm in particular, but also highlight the intricacies of diagnostic inferences from implicit measures in general.

Twenty-Five Years of Research Using Implicit Measures

The year 2020 marks the 25th anniversary of two seminal publications that have set the foundation for an exponentially growing body of research using implicit measures: Fazio, Jackson, Dunton, and Williams's (1995) work using evaluative priming to measure racial attitudes, and Greenwald and Banaji's (1995) review of implicit social cognition research that served as the basis for the development of the Implicit Association Test (IAT). The current article provides an overview of (1) two conceptual roots that continue to shape interpretations of implicit measures; (2) conflicting interpretations of the term implicit; (3) different kinds of dissociations between implicit and explicit measures; (4) theoretical developments inspired by these dissociations; and (5) research that used implicit measures to address domain-specific and applied questions. We conclude with a discussion of challenges and open questions that remain to be addressed, offering guidance for the next generation of research using implicit measures.

To Run or Not to Run? Automatic Evaluations and Reflective Attitudes Toward Exercise.

Using an evaluative priming procedure, this study tested whether automatic evaluations of running differ among groups based on their amount of exercise and whether they were runners or not. Ninety-five participants (26 ± 5.06years; 46% female) were divided into five groups: an inactive group, active exercisers, highly active exercisers, active runners, and highly active runners. A priming effect score was calculated based on the concept of response facilitation or inhibition: the reaction is faster when the target and prime are valence congruent and becomes slower if they are incongruent. The highly active runner group differed significantly from the inactive group (p < .01) and from the active exerciser group (p < .05). Furthermore, reflective evaluations were measured via questionnaires. The results show that priming effect scores can detect automatic evaluations of running, and they differ not only because of the amounts of physical exercise but also because of their preferred type of exercise.

Testing the judgment-related account for the extinction of evaluative conditioning

ABSTRACT Evaluative conditioning (EC) effects refer to changes in the liking of a neutral (conditioned) stimulus (CS) due to pairing with an affective (unconditioned) stimulus (US). Some research found that EC effects are resistant to presentations of the CS without the US, whereas other studies found evidence for extinction effects. A recent study found extinction of EC only when participants rated the CS before and after the CS-only presentations, but not when CS evaluation was measured once or indirectly with the evaluative priming task. In two experiments (total N = 2,181), we found no evidence that indirectly measured evaluation is sensitive to extinction, using an indirect evaluation measure with high sensitivity – the Implicit Association Test. However, unlike previous research, we found that evaluation of any stimuli (and not only the CS) before the CS-only presentations decreases self-reported EC effects. Our results are compatible with the conclusion that the extinction of EC is limited to evaluation measured directly. We discuss the theoretical implications of these results, and conclude that the specific conditions (and mechanisms) that change the direct evaluative response are yet to be clarified.

Goal-dependent automatic processing of potency: A potency-priming paradigm

Valence, arousal and potency can be conceived of as basic dimensions of meaning that are frequently activated. As an effect of frequent usage, research on evaluative priming has demonstrated that valence can be processed automatically. The present study addresses the question of whether potency as a frequently processed dimension is automatically activated, too. In four experiments we employed a category priming procedure with response window technique to test the assumption that shortly presented primes automatically influence responses to targets and/or the encoding of targets. Stimuli varying in potency (i.e., high vs. low potency) were presented as primes and targets with a short stimulus onset asynchrony (30 ms) whereas potency was the task-relevant stimulus feature. In Experiments 1a/b, participants conducted a strong-weak categorization on adjectives varying in potency and valence (Experiment 1a) or potency and arousal (Experiment 1b). In Experiment 2, we examined if novel primes also contribute to the potency priming effect. In Experiment 3, we tried to replicate the effect of novel primes and tested if potency-priming depends on the task. Results across all experiments revealed higher rates of correct responses on trials where prime and target were compatible in terms of their potency compared to trials with incompatible prime-target combinations. Since the priming effect emerges for novel primes, too, it is likely that prime potency is processed automatically. Moreover, the fact that potency priming was modulated by the task-goal further highlights the automaticity of this process.