Halo Error Research Articles

Effectiveness of Employees’ Performance Appraisal in the Nigerian Public Sector: An Empirical Study

The study identified types of Performance Appraisal (PA) systems in use in Nigerian public sector, investigated effectiveness of PA systems in the sector and examined factors affecting effectiveness of PA systems in the public sector. Purposive sampling technique was used in selecting top, middle and lower levels staff from the various administrative departments within the selected Ministries, Departments and Agencies (MDAs). The result identified annual review, general performance appraisal, evaluation studies, inventory turnover, performance reviews, 4-block metric reviews, performance improvement plans, employee self-appraisal and 360-degree performance appraisal systems as being used in the Nigerian public sector. However, only annual review, inventory turnover, performance reviews, employee self-appraisal and 360-degree appraisal are fairly effective on paper but not in practice. The finding also identified lack of objectivity, halo error, leniency, central tendency error, recent behavior bias, personal bias (stereotyping), manipulating the evaluation, employee anxiety, shifting standards, overall ratings, horn error, strictness and politicisation of rating as factors hindering effectiveness of the appraisal system in Nigerian Public sector. The study concluded that there are employee appraisal systems in place in the Nigerian Public Sector but they are not effectively implemented.

The biasing impact of positive instructor reputation on student evaluations of teaching

A naturally-occurring intervention in a longitudinal field setting (4 months) was used to examine the presence and biasing impact of a positive reputation on subsequent ratings of work performance (student evaluations of teaching). During pre-semester interactions, first-year MBA students received information from second-year MBAs about their upcoming professors and classes. Favorable information about the two professors and course examined in the present study caused a positive reputation. Results indicated that despite four months of experiencing actual performance, the positive reputation hindered students’ decision-making process resulting in biasedly inflated ratings of instructor performance and halo error judgments of course materials, grading, and amount learned. The problematic implications of using biased student evaluations of teaching to measure faculty performance is discussed, along with suggestions of ways to mitigate against overreliance on this evaluation method and to possibly minimize reputational effects.

Consumer behaviour towards Russian brands of foreign FMCG companies

Goal: this study aims to explore the existent theoretical concepts that may explain the behavioural response of consumers in relation to various cross-categorical fast-moving consumer goods brands that constitute brand portfolios of international FMCG corporations. Namely, these concepts include the halo error, foreign-local bias, country-of-origin effects. The research grasps if changes in the consumer attitude exist when it comes to misidentification of brands by country of origin (the outcome from foreign-local bias), which is resulted by the occurrence of halo effect. Methodology: the research suggests quantitative methodology to explore the customer viewpoints. The data was collected through online structured interviews with ordinary visitors of grocery shopping malls, analysed and interpreted under the structural equation modelling statistical technique. Findings: from the customer standpoint, the research suggests that Russian consumer experience halo effect from the pseudo-local FMCG brands. Besides, this cognitive bias is shown to be positively related with the age of consumers, meaning that the older consumers are exposed to experience halo effect. The misidentification of FMCG brands by country of origin is directly influenced by the occurrence of halo effect among domestic consumers, but is not shown to drive their subconscious orientation towards pseudo-local FMCG brands. Originality and contribution the authors: the research helps to identify the system of relationships that could explain the consumers’ subjective orientation towards pseudo-local FMCG brands. As not all the hypothesised paths being supported by the data, the significant relationships on consumers’ attitudes are identified.

Feedback to support examiners’ understanding of the standard-setting process and the performance of students: AMEE Guide No. 145

The ratings that judges or examiners use for determining pass marks and students' performance on OSCEs serve a number of essential functions in medical education assessment, and their validity is a pivotal issue. However, some types of errors often occur in ratings that require special efforts to minimise. Rater characteristics (e.g. generosity error, severity error, central tendency error or halo error) may present a source of performance irrelevant variance. Prior literature shows the fundamental problems in student performance measurement attached to judges’ or examiners’ errors. It also indicates that the control of such errors supports a robust and credible pass mark and thus, accurate student marks. Therefore, for a standard-setter who identifies the pass mark and an examiner who rates student performance in OSCEs, proper, user-friendly feedback on their standard-setting and ratings is essential for reducing bias. This feedback provides useful avenues for understanding why performance ratings may be irregular and how to improve the quality of ratings. This AMEE Guide discusses various methods of feedback to support examiners' understanding of the performance of students and the standard-setting process with an effort to make inferences from assessments fair, valid and reliable.

Effectiveness of source‐monitoring training in reducing halo error and negativity bias in a performance appraisal setting

AbstractThe goal of the present study was to examine the effectiveness of a source‐monitoring procedure in reducing halo error and negativity bias in a performance appraisal setting. Participants watched videotaped performance segments of an instructor presumably taking place over a school year (N = 126). Then, participants in the source‐monitoring training condition received instructions teaching them how to differentiate memories that evoke detailed representations (remember judgments) from memories that evoke mere feelings of familiarity (know judgments). With this distinction in mind, participants in the training condition made significantly more accurate judgments than participants in the control condition. The training was successful in increasing the accuracy of performance ratings and reducing halo error and the negativity bias. Recognition memory measures showed that improvements were due to increased reliance on specific memory traces in the training condition. There was evidence of heuristic reliance on general impressions in the control condition.

Past performance contaminates the ADAPTS measurement

Theory suggests that adaptive selling behaviors improve sales performance. While this causation appears to be supported using the ADAPTS measurement scale, an alternative explanation is that perception of past performance predicts response to the ADAPTS scale. The ADAPTS items often contain abstract concepts such as “sales approach” and “not working” which may be difficult for respondents to interpret. Consistent with a halo error, salespeople may draw upon their overall attitude toward their selling ability to help them answer abstract ADAPTS items. Because past performance contributes to overall attitude toward selling ability, past performance could be biasing ADAPTS ratings through a halo error. This research consists of three studies. The first study uses a qualitative methodology to investigate ADAPTS item interpretation. The second study uses a time-lagged experimental methodology to demonstrate that past performance influences ADAPTS ratings above and beyond actual adaptive behaviors. The final study uses a survey methodology to show that sales self-efficacy mediates the relationship between self-rate performance and ADAPTS, even while controlling for an acquiescence bias. The results of the three studies demonstrate that ADAPTS contains significant bias. Alternative methods for operationalizing adaptive selling behaviors are suggested.

人事評定者訓練におけるハロー効果の程度のフィードバック効果 : 2次因子分析によるハロー効果の測定(第2報)(理論・技術)

人事評定者訓練におけるハロー効果の程度のフィードバック効果 : 2次因子分析によるハロー効果の測定(第2報)(理論・技術)

Choosing one, ignoring the other: Binary choice-making and easily accessible information under uncertainty

(ProQuest: ... denotes formulae omitted.)When we make a decision, we need information (Einhorn & Hogarth, 1981). Psychological research has shown that people often overestimate the correctness (or validity) of information which they have accessed quickly, or when it is the only information available to them (e.g., availability heuristic, confirmation bias, cue familiarity, halo error, etc.). This tendency becomes even more salient in an uncertain situation in which the necessary information is less available (Tversky & Kahneman, 1974), such as hard/easy effect (Fischhoff, Slovic, & Lichteinstein, 1977; Lichtenstein, Fischhoff, & Philips, 1982; Suantak, Bolger, & Ferrell, 1996).According to the information accessibility hypothesis (or model; Koriat, 1993; Koriat, 1995), people get a subjective feeling of correctness from information when that information easily comes to mind. Also, people overestimate the validity of information when it is the sole information available to them (Koriat, Lichtenstein, & Fischhoff, 1980). Overall, people's subjective beliefs about information are not only influenced by how they evaluate the information, but also whether their mind has accessed the information easily or deliberately, and whether the information available to them is with or without other information.The idea of the current research is based on the well-known phenomenon in which information can subjectively facilitate judgment but does not significantly improve objective performance, if the information is easily accessible, or available without contradictory information (Benjamin, Bjork, & Schwartz, 1998; Kelly & Lindsay, 1993; Koriat et al. 1980). Specifically, the current research has investigated whether this phenomenon can also be found in binary-choice situations-where a person chooses one correct answer from two alternatives-and what psychological processes lie behind this phenomenon.Choosing between two alternatives is an extremely important and also frequent cognitive and behavioral activity, whether it's predicting sports outcomes, voting in the US presidential election, or encountering a sudden split in the highway. Indeed, any yes or no decision is essentially a choice between two and any decision that involves many alternatives is often reduced to a choice between the two best alternatives. Thus, if choice-making is overly influenced by psychological factors which do not always help us to choose the best alternative, they are worth investigating.The current research attempted methodological and theoretical improvements as well. While much of the previous research used complex experimental methods - often involving bulky tasks, such as general knowledge questions (Koriat, 1995; Fischhoff et al., 1977), writing supportive and opposing ideas (Koriat et al., 1980), measuring different types of performanceprediction (Costermans, Lories, & Ansay, 1992), etc. - the current research tested the same hypothesis, and demonstrated a comparable phenomenon, while employing a simple true experimental method. It directly manipulated the accessibility (or availability) of the information between a control and experimental condition, and derived quantitatively measurable and statistically significant behavioral differences between them. Furthermore, by modulating the manipulation in several ways through multiple experiments, it became more parsimonious to identify the psychological processes behind the phenomenon: While easily accessible or solely available information can facilitate judgment, it does not improve objective performance to the same degree. This theoretical improvement was attained by adopting the paradigm, choosing one correct answer from two alternatives, in the current research.The current research is based on four psychological concepts, from which four specific questions have been derived. These concepts and questions are as follows:1. …

Analyzing Peer Assessment Claims for Team Projects in Engineering Courses

Ten distinct claims about peer assessment system user perceptions or actions were found to have been made by researchers regarding peer assessments in a team based project. The claims include: the presence of “Free Riders”, the ability for peer assessments to motivate students, biased assessments because of social acceptance, raters noticing peers’ lack of appropriate skills, positive rater reactions to the specific peer assessment system used, improvements in confidence and skill in rating because of practice rating exercises, biased assessments because of Halo error and social pressures, raters resisting peer assessments because of perceived bias, raters perceiving peer assessments to be unhelpful, and raters hesitation to assess peers because of perceived lack of authority. Several thousand anonymous and volunteer verbatim comments from a peer assessment system used in engineering courses were categorized based on the above claims, and analyzed in an attempt to support or dispute these claims. Only two claims in this study were found to be strongly supported in the verbatim comments made when completing peer reviews (Free Riders and Missing Skills) and no claims on peer assessment can be disputed based on the available evidence. The presence of Free Riders is the most significant peer assessment problem identified in the verbatim comments that instructors must address.

Why Won’t Task Conflict Cooperate? Deciphering Stubborn Results

Theory suggests that task conflict should be functional for team performance on non-routine tasks unless it becomes excessive, yet empirical support has been mixed. Why are the functional effects of task conflict so inconsistent? Why do they appear in some instances but not in others? In essence, why does task conflict work well for some teams but not work well for others? We review the empirical literature on task conflict, including four meta-analyses, and offer explanations for the observed results. High correlations between task, relationship, and process conflict make it difficult to isolate the effects of task conflict. There are a number of measurement and data analysis issues that warrant attention, including the possibility of halo error in ratings that may be exacerbated by the language in the measures. Furthermore, theory predicts a curvilinear relationship between task conflict and team performance, but most empirical studies have not tested for one, nor have they looked for reciprocal or interaction effects. A number of relevant temporal issues could also affect the results. We discuss boundary conditions and potential moderators that are in need of more research. At the group level, these include values congruence, goal alignment, norms for debate, the group’s performance history and conflict history, and the complex relationship between conflict and trust. A number of individual difference variables that may affect the relationship between task conflict and performance are also discussed. Finally, research suggests that conflict management approaches are of critical importance in determining how task conflict will influence performance.

Group Assessments of Resident Physicians Improve Reliability and Decrease Halo Error

Individual faculty assessments of resident competency are complicated by inconsistent application of standards, lack of reliability, and the "halo" effect. We determined whether the addition of faculty group assessments of residents in an ambulatory clinic, compared with individual faculty-of-resident assessments alone, have better reliability and reduced halo effects. This prospective, longitudinal study was performed in the outpatient continuity clinics of a large internal medicine residency program. Faculty-on-resident and group faculty-on-resident assessment scores were used for comparison. Overall mean scores were significantly higher for group than individual assessments (3.92 ± 0.51 vs. 3.83 ± 0.38, p = 0.0001). Overall inter-rater reliability increased when combining group and individual assessments compared to individual assessments alone (intraclass correlation coefficient, 95% CI = 0.828, 0.785-0.866 vs. 0.749, 0.686-0.804). Inter-item correlations were less for group (0.49) than individual (0.68) assessments. This study demonstrates improved inter-rater reliability and reduced range restriction (halo effect) of resident assessment across multiple performance domains by adding the group assessment method to traditional individual faculty-on-resident assessment. This feasible model could help graduate medical education programs achieve more reliable and discriminating resident assessments.

Comparing Current and Former Student Evaluations of Course and Instructor Quality

Halo error, ratings of instruction quality, and attitudes toward completing teaching evaluations were examined across two studies of undergraduate communication students. In both studies, comparisons between current students (i.e., students currently enrolled in a given instructor's course) and former students (i.e., students who took an instructor more than one semester ago) were made along evaluative criteria. Results indicated former students rated instructors and course content less positively and exhibited greater variability (i.e., less halo) among study factors compared to current students. Attitudes toward completing student evaluations were positively related to ratings of course instruction. Partial support was found for the hypothesis that proposed current students would use dispositional cues (e.g., instructor immediacy), whereas former students would use more situational cues (e.g., course difficulty) when rating course and instructor quality.

Research Note—Should Consumers Use the Halo to Form Product Evaluations?

In purchase situations where attribute information is either missing or difficult to judge, a well-known heuristic that consumers use to form evaluations is the halo effect. The psychology literature has widely considered the halo a reflection of consumers' inability to discriminate between different attributes and have therefore labeled it the “halo error” or the “logical error.” The objective of this paper is to offer a rationale for the halo effect. We use a decision-theory framework to show that the halo is consistent with the goal of minimizing estimation risk. Contrary to conventional wisdom, we demonstrate that a decision using the halo has lower estimation risk compared to not using the halo heuristic. Therefore, using the halo results in utility maximization and is indicative of rational behavior.

Is There a General Factor in Ratings of Job Performance? A Meta-Analytic Framework for Disentangling Substantive and Error Influences.

A database integrating 90 years of empirical studies reporting intercorrelations among rated job performance dimensions was used to test the hypothesis of a general factor in job performance. After controlling for halo error and 3 other sources of measurement error, there remained a general factor in job performance ratings at the construct level accounting for 60% of total variance. Construct-level correlations among rated dimensions of job performance were substantially inflated by halo for both supervisory (33%) and peer (63%) intrarater correlations. These findings have important implications for the measurement of job performance and for theories of job performance.

IS HALO HELPFUL? EFFECTS OF INDUCING HALO ON PERFORMANCE RATING ACCURACY

Halo error, although traditionally viewed as being detrimental to the quality of social cognition, has been purported to have potentially beneficial effects in some situations (Murphy, Jako, & Anhalt, 1993; Nathan & Tippins, 1990). Consequently, Murphy et al. called for investigations on the effects of inducing halo. The present experiment tested the effect of inducing halo on performance rating accuracy. Participants (N=160) were randomly assigned to either the induced halo or control condition. Participants observed videotaped performances then made job performance ratings after a 24-hour delay. In spite of a high degree of power, the effect of the halo induction on rating accuracy was not significant. Thus, it appears plausible that any potentially beneficial effects of halo on performance ratings may be limited to naturally occurring halo rather than to induced halo.

Comment on Halo Effects in Rating and Evaluation Research

This paper considers the existence of halo effects in individuals' evaluations of target communicators across different dimensions. Halo effects result from raters' inability to discriminate among conceptually distinct and theoretically independent aspects of a target's behavior. Halo error has also been defined as the influence of global attitudes on evaluations of individual attributes of a person (Nisbett & Wilson, 1977). The primary consequence of this “top-down processing” is to cause dimensional rating intercorrelations to be inflated. The first part of this article discusses current conceptions of halo error and suggests several potential sources of halo error. The measurement of true and illusory halo is then considered and recommendations for minimizing halo error in communication measurement are outlined.

Improving the measurement of customer satisfaction: a test of three methods to reduce halo

Many firms measure customer satisfaction on an attribute‐by‐attribute level. Past research has shown that halo errors can pose a serious threat to the interpretability of such data. Examines three factors that potentially reduce halo, using a combination of an experimental and quasi‐experimental research design. Three conclusions were drawn. First, measurement after consumption showed less halo than delayed measurement. Second, relative rating scales contained less halo than standard satisfaction scales. Third, an interaction effect was found between the number of attributes to be evaluated and the rating scale used. The evaluation of many attributes reduced halo in comparison to an evaluation of few attributes when a standard satisfaction rating scale was used. However, when the more complex relative rating scale was used, halo was not reduced when subjects had to evaluate a large number of attributes, perhaps due to the increased complexity of the task.

Reducing the Effects of Performance Expectations on Behavioral Ratings

In this study, we develop and test two strategies for reducing the effects of performance expectations on behavioral ratings. A 3 × 3 experimental design (N = 169), manipulating preobservation performance cues (positive, negative, or none) and the type of intervention (halo error training, structured recall memory, or none), was conducted. The results of this study indicate that both interventions reduced the effects of performance expectations on behavioral ratings. However, analyses of rating accuracy and measures from both recognition memory and recall memory tests suggest that the structured recall memory intervention has distinct advantages. These analyses indicate that the structured recall intervention can reduce raters' reliance on heuristics and increase the correspondence between raters' memory and their subsequent ratings.

Rater bias in psychological research: When is it a problem and what can we do about it?

Rater bias is a substantial source of error in psychological research. Bias distorts observed effect sizes beyond the expected level of attenuation due to intrarater error, and the impact of bias is not accurately estimated using conventional methods of correction for attenuation. Using a model based on multivariate generalizability theory, this article illustrates how bias affects research results. The model identifies 4 types of bias that may affect findings in research using observer ratings, including the biases traditionally termed leniency and halo errors. The impact of bias depends on which of 4 classes of rating design is used, and formulas are derived for correcting observed effect sizes for attenuation (due to bias variance) and inflation (due to bias covariance) in each of these classes. The rater bias model suggests procedures for researchers seeking to minimize adverse impact of bias on study findings.

THE CONSTANT ERROR OF THE HALO IN EDUCATIONAL OUTCOMES RESEARCH

Students' reports of their learning anddevelopment play an important role in research andassessment in higher education. Assessment researchfrequently asks students questions about gains madeduring college to identify dimensions of gains andthen examines relationships between college experiencesand gains. A growing body of research suggests thatcorrelations between ratings of gains and college experiences may be an artifact of a constanterror of the halo. The present research examines whetherhalo error underlies students' self reports of gains,the significance of the halo error, and the effect of halo error on relationships between collegeexperiences and educational outcomes. Results confirmthat halo error may be an important component instudents' ratings of their learning and development. Moreover, halo error may obscure relationshipsbetween college experiences and educationaloutcomes.