Rank Feedback Research Articles

Does supportive feedback on class rank improve scores for intermediate-level microeconomics?

Instructors may use low-cost, light-touch strategies to help students achieve optimal effort in demanding upper-level courses. The authors of this study exploit an intervention that provides a series of personalized feedback emails to students about their relative performance, which is tied to approving messages or tips that encourage improvement. The authors analyze the efficacy of this supportive rank feedback in an intermediate-level microeconomics theory course, given the challenges it poses to students. Evidence shows gains from feedback across the grade distribution, but they are short-lived and decline toward zero as the course progresses.

Combined Effects of Social Exclusion and Social Rank Feedback on Risky Decision-Making Across Adolescence.

Adolescents' need to belong and concerns about social status are thought to increase risk-taking, however, not much is known about how feedback about social rank and the effects of social exclusion moderate risky decision-making. To this end, the present study examined how social rank feedback moderates the effects of social exclusion on risky decisions during adolescence. The experimental study included a total of 122 participants (11-19 years; 44% female). Participants were randomly assigned to receive either individual or social rank feedback in the Columbia Card Task after social inclusion and exclusion via the Cyberball paradigm. Contrary to expectations, social exclusion led to more cautious decision-making. Mid-adolescents were most influenced by the combination of social exclusion and social rank feedback, while late adolescents became more cautious with individual feedback. These findings suggest that peer influences also have adaptive effects, increasing sensitivity to risk information, with developmental differences in the role of social rank.

Gender, choice of task, and the effect of feedback on competition: An experiment

We conduct a laboratory experiment to examine gender differences in task choices and competitiveness: Individuals have the option to self-select into one of two stereotypically different tasks and subsequently decide whether or not to engage in competition while receiving one of three different feedback treatments. Compared to a control setting, we study the effect of providing relative performance feedback (rankings), and additionally the effect of providing information about the gender of the competitors. We find a significant gender gap in the choice of the male task, only when presenting the ranking in addition to the gender composition of the group. Turning to the decision to enter competition, we observe that task choice, combined with ranking feedback on performance, reduces the gender gap in competition entry in both tasks compared to the control. The dynamics over treatments reveal that men primarily respond to feedback in the male task, while women respond to feedback in the word task. These findings highlight that gender differences in task choice and competitiveness are contingent on feedback, the underlying task, and the task choice set.

Can leagues mitigate the demoralization effect of rank feedback? A randomized controlled trial

In a randomized controlled trial involving hundreds of university students, we provide relative performance feedback specifically designed to reduce low performers’ demoralization, by dynamically assigning students to small leaderboard groups that share a similar score in a semester-long online assignment. Treated students appear 2.6% more likely to go beyond the call-of-duty on their assignment by mid-semester. For low performers, this translates in 0.27 SDs higher exam grades, more stress, increased effort and lower procrastination. High performers are happier, procrastinate less and overachieve in the assignment on which they are ranked, but ultimately also score 0.25 SDs lower exam grades.

Relative Performance Feedback in Education: Evidence from a Randomised Controlled Trial*

Abstract In a one-year randomised controlled trial involving thousands of university students, we provide real-time private feedback on relative performance in a semester-long online assignment. Within this set-up, our experimental design cleanly identifies the behavioural response to rank incentives (i.e., incentives stemming from an inherent preference for high rank). We find that rank incentives boost performance in the related course assignment, but also the average course exams grade by 0.21 SDs. These beneficial effects remain sizeable across all quantiles and extend beyond the intervention period. Furthermore, rank feedback stimulates social learning, i.e., rank incentives make students engage more in peer interactions.

Rivals without a cause? Relative performance feedback creates destructive competition despite aligned incentives

AbstractWhether people compete or cooperate with each other has consequences for their own performance and that of organizations. To explain why people compete or cooperate, previous research has focused on two main factors: situational outcome structures and personality types. Here, we propose that—above and beyond these two factors—situational cues, such as the format in which people receive feedback, strongly affect whether they act competitively, cooperatively, or individualistically. Results of a laboratory experiment support our theorizing: After receiving ranking feedback, both students and experienced managers treated group situations with cooperative outcome structures as competitive and were in consequence willing to forgo guaranteed financial gains to pursue a—financially irrelevant—better rank. Conversely, in dilemma situations, feedback based on the joint group outcome led to more cooperation than ranking feedback. Our study contributes to research on competition, cooperation, interdependence theory, forced ranking, and the design of information environments.

Can Rank Feedback Avoid Demoralization? A Randomized Controlled Trial

In a randomized controlled trial involving hundreds of university students, we provide relative performance feedback specifically designed to reduce low performers' demoralization, by dynamically assigning students to small leaderboard groups that share a similar score in a semester-long online assignment. Treated students appear 2.6% more likely to go beyond the call-of-duty on their assignment by mid-semester. For low performers, this translates in 0.27 SDs higher exam grades, more stress, increased effort and lower procrastination. High performers are happier, procrastinate less and overachieve in the assignment on which they are ranked, but ultimately also score 0.25 SDs lower exam grades.

Designing a smart information system: the influence of feedback on energy conservation persuasion

ABSTRACT This study explored whether a smart information system can motivate users to conserve energy in the home environment. Three kinds of feedback information based on user decisions were discussed: emotion feedback with a happy mood, average feedback of a comparison group and ranking feedback within a comparison group. An experiment validated that all the feedbacks had a positive influence on users’ energy conservation performance. Users’ positive attitude toward the social group that they lived in and compared with would increased users’ environmentally friendly behaviours. Finally, several suggestions on the design of energy feedback are provided.

On the Effects of Relative Performance Feedback in a Risk Elicitation Task: Evidence From a Lab Experiment

This paper experimentally investigates the influence of different non-payoff-relevant feedback regimes on motivation, competitiveness and risk-taking behaviors in a risk elicitation task. We explore four feedback regimes: private feedback, full ranking feedback, top-5 ranking feedback, and non-ranking feedback based on the median performance of peers. We found that providing relative performance feedback increases risk-taking behavior, especially when using a top-5 ranking. Feedback regimes that provide a user with rank-based feedback have adverse effects on the users' attitudes as they increase competitiveness while failing to increase intrinsic motivation and performance motivation. In contrast, non-ranking feedback like the median feedback increases intrinsic motivation and does not influence competitiveness and risk-taking. Across all treatment groups, introducing relative performance feedback rules out the importance of a monetary compensation. From a management perspective, these results suggest that in situations where high risk decisions are problematic, non-ranking feedback should be preferred over rank-based feedback regimes.

Room for advancement: The regulatory fit of bottom-rank intermediate feedback.

This research tests the hypothesis that promotion‐focused individuals experience regulatory fit from bottom rank, intermediate performance‐feedback. Prior research suggests promotion‐focused individuals experience fit in high social ranks (power). Bottom performance ranks may appear psychologically opposite to high power, which might lead one to expect that promotion‐focused individuals experience fit at top ranks. We propose that the opposite occurs in intermediate performance ranking feedback, in that promotion‐focused individuals experience regulatory fit at a bottom rank, because bottom rank implies having something to gain (yielding eagerness), whereas top rank implies having something to lose (yielding vigilance). Study 1 (N = 261) supports the notion that ranks affect eagerness/vigilance. Study 2 (N = 199) extends these findings by examining engagement from regulatory fit.

Pushing Through or Slacking Off? Heterogeneity in the Reaction to Rank Feedback

This paper studies heterogeneity in the reaction to rank feedback. In a laboratory experiment, individuals take part in a series of dynamic real-effort contests with intermediate feedback. To solve the identification problem in estimating the causal effect of rank feedback on subsequent effort provision we implement a random multiplier in the first round of each contest. The realization of this multiplier then serves as a valid instrument for rank feedback. While rank feedback has a robust effect on subsequent effort provision on average, an explicit analysis of between-subject heterogeneity reveals that a substantial fraction of participants in fact react entirely opposite than the aggregated results indicate. We further show that this heterogeneity has consequences for overall outcomes, thereby arguing that heterogeneous sensitivities to rank feedback could have implications for the design of various policies in education and organizations.

Mining taxi trajectories for most suitable stations of sharing bikes to ease traffic congestion

Sharing bike service is a new emerging public transportation, which has been the hottest topic for months. Sharing bike service provides flexible demand-oriented transit services for city commuters. However, as large amount of sharing bikes flood into big cities, problems caused by chaotic order of sharing bikes are emerging slowly. The authors aim to draw support from taxi trajectories to analyse current traffic condition and improve it with sharing bikes. In this study, the authors propose a traffic congestion finding framework, called CF. In CF, derived from the points density-based clustering method of inspiration, the authors propose a new clustering method (CF-Dbscan) and successfully applied it to the clustering of trajectories. A road network matching algorithm (CF-Matching) helps to match GPS points to road net even if points are in low-sampling-rate. They also employ a ranking feedback mathematical model to adjust the number of sharing bikes of different stations to meet people's demand and reduce redundancy. The first experiment proves that the proposed clustering algorithm performs better than traditional DBSCAN. Another experiment is conducted to verify the effectiveness of the proposed framework in reducing traffic congestion. The experimental results prove that with the proposed framework the authors can achieve the purpose of easing traffic congestion.

The effect of social rank feedback on risk taking and associated reward processes in adolescent girls.

The onset of adolescence is associated with an increased tendency to engage in risky behaviors and a developmental shift toward peers that contributes to increased prioritization for learning about and achieving social status. There is relatively little understanding about the specific links between these adolescent-typical phenomena, particularly regarding their neural underpinnings. Based on existing models that suggest the role of puberty in promoting adolescent status-seeking and risk-taking tendencies, we investigated the relation of pubertal hormones with behavioral and neural responses to status-relevant social information in the context of risk taking. We used a probabilistic decision task in which 11- to 13-year-old girls chose to take a risk, or not, while receiving either social rank or monetary performance feedback. While feedback type did not differentially influence risk-taking behavior, whole-brain imaging results showed that activation in the anterior insula was increased for risk taking in the social rank feedback condition compared to the monetary feedback condition. This heightened activation was more pronounced in girls with higher estradiol levels. These findings suggest that brain processes involved in adolescent risky decisions may be influenced by the desire for social-status enhancement and provide preliminary evidence for the role of pubertal hormones in enhancing this adolescent-typical social sensitivity.

Sequential Search and Learning from Rank Feedback: Theory and Experimental Evidence

This paper studies the effect of limited information in a sequential search setting where a single selection is to be made from a set of random potential options. We consider both a full-information problem, where the decision maker observes the exact value of each option as she searches, and a partial-information problem, in which the decision maker only learns the rank of the current option relative to the options that have already been observed. We develop a model that allows for a sharp contrast between search behavior in the two information settings, both theoretically and empirically. We present the results of an experiment that tests, and supports, the key prediction of our model analysis—limited information induces longer search. Our data further suggest systematic deviations from the theoretical benchmarks in both informational settings. Importantly, subjects in our partial-information conditions are prone to stop prematurely during early stages of the search process and to suboptimally continue the search during late stages. We propose a simple model that succinctly captures the interplay of two symmetric choice and judgment biases that have asymmetric (but opposing) effects on the length of search. Data, as supplemental material, are available at http://dx.doi.org/10.1287/mnsc.2014.1902 . This paper was accepted by Teck-Hua Ho, behavioral economics.

Sequential Search and Learning from Rank Feedback: Theory and Experimental Evidence

This paper studies the effect of limited information in a sequential search setting where a single selection is to be made from a set of random potential options. We consider both a full-information problem, where the decision maker observes the exact value of each option as she searches, and a partial-information problem, in which the decision maker only learns the rank of the current option relative to the options that have already been observed. We develop a model which allows for a sharp contrast between search behavior in the two information settings, both theoretically and empirically. We present the results of an experiment that tests, and supports, the key prediction of our model analysis – limited information induces longer search. Our data further suggest systematic deviations from the theoretical benchmarks in both informational settings. Importantly, subjects in our partial-information conditions are prone to stop prematurely during early stages of the search process and to sub-optimally continue the search during late stages. We propose a simple model that succinctly captures the interplay of two symmetric choice and judgment biases that have asymmetric (but opposing) effects on the length of search.

A Laboratory Investigation of Rank Feedback in Procurement Auctions

A popular procurement auction format is one in which bidders compete during a live auction event but observe only the rank of their own bid and not the price bids of their competitors. We investigate the performance of auctions with rank feedback in a simple setting for which analytical benchmarks are readily available. We test these benchmarks in the laboratory by comparing the performance of auctions with rank-based feedback to auctions with full-price feedback as well as to auctions with no price feedback (sealed-bid auctions). When bidders are risk-neutral expected-profit maximizers, the buyer's expected costs should be the same under rank and full-price feedback; moreover, expected buyer costs should be the same as in a sealed-bid auction. However, when we test this theoretical equality in a controlled laboratory setting we find that, consistent with practitioners' beliefs but contrary to our model, rank feedback results in lower average prices than full-price feedback. We identify two behavioral reasons for the difference. The first explanation is based on the similarity of the bidders' problem in a sealed-bid first-price auction and an open-bid auction with rank feedback. The second explanation incorporates the use of jump bids motivated by bidder impatience.

Management Insights

Management Insights

Semi Analytical Solution of Constrained Output Feedback Pole Placement in Multi-Input Systems

Pole-placement is one of the oldest control design method used to modify the behavior of a given plant dynamics. In the literature, analytical solution of output feedback pole placement is available only for two-input two-output systems. For general linear time-invariant, multi-input, multi-output (MIMO) systems, static (constant gain) output feedback pole placement is an open non-deterministic polynomial-time (NP) hard problem. This paper proposes a semi-analytical solution of constrained (minimum number of nonzero gains) output feedback pole placement problem in MIMO systems. To get the semi-analytical solution, first an analytical formula is developed for pole-placement in single-input, multi-output (SIMO) systems. This formula is an extension of Bass-Gura's formula used for state feedback pole placement in single-input systems. Along with the gains needed for partial pole-placement, this novel formula computes the coefficients of characteristic polynomial corresponding to the remaining poles of the plant. Using this analytical formula a novel iterative procedure is developed to place all the poles of a multi-input system using output feedback. Proposed iterative method needs only the solution of linear equations at each step of iteration. Starting from the known initial guess, it searches for the solution in a lower dimension space compared to other numerical methods. The algorithm computes a full rank feedback gain matrix. For a decoupled m-input system the proposed method computes the output feedback gains in exactly m-steps, non-iteratively.

Reputation Model with Feedback of Ranking for P2P Systems

Some peers may receive service and information of low-quality from other peers in peer-to-peer(or P2P) networks.Reputation evaluation is the normal method used to reduce the above phenomena.P2P reputation,based on score feedback,is defective because it can not distinguish the malicious feedback from the erring feedback returned by honest peers.It needs long time to converge the reputation and evaluate feedback.It is inflexible and unnatural to depict the reputation of a peer through a lot of numbers.In fact,the reputation is used to determine the rank of the peers.A reputation model called RbRf(reputation based ranking feedback) based on the rank feedback,is presented in this paper.Mathematical models unfolds in this paper show that the influence of erring feedbacks attenuates with the exponential function of RbRf.The influence of unintended malicious feedbacks is attenuated with the polynomial function in RbRf.The intended collusive feedbacks are counteracted by the correct information introduced by these feedbacks.In summary,the defection of score feedback,such as the need of a second evaluation of the trust of feedback,does not in RbRf any longer because the RbRf uses rank feedback,instead of score feedback,and the RbRf can achieve a better effect when resisting to malicious attacks.All results are verified by experimental data.

Teachers’ Beliefs and Practice Concerning Feedback Strategies

This study aims at surveying what strategies teachers believe they use in giving feedback, what strategies teachers actually use in giving feedback and the relationship between their perceptions and the reality. The six subjects were teachers who taught Fundamental English II at King Mongkut’s University of Technology Thonburi (KMUTT). Each of them was observed teaching once and then asked to rank feedback strategies by perceived frequency of use. The results show that some of the perceptions about frequency of use match the actual strategies they used in class but some of them do not. Some feedback strategies, such as zero feedback and repeating student utterance, were used more frequently in actual practice than teachers believed, possibly since those feedback strategies do not require a conscious thinking process; on the other hand, the strategy summarizing strategies was used less in practice, perhaps because this strategy needs a great deal of conscious effort.