Unfair Ratings Research Articles

Performance evaluation inflation and compression

We provide a behavioral account of subjective performance evaluation inflation (i.e., leniency bias) and compression (i.e., centrality bias). When a manager observes noisy signals of employee performance and the manager strives to produce accurate ratings but feels worse about unfavorable errors than about favorable errors, the manager’s selfishly optimal ratings will be biased upwards. Both the uncertainty about performance and the asymmetry in the manager’s utility are necessary conditions for performance evaluation inflation. Moreover, the extent of the bias is increasing in the variance of the performance signal and in the asymmetry in aversion to unfair ratings. Uncertainty about performance also leads to compressed ratings. These results suggest that performance appraisals based on well-defined unambiguous criteria will have less bias. Additionally, we demonstrate that employer and employee can account for biased performance evaluations when they agree to a contract, and thus, to the extent leniency bias and centrality bias persist, these biases hurt employee performance and lower firm productivity.

Can Online Trading Survive Bad Mouthing? An Experimental Investigation

We use controlled lab experiments to test whether the market efficiency being affected by unfair negative ratings, and whether announcing the percentage of unfair ratings in the market makes any difference. We find that the market efficiency is higher than no rating information case even when unfair ratings and ambiguous ratings are present. We also find that buyers behave differently when the unfair rating information is known and when the information is unknown; however, comparing with the market with fair ratings, there is no statistically significant difference in the market efficiency whenever the percentage of unfair ratings is known or unknown.

Effective Usage of Computational Trust Models in Rational Environments

Computational reputation-based trust models using statistical learning have been intensively studied for distributed systems where peers behave maliciously. However practical applications of such models in environments with both malicious and rational behaviors are still very little understood. In this article, we study the relation between their accuracy measures and their ability to enforce cooperation among participants and discourage selfish behaviors. We provide theoretical results that show the conditions under which cooperation emerges when using computational trust models with a given accuracy, and how cooperation can still be sustained while reducing the cost and accuracy of those models. Specifically, we propose a peer selection protocol that uses a computational trust model as a dishonesty detector to filter out unfair ratings. We prove that such a model with reasonable misclassification error bound in identifying malicious ratings can effectively build trust and cooperation in the system, considering rationality of participants. These results reveal two interesting observations. First, the key to the success of a reputation system in a rational environment is not a sophisticated trust-learning mechanism, but an effective identity-management scheme to prevent whitewashing behaviors. Second, given an appropriate identity-management mechanism, a reputation-based trust model with a moderate accuracy bound can be used to effectively enforce cooperation in systems with both rational and malicious participants. As a result, in heterogeneous environments where peers use different algorithms to detect misbehavior of potential partners, cooperation may still emerge. We verify and extend these theoretical results to a variety of settings involving honest, malicious, and strategic players through extensive simulation. These results will enable a much more targeted, cost-effective and realistic design for decentralized trust management systems, such as needed for peer-to-peer, electronic commerce, or community systems.

AN OPTIMAL TRUST MANAGEMENT METHOD TO PROTECT PRIVACY AND STRENGTHEN OBJECTIVITY IN UTILITY COMPUTING SERVICES

Utility computing services constitute business opportunities in the customer-to-customer (C2C) marketplace. Providers deal with the risks and uncertainties faced by users while transacting with unfamiliar counterparts. Ensuring trustworthy transactions among unfamiliar users is an important condition of utility-computing service. Many peer-to-peer (P2P) sites utilize reputation systems; however, such systems are subject to lack of reviewer objectivity and robustness against attacks as well as unfair ratings. In this study, we propose the manual adjustment of reputation ratings by means of transaction monitoring and establish that the resulting enhanced objectivity and robustness in utility-computing service transactions will strengthen the trust management system. We also propose the optimal level of monitoring and penalizing activities based on the level of privacy concerns among users and the appropriate complementary service.

Service selection in stochastic environments: a learning-automaton based solution

In this paper, we propose a novel solution to the problem of identifying services of high quality. The reported solutions to this problem have, in one way or the other, resorted to using so-called "Reputation Systems" (RSs). Although these systems can offer generic recommendations by aggregating user-provided opinions about the quality of the services under consideration, they are, understandably, prone to "ballot stuffing" and "badmouthing" in a competitive marketplace. In general, unfair ratings may degrade the trustworthiness of RSs, and additionally, changes in the quality of service, over time, can render previous ratings unreliable. As opposed to the reported solutions, in this paper, we propose to solve the problem using tools provided by Learning Automata (LA), which have proven properties capable of learning the optimal action when operating in unknown stochastic environments. Furthermore, they combine rapid and accurate convergence with low computational complexity. In addition to its computational simplicity, unlike most reported approaches, our scheme does not require prior knowledge of the degree of any of the above mentioned problems associated with RSs. Instead, it gradually learns the identity and characteristics of the users which provide fair ratings, and of those who provide unfair ratings, even when these are a consequence of them making unintentional mistakes. Comprehensive empirical results show that our LA-based scheme efficiently handles any degree of unfair ratings (as long as these ratings are binary--the extension to non-binary ratings is "trivial", if we use the S-model of LA computations instead of the P-model). Furthermore, if the quality of services and/or the trustworthiness of the users change, our scheme is able to robustly track such changes over time. Finally, the scheme is ideal for decentralized processing. Accordingly, we believe that our LA-based scheme forms a promising basis for improving the performance of RSs in general.

Extensive Experimental Validation of a Personalized Approach for Coping with Unfair Ratings in Reputation Systems

The unfair rating problem exists when a buying agent models the trustworthiness of selling agents by also relying on ratings of the sellers from other buyers in electronic marketplaces, that is in a reputation system. In this article, we first analyze the capabilities of existing approaches for coping with unfair ratings in different challenging scenarios, including ones where the majority of buyers are dishonest, buyers lack personal experience with sellers, sellers may vary their behavior, and buyers may provide a large number of ratings. We then present a personalized modeling approach (PMA) that has all these capabilities. Our approach allows a buyer to model both the private reputation and public reputation of other buyers to determine whether these buyers' ratings are fair. More importantly, in this work, we focus on experimental comparison of our approach with two key models in a simulated dynamic e-marketplace environment. We specifically examine the above mentioned scenarios to confirm our analysis and to demonstrate the capabilities of our approach. Our study thus provides the extensive experimental support for the personalized approach that can be effectively employed by reputation systems to cope with unfair ratings.

Trust-based web service selection in virtual communities

In the present web service architecture, selecting which web service to use is still a task which requires a high level of human intervention. In this paper, we propose a trust-based web service selection process which aspires to automate part of the web service selection task to decrease the workload of human designers in the process. Most of the existing web service selection mechanisms are capability oriented or function oriented. We propose a trust-based web service selection approach to augment the current web service mechanisms. A trust model from our previous research, which offers strong protection against the adverse effect of unfair ratings from witnesses, is incorporated into an agent augmented service oriented virtual community. An experiment based on the virtual community system is conducted to study the effectiveness of the model in enabling trust agents to select highly trustworthy web services with minimal human supervision.

Securing Rating Aggregation Systems Using Statistical Detectors and Trust

Online feedback-based rating systems are gaining popularity. Dealing with unfair ratings in such systems has been recognized as an important but difficult problem. This problem is challenging especially when the number of regular ratings is relatively small and unfair ratings can contribute to a significant portion of the overall ratings. Furthermore, the lack of unfair rating data from real human users is another obstacle toward realistic evaluation of defense mechanisms. In this paper, we propose a set of statistical methods to jointly detect collaborative unfair ratings in product-rating type online rating systems. Based on detection, a framework of trust-assisted rating aggregation system is developed. Furthermore, we collect unfair rating data from real human users through a rating challenge. The proposed system is evaluated through simulations as well as experiments using real attack data. Compared with existing schemes, the proposed system can significantly reduce negative impact from unfair ratings.

Dishonest Behaviors in Online Rating Systems: Cyber Competition, Attack Models, and Attack Generator

Recently, online rating systems are gaining popularity. Dealing with unfair ratings in such systems has been recognized as an important but challenging problem. Many unfair rating detection approaches have been developed and evaluated against simple attack models. However, the lack of unfair rating data from real human users and realistic attack behavior models has become an obstacle toward developing reliable rating systems. To solve this problem, we design and launch a rating challenge to collect unfair rating data from real human users. In order to broaden the scope of the data collection, we also develop a comprehensive signal-based unfair rating detection system. Based on the analysis of real attack data, we discover important features in unfair ratings, build attack models, and develop an unfair rating generator. The models and generator developed in this paper can be directly used to test current rating aggregation systems, as well as to assist the design of future rating systems.

On Desideratum for B2C E-Commerce Reputation Systems

This paper reviews existing approaches to reputation systems, their constraints as well as available solutions. Furthermore, it presents and evaluates a novel and comprehensive reputation model devoted to the distributed reputation system for Business-to-Consumer (B2C) E-commerce applications that overcomes the discussed drawbacks. The algorithm offers a comprehensive approach as considers a number of issues that have a bearing on trust and reputation such as age of ratings, transaction value, credibility of referees, number of malicious incidents, collusion and unfair ratings. Moreover, it also extends the existing frameworks based on information about past behaviour, with other aspects affecting online trading decisions which relate to the characteristic of the providers, such as existence of trust mark seals, payment intermediaries, privacy statements, security/privacy strategies, purchase protection/insurance, alternative dispute resolutions as well as the existence of first party information.

The Commission for Social Care Inspection's Quality Rating Review Service: Fair or foul?

Abstract The Commission for Social Care Inspection (CSCI) has consistently refused to accept that there should be independent scrutiny of its powers to quality rate care providers. While it is difficult to question the rationale or even the ability of CSCI to issue quality ratings, there remain concerns within the sector about its ability to do this consistently and fairly. The largest single issue raised in consultation was the consistency of inspection and the skills and competencies of inspectors. CSCI has now introduced the Quality Rating Review Service (QRRS) as part of its quality assurance strategy. This paper considers whether this system can work effectively by critically assessing its procedural limitations and its lack of independence. The paper concludes that to overcome the defects within QRRS, providers should be entitled to appeal unfair quality ratings to the Care Standards Tribunal.

Evaluating the trustworthiness of advice about seller agents in e-marketplaces: A personalized approach

In this paper, we present a model for evaluating the trustworthiness of advice about seller agents in electronic marketplaces. In particular, we propose a novel personalized approach for effectively handling unfair ratings of sellers provided to buyer agents from other buyers (called advisors). Our approach offers flexibility for buyers to weight their value for private and public knowledge about advisors. A personalized approach is proposed as well for buyers to model the trustworthiness of sellers, based on the advice provided. Experimental results demonstrate that our approach can effectively model trustworthiness for both advisors and sellers, even when there are large numbers of unfair ratings.

A COMPREHENSIVE APPROACH FOR SHARING SEMANTIC WEB TRUST RATINGS

In the context of the Semantic Web, it may be beneficial for a user (consumer) to receive ratings from other users (advisors) regarding the reliability of an information source (provider). We offer a method for building more effective social networks of trust by critiquing the ratings provided by the advisors. Our approach models the consumer's private reputations of advisors based on ratings for providers whom the consumer has had experience with. It models public reputations of the advisors according to all ratings from these advisors for providers, including those who are unknown to the consumer. We then combine private and public reputations by assigning weights for each of them. Experimental results demonstrate that our approach is robust even when there are large numbers of advisors providing large numbers of unfair ratings. We show that we can effectively model the trustworthiness of advisors even when the population of providers grows increasingly large and discuss how our approach is beneficial in modeling providers. As such, we present a framework for sharing ratings of possibly unreliable sources, of value as users on the Semantic Web attempt to critique the trustworthiness of the information they seek.

A Trustworthiness Based Ad-Hoc Secure Interoperation Method

A trustworthiness-based ad-hoc secure interoperation method is proposed, in which the concept of trustworthiness is introduced to describe the probability of proper collaboration. The trustworthiness of an autonomic domain on a user is decided jointly by direct experiences of interactions and other domains' assessments on the user. Only the users who satisfy the requirements of target domains' trust policies have the privileges to access entry roles. Records of a user's malicious actions will decrease his trustworthiness and then accordingly reduce his privileges. Target domain uses weighted majority algorithm to update recommender's trustworthiness, which is reduced by unfair ratings. Experimental results show that this method can effectively resist cheating and malicious actions.

Implicit user credibility extraction for reputation rating mechanism in B2C e-commerce

Reputation systems are emerging as one of the promising solutions for building trust among market participants in e-commerce. Finding ways to avoid or reduce the influence of unfair ratings is a fundamental problem in reputation systems. We propose an implicit reputation rating mechanism suitable for B2C e-commerce. The conceptual framework of the mechanism is based on the source credibility model in consumer psychology. We have experimentally evaluated the performance of the mechanism by comparing with the other benchmark rating mechanisms. The experimental results provide evidence that the general users' opinions can be predicted more effectively by only a small number of users selected by our proposed mechanism.

Globally Decoupled Reputations for Large Distributed Networks

Reputation systems help establish social control in peer-to-peer networks. To be truly effective, however, a reputation system should counter attacks that compromise the reliability of user ratings. Existing reputation approaches either average a peer's lifetime ratings or account for rating credibility by weighing each piece of feedback by the reputation of its source. While these systems improve cooperation in a P2P network, they are extremely vulnerable to unfair ratings attacks. In this paper, we recommend that reputation systems decouple a peer'sservice providerreputation from itsservice recommenderreputation, thereby, making reputations more resistant to tampering. We propose a scalable approach to system-wide decoupled service and feedback reputations and demonstrate the effectiveness of our model against previous nondecoupled reputation approaches. Our results indicate that decoupled approache significantly improves reputation accuracy, resulting in more successful transactions. Furthermore, we demonstrate the effectiveness and scalability of our decoupled approach as compared to PeerTrust, an alternative mechanism proposed for decoupled reputations. Our results are compiled from comprehensive logs collected from Maze, a large file-sharing system with over 1.4 million users supporting searches on 226TB of data.

An adaptive trust model of web services

This paper proposes a dynamic Web service trust(WS Trust) model, and some corresponding trust metric evaluation algorithms. The main goal is to evaluate the trust-worthiness and predict the future behaviors of entities in order to help users find trustworthy Web service providers and prevent users from providing unfair ratings against service providers.

Development, Colonial Rule, and Intergroup Conflict in a Chinese Village in Hong Kong

u P UNTIL THE LATE 1960s the rural areas of Hong Kong, called the New Territories, lagged far behind the urban areas in socioeconomic development. Since the early 1970s, however, the situation has drastically changed. Overpopulation in urban Hong Kong has impelled the government to look for land in the New Territories in order to meet the increasing need for housing, industrial sites, and room for recreational and sports facilities. The process of acquiring land in the New Territories involves the requisition of privately owned land at a rate of compensation set down by the government, a rate that is normally far below the market rate. In the process of land requisition, the government inevitably runs into conflict with local people, many of whom are opposed to the development of the New Territories, either because of the perceived unfair rate of compensation or because they want to preserve the rural character of the area. Consequently, along with the planned development of the New Territories by the government has come the rise of conflicts over land resumption. Such conflicts have to be resolved if an orderly developmental process is to be attained. Because the government is the dominant actor in the process of planned development, its role in conflict resolution is crucial. Planned development in the New Territories has brought to the surface another category of conflict that is the inadvertent result of past administrative measures. This is the conflict between two groups of local people: the original inhabitants and the outsiders in the villages. When the New Territories were leased from China in 1898, the British distinguished between two types of inhabitants in the area and granted special rights to one of them that were denied the other. The original inhabitants, called the pun-ti-yan, are those, or the descendants of those, who lived in the New Territories before o r in 1898, when the area was leased from China. The outsiders are those who have moved into the New Territories after that date. It is apparent that such a differentiation is merely administrative,