User Perceived Qualities and Acceptance of Recommender Systems

Abstract

Recommender systems have become important, as users are faced with an ever-increasing amount of information available on internet. Much of the research work on the topic has been focused on recommendation techniques, aiming at improving the accuracy of recommended items. Today, researchers use accuracy-metrics for evaluating goodness, when in fact these do not capture users' expectations and criteria for evaluating recommendation usefulness. We must ask ourselves whether a less accurate recommendation is necessarily a less valuable one for the user. To support this, we centre our investigations in this thesis on users, and explore their acceptance behaviours when using recommendations, and their perceived qualities. We present results in four areas. First, we study users' perceptions leading to the acceptance of recommendations and the possible long-term adoption of the system. We run two user studies using two online music recommenders relying on different recommendation techniques. Our results show that the perceived usefulness in terms of quality, and the perceived ease of use in terms of effort, are directly correlated with the users' acceptance of the recommendations. The results also show the necessity for low-involvement recommenders to be highly reactive, helping to take the users' search context into account. Secondly, we evaluate a behavioural recommender, where recommendations are made from implicitly expressed user preferences. We take profile sizes into account and compare such recommendations to an explicit search & browse interface. Our experiment reveals that users perceive the smaller effort required to use a behavioural recommender, but find the explicit solution to yield more diverse suggestions and gives them more control. Overall, users perceive both approaches as being satisfactory, providing the profile size is big enough. Thirdly, we analyse the impact on users' perceptions of a visual rendering. We designed an iconised representation of compound critiques, usually textual, and observed the differences in users' appreciation. Our results reveal that users prefer the visual interface, that it reduces their interaction efforts, and that users are attracted to apply the critiques more frequently in complex product domains, which have more product-features. In a fourth area, we examine the role of diversity of recommendations in users' acceptance. A first study shows that diversity is the dimension which most influences users' satisfaction. We also highlight that users have more confidence in their choice using an organised layout interface for the same perceived ease of use as with a list view, even though the organised layout creates longer interactions. For the first time in a study, we show that diversity correlates with the trust of users. In a second study, we use an eye-tracker to carry out an in-depth study of users' decision process. We show how the influence of a recommender increases throughout a user's purchase decision process until the decision is close to being taken. At this moment, we observed that users rely on the recommender to enhance their confidence in the purchase decision, and that they need diversity to prioritise the suggestions. To end our work, we propose a theoretical diversity-model for maximising users' overall satisfaction by balancing users' needs for recommendation accuracy and diversity throughout the decision process. In addition, we derive a set of design guidelines from all of the experimental results. They are elaborated around four primary axes: user effort, purchase intentions, complex systems and diversity.