Calista: A deep learning-based system for understanding and evaluating website aesthetics

Abstract

Website aesthetics play an important role in attracting users and customers, as well as in enhancing user experience. In this work, we propose a tool that performs automatic evaluation of website aesthetics using deep learning models that display high correlation to human perception. These models were developed using two different datasets. The first dataset was created by employing a rating-based ranking approach and contains user judgments on websites in the form of an explicit numerical value on a scale. Using the first dataset, we developed models following three different approaches and managed to outperform previous works. In addition, we created a new dataset by employing a comparison-based ranking approach, which is a more reliable dataset in the sense that it follows a more “natural” data collection method. In this case, users were asked to compare two websites at a time and choose which is more attractive. Data collection was performed via a web application especially designed and developed for this purpose. In the experiments conducted, we evaluated each model and compared the two data collection methods. This work aims to illustrate the effectiveness of deep learning as a solution to the problem as well as to highlight the importance of comparison-based ranking in order to achieve reliable results. In order to further promote our work, we also developed a tool that scores the aesthetics of a website, simply by providing the website URL. We argue that such a tool will serve as a reliable guide in the hands of designers and developers during the design process.