Why Does Collaborative Filtering Work? Recommendation Model Validation and Selection By Analyzing Bipartite Random Graphs

Abstract

A large number of collaborative filtering (CF) algorithms have been proposed in the literature as the core of automated recommender systems. However, the underlying justification for these algorithms is lacking and their relative performances are typically domain- and data-dependent. In this paper, we aim to develop initial understanding of the validation and model/algorithm selection issues based on the graph topological modeling methodology. By representing the input data in the form of consumer-product interactions such as purchases and ratings as a bipartite graph, we develop bipartite graph topological measures to capture patterns that exist in the input data relevant to recommendation. Using a simulation approach, we observe the deviations of these topological measures for given recommendation datasets from the expected values for simulated random datasets. These deviations help explain why certain CF algorithms work for the given datasets. They can also serve as the basis for a comprehensive model selection framework that chooses appropriate CF algorithms given the characteristics of the dataset under study. We validate our approach using two real-world e-commerce datasets.