Non-intrusive user modelling and behaviour prediction in museums

Abstract

This thesis investigates non-intrusive user modelling techniques for predicting museum visitors' movements and interests in exhibits. Our research is motivated by the need to provide automated support to visitors of museums. Such support is needed as visitors can be overwhelmed by the vast amount of information in museum spaces, making it difficult to select personally interesting exhibits. To assist visitors in this selection process, computer-based technology can process non-intrusive observations of visitors' movements in the physical museum to provide input to our models. Our models in turn will eventually enable personalised exhibit recommendations based on the predictions they generate. The physicality of the museum domain poses practical challenges for developing predictive user models. For example, datasets of visitor pathways through a museum are difficult to obtain prior to deploying positioning technology in the physical museum space. However, such datasets are necessary to assess different modelling techniques. This thesis describes an approach for computer-supported semi-automated collection of visitor pathways by observationally tracking visitors in a museum. We used this approach to conduct a data collection at Melbourne Museum (Melbourne, Australia). The resultant dataset of 158 complete visit trajectories serves as a basis for evaluating our user models. For predicting visitor pathways, we discuss distance-based transition models derived from the spatial layout of the museum, and develop frequency-based transition models derived from non-intrusive observations of other visitors' previous movements. These models are then used to predict a visitor's next few most likely exhibits as a ranked set and sequence. Our results show that the frequency-based models mostly outperform the distance-based baselines, which suggests that other people's movements are better predictors of a visitor's movements than the spatial layout of the museum. Additionally, our results indicate that sequence-based prediction outperforms set-based prediction when predicting more than one next exhibit, which suggests that sequence information aids prediction. To measure interest, we transform a visitor's previous viewing durations at museum exhibits into implicit exhibit ratings. These ratings serve as input to two nearest-neighbour collaborative filters and two content-based models for interest prediction. We also develop an interest model based on the theory of spatial processes, which models visitors' rating vectors as independent Gaussian random vectors, but shares the mean vector and exhibit-to-exhibit covariance matrix across visitors. This covariance matrix has a special structure, which requires a notion of distance between exhibits. We develop models of museum exhibit distance derived from viewing-time similarity, semantic similarity, and walking distance. Our results suggest that utilising walking and semantic distances between exhibits enables more accurate predictions of a visitor's interests in unseen exhibits than using distances derived from observed exhibit viewing times. Our evaluation also shows that content-based interest prediction yields better results than nearest-neighbour collaborative prediction, and that our model based on spatial processes attains the highest predictive accuracy overall. We also explore ways of improving the performance of our pathway and interest models by means of model hybridisation: (1) we incorporate a visitor's interests in exhibits into one of our models for pathway prediction; and (2) propose a generic user- and item-aware weighting scheme for linearly combining predictive user models, which is used to combine two variants of our interest model based on spatial processes. Personalising the museum experience is a challenging task, as predictions differ from recommendations (we do not want to recommend exhibits that visitors are going to see anyway). This is in contrast to traditional recommender systems for the virtual domain, where predictions regarding a user's interests directly determine the ranking of items and recommendations. To round off the thesis, we suggest an approach for generating interesting exhibit recommendations based on the predictions of our models. This approach compares the exhibits predicted to be of interest to a visitor (generated by our interest models) with a prediction of the visitor's short-term pathway through the museum (generated by our pathway models), and supports the recommendation of personally interesting exhibits that are not going to be seen immediately if the predicted pathway is followed. The key contributions of this thesis are as follows: - A computer-supported approach for recording, visualising and analysing the movements and viewing behaviour of museum visitors - Models for predicting visitors' next few most likely exhibits from non-intrusive observations of the visitors' previous movements through the museum - Models for predicting visitors' interests in exhibits from non-intrusive observations of the visitors' previous viewing behaviour in the museum - Ways of improving predictive accuracy by means of model hybridisation