Show Me Your Face: Towards an Automated Method to Provide Timely Guidance in Visual Analytics.

Davide Ceneda,Silvia Miksch,Alessio Arleo,Theresia Gschwandtner

doi:10.1109/tvcg.2021.3094870

Davide Ceneda, Silvia Miksch + Show 2 more

Open Access

https://doi.org/10.1109/tvcg.2021.3094870

Copy DOI

Abstract

Providing guidance during a Visual Analytics session can support analysts in pursuing their goals more efficiently. However, the effectiveness of guidance depends on many factors: Determining the right timing to provide it is one of them. Although in complex analysis scenarios choosing the right timing could make the difference between a dependable and a superfluous guidance, an analysis of the literature suggests that this problem did not receive enough attention. In this paper, we describe a methodology to determine moments in which guidance is needed. Our assumption is that the need of guidance would influence the user state-of-mind, as in distress situations during the analytical process, and we hypothesize that such moments could be identified by analyzing the user's facial expressions. We propose a framework composed by a facial recognition software and a machine learning model trained to detect when to provide guidance according to changes of the user facial expressions. We trained the model by interviewing eight analysts during their work and ranked multiple facial features based on their relative importance in determining the need of guidance. Finally, we show that by applying only minor modifications to its architecture, our prototype was able to detect a need of guidance on the fly and made our methodology well suited also for real-time analysis sessions. The results of our evaluations show that our methodology is indeed effective in determining when a need of guidance is present, which constitutes a prerequisite to providing timely and effective guidance in VA.

Highlights

Visual Analytics (VA) aims at enabling a better collaboration between humans and analytical systems by means of interactive visual interfaces [1]
We describe the training and use of a machine learning model (ML, Random Forrest) to analyze facial expressions and automatically identify the need for guidance in real time (Section 4)
We evaluate our methodology by evaluating the trained model with both test data and in a realtime analysis scenario (Section 5) discussing its advantages, possible limitations, and illustrating how subjective traits, such as the willingness of a user to accept the guidance, are factors that play a crucial role in determination of the most appropriate timing to initiate guidance (Section 6)

Summary

Introduction

Visual Analytics (VA) aims at enabling a better collaboration between humans and analytical systems by means of interactive visual interfaces [1]. Despite being straightforward to understand, the VA process hides challenges at any of its steps making it difficult to apply in practice. For this reason, in parallel to the development of VA methods, scientists have been studying approaches to help analysts using them. The science of assisting analysts has roots in interaction science and visual interface design [2], [3]. All their nuances can be grouped together under the term guidance [4]. Choosing a wrong timing may mislead and sway the analyst and interfere with the analysis as a whole

Objectives

Methods

Results

Discussion

Conclusion