USER ENGAGEMENT COMBINED ASSESSMENT SYSTEM

Abstract

The article discusses methods for assessing engagement as a characteristic of human activity. Existing systems for assessing engagement that are designed to assess either the activities of a software user or the quality of the software itself with which this user interacts are described. A method for the combined assessment of the engagement of software users based on heterogeneous data obtained during the user’s interaction with the software is described. Engagement indicators are proposed, defined as numerical interpretations of oculographic data, data on the emotional state of the user, and web analytics data. Due to the subjective nature of any interpretation, the combined assessment of engagement is defined using a fuzzy inference mechanism. Three linguistic variables characterizing each indicator are defined. A base of fuzzy production rules for a fuzzy inference system and a method for combining them based on fuzzy rules have been developed. The proposed method is implemented in a system that includes subsystems for collecting and sending user data and evaluating engagement, interacting with each other through the API. An experiment conducted to evaluate the effectiveness of the proposed method using the developed system is described. The experiment consisted in assessing the engagement of the software user, first by each indicator separately, then by their combination, and analyzing the degree of compliance of these assessments with the assessments received from an external expert and the user himself (self-assessment). An analysis of the results of the experiment showed that 63 % of the ratings obtained by the combined assessment method coincided with at least one of the assessments received from an external expert or the user himself, compared with 55 % of coincidences obtained using only the oculographic indicator, and 17 % of coincidences obtained using the “emotional involvement” indicator. Thus, the proposed method provides a higher degree of confidence in the results of the engagement assessment for making a decision based on this assessment than methods for assessing involvement that use each indicator separately.