Measure User Engagement Research Articles

The Examination of the User Engagement Scale (UES) in Small Medium Enterprise Social Media Usage: A Survey-Based Quantitative Study

Social networks have proven to be an essential marketing tool for the success of any product, service, or business. User participation affects the increase in revenue gain and creates long-term profit. The User Engagement Scale (UES) is one of the tools developed to measure user engagement and has been used in various digital domains. The UES intends to compute six dimensions of engagement: aesthetic appeal, perceived usability, focused attention, novelty, felt involvement, and endurability. This study investigates and verifies the three-factor structure of the UES. We used PCA to perform the analysis. The original data will be reanalyzed using UES, which consists of 220 valid responses. The result shows that the UES examination indicates good reliability in three factors. Factor 1 encompasses the feeling of involvement (FI), aesthetic appeal (AE), novelty (NO), and endurability (EN). Factor 2 aggregates the perceived usability (PU) elements. Factor 3 pertains to focused attention (FA) items. Our findings indicate that the User Engagement Scale is a valuable and suitable measurement tool for assessing user engagement in the context of social media within small and medium enterprises.

A twitter analysis of patient and family experience in pediatric spine surgery.

There is little data on patient and caregiver perceptions of spine surgery in children and youth. This study aims to characterize the personal experiences of patients, caregivers, and family members surrounding pediatric spine surgery through a qualitative and quantitative social media analysis. The Twitter application programming interface was searched for keywords related to pediatric spine surgery from inception to March 2022. Relevant tweets and accounts were extracted and subsequently classified using thematic labels. Tweet metadata was collected to measure user engagement via multivariable regression. Sentiment analysis using Natural Language Processing was performed on all tweets with a focus on tweets discussing the personal experiences of patients and caregivers. 2424 tweets from 1847 individual accounts were retrieved for analysis. Patients and caregivers represented 1459 (79.0%) of all accounts. Posts discussed the personal experiences of patients and caregivers in 83.5% of tweets. Pediatric spine surgery research was discussed in few posts (n=90, 3.7%). Within the personal experience category, 975 (48.17%) tweets were positive, 516 (25.49%) were negative, and 533 (26.34%) were neutral. Presence of a tag (beta: -6.1, 95% CI -9.7 to -2.5) and baseline follower count (beta<0.001, 95% CI <0.001 to <0.001) significantly affected tweet engagement negatively and positively, respectively. Patients and caregivers actively discuss topics related to pediatric spine surgery on Twitter. Posts discussing personal experience are most prevalent, while posts on research are scarce, unlike previous social media studies. Pediatric spine surgeons can leverage this dialogue to better understand the worries and needs of patients and their families.

Exploring Clustering Techniques for Analyzing User Engagement Patterns in Twitter Data

Social media platforms have revolutionized information exchange and socialization in today’s world. Twitter, as one of the prominent platforms, enables users to connect with others and express their opinions. This study focuses on analyzing user engagement levels on Twitter using graph mining and clustering techniques. We measure user engagement based on various tweet attributes, including retweets, replies, and more. Specifically, we explore the strength of user connections in Twitter networks by examining the diversity of edges. Our approach incorporates graph mining models that assign different weights to evaluate the significance of each connection. Additionally, clustering techniques are employed to group users based on their engagement patterns and behaviors. Statistical analysis was conducted to assess the similarity between user profiles, as well as attributes, such as friendship, followings, and interactions within the Twitter social network. The findings highlight the discovery of closely linked user groups and the identification of distinct clusters based on engagement levels. This research emphasizes the importance of understanding both individual and group behaviors in comprehending user engagement dynamics on Twitter.

Cluster-N-Engage: A New Framework for Measuring User Engagement of Website With User Navigational Behavior

Cluster-N-Engage: A New Framework for Measuring User Engagement of Website With User Navigational Behavior

Towards an integrated framework to measure user engagement with interactive or physical products

Building great products or services is not easy; users want products and services that exceed their expectations and evolve with their needs; it is not just about building the right features. Knowing the user engagement (UE) towards a physical, virtual product or service can give valuable information that could be used as feedback for the design, enhancing its chances of success. In the context of user-centered design, UE is the assessment of the user experience characterized by the study of the individual's cognitive, affective, and behavioral response to some stimulus, such as a product, a service, or a website. UE considers not only the users’ requirements and wishes but also their perceptions and reactions during and after an interaction with a product, system, or service. Many studies looking to quantify the UE are available. Still, a framework that provides a generic view of the most commonly used methods and metrics to measure UE does not yet exist in the literature. Aiming to understand the UE better, in this research, we developed a conceptual framework summarizing the available metrics and techniques used across different contexts, including good practices of self-report methods and physiological approaches. We expect this study will allow future researchers, developers, and designers to consider the UE as one of the most prominent product/service success indicators and use this guideline to find the more appropriate method, technique, and metric for its measurement.

Measuring user engagement with low credibility media sources in a controversial online debate

We quantify social media user engagement with low-credibility online news media sources using a simple and intuitive methodology, that we showcase with an empirical case study of the Twitter debate on immigration in Italy. By assigning the Twitter users an Untrustworthiness (U) score based on how frequently they engage with unreliable media outlets and cross-checking it with a qualitative political annotation of the communities, we show that such information consumption is not equally distributed across the Twitter users. Indeed, we identify clusters characterised by a very high presence of accounts that frequently share content from less reliable news sources. The users with high U are more keen to interact with bot-like accounts that tend to inject more unreliable content into the network and to retweet that content. Thus, our methodology applied to this real-world network provides evidence, in an easy and straightforward way, that there is strong interplay between accounts that display higher bot-like activity and users more focused on news from unreliable sources and that this influences the diffusion of this information across the network.

Measuring User Engagement with a Socially Connected, Gamified Health Promotion Mobile App.

Participant engagement is an important consideration in mHealth interventions and there are no standardised measurements available to guide researchers. This paper describes the engagement index customised for the Milk Man app, a mobile app designed to engage fathers with breastfeeding and parenting information. Participants were recruited from maternity hospitals in Perth, Western Australia. An engagement index with scores ranging from 0 to 100 was calculated. Kaplan Meier survival analysis was used to determine difference in duration of exclusive breastfeeding, and Pearson’s chi square analysis was conducted to investigate the association of engagement level with demographic characteristics and exclusive breastfeeding at 6 weeks. While overall, partners of participants who installed Milk Man were less likely to have ceased exclusive breastfeeding at any time point from birth to six weeks postpartum, this result was modest and of borderline significance (log rank test p = 0.052; Breslow p = 0.046; Tarone-Ware p = 0.049). The mean engagement score was 29.7% (range 1–80%), median 27.6%. Engagement level had no impact on duration of exclusive breastfeeding and demographic factors were not associated with engagement level. This research demonstrates a range of metrics that can be used to quantify participant engagement. However, more research is needed to identify ways of measuring effective engagement.

FaceEngage: Robust Estimation of Gameplay Engagement from User-Contributed (YouTube) Videos

Measuring user engagement in interactive tasks can facilitate numerous applications toward optimizing user experience, ranging from eLearning to gaming. However, a significant challenge is the lack of non-contact engagement estimation methods that are robust in unconstrained environments. We present FaceEngage, a non-intrusive engagement estimator leveraging user facial recordings during actual gameplay in naturalistic conditions. Our contributions are three-fold. First, we show the potential of using front-facing videos as training data to build the engagement estimator. We compile FaceEngage Dataset with over 700 picture-in-picture, realisitic, and user-contributed YouTube gaming videos (i.e., with both full-screen game scenes and time-synchronized user facial recordings in subwindows). Second, we develop FaceEngage system, that captures relevant gamer facial features from front-facing recordings to infer task engagement. We implement two FaceEngage pipelines: an estimator trained on user facial motion features inspired by prior psychological works, and a deep learning-enabled estimator. Lastly, we conduct extensive experiments and conclude: (i) certain user facial motion cues (e.g., blink rates, head movements) are engagement-indicative; (ii) our deep learning-enabled FaceEngage pipeline can automatically extract more informative features, outperforming the facial motion feature-based pipeline; (iii) FaceEngage is robust to various video lengths, users/game genres and interpretable. Despite the challenging nature of realistic videos, FaceEngage attains the accuracy of 83.8 percent and leave-one-user-out precision of 79.9 percent, both of which are superior to our face motion-based model.

Posts Characterization and User Engagement

Many health organizations use Facebook as a platform to engage with users. This has resulted in many research studies conducted on this platform. One popular type of research study is to characterize posts and measure user engagement levels . In this study, 100 post on the Mental Health Foundation of New Zealand Facebook page was analyzed this purpose. A mixed method approach was used. Quantitative descriptive statistics was used to analyze levels of engagement whilst qualitative content analysis was used to characterize posts into themes. Preliminary results indicate most posts fit in the awareness theme followed by the campaign theme. High levels of user engagements was observed for posts related to the awareness and others theme. Results of this study makes the suggestion for the implementation of intervention type awareness posts. A recommendation is also to made that the awareness posts promote mental health education and communication. This research study adds new knowledge to the area of posts characterization and user engagement levels on a mental health Facebook page.

Posts Characterization and User Engagement A Preliminary Study on a Mental Health Facebook Page in New Zealand

Many health organizations use Facebook as a platform to engage with users. This has resulted in many research studies conducted on this platform. One popular type of research study is to characterize posts and measure user engagement levels . In this study, 100 post on the Mental Health Foundation of New Zealand Facebook page was analyzed this purpose. A mixed method approach was used. Quantitative descriptive statistics was used to analyze levels of engagement whilst qualitative content analysis was used to characterize posts into themes. Preliminary results indicate most posts fit in the awareness theme followed by the campaign theme. High levels of user engagements was observed for posts related to the awareness and others theme. Results of this study makes the suggestion for the implementation of intervention type awareness posts. A recommendation is also to made that the awareness posts promote mental health education and communication. This research study adds new knowledge to the area of posts characterization and user engagement levels on a mental health Facebook page.

What and How long: Prediction of Mobile App Engagement

User engagement is crucial to the long-term success of a mobile app. Several metrics, such as dwell time, have been used for measuring user engagement. However, how to effectively predict user engagement in the context of mobile apps is still an open research question. For example, do the mobile usage contexts (e.g., time of day) in which users access mobile apps impact their dwell time? Answers to such questions could help mobile operating system and publishers to optimize advertising and service placement. In this article, we first conduct an empirical study for assessing how user characteristics, temporal features, and the short/long-term contexts contribute to gains in predicting users’ app dwell time on the population level. The comprehensive analysis is conducted on large app usage logs collected through a mobile advertising company. The dataset covers more than 12K anonymous users and 1.3 million log events. Based on the analysis, we further investigate a novel mobile app engagement prediction problem—can we predict simultaneously what app the user will use next and how long he/she will stay on that app? We propose several strategies for this joint prediction problem and demonstrate that our model can improve the performance significantly when compared with the state-of-the-art baselines. Our work can help mobile system developers in designing a better and more engagement-aware mobile app user experience.

#PowerInNumbers: How Digital Libraries Use Collaborative Social Media Campaigns to Promote Collections

Social media provides ample opportunity for digital libraries to work together to promote their online collections to users. Despite this, institutions rarely connect on social networking platforms, and there is little research of the outcomes when they do. This article examines the benefits of the monthly collaborative social media campaign #ArchivesHashtagParty, launched by the National Archives and Records Administration in August 2017, during the month of April 2021. Using Twitter analytics and personal observations, the author explores how users engage with the campaign and interact with material, how digital libraries work together as a community to promote their collections, and whether this campaign benefits participating libraries. Using return on investment to measure user engagement and increase in followers, this study finds that participating in this one-day collaborative campaign can benefit digital libraries of all sizes and is a valuable addition to social media strategies.

El engagement y la sorpresa en la comunicación digital de las marcas

Purpose: Two studies were carried out. The main objective of study 1 was to find out the emotional effects and engagement that «GIFs» communication resources provoke in users. The main objective of study 2 was to determine the implicit attitudes that participants have towards well-known brands. Design / Methodology /Approach: The first study uses a combination of biometric tools of automated observation of facial expressions, skin conductance and pupil direction to observe the emotional state that GIFs of well-known brands elicit in participants, and compare it to the effects caused by static images of the same brands. The second study uses the Implicit Association Test to observe unconscious associations related to well-known brands. Results: The results of the first study show that GIFs achieve user engagement and provoke a «feel-good-pleasure state» (Russel’s Circumplex Model, 1980), but not the expected enthusiastic state. The results of the second study indicate that participants have the belief that wellknown brands are quality brands, while they do not associate this attribute with non-famous brands. The correlation found between the results of the two studies reveals that GIFs, while provoking positive emotions and generating engagement, do not achieve an enthusiastic state when brands are internalised as quality brands. Limitations / implications: The research method used has shown its usefulness in measuring user engagement in viewing GIFs using metrics such as the sign and intensity of emotion, and neuromarketing tools. The experimental study had its main limitation in not being able to observe the emotional effects generated by GIFs that allow user interaction. Originality / contribution: Well-known brands, in addition to using these innovative visual resources, have to use them in a different way to generate surprise and manage to place potential customers in the «enthusiastic emotional state» of the Circumplex Model (Russel,1980).

Asymmetric Network Effects in the Integration of Digital Platforms

We have witnessed an increasing number of mergers and acquisitions (M&A) between two-sided digital platforms, generally involving a unidirectional resource transfer policy from the acquired to the acquiring platform. This scenario may incur an asymmetric network effect that has not been explored in the extant literature. Using a unique dataset from Dianping and Meituan, two digital platforms similar to Yelp, we explore whether and how the unidirectional data integration policy causes a positive network effect, as reflected by the increments of user engagement on each side of the M&A. We measure user engagement in two dimensions, namely, quantity measures (e.g., number of total reviews) and quality measures (e.g., number of reviews with pictures and length of review). The results show that the policy enhances both measures of user engagement for the acquired and acquiring platforms but that its effects on user engagement on the acquiring platform are larger than those on the acquired platform. We propose three mechanisms: the network size effect, self-image management effect, and social learning effect to explain the possible impacts of platform integration on user engagement, and we conduct single-group interrupted time-series analyses to test the effects. We further conduct three sets of supplementary analyses. We find that, first, the policy not only expands its user base but also boosts the contribution enthusiasm of current users. Second, we clarify how the benefits of platform integration are shaped by merchant features such as per capita consumption and whether a merchant has chain stores. Third, we focus on the multi-homing phenomenon of merchants and find that multi-homing merchants benefit more from platform integration. The findings of this study deepen our understanding of the connections between unidirectional resource transfer and the asymmetric network effect in the context of the integration of two digital platforms.

Optimizing for Engagement Can Be Harmful. There Are Alternatives

Reports on the topic of personalization systems. These systems, including content filtering and recommender engines, have become the cornerstone of social media and digital business strategy. Back in 2016, Netflix estimated its recommendation engine brought $1 billion in value to the company. On Amazon, 35% of purchases come via recommendations (https://www. mckinsey.com/industries/retail/our-insights/howretailers- can-keep-up-with-consumers). And, of course, Facebook, Twitter, and other social media companies carefully measure user engagement with content and have finely tuned algorithms that present it in an order that is most engaging and that suggest more content designed to keep people in the systems longer.

Applicability of the User Engagement Scale to Mobile Health: A Survey-Based Quantitative Study.

BackgroundThere has recently been exponential growth in the development and use of health apps on mobile phones. As with most mobile apps, however, the majority of users abandon them quickly and after minimal use. One of the most critical factors for the success of a health app is how to support users’ commitment to their health. Despite increased interest from researchers in mobile health, few studies have examined the measurement of user engagement with health apps.ObjectiveUser engagement is a multidimensional, complex phenomenon. The aim of this study was to understand the concept of user engagement and, in particular, to demonstrate the applicability of a user engagement scale (UES) to mobile health apps.MethodsTo determine the measurability of user engagement in a mobile health context, a UES was employed, which is a psychometric tool to measure user engagement with a digital system. This was adapted to Ada, developed by Ada Health, an artificial intelligence–powered personalized health guide that helps people understand their health. A principal component analysis (PCA) with varimax rotation was conducted on 30 items. In addition, sum scores as means of each subscale were calculated.ResultsSurvey data from 73 Ada users were analyzed. PCA was determined to be suitable, as verified by the sampling adequacy of Kaiser-Meyer-Olkin=0.858, a significant Bartlett test of sphericity (χ2300=1127.1; P<.001), and communalities mostly within the 0.7 range. Although 5 items had to be removed because of low factor loadings, the results of the remaining 25 items revealed 4 attributes: perceived usability, aesthetic appeal, reward, and focused attention. Ada users showed the highest engagement level with perceived usability, with a value of 294, followed by aesthetic appeal, reward, and focused attention.ConclusionsAlthough the UES was deployed in German and adapted to another digital domain, PCA yielded consistent subscales and a 4-factor structure. This indicates that user engagement with health apps can be assessed with the German version of the UES. These results can benefit related mobile health app engagement research and may be of importance to marketers and app developers.

Stages of User Engagement on Social Commerce Platforms: Analysis with the Navigational Clickstream Data

ABSTRACTSocial commerce platforms have gained prominence in e-commerce, as social media has become an integral part of users’ online activities. Therefore, firms have been either developing or utilizing social commerce platforms to increase user engagement by adding social shopping facility onto their electronic commerce platforms. However, managing user engagement and user interaction becomes complex when e-commerce platforms are transformed into social commerce platforms. In this study, we operationalize four distinct stages of the social commerce platform, namely, social identification, social interaction, social shopping, and transaction based on salience theory. Using clickstream data, we empirically measure user engagement in these four states by modeling users’ incidence and time spent. Drawing from the PageRank algorithm, we capture the importance of ranking and distance on user engagement. The model also accounts for the effects of situational variables such as weekend; holiday; time of day; and user characteristics, such as gender and social media setting. Our results suggest that ranking and distance have significant effects on users’ incidence as well as time spent on social commerce platforms. The insights from this study can be helpful in designing the social commerce platform effectively using only the customers’ path navigational clickstream data from the parent social commerce platform.

CrowdWatcher: an open-source platform to catch the eye of the crowd

This paper presents the open-source eye tracking platform CrowdWatcher. It enables researchers to measure gaze location and user engagement in a crowdsourcing context through traditional RGB webcams. The proposed platform particularly advances the field of Quality of Experience (QoE) research, as it allows the experimenter to collect remotely and with very limited effort novel information from crowds of participants, such as their commitment towards a task, attention and decision-making processes. Two different experiments are described that were conducted to demonstrate the platform’s potential. The first experiment addresses the measurement of participants’ behavior while performing a movie selection task. Results show that the platform provides complementary information to traditional self-reported data by taking gaze analysis into account. This is of particular relevance, since in a crowdsourcing context decision processes and attention are difficult to assess, and there is often limited control over the engagement of the test user with the task. A second experiment is conducted in the scenario of a multimedia QoE test. Prediction accuracy is compared to a professional infrared eye tracker. While CrowdWatcher performs less well than the professional eye tracker, it is still able to collect valuable gaze information in the far more challenging environment of crowdsourcing. As an outlook to further application domains, the usage of the platform to measure user engagement allows participants who do not pay attention to the task to be identified.

A practical approach to measuring user engagement with the refined user engagement scale (UES) and new UES short form

User engagement (UE) and its measurement have been of increasing interest in human-computer interaction (HCI). The User Engagement Scale (UES) is one tool developed to measure UE, and has been used in a variety of digital domains. The original UES consisted of 31-items and purported to measure six dimensions of engagement: aesthetic appeal, focused attention, novelty, perceived usability, felt involvement, and endurability. A recent synthesis of the literature questioned the original six-factors. Further, the ways in which the UES has been implemented in studies suggests there may be a need for a briefer version of the questionnaire and more effective documentation to guide its use and analysis. This research investigated and verified a four-factor structure of the UES and proposed a Short Form (SF). We employed contemporary statistical tools that were unavailable during the UES’ development to re-analyze the original data, consisting of 427 and 779 valid responses across two studies, and examined new data (N=344) gathered as part of a three-year digital library project. In this paper we detail our analyses, present a revised long and short form (SF) version of the UES, and offer guidance for researchers interested in adopting the UES and UES-SF in their own studies.

CCLTracker Framework: Monitoring user learning and activity in web based citizen science projects

Analytics tools have been widely used over the last years for the development of web-based application and services. Analytics data allows improving user interfaces through planning, executing, and evaluating actions intended to increase user engagement. Measuring and improving user engagement in citizen science projects is not different from other web applications such as on-line shopping, newspapers, or sites for recommending music or movies. However, citizen science projects also aim to produce learning outcomes on the participants. Current analytics tools do not present sufficient information regarding user behaviour with the application, thus making measuring engagement and learning outcomes difficult. This paper presents the CCLTracker analytics framework that is intended to overcome current limitations in analytics tools, by providing an API for monitoring user activities such as time spent watching a video, time to complete a task, or how far down a page is scrolled. CCLTracker has been integrated in 3 different citizen science projects which have proved its value for measuring user engagement and learning.