Assessing the Psychometric Properties of the Digital Behavior Change Intervention Engagement Scale in Users of an App for Reducing Alcohol Consumption: Evaluation Study.

Olga Perski,Susan Michie,Claire Garnett,Ann Blandford,Jim Lumsden,Robert West

doi:10.2196/16197

Abstract

BackgroundThe level and type of engagement with digital behavior change interventions (DBCIs) are likely to influence their effectiveness, but validated self-report measures of engagement are lacking. The DBCI Engagement Scale was designed to assess behavioral (ie, amount, depth of use) and experiential (ie, attention, interest, enjoyment) dimensions of engagement.ObjectiveWe aimed to assess the psychometric properties of the DBCI Engagement Scale in users of a smartphone app for reducing alcohol consumption.MethodsParticipants (N=147) were UK-based, adult, excessive drinkers recruited via an online research platform. Participants downloaded the Drink Less app and completed the scale immediately after their first login in exchange for a financial reward. Criterion variables included the objectively recorded amount of use, depth of use, and subsequent login. Five types of validity (ie, construct, criterion, predictive, incremental, divergent) were examined in exploratory factor, correlational, and regression analyses. The Cronbach alpha was calculated to assess the scale’s internal reliability. Covariates included motivation to reduce alcohol consumption.ResultsResponses on the DBCI Engagement Scale could be characterized in terms of two largely independent subscales related to experience and behavior. The experiential and behavioral subscales showed high (α=.78) and moderate (α=.45) internal reliability, respectively. Total scale scores predicted future behavioral engagement (ie, subsequent login) with and without adjusting for users’ motivation to reduce alcohol consumption (adjusted odds ratio [ORadj]=1.14; 95% CI 1.03-1.27; P=.01), which was driven by the experiential (ORadj=1.19; 95% CI 1.05-1.34; P=.006) but not the behavioral subscale.ConclusionsThe DBCI Engagement Scale assesses behavioral and experiential aspects of engagement. The behavioral subscale may not be a valid indicator of behavioral engagement. The experiential subscale can predict subsequent behavioral engagement with an app for reducing alcohol consumption. Further refinements and validation of the scale in larger samples and across different DBCIs are needed.

Highlights

Some level of engagement with digital behavior change interventions (DBCIs) is necessary for the effectiveness of such interventions [1]
The behavioral subscale may not be a valid indicator of behavioral engagement
Studies conducted across different settings and target behaviors report a positive association of DBCI engagement and intervention effectiveness [4,5], suggesting that these variables may be linked via a dose-response function [1,6]

Summary

Introduction

Some level of engagement with digital behavior change interventions (DBCIs) is necessary for the effectiveness of such interventions [1]. Observed levels of engagement with DBCIs are often considered too limited to support behavior change [2]. Studies conducted across different settings and target behaviors report a positive association of DBCI engagement and intervention effectiveness [4,5], suggesting that these variables may be linked via a dose-response function [1,6]. Attempts have been made to characterize the function linking engagement with intervention effectiveness [1,7,8,9], but progress is hindered due to the use of different definitions and measures of engagement across studies. The level and type of engagement with digital behavior change interventions (DBCIs) are likely to influence their effectiveness, but validated self-report measures of engagement are lacking. The DBCI Engagement Scale was designed to assess behavioral (ie, amount, depth of use) and experiential (ie, attention, interest, enjoyment) dimensions of engagement

Methods

Results

Discussion

Conclusion