Abstract
Trials of digital interventions can yield extensive, in-depth usage data, yet usage analyses tend to focus on broad descriptive summaries of how an intervention has been used by the whole sample. This paper proposes a novel framework to guide systematic, fine-grained usage analyses that better enables understanding of how an intervention works, when, and for whom. The framework comprises three stages to assist in the following: (1) familiarization with the intervention and its relationship to the captured data, (2) identification of meaningful measures of usage and specifying research questions to guide systematic analyses of usage data, and (3) preparation of datasheets and consideration of available analytical methods with which to examine the data. The framework can be applied to inform data capture during the development of a digital intervention and/or in the analysis of data after the completion of an evaluation trial. We will demonstrate how the framework shaped preparation and aided efficient data capture for a digital intervention to lower transmission of cold and flu viruses in the home, as well as how it informed a systematic, in-depth analysis of usage data collected from a separate digital intervention designed to promote self-management of colds and flu. The Analyzing and Measuring Usage and Engagement Data (AMUsED) framework guides systematic and efficient in-depth usage analyses that will support standardized reporting with transparent and replicable findings. These detailed findings may also enable examination of what constitutes effective engagement with particular interventions.
Highlights
Digital interventions are intended to support positive change in a range of health-related outcomes, including psychological, behavioral, educational, social, and environmental [1,2,3]
This paper proposes a novel framework to structure the process of analyzing usage associated with a digital intervention by doing the following: (1) drawing together potential measures of usage and identifying which are meaningful to the intervention, (2) generating specific research questions to act as testable hypotheses, and (3) supporting data preparation and selection of methods for analysis
The framework has been applied to two digital interventions: Germ Defence promotes respiratory tract infections (RTIs) prevention and Internet Dr supports self-management of RTI symptoms
Summary
Digital interventions are intended to support positive change in a range of health-related outcomes, including psychological, behavioral, educational, social, and environmental [1,2,3]. They may be delivered using any digital device (eg, phone and computer), making them cost-effective for providers [4,5]. Trialing a digital intervention can yield complex, large-scale datasets containing detailed usage data. If analyzed appropriately, this data is able to provide invaluable detail on how users interact with the intervention and inform our understanding of engagement. As a key element of engagement, in-depth and consistently applied usage analyses are capable of providing invaluable insight into the field of engagement with digital interventions
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have