Usage Metrics of Web-Based Interventions Evaluated in Randomized Controlled Trials: Systematic Review.

Elena Koneska,Susanna Dodd,Paula R Williamson,Duncan Appelbe

doi:10.2196/15474

Abstract

BackgroundThe evaluation of web-based interventions (defined as an intervention that can be downloaded or accessed on the internet through a web browser) in randomized controlled trials (RCTs) has increased over the past two decades. Little is known about how participants’ use of the intervention is measured, reported, and analyzed in these studies.ObjectiveThis study aimed to review the evaluation of web-based interventions in RCTs, assessing study characteristics and the methods used to record, and adjust for, intervention usage.MethodsA systematic review of the literature was undertaken to identify all published reports of RCTs that involved a web-based intervention. A random sample of 100 published trials was selected for detailed data extraction. Information on trial characteristics was extracted, including whether web usage data were recorded, and if so, the methods used to gather these data and whether these data were used to inform efficacy analyses.ResultsA PubMed search identified 812 trials of web-based interventions published up to the end of 2017 and demonstrated a growing trend over time. Of the 100 studies reviewed, 90 studies collected web usage data, but more than half (49/90, 54%) of these studies did not state the method used for recording web usage. Only four studies attempted to check on the reliability of their web usage data collection methods. A total of 39% (35/90) studies reported patterns or levels of web intervention use, of which 21% (19/90) studies adjusted for intervention use in their outcome analysis, but only two of these used appropriate statistical methods.ConclusionsTrialists frequently report a measure of web-based intervention usage but do not always report the collection method or provide enough detail on their analysis of web usage. Appropriate statistical methods to account for intervention use are rarely used and are not well reported even in the very few trials in which they are used. The number of trialists who attempt to check on the reliability of their web usage collection methods is extremely low.

Highlights

Randomized Controlled TrialsA randomized controlled trial (RCT) is used to assess the efficacy or effectiveness of an intervention by randomly dividing trial participants into experimental or control treatment arms, thereby providing a fair comparison for the unbiased assessment of treatment effects [1,2,3,4]
The PubMed search for systematic reviews of web-based intervention trials identified 271 citations, 123 of which were found to be eligible following a review of titles and abstracts
This study suggests that there is a need for greater adherence to reporting guidelines in publications of web-based intervention trials

Summary

Introduction

Randomized Controlled TrialsA randomized controlled trial (RCT) is used to assess the efficacy or effectiveness of an intervention by randomly dividing trial participants into experimental or control treatment arms, thereby providing a fair comparison for the unbiased assessment of treatment effects [1,2,3,4]. With an estimated 4.4 billion people being active internet users as of April 2019 [8], an increasing proportion of the global population are potential users of web-based interventions, given the convenience and flexibility of such interventions. As such, these interventions have enormous potential to improve health and health care delivery and can be accessible to patients [1,9,10,11]. The evaluation of web-based interventions (defined as an intervention that can be downloaded or accessed on the internet through a web browser) in randomized controlled trials (RCTs) has increased over the past two decades. The number of trialists who attempt to check on the reliability of their web usage collection methods is extremely low

Methods

Results

Discussion

Conclusion