Evaluating use of web-based interventions: an example of a Dutch sexual health intervention.

Abstract

With the current increase in web-based interventions, the question of how to measure, and consequently improve engagement in such interventions is gaining more importance. Modern day web analytics tools make it easy to monitor use of web-based interventions. However, in this article, we propose that it would be more meaningful to first examine how the developers envisioned the use of the intervention to establish behavior change (i.e. intended use), before looking into how the intervention is ultimately used with web analytics (i.e. actual use). Such an approach responds to the regularly expressed concern that behavioral interventions are often poorly described, leading to less meaningful evaluations as it is not clear what exactly is being evaluated. Using a page on chlamydia prevention (104 557 pageviews in 2020) from a Dutch sexual health intervention (Sense), we demonstrate the value of acyclic behavior change diagrams (ABCDs) as a method to visualize intended use of an intervention. ABCDs show at a glance how behavior change principles are applied in an intervention and target determinants of behavior. Based on this ABCD, we investigate actual use of the intervention, using web analytics tool Matomo. Despite being intended to stimulate STI-testing, only 14% of the 35 347 transfers from this page led to the STI-testing page and a high bounce rate (79%) and relatively high exit rate were reported (69%). Recommendations to further interpret the data are given. This real-life example demonstrates the potential of combining ABCDs and Matomo as methods to gain insight into use of web-based interventions.