Using science to sell apps: Evaluation of mental health app store quality claims

Mark Erik Larsen,John Torous,Jennifer Nicholas,Emily Li,Louise Birrell,Bill Reda,Kit Huckvale

doi:10.1038/s41746-019-0093-1

Abstract

Despite the emergence of curated app libraries for mental health apps, personal searches by consumers remain a common method for discovering apps. App store descriptions therefore represent a key channel to inform consumer choice. This study examined the claims invoked through these app store descriptions, the extent to which scientific language is used to support such claims, and the corresponding evidence in the literature. Google Play and iTunes were searched for apps related to depression, self-harm, substance use, anxiety, and schizophrenia. The descriptions of the top-ranking, consumer-focused apps were coded to identify claims of acceptability and effectiveness, and forms of supporting statement. For apps which invoked ostensibly scientific principles, a literature search was conducted to assess their credibility. Seventy-three apps were coded, and the majority (64%) claimed effectiveness at diagnosing a mental health condition, or improving symptoms, mood or self-management. Scientific language was most frequently used to support these effectiveness claims (44%), although this included techniques not validated by literature searches (8/24 = 33%). Two apps described low-quality, primary evidence to support the use of the app. Only one app included a citation to published literature. A minority of apps (14%) described design or development involving lived experience, and none referenced certification or accreditation processes such as app libraries. Scientific language was the most frequently invoked form of support for use of mental health apps; however, high-quality evidence is not commonly described. Improved knowledge translation strategies may improve the adoption of other strategies, such as certification or lived experience co-design.

Highlights

Recent reviews have found mobile health apps to be effective in reducing symptoms of depression[1] and anxiety;[2] authors acknowledge the disparity between apps with research evidence and the apps currently available to – and used by – consumers
There is an increasing interest in accreditation processes,[8] app libraries[9,10] and frameworks to support clinicians in recommending mental health apps,[11] personal searches on commercial app stores operated by the major smartphone platform providers remain a common method for discovering mental health apps.[12]
This study aims to extend this preliminary analysis to further understand how scientific evidence is currently used to market and sell mental health apps by (i) examining the types of claims made by mental health apps and, estimating the proportion of apps that invoke claims of effectiveness; (ii) describing the types of supporting statements used to justify claims and, estimating the proportion of apps which invoke scientific principles; and (iii) assessing the credibility of scientific principles that are used as supporting statements

Summary

Introduction

Recent reviews have found mobile health (mHealth) apps to be effective in reducing symptoms of depression[1] and anxiety;[2] authors acknowledge the disparity between apps with research evidence and the apps currently available to – and used by – consumers. There is an increasing interest in accreditation processes,[8] app libraries[9,10] and frameworks to support clinicians in recommending mental health apps,[11] personal searches on commercial app stores operated by the major smartphone platform providers remain a common method for discovering mental health apps.[12]. In this setting, marketing materials provided by developers are a principal source of information to inform consumer or clinician choice. The format of this material is standardised for commercial app stores, consisting of a written app description and, optionally, screenshots or videos of app functions. A preliminary investigation by the authors previously reported that, for apps clinically relevant for depression, 38% of app store descriptions included wording related to claims of effectiveness, whereas only 2.6% provided evidence to substantiate such claims.[13]

Objectives

Methods

Results