Content validity and methodological considerations in ecological momentary assessment studies on physical activity and sedentary behaviour: a systematic review

L Degroote,D Van Dyck,I De Bourdeaudhuij,G Crombez,A Desmet

doi:10.1186/s12966-020-00932-9

Abstract

BackgroundEcological momentary assessment (EMA) is a method of collecting real-time data based on repeated measures and observations that take place in participant’s daily environment. EMA has many advantages over more traditional, retrospective questionnaires. However, EMA faces some challenges to reach its full potential. The aims of this systematic review are to (1) investigate whether and how content validity of the items (i.e. the specific questions that are part of a larger EMA questionnaire) used in EMA studies on physical activity and sedentary behaviour was assessed, and (2) provide an overview of important methodological considerations of EMA in measuring physical activity and sedentary behaviour.MethodsThirty papers (twenty unique studies) were systematically reviewed and variables were coded and analysed within the following 4 domains: (1) Content validity, (2) Sampling approach, (3) Data input modalities and (4) Degree of EMA completion.ResultsOnly about half of the studies reported the specific items (n = 12) and the source of the items (n = 11). None of the studies specifically assessed the content validity of the items used. Only a minority (n = 5) of the studies reported any training, and one tested the comprehensibility of the EMA items. A wide variability was found in the design and methodology of the EMA. A minority of the studies (n = 7) reported a rationale for the used prompt frequency, time selection, and monitoring period. Retrospective assessment periods varied from ‘now’ to ‘in the last 3.5 hours’. In some studies there was a possibility to delay (n = 6) or deactivate (n = 10) the prompt, and some provided reminders after the first prompt (n = 9).ConclusionsAlmost no EMA studies reported the content validation of the items used. We recommend using the COSMIN checklist (COnsensus-based Standards for the selection of health Measurement INstruments) to report on the content validity of EMA items. Furthermore, as often no rationale was provided for several methodological decisions, the following three recommendations are made. First, provide a rationale for choosing the sampling modalities. Second, to ensure assessment ‘in the moment’, think carefully about the retrospective assessment period, reminders, and deactivation of the prompt. Third, as high completion rates are important for representativeness of the data and generalizability of the findings, report completion rates.Trial registrationThis review is registered in PROSPERO, the International prospective register of systematic reviews (registration number: CRD42017077996).

Highlights

Ecological momentary assessment (EMA) is a method of collecting real-time data based on repeated measures and observations that take place in participant’s daily environment
We only focus on content validity of EMA as this is a main premise for all other forms of validity, including construct validity
Inclusion criteria To be included, studies had to meet the following inclusion criteria: (1) to use EMA to assess physical activity or/and sedentary behaviour and/or their psychosocial and environmental determinants; (2) to use an electronic device as platform for the EMA; excluding studies using pencil and paper; (3) to comprise healthy people; excluding studies in clinical samples, e.g. people with insomnia, posttraumatic stress disorder, schizophrenia, etc. as this review focused on health promotion

Summary

Introduction

Ecological momentary assessment (EMA) is a method of collecting real-time data based on repeated measures and observations that take place in participant’s daily environment. A detailed analysis of the psychological, social and physical environmental determinants in context may provide a profound understanding about why, how, and when these behaviours are elicited [11–13]. Many studies use self-report questionnaires, in which participants reflect on their level of activity and its determinants, typically over an extended time period (last day/week) [17]. These self-report methods are prone to biases [18, 19]. Variation in behaviours and determinants over time and context may be overlooked

Methods

Results

Discussion

Conclusion