The accuracy and validity of self-reported social media use measures among adolescents

Tim Verbeij,J Loes Pouwels,Ine Beyens,Patti M Valkenburg

doi:10.1016/j.chbr.2021.100090

Tim Verbeij, J Loes Pouwels + Show 2 more

Open Access

https://doi.org/10.1016/j.chbr.2021.100090

Copy DOI

Abstract

A growing number of studies have tried to assess the effects of social media on adolescents, who are among the most avid social media users. To establish the effects of social media use, we need accurate and valid instruments to measure adolescents’ time spent with these media. The aim of this preregistered study was to examine the accuracy and convergent validity of retrospective surveys and experience sampling method (ESM) surveys, by comparing adolescents’ responses to these self-report measures with their digital trace data. The sample consisted of 125 adolescents (48% girls; Mage = 14.1) with Android smartphones. In both retrospective surveys and ESM, adolescents overestimated their time spent on social media. They more accurately estimated their time spent on platforms that are used in a less fragmented way (Instagram) than on platforms that are used in a more fragmented way (Snapchat). The between-person convergent validity of adolescents’ time estimates according to retrospective surveys and ESM reached the threshold for minimum acceptable convergent validity (r ranged from .55 to .65). The within-person convergent validity of adolescents’ ESM estimates of their time spent on social media was unacceptable (r = .32). The between- and within-person convergent validity of ESM estimates decreased over time (i.e., fatigue effect).

Highlights

A growing number of studies have tried to assess the effects of social media on adolescents, who are among the most avid social media users
Since most adolescents use multiple social media platforms in functionally complementary ways (Waterloo, Baumgartner, Peter, & Valkenburg, 2017), we investigated whether the accuracy and convergent validity of self-report measures of time spent on social media differs across the three mostly used social media platforms among Dutch adolescents: Instagram, WhatsApp, and Snapchat
Comparing the accuracy across selfreport measures, retrospective survey estimates in both a typical week (d 1⁄4 .60) and a previous week (d 1⁄4 .39) were more accurate than adolescents’ experience sampling method (ESM) estimates (d 1⁄4 .91) of their time spent on social media as they yielded smaller differences with digital trace data

Summary

Introduction

A growing number of studies have tried to assess the effects of social media on adolescents, who are among the most avid social media users. Several researchers have called for ESM studies to investigate the effects of social media use because they believe that ESM measures reduce recall bias, so that they provide more accurate and valid estimates of adolescents’ time spent on social media than retrospective surveys (Dienlin & Johannes, 2020; Griffioen et al, 2020; Naab et al, 2018; Orben & Przybylski, 2019) This assumption has never been investigated, because previous studies have compared digital trace data with either ESM data (Deng et al, 2019) or retrospective survey estimates (e.g., Junco, 2013; Sewall et al, 2020). We investigated two opposing hypotheses regarding such time effects: the learning effect hypothesis, which proposes more accuracy and higher convergent validity across time and the fatigue effect hypothesis, which proposes less accuracy and lower convergent validity across time

Objectives

Methods

Results