How well do network models predict observations? On the importance of predictability in network models

Jonas M B Haslbeck,Lourens J Waldorp

doi:10.3758/s13428-017-0910-x

Abstract

Network models are an increasingly popular way to abstract complex psychological phenomena. While studying the structure of network models has led to many important insights, little attention has been paid to how well they predict observations. This is despite the fact that predictability is crucial for judging the practical relevance of edges: for instance in clinical practice, predictability of a symptom indicates whether an intervention on that symptom through the symptom network is promising. We close this methodological gap by introducing nodewise predictability, which quantifies how well a given node can be predicted by all other nodes it is connected to in the network. In addition, we provide fully reproducible code examples of how to compute and visualize nodewise predictability both for cross-sectional and time series data.

Highlights

Network models graphically describe interactions between a potentially large number variables: each variable is represented as a dot and interactions are represented by lines connecting the nodes
We would like to know how close these are to the observed values in the data. Because it is of interest how well a given node can be predicted by all other nodes in the network, we need to remove any effects of the intercept and the marginal
Given the above definition of measures of predictability, it is evident that there is a close relationship between the parameters of the network model and predictability: if a node is not connected to any other node, the explained variance/ normalized accuracy of this node has to be 0

Summary

Introduction

Network models graphically describe interactions between a potentially large number variables: each variable is represented as a dot (node) and interactions are represented by lines (edges) connecting the nodes Most applications are in the field of clinical psychology (e.g., Fried et al, 2015; Fried, Epskamp, Nesse, Tuerlinckx, & Borsboom, 2016; Beard et al, 2016; McNally et al, 2015; Boschloo et al, 2015) but network models are applied in health psychology (Kossakowski, Epskamp, et al, 2016) and personality psychology (Cramer et al, 2012; Costantini et al, 2015) While initially they were used to model cross-sectional data, there is increasing interest in analyzing data obtained using the experience sampling method (ESM), which consists of repeated measurements of the same person (e.g., Bringmann et al, 2013; Pe et al, 2015). The data in our applications are from two published studies and will be downloaded automatically with the provided code

Methods

Findings

Discussion