Influence.ME: Tools for Detecting Influential Data in Mixed Effects Models

Abstract

influence.ME provides tools for de- tecting influential data in mixed effects mod- els. The application of these models has become common practice, but the development of diag- nostic tools has lagged behind. influence.ME calculates standardized measures of influential data for the point estimates of generalized mixed effects models, such as DFBETAS, Cook's dis- tance, as well as percentile change and a test for changing levels of significance. influence.ME calculates these measures of influence while ac- counting for the nesting structure of the data. The package and measures of influential data are introduced, a practical example is given, and strategies for dealing with influential data are suggested. The application of mixed effects regression models has become common practice in the field of social sci- ences. As used in the social sciences, mixed effects re- gression models take into account that observations on individual respondents are nested within higher- level groups such as schools, classrooms, states, and countries (Snijders and Bosker, 1999), and are often referred to as multilevel regression models. Despite these models' increasing popularity, diagnostic tools to evaluate fitted models lag behind. We introduce influence.ME (Nieuwenhuis, Pelzer, and te Grotenhuis, 2012), an R-package that provides tools for detecting influential cases in mixed effects regression models estimated with lme4 (Bates and Maechler, 2010). It is commonly accepted that tests for influential data should be performed on regression models, especially when estimates are based on a relatively small number of cases. How- ever, most existing procedures do not account for the nesting structure of the data. As a result, these existing procedures fail to detect that higher-level cases may be influential on estimates of variables measured at specifically that level. In this paper, we outline the basic rationale on de- tecting influential data, describe standardized mea- sures of influence, provide a practical example of the analysis of students in 23 schools, and discuss strate- gies for dealing with influential cases. Testing for influential cases in mixed effects regression models is important, because influential data negatively in- fluence the statistical fit and generalizability of the model. In social science applications of mixed mod- els the testing for influential data is especially im- portant, since these models are frequently based on large numbers of observations at the individual level while the number of higher level groups is relatively small. For instance, Van der Meer, te Grotenhuis, and Pelzer (2010) were unable to find any country-level comparative studies involving more than 54 coun- tries. With such a relatively low number of coun- tries, a single country can easily be overly influen- tial on the parameter estimates of one or more of the country-level variables.