What Are Degrees of Freedom?

Abstract

As we were teaching a multivariate statistics course for doctoral students, one of the students in the class asked, What are degrees of freedom? I know it is not good to lose degrees of freedom, but what are they? Other students in the class waited for a clear-cut response. As we tried to give a textbook answer, we were not satisfied and we did not get the sense that our students understood. We looked through our statistics books to determine whether we could find a more clear way to explain this term to social work students. The wide variety of language used to define degrees of freedom is enough to confuse any social worker! Definitions range from the broad, Degrees of freedom are the number of values in a distribution that are free to vary for any particular (Healey, 1990, p. 214), to the technical: Statisticians start with the number of terms in the sum [of squares], then subtract the number of mean values that were calculated along the way. The result is called the degrees of freedom, for reasons that reside, believe it or not, in the theory of thermodynamics. (Norman & Streiner, 2003, p. 43) Authors who have tried to be more specific have defined degrees of freedom in relation to sample size (Trochim, 2005; Weinbach & Grinnell, 2004), cell size (Salkind, 2004), the number of relationships in the data (Walker, 1940), and the difference in dimensionalities of the parameter spaces (Good, 1973). The most common definition includes the number or pieces of information that are free to vary (Healey, 1990; Jaccard & Becker, 1990; Pagano, 2004; Warner, 2008; Wonnacott & Wonnacott, 1990). These specifications do not seem to augment students' understanding of this term. Hence, degrees of freedom are conceptually difficult but are important to report to understand statistical analysis. For example, without degrees of freedom, we are unable to calculate or to understand any underlying population variability. Also, in a bivariate and multivariate analysis, degrees of freedom are a function of sample size, number of variables, and number of parameters to be estimated; therefore, degrees of freedom are also associated with statistical power. This research note is intended to comprehensively define degrees of freedom, to explain how they are calculated, and to give examples of the different types of degrees of freedom in some commonly used analyses. DEGREES OF FREEDOM DEFINED In any statistical analysis the goal is to understand how the variables (or parameters to be estimated) and observations are linked. Hence, degrees of freedom are a function of both sample size (N) (Trochim, 2005) and the number of independent variables (k) in one's model (Toothaker & Miller, 1996; Walker, 1940; Yu, 1997). The degrees of freedom are equal to the number of independent observations (N), or the number of subjects in the data, minus the number of parameters (k) estimated (Toothaker & Miller, 1996; Walker, 1940). A parameter (for example, slope) to be estimated is related to the value of an independent variable and included in a statistical equation (an additional parameter is estimated for an intercept in a general linear model). A researcher may estimate parameters using different amounts or pieces of information, and the number of independent pieces of information he or she uses to estimate a statistic or a parameter are called the degrees of freedom (df) (HyperStat Online, n.d.). For example, a researcher records income of N number of individuals from a community. Here he or she has Nindependent pieces of information (that is, N points of incomes) and one variable called income (k);in subsequent analysis of this data set, degrees of freedom are associated with both N and k. For instance, if this researcher wants to calculate sample variance to understand the extent to which incomes vary in this community, the degrees of freedom equal N - k. …