Examining the Influence of Interval Length on the Dependability of Observational Estimates

Abstract

Systematic direct observation is a tool commonly employed by school psychologists to investigate student behavior. As these data are used for educational decision-making, ensuring the psychometric adequacy of the obtained data is an important consideration. Given that procedural aspects of systematic direct observation have been shown to influence the psychometric properties of obtained data, this study was designed to explore how interval length influences the dependability of academic engagement data when using a momentary time sampling procedure. Twenty seventh-grade students were each observed for two 15-min sessions during math instruction. A series of generalizability studies were conducted to examine how manipulations to interval length influenced reliability-like coefficients. In general, shorter interval lengths (i.e., 10 s, 15 s) were shown to produce higher levels of dependability. For example, an acceptable level of dependability (i.e., ϕ = .70) required twice as many 30-min observations when utilizing 20- or 30-s sampling as were required when utilizing 10- or 15-s sampling. Furthermore, whereas an acceptable level of dependability (i.e., ϕ = .70) could not be obtained using any interval length when conducting a single observation, this criterion was met using either 10- or 15-s sampling across two 30-min observations.