Considering Systematic Direct Observation after a Century of Research—Commentary on the Special Issue

Abstract

Systematic Direct Observation (SDO) has played a pivotal role in the field of Emotional and/or Behavioral Disorders (EBD) since its inception as a key part of more about the behaviors, contexts that impact them, and the effective supports necessary for this population. Thinking more critically about how we have, and should continue to use, this methodology is an ongoing charge for all of us. As we prepared ourselves to do this for the purposes of this article, and our own curiosity, we took a bit of time to explore the not-sorecent writings on the science of direct observation. Early on, we came across an outstanding article by Jersild and Meigs (1939). We strongly recommend that any scholar interested in the topic of SDO assessment research read that article. Essentially, it serves as a well-articulated reminder that the state of SDO research has changed little since 1939. While there are certainly many studies utilizing SDO, and with an ever-increasing population, Jersild and Meigs' hope for a more systematic investigation of SDO remains elusive. The extensive list of questions is as relevant today as it was then. Further, their recommendations are conceptually consistent with issues put forth in this special issue and ones we will champion in this commentary.As we fast-forward 75 years, one of the consistent hallmarks of EBD research and practice is the general consensus that SDO is the method (Baer, Wolf, & Risley, 1987). What does that exactly mean and how did this understanding come to exist? Is this belief fully defensible based on a comprehensive literature base that has investigated SDO through a clearly understood process for evaluating effective assessment procedures? Rather, is much of our literature base and corresponding assumptions based on cumulative data? In other words, have we yet to realize under what conditions and when SDO is valid and reliable? Moreover, which specific SDO coding schemes are valid and reliable in what conditions? And in some cases, has SDO become the default gold standard assessment method in the absence of other valid and reliable options? In this commentary we consider these century-old issues, the contribution of the other articles in the special issue to this question, and suggest an increased emphasis toward a systematic line of research to fully consider the psychometric properties of specific SDO for targeted purposes.Despite almost a century of documented research on SDO in child behavior (see Jersild & Meigs, 1939, Lewis et al., this issue), the field continues to seek more specific coding schemes that are reliable and valid. This may be in part because the tendency has been to generalize SDO research across populations and settings, as opposed to emphasizing more targeted research specifying unique coding schemes. Lewis and colleagues provide a descriptive account of this phenomenon within this issue as they highlight how SDOs are traditionally chosen almost exclusively based on the current dimension of the target behavior. Many SDO coding manuals come with procedures for data collection and operational definitions; but few come with documentation indicating the unique purpose for which the coding scheme-has been validated. Messick (1980) cautioned this and provided some insights. He operationally defined frameworks to systematically address various forms of validity, and perhaps even more relevant for the current discussion, identify the purpose of the assessment.Not all SDO is really developed or intended for use in applied settings; rather, such specific SDO codes serve a specific research paradigm. Others may be uniquely suited to capture multiple interaction codes for very specific contexts (e.g., teacher/student interaction effects), but are not valid for capturing the same student behavior within a discrete trial setting. Similarly, some SDO coding systems that are valid for students with extreme overt behaviors may not maintain validity or reliability when used with students with low incidence behavior, depression or even different age groups. …