Keeping Pace with Methodological and Analytical Advances in Social Research

Abstract

Most educators and scholars rely on the methodological and analytical content they received during their doctoral training to build and sustain their research careers. Thus, when designing new investigations, faculty members often rely on the design, measurement, and analytical techniques they learned during their formative training. In the past, this strategy worked well for most investigators in social work and other social sciences. In recent years, however, an increase in the sophistication of quantitative techniques and a proliferation of the software used to analyze complex data mean that not only content but also research techniques mastered during doctoral training are outdated more quickly than in the past. In fact, methodological and analytical approaches to social research, which evolve constantly, render old methods less useful or even inappropriate in some cases. Therefore, keeping pace with relevant advances in quantitative methodology and analysis is critical to competing successfully for external funding and advancing knowledge about individual and social problems. DESIGN AND ANALYTICAL ADVANCES Many new and sophisticated design, measurement, and analysis strategies have been introduced in social research in the past decade. For example, recent methodological and analytical developments have made possible overcoming challenges like attrition in efficacy and effectiveness trials. Once an unknown and uncertain process, multiple imputation strategies for handling missing data have become common practice in longitudinal studies (Schafer & Graham, 2002). Such strategies are particularly common today in randomized trials assessing the efficacy or effectiveness of prevention programs or social interventions (Graham, Olchowski, & Gilreath, 2007). Similarly, innovative methods to deal with selection effects that use techniques such as propensity analyses have been introduced in intervention research (Fraser, 2004). Yet another example is evident in advances in estimating power, a thorny problem that has been made less complicated by advances in specialized software programs (Raudenbush & Bryk, 2002). Achieving mastery of new analytical techniques may be challenging to experienced investigators who did not previously receive adequate training in similar approaches. For instance, once the exception, statistical methods known variously as multilevel, hierarchical, or mixed models are now used frequently to answer increasingly complicated research questions related to the onset and persistence of selected behaviors and outcomes. Similarly, an increase in group-randomized trials (GRTs) has led to numerous analytical advances necessary to analyze data collected in schools and community prevention experiments (for example, Jenson & Dieterich, 2007; Jenson, Dieterich, Rinner, Washington, & Burgoyne, 2006). That is, when randomization is done at the group level, treatment effects need to be analyzed at the group level. For example, in a study where schools are randomly assigned to treatment and control conditions, the denominator degrees of freedom for a test of the difference between the means of the control and experimental schools is not based on the number of students but rather on the number of schools in the study. Thus, the appropriate statistical models in a GILT partition both random and fixed effects at the student-, classroom-, and school-levels. It is important to note that the use of such models has a significant effect on the sample size and power requirements in a study. This brief discussion barely scratches the surface of recent developments in quantitative methodology. Additional issues and recommendations pertaining to design sampling, power, and analysis are found in the recent social work literature (for example, Fraser, 2004; Jenson et al., 2006; LeCroy & Krysik, 2007; Nash, Kupper, & Fraser, 2004). Readers may also wish to examine important papers from other disciplines that summarize new approaches to preventing or handling complex methodological and analytical issues in social research (for example, Muthen, 2002; Murray, Varnell, & Blitstein, 2004; Raudenbush & Bryk, 2002; Singer & Willett, 2003). …