Some Methodological Problems, Solutions and Findings from Evaluating Risk Reduction Projects

Abstract

Three methodological problems and solutions used in comparing the effects of five risk reduction projects in Florida are reported. Some findings are also included to show how solutions effect outcome and because they were considered useful to the field of drug education. The question whether the five projects should have different stated objectives because they expected baseline data on similar target groups to be different was resolved by permitting these differences. Baseline data, however, showed no significant difference from project to project for similar target groups. A better design for future projects would be two sets of objectives, one unique to each project and one facilitating inter-project comparisons. A related problem was whether the same instrument could measure changes which were expected to be different because of different baselines, different target groups and different treatment methods from project to project. One instrument with two sections was used: one section to measure changes unique to each project and one for changes expected to be common to all projects. This was a satisfactory solution, but a longer instrument with more common items was suggested for future projects. A third problem was that each project differed in both length of program and type of program. Analysis of findings used content analysis to hold one of these constant while noting changes in effects from variations in the other. Because of the small number of sites, this was only partially successful. Effects of projects in the future would be more comparable if at least two projects had either identical program types while varying program length or vice versa. Changes in knowledge of effects of smoking and drinking appear most likely related to programs of “mixed” type containing educational elements aimed at producing positive self-awareness, and including exposure to a range of enjoyable and rewarding non-drug activities.