Letter to the editor re Pedersen et al.

Abstract

Dear Editor,With interest we read the article by Pedersen et al. on realisticevaluation as a new way to design and evaluate occupationalsafety interventions in Safety Science (Pedersen et al., 2012). SinceCochrane Reviews are staged as non-realistic evaluations in thisarticle, it provides interesting food for thought and discussion.We believe that the distinction between realistic and non-realisticevaluation is not very helpful in guiding our understanding of whatworks in injury prevention. Depending on the type of problem andintervention at hand, different research designs should be usedthat are most suited to solve the specific problems.The authors equate safety interventions with complex organisa-tional interventions. However, according to our experience, safetyinterventions can range from simple technical measures tocomplex organisational measures. Simple technical interventionscan easily be evaluated with a randomised controlled design. Sincethere is general consensus in science that this gives the most validresults, we think that, where appropriate, this design should beused without question. The use of blunt needles by surgeons toprevent needlestick injuries is a good example of such an interven-tion carried out in the real world (Nordkam et al., 2005). For morecomplex interventions, especially those that are introduced at thegroup level, randomisation is more difficult but not impossiblesuch as in cluster-randomisation. For example, many complexorganisational interventions for stress management have beenevaluatedin this way (Marineet al., 2006). However,we fully agreethat the content, the context and the implementation of the inter-vention becomes a more prominent factor when the intervention ismore complex. These items get full attention in Cochrane System-atic Reviews through the detailed description of the variousfeatures of the included studies and via subgroup analyses. A goodexample is the Cochrane Review about prevention of noise-induced hearing loss, where lack of implementation of technicalmeasures explains the lack of an intervention effect (Verbeeket al., 2009). This has also been used as an argument by the Occu-pational Safety and Health administration in the US to proposestricter enforcement of legislation to better prevent noise-inducedhearing loss.We find problematic that throughout the article the authorsmake no distinction between primary evaluation studies andsystematic reviews. However, the problems involved in evaluatinga single intervention are quite different from those met whensynthesizing the results of multiple intervention studies. The infor-mation provided on contextual factors in a review depends on theamount of information provided by the authors of primary articles.It is up to the reader of a Cochrane review to bring in his or herexpertise about the local context to enable the application to prac-tice of evidence that has been synthesized in a review. One shouldnot equate Cochrane Systematic Reviews with randomised con-trolled trials, as the authors implicitly do. This is especiallypoignant as the cited Cochrane review does not contain any RCTs.There are more than 80 Cochrane Systematic Reviews currentlyavailable on occupational health and safety topics (see: osh.cochra-ne.org). Many of them include non-randomised studies such ascontrolled-before-after studies and interrupted time-series.The authors argue that it is seldom possible and not ethical torandomise enterprises. However, there is no difference in theethics of randomizing patients or enterprises. There are manyexamples available where this has been successfully achieved.Instead of contrasting evaluation methods, it would be morehelpful to show what kinds of primary studies could improve ourknowledge of what works in, say, injury prevention in the con-struction industry. Even though we included very low qualitystudy designs in our review, we found that very few evaluationstudieshad beenconducted (van der Molen et al., 2007). Given thatsensible controlled intervention studies are carried out, theCochrane Systematic Review still remains the most powerful, validand candid tool for pulling their results together.References