Your opinions and letters

Abstract

We welcome Alan Pearson's response to our article. He raises three extremely important issues: defining a question for a systematic review; the difficulties in conducting a critical appraisal of systematic reviews; the limited nature of evidence that is normally included in systematic reviews. The type of question asked will undoubtedly determine the scope of a literature review. We agree that narrowly defined questions make for easier and probably more rigorous reviews. The problem is that narrowly defined questions often lead to narrowly focused answers. These may not be the types of answers required by policy makers who are trying to respond to the complexities of providing public health interventions in the real world. They require evidence across a wide set of topics relevant to policy. Thus the questions asked by policy makers tend to be broad and ultimately more difficult to answer. This raises an important question of the relationship between policy makers involved in commissioning research and the research community. The Scottish Executive issued the original tender document for our review. It set out to commission a systematic review of the evidence for public health nursing. Public health nursing was described as any nursing that contributes to public health. We tried to limit the scope of the review by refining the question and setting inclusion and exclusion criteria. This was achieved partly through discussion with the literature research committee, which was specially convened to guide the review. The committee consisted of the commissioners, academics and public health specialists. Nevertheless, the scope of the review was extremely wide and given the time that was allocated to conducting the review, it remained extremely ambitious. There is a strong argument for better communication between those who commission reviews and the research community. We believe this should happen at the planning stage when the commissioners are developing a research tender. Commissioners could also be provided with further training in the methods that are utilized in conducting a systematic review. Such measures may lead to more realistic research studies that satisfy the demands of policy makers and scientific research (Innvaer et al. 2002). Judging the quality of reviews is extremely important. The findings of any review rest on the quality of the research that underpins it. Reaching a decision about the quality of a review is extremely difficult. As Alan Pearson indicates there are many checklists available that assist this task. To our knowledge very few, if any, have been properly assessed as valid or reliable. Thus, assessment often involves a great deal of academic judgement. We agree this process should become more transparent and that it would enhance rigour if reviewers provided more detailed reports on how their decisions were reached. We also think more effort should be devoted to designing objective and standard checklists. This requires further research and might include techniques developed by Jadad et al. (1996) in developing an instrument for judging the quality of randomized trials and the work of Joanna Briggs Institute for qualitative research. Our main interest was to determine the effectiveness of public health nursing which meant focusing on controlled studies. Qualitative research was not included in our review because it does not fulfil the accepted criteria for causality. We fully recognize the limitations of this decision. Controlled studies, while answering the question does it work, often lack the power to determine why interventions succeed or fail. Qualitative research, however, is extremely powerful in providing contextual clues that identify why a particular intervention worked at a given time and why some people benefited whilst others did not. Qualitative research can also be used to identify outcomes that are not readily amenable to measurement such as, quality of life, social well-being, and social integration. Further, there are areas of public health where controlled studies are difficult to conduct, such as the impact of housing on health (Thomson et al. 2003). More importantly there are other types of questions asked by health professionals that do not focus on the issue of cause and effect and are better addressed by qualitative research. Thus, as Alan Pearson argues, nursing research will undoubtedly take a pluralist approach in accepting different types of evidence as legitimate (Pearson 2003). We welcome Alan Pearson's comments. Systematic reviews are important in shaping public health nursing policy. Better communication between policy makers and researchers will improve the usefulness of reviews whilst satisfying academic rigour. Greater effort is also required to develop objective assessment criteria by which reviews are assessed. Finally, the strengths of qualitative research deserve greater recognition and this type of evidence should occupy a more prominent position in systematic reviews.