Critical Appraisal Process Research Articles

An Interdisciplinary Course for Trainee Psychiatrists: Feedback and Implications

To describe a course for trainee psychiatrists, designed to encourage critical thinking within an interdisciplinary framework. Trainees' responses to the course, and the implications of these for teaching, learning and clinical practice, are considered. Trainees were interviewed, and their responses subjected to content analysis. To some extent, the course appears to have met its objectives. However, some trainees found parts of the course content threatening, and reported negative perceptions of teaching staff and the process of critical appraisal. Challenging taken for granted beliefs in psychiatry is a fundamental skill, which should be fostered in trainee psychiatrists if the profession is to move forwards as an intellectual and clinical discipline. The lessons learned from the course we describe may be useful to others who wish to pursue integrated, interdisciplinary teaching methods.

Critically appraising qualitative research for systematic reviews: defusing the methodological cluster bombs

English This article examines the place of qualitative research within systematic reviews as evidence for policy, discusses the critical appraisal process as applied to qualitative research, and gives illustrative examples of sound qualitative studies from the health policy field. A checklist is a useful tool for the quality evaluation of qualitative research, facilitating comparison of the strengths and weaknesses of different studies, and should stimulate debate between reviewers as part of the broader critical appraisal process. Critical appraisal must be central to research syntheses, thus enabling reviewers to make only good-quality qualitative evidence accessible for policy makers and practitioners.

A critical appraisal of internet resources on colorectal cancer

The significance of the internet as a source of healthcare information for professionals and public is widely recognized. Access to good quality information is very important for informed decisions making on healthcare issues. This is an introduction to the critical appraisal of clinical information on the internet, discussion of the significance of quality checking of healthcare internet resources and the application of critical appraisal tools to selected internet resources on colorectal cancer care. Internet resources were identified on the basis of a general criterion that it provides information on colorectal cancer and selected on the basis of explicit inclusion and exclusion criteria. Each selected website is assessed for quality of information with validated criteria. The search strategy identified 10 well known internet resources on healthcare information (e.g. Cochrane, NICE) and 12 websites which are dedicated to cancer related information (e.g. CancerBacup, cancer.gov). The websites of NHS Direct, cancerBACUP, cancer.gov, NICE and Cancer Survivors Network were qualified as resources providing good quality information. The critical appraisal process assures quality of the available information and the tool is applicable to all healthcare related internet resources. The process is discussed with respect to colorectal cancer information resources and will be equally useful to healthcare professional and public. The healthcare professionals can apply the tool in practice and advise the public more confidently on internet information.

Prevalence of Diabetic Eye disease in Tayside, Scotland (P‐DETS) Study: methodology

To describe the use of a validated diabetes register for sampling frame generation and assessment of the representative nature of participants in a fieldwork study of diabetic eye disease. We performed an observational, cross-sectional fieldwork study of diabetic retinal disease using reference standard eye examination. We sampled the entire diabetic population using the Diabetes Audit and Research in Tayside Study (DARTS) diabetes register. The study population comprised 4825 diabetic patients aged over 16 years registered with one of 166 general practitioners (GPs) in 41 practices in Tayside in October 1999. This represented 61.1% of the Tayside diabetic population (7903). A total of 586 (66%; 95% confidence interval 63, 70) patients were examined from a sampling frame of 882 living patients registered with a Tayside GP. Demographic and disease parameters recorded on the DARTS patient register allowed comparison between participants and non-participants. This study shows the clear benefit of using a complete diabetic population as a sampling frame. This allows potential selection bias and external validity to be evaluated using routine data sources. Studies performed and reported in this way will aid the critical appraisal process.

Your opinions and letters

Your opinions and letters

Health impact assessment, integration and critical appraisal

There is little existing debate about what constitutes either a good quality health impact assessment (IA) or a good quality integration of health with environmental and social assessment (EIA, SIA). A critical appraisal process is required to assure the quality of each. The appraisal considers procedural and methodological components. It includes a comparison of the completed report with the terms of reference on which it is based. It examines the competence and experience of the assessment team, the role of the steering committee and local community, the determinants of health, and the way in which integration is achieved. There are both parallel and sequential components to integration because the outputs of EIA and SIA are often inputs to HIA. A case study from Sakhalin Energy Investment Company concludes that integration added value and that there were additional opportunities for community involvement and integrated mitigation.

Developing critical appraisal skills amongst staff in a hospital trust

This project was designed to encourage health professionals to use available evidence in their practice by facilitating the development of critical appraisal skills and by making them aware of the practical benefits of using evidence in practice. A partn ership was developed to roll out a workshop programme to enable all members of the health care team within a variety of clinical areas locate, appraise and utilise appropriate evidence in their speciality and to provide the Hospital's professional develo pment team with the necessary experience to conduct subsequent programmes. Six different modules were provided including an introductory session and five other modules each covering a portion of the critical appraisal process. 45 staff including 19 nurse s participated and all were surveyed in the post workshop evaluations. Comments were very positive and the sessions appeared to have enthused the participants. There were some limitations to the project, including fluctuating attendance, but the main out come of this project is that the team realised that in order to be more effective in their approach it would be most appropriate to single out clinical specialities and design workshops entirely applicable to their interests.

Resident Utilization of Information Technology: A Randomized Trial of Clinical Question Formation

In the current health care environment, it is expected that individual patient decisions will be made using clinical judgment, expertise, and information from well-designed clinical trials. There is evidence that caring for patients using information from valid clinical trials can improve patient outcomes.1–3 However, the process of moving from an individual patient encounter to practicing evidence-based medicine (EBM) requires explicit skills that are rarely taught. Specifically, clinicians must identify the need for information, formulate questions, search for information, appraise literature, and then apply scientific evidence to the clinical decision at hand. Most physicians, either while in training or in practice, have been exposed to the skills and knowledge needed to critically appraise a scientific report. For example, the User's Guide to the Medical Literature series has clearly outlined how to efficiently appraise most types of scientific reports and trials.4–7 These skills have been used to effectively teach evaluation of the medical literature.8 Established teaching traditions, such as medical journal clubs, have been used to motivate critical reading in house officers.9 However, the critical appraisal process is dependent on asking the right questions and finding the relevant evidence. These important first steps in the EBM process are rarely discussed or practiced. Physicians often form questions from patient encounters,10–12 but the generation of a question does not necessarily lead to a search of the literature.11 One impediment to searching is the lack of a focused and searchable question. The difficulties in formulating questions and moving to the search are troubling, because evidence does exist to treat patients12 and patient outcomes are enhanced when the best available evidence is applied to their care.1–3 Therefore, physicians in training need to be taught the skills of EBM, and studies are needed to show how these educational interventions affect behavior. We designed a randomized controlled trial to objectively measure the effect of an educational intervention on the first steps of the EBM process, moving from a clinical question to a search of the medical literature. We sought to teach residents how to build clinically relevant questions that would facilitate searching. Without an appropriate question, searching is not likely to be fruitful. Our primary hypothesis was that residents taught to frame effective clinical questions would increase their use of medline.

Recommendations for management of trigeminal nerve defects based on a critical appraisal of the literature

Purpose: Management of trigeminal nerve injuries continues to challenge oral and maxillofacial surgeons. The purpose of this review article is to apply the principles of evidence-based medicine (E-BM) to determine the optimal operative technique for managing defects involving the inferior alveolar (IAN) or lingual nerves when direct (ie, primary) repair is not feasible. Methods: To address the research purpose, the four steps of the E-BM critical appraisal process were applied: 1) identify the clinical problem, 2) efficiently search the literature, 3) select relevant articles and apply rules of evidence, and 4) apply the findings to patient care. Parameters for the literature search included using Medlineto identify English language articles, publication dates from 1986 through 1996, and studies involving human subjects. Results: The studies reviewed showed that the clinical literature on operative management of trigeminal nerve injuries is sparse, preoperative and postoperative neurosensory examinations are poorly documented, and the data are derived completely from reports using case series methods. Given these limitations, the available literature suggests that 1) tension-free, primary (direct) suture repair of an injured nerve, if possible, provides optimal results; 2) if direct repair is not possible, autogenous nerve grafts should be used for acute injuries, for example, immediate nerve repair after tumor resection or at the time of acute repair after traumatic injury; and 3) if direct repair is not possible, autogenous nerve grafts or hollow conduits (entubulization) to bridge the defect are equally successful for delayed reconstruction of gaps of 3 cm or smaller. Conclusions: Based on the weakness of the current literature, recommendations for future research include 1) better standardization and documentation of sensory deficits resulting from nerve injuries and their recovery, 2) using multicenter studies to accumulate large samples of patients rapidly, 3) using case series or prospective cohort study designs to assess the value of operative management of nerve injuries, and 4) progressing to randomized clinical trials to ascertain the optimal operative management of nerve injuries.

PULMONARY ARTERY CATHETERS IN THE CRITICALLY ILL: An Overview Using the Methodology of Evidence-Based Medicine

Evidence-based approaches to assessing the clinical literature are used increasingly in issues relating to critical care medicine. As we discussed previously, this approach attempts to provide a logical and convenient framework from which the quality and relevance of clinical studies may be assessed in an unbiased manner. An evidence-based approach also allows the reader to differentiate between solid evidence and evidence that is based on a presumed mechanism, standard practice, or conventional wisdom. Evidence-based medicine that deemphasizes intuition, unsystemic clinical experience, and pathophysiologic rationale is sufficient grounds for clinical decision making and stresses the examination of evidence from clinical research. Importantly, it is the objective nature by which the evidence-based medicine paradigm approaches the questions of "What are we doing" and "How can we do better," that causes health care providers and funding agencies to increasingly adopt this paradigm as a primary principle. The role of evidence-based medicine, therefore, is not to discount expert opinion but, wherever possible, to require that recommendations be based on the results of rigorous and controlled scientific study. We introduced this article by highlighting the growing imbalance between resources and patient needs in the critical care environment. At the level of diagnostic technology and therapeutic care plans, critical care professionals increasingly are asked to identify strategies to improve efficiencies-approaches with rigorous costs that at the same time promote better patient care. Formalized technology assessment is one of the mechanisms to accomplish this. Using critical appraisal within the context of evidence-based medicine is one of the mechanisms by which data can be synthesized to describe technology assessment. In this example, we used controversies surrounding use of pulmonary artery catheterization in critically ill patients to highlight some of the principles of evidence-based medicine at the bedside. We introduced how a particular case example can be used to ask the question, "What is the evidence that a particular diagnostic technology or therapeutic modality improves outcome?" In the case of PACs, no one would argue that a diagnostic technology a priori leads to improved outcome; rather, we have to presume that given good data, the care provider will apply the most appropriate management strategy. In this case example, however, we demonstrate how the critical appraisal process should identify a search strategy to find the most appropriate evidence to support the questioning process. From this, the evidence can be critically appraised and tabulated. In the case of the PAC, Table 13 demonstrates that there are few data to identify a grade A recommendation that the PAC should be used as part of the care process in critically ill patients. Finding that there is little evidence to support the use of pulmonary artery catheterization in the clinical literature does not mean that this diagnostic technology is neither efficacious nor effective. It might well be that information provided by PACs is important in the care process. However, what this exercise has taught us is that there is little objective evidence to support this conclusion. The challenge to critical care practitioners is not only to apply the evidence-based processes more frequently to our environment but also to use the information to separate out clearly what is fact versus opinion. Where there is little evidence to support a particular clinical practice, as we have demonstrated with the PAC review earlier, the challenge to the clinician should be the design and conduct of clinical trials clarifying debate between opinion and evidence.

In search of the evidence: informing effective practice

Despite the abundant journal literature the challenge for the health care professional is in retrieving clinically relevant information. Critical appraisal is a useful tool in meeting this challenge. This article examines the five stages of the critical appraisal process and the information implications associated with each stage. The role of health information workers, together with their training and development requirements, is also outlined. Clinicians and information workers need to work together to provide further evidence in support of the effectiveness of literature‐based approaches and their contribution to patient care.

An Initial Evaluation of Effective Health Care Bulletins as Instruments of Effective Dissemination

Reports a postal survey of 175 senior district health authority managers to determine the extent to which the Effective Health Care (EHC) bulletin No. 1 has effectively disseminated the results of a systematic review of the effectiveness of population‐based bone density screening for osteoporosis to health care commissioners in district health authorities; and a supplement outlining the process of critical appraisal. The information in the bulletin was perceived by a large majority as “correct”. A positive correlation was found between confidence to act and having read about the process of critical appraisal. Confidence to act was strongly correlated with a stated intention to use the bulletin in situations relating to the purchase of bone density screening for osteoporosis and also with the view that the bulletin would be influential in local discussions. Further research is needed to assess the impact on decision making.