The role of simulator-based assessments in physician competency evaluations

Abstract

Members of a self-regulating profession such as medicine have an obligation to their professional body to ensure that their performance meets the expectations of a competent professional. The professions and the regulatory bodies that govern them have an obligation to the public to ensure that the members possess the necessary competencies for safe practice and to make certain that they are engaged in developing and updating these competencies. In the past, it was assumed that physicians in practice participated in professional development through a process of self-assessment and self-directed learning. Unfortunately, the evidence suggests that many physicians made little effort to do so. Perhaps many of them perceived no deficiencies in their knowledge base or practice patterns. However, people tend to hold overly favourable views of their own knowledge and abilities, and the least competent tend to have the most inflated self-assessments. As well, there is a poor correlation between physicians’ selfassessments and those provided by external reviewers, indicating a limited ability on the part of physicians to selfdetermine their true personal need for education. Physicians with more experience are generally believed to have accumulated knowledge and skills during their years in practice, and such experience is thought to be equated with enhanced care. In fact, there may be an inverse relationship between the number of years that a physician has been in practice and the quality of care that is provided. Choudry et al. performed a systematic review of studies that compared the medical knowledge of physicians and the health care they provided with their age and years in practice, and they concluded that the quality of care delivered may deteriorate rather than improve with increasing physician age and duration of practice. Obviously, a mechanism is needed both to ensure continuing competency among practicing physicians and to meet the public’s insistence that this process be realized. The question is not whether periodic practice assessments and re-validation will become a fact of life for medical practitioners; they are already. However, the metric will no longer be the number of hours of continuing medical education that a practitioner can lay claim to, as this approach was in the early iterations of maintaining competency programs. As well, the simple and subjective selfassessments that have been the staple of many competency assurance programs are likely to give way in the near future to programs that include norm-referenced self-audits and assessments by peer reviewers. To achieve the goals of public accountability, practice assessments must be carefully structured and externally audited, with considerable rigor given to the establishment of appropriate norms for benchmarking. To be fair to physicians, efforts must be made to ensure that expectations for practice are measured against accurate norms. Arbitrary or unvalidated benchmarks for care, even if agreed to by selfor externally appointed reviewing bodies, serve no one well in this process. In addition to being fundamentally fair and affording the participants the expectation of an appropriate process for assessment, a system of review will be required that is valid, reliable, and reasonable. In this issue of the Journal, Borges et al. reported that experienced anesthesiologists demonstrated major E. Crosby, MD (&) Department of Anesthesiology, University of Ottawa, The Ottawa Hospital – General Campus, Suite 1401, 501 Smyth Road, Ottawa, ON K1H 8L6, Canada e-mail: ecrosby@sympatico.ca