Developing effective, efficient and practical methods of assessing the professional skills of physicians in practice

Abstract

These are extraordinary times for medical education in the United States. We are witnessing dramatic changes in licensing and specialty certification and in maintenance of certification. In the licensing arena, the US Medical Licensure Examination (USMLE) is adding an objective structured clinical examination (OSCE)-type assessment, described as a clinical skills examination (CSE), to the second of the existing 3 steps to licensure. The graduating classes of 2005 will be the first to be required to pass the CSE. Students must successfully complete Step 1 (usually taken sometime during or after Year 2 of medical school) and Step 2 (usually taken during Year 4 of medical school) in order to be eligible to take the CSE. The logistics of how to administer the CSE to 20 000 + examinees per year in a manner that meets the standards established for such a high stakes examination are both expensive and daunting. The cost of administering the examination is estimated at close to $1000 per candidate, excluding their travel expenses. The development of testing sites for the CSE has been a challenge for the National Board of Medical Examiners (NBME), with dedicated sites being developed at up to 6 locations dispersed throughout the USA. The launch of the CSE is a clear indication that the medical profession is dedicated to ensuring that the standards of the profession are upheld. On another front, the Accreditation Council for Graduate Medical Education (ACGME) and the American Board of Medical Specialties (ABMS) have each adopted the same 6 competencies. These competencies comprise: patient care that is compassionate, appropriate and effective for the treatment of health problems and the promotion of health; medical knowledge about established and evolving biomedical, clinical and cognate (e.g. epidemiological and social-behavioural) sciences and the application of this knowledge to patient care; practice-based learning and improvement that involves investigation and evaluation of the subject's own patient care, appraisal and assimilation of scientific evidence, and improvements in patient care; interpersonal and communication skills that result in effective information exchange and teaming with patients, their families, and other health professionals; professionalism, as manifested through a commitment to carrying out professional responsibilities, adherence to ethical principles, and sensitivity to a diverse patient population, and systems-based practice, as manifested by actions that demonstrate an awareness of and responsiveness to the larger context and system of health care and the ability to effectively call on system resources to provide care that is of optimal value. These organisations are doing more than paying lip service to implementing the competencies. The ACGME, the organisation that accredits residency programmes in the USA, is to require that residents be assessed on the 6 competencies. While the precise mechanism by which programmes will be required to assess residents on the competencies is not being legislated for, meaningful assessment of the competencies is being mandated. Similarly, the ABMS is phasing in requirements that candidates be assessed on the 6 competencies for specialty certification and maintenance of certification. Over time, it is anticipated that the requirements will be made increasingly precise. As with the CSE, the logistics of assessing individuals on the 6 competencies represent a significant challenge. A toolbox of instruments that can be used to assess the competencies has been compiled and is available on-line from the ABMS/ACGME at the following web address: http://www.acgme.org/Outcome/assess/Toolbox.pdf. The ABMS member boards' maintenance of certification programmes further challenges the process because of the need to assess physician skills with as little disruption to their medical practice as possible. In brief, this challenge involves the development of reliable and valid measurements of the 6 competencies that can be administered with little disruption to the health care system and at low cost. In addition to the toolbox, the website above also provides examples of instruments that have been used to assess several of the 6 competencies. The NBME, ACGME and ABMS are to be commended for their commitment to maintaining the standards of the profession. In recognition of his efforts in bringing about the adoption of the 6 competencies by the ACGME and ABMS, Dr David Nahrwald was presented with the 2003 Hubbard Award for excellence in evaluation. While this is a remarkable beginning, there remains much work to be done. The efforts of the ABMS, ACGME and NBME to promote excellence in medicine through valid assessment may make it more feasible to develop new and innovative ways of assessing more than knowledge on a large scale. In the following articles, Dr David Leach describes the genesis of the 6 competencies and some of the lessons learned from the process. In a companion article, Dr Sheldon Horowitz et al. describe the efforts of the ABMS in promoting the 6 competencies and plans for developing effective, efficient and practical methods of assessing physicians in practice.