To the Editor: Addressing Limitations of a Standardized Letter of Recommendation.

Abstract

We would like to thank the authors of the recent article “A Call to Action for Standardizing Letters of Recommendation”1 for their thoughtful summary of the current evidence for standardized letters for residency applications and their call for adoption across more specialties. While we agree that standardized letters overall demonstrate an improvement over the narrative letter of recommendation and that their adoption will likely lead to improvements in the residency selection process, the standardized letter should not be viewed as a panacea for discriminatory practices. Respectfully, their article does not address the negative aspects of standardized letters—particularly the structural bias in this evaluation tool. Further, without acknowledging and addressing the downside of tools currently in use, we lose an opportunity to improve these tools as we seek an equitable future in medical education.Specifically, the authors mention the need to address bias, “which may affect UIM (underrepresented in medicine) applicants.” This statement is unassailable; however, changing to a standardized letter alone will not mitigate racial/ethnic bias. Our recent article provided evidence that the emergency medicine standardized letter of evaluation (EM SLOE) demonstrates differences in rankings by race/ethnicity.2 Additionally, studies have highlighted gender bias in EM SLOE comparative rankings as well as in the narrative portion of the letter.3,4 Calls for all specialties to create a SLOE need to acknowledge these limitations. Conceding the limitations of the SLOE is vital so that those creating new standardized letters for their specialties can work to mitigate existing bias. Our article details an approach that may help to mitigate bias in new standardized assessment tools.2Further, the authors here do acknowledge that standardized letters do not stop authors from “inflating students' qualifications,” but stop short of acknowledging the limited validity evidence currently supporting the EM SLOE.5 Fully appreciating the limitations of this tool will help future users design a standardized letter that has better value in measuring what program directors value in a residency applicant.We agree with the authors that more specialties should look to implement a standardized letter to improve the residency application process. However, it is important to learn from the past so that we may create the best possible tools to promote equity within medical education.We express our gratitude to the authors of the article, “Racial Bias on the Emergency Medicine Standardized Letter of Evaluation,” published in this journal around the same time as our Perspectives article. The findings shared by Kukulski et al are critical to continue assessing the validity of the emergency medicine standardized letter of evaluation (EM SLOE) and highlight key areas in need of further improvement. We agree that the development, implementation, and continuous refinement of SLOEs for other specialties must be informed by the totality of the available evidence in the literature. While it is encouraging that prior studies have found evidence that SLOEs potentially help mitigate gender bias and improve discrimination between applicants, they must be designed to facilitate and promote a holistic review of each applicant.1-3 The threats to validity posed by systemic racism and individual implicit biases must be identified, assessed, and addressed throughout the process, from design to implementation, with an important focus on faculty development integrated within an anti-racist framework.4 We pose our call to action to graduate medical education program leaders and educational scholars to collaborate, not only in the process of developing and integrating further SLOEs, but also rigorously evaluating these tools to promote continuous refinement. We thank our colleagues who authored “Racial Bias on the Emergency Medicine Standardized Letter of Evaluation” for being on the leading edge of this important frontier.Melissa M. Tavarez, MD, MSAline Baghdassarian, MD, MPHJessica Bailey, MDDerya Caglar, MDMichelle Eckerle, MD, MPHAndrea Fang, MDKatherine McVety, MDJoshua Nagler, MD, MHPEdThuy L. Ngo, DO, MEdJerri A. Rose, MDCindy Ganis Roskind, MDFrances Turcotte Benedict, MD, MPHJo-Ann O. Nesiama, MD, MSAnita A. Thomas, MD MPHMelissa L. Langhan, MD, MHS