US Radiology Research Articles

An open-source fine-tuned large language model for radiological impression generation: a multi-reader performance study

BackgroundThe impression section integrates key findings of a radiology report but can be subjective and variable. We sought to fine-tune and evaluate an open-source Large Language Model (LLM) in automatically generating impressions from the remainder of a radiology report across different imaging modalities and hospitals.MethodsIn this institutional review board-approved retrospective study, we collated a dataset of CT, US, and MRI radiology reports from the University of California San Francisco Medical Center (UCSFMC) (n = 372,716) and the Zuckerberg San Francisco General (ZSFG) Hospital and Trauma Center (n = 60,049), both under a single institution. The Recall-Oriented Understudy for Gisting Evaluation (ROUGE) score, an automatic natural language evaluation metric that measures word overlap, was used for automatic natural language evaluation. A reader study with five cardiothoracic radiologists was performed to more strictly evaluate the model’s performance on a specific modality (CT chest exams) with a radiologist subspecialist baseline. We stratified the results of the reader performance study based on the diagnosis category and the original impression length to gauge case complexity.ResultsThe LLM achieved ROUGE-L scores of 46.51, 44.2, and 50.96 on UCSFMC and upon external validation, ROUGE-L scores of 40.74, 37.89, and 24.61 on ZSFG across the CT, US, and MRI modalities respectively, implying a substantial degree of overlap between the model-generated impressions and impressions written by the subspecialist attending radiologists, but with a degree of degradation upon external validation. In our reader study, the model-generated impressions achieved overall mean scores of 3.56/4, 3.92/4, 3.37/4, 18.29 s,12.32 words, and 84 while the original impression written by a subspecialist radiologist achieved overall mean scores of 3.75/4, 3.87/4, 3.54/4, 12.2 s, 5.74 words, and 89 for clinical accuracy, grammatical accuracy, stylistic quality, edit time, edit distance, and ROUGE-L score respectively. The LLM achieved the highest clinical accuracy ratings for acute/emergent findings and on shorter impressions.ConclusionsAn open-source fine-tuned LLM can generate impressions to a satisfactory level of clinical accuracy, grammatical accuracy, and stylistic quality. Our reader performance study demonstrates the potential of large language models in drafting radiology report impressions that can aid in streamlining radiologists’ workflows.

Status of LGBTQ+ Inclusion: Multi-Institution Assessment of US Radiology Residencies

IntroductionAn inclusive residency program is crucial to the recruitment and retention of competitive and diverse applicants. The radiology lesbian, gay, bisexual, transgender, queer or questioning, or another diverse gender identity (LGBTQ+) inclusion audit was published in 2022, which provided a road map for assessing the inclusivity of a program’s policies, facilities, culture, and engagement. In this multi-institutional trial, we detail the results of the LGBTQ+ inclusion audit for nine US radiology residency programs. MethodsA volunteer cohort of academic radiology programs was recruited through the Radiology Residency Education Research Alliance. The LGBTQ+ inclusion audit was modified to apply to a multi-institutional study. Participating programs performed the audit from December 2023 to February 2024. Pre- and postaudit surveys were distributed to capture participants subjective assessment of inclusivity at their programs. ResultsNine US radiology residency programs completed the audit. Audit scores ranged from 6 out of 10 to 9 out of 10; no program received a perfect 10 out of 10 score. Inclusive facilities and institutional culture scored highly, with eight of nine programs reporting all milestones met in these areas. The lowest-performing areas were department culture and community engagement with only three of nine and four of nine programs reporting “all milestones met.” After the audit, programs overall reported improved self-perceived inclusivity, with postaudit survey scores increasing in areas related to department policy, inclusive facilities, and LGBTQ+ community outreach. DiscussionParticipating radiology residency programs demonstrated an overall high level of LGBTQ+ inclusivity; however, all programs identified inclusion shortcomings in department policy and practice. Intentional review of a department can be a helpful tool to promote a welcoming and healthy environment for a diverse radiology practice.

Diversity, Equity, Inclusion in US Radiology: Current Status and Legislative Trends

Diversity, Equity, Inclusion in US Radiology: Current Status and Legislative Trends

Discrimination faced by radiology residents: an analysis of experiences and mitigation strategies

Rationale and objectivesLiterature shows that discrimination has been pervasive in the field of medicine. The aim of this study was to collect experiences related to discrimination among US radiology residents, including type and source, as well as the residents’ perception on lectures about discrimination and harassment. We also explored the barriers to reporting, and suggested strategies to overcome them. Material and methodsFollowing Institutional Review Board (IRB) approval, an online survey was sent to program directors and coordinators across the US, who were asked to forward the link to their radiology residents. A reminder email was sent over a period of 4 months. The participants were reassured the survey was confidential and anonymous. ResultsAmong the respondents, the most reported types of discrimination were based in gender, race and nationality, the majority of which not being reported. The most common perpetrators were attending radiologists, co-residents, technologists, and patients. The main barriers for reporting were fear of retaliation, confidentiality concerns, and skepticism about a positive outcome. ConclusionOur study examines some experiences of discrimination shared by residents during their training, with gender and race being the most common causes. This sheds light into a hidden and unspoken issue and highlights the need for more active discussions in radiology on microaggressions and implicit bias. Our data can guide future studies as well as residency programs to build effective strategies to address discrimination, aiming for sustainable changes.

US Radiology Resident Perceptions of Current Well-Being Programming: A Case Study

US Radiology Resident Perceptions of Current Well-Being Programming: A Case Study

The Influence of Extracurricular Activities on Radiology Resident Selection Decisions

PurposeExtracurricular activities (EAs) listed on radiology residency applications can signal traits and characteristics desired in holistic reviews. The authors conducted an objective analysis to determine the influence of EAs on resident selection decisions. MethodsA discrete-choice experiment was designed to model radiology resident selection and determine the relative weights of EAs among academic and demographic application factors. Faculty members involved in resident selection at 30 US radiology programs chose between hypothetical pairs of applicant profiles between October 2021 and February 2022. Each applicant profile included one of 22 EAs chosen for study. A conditional logistic regression model assessed the relative weights of the attributes and odds ratios (ORs) were calculated. ResultsTwo hundred forty-four participants completed the exercise. Community-service EAs were ranked most highly by participants. LGBTQ Pride Alliance (OR, 1.56; 95% confidence interval [CI], 1.14-2.15; P = .006) and Young Republicans (OR, 0.60; 95% CI, 0.43-0.82; P = .001) significantly influenced decisions. The highest ranked EAs were significantly preferred over the lowest ranked EAs (OR, 1.916; 95% CI, 1.671-2.197; P < .001). Participants preferred EAs that reflected active over passive engagement (OR, 1.154; 95% CI, 1.022-1.304; P = .021) and progressive over conservative ideology (OR, 1.280; 95% CI, 1.133-1.447; P < .001). Participants who ranked progressive EAs more highly preferred applicants with progressive EAs (P < .05 for all). ConclusionsThe influence of EAs on resident selection decisions is significant and likely to gain importance in resident selection as medical student performance metrics are further eliminated. Applicants and selection committees should consider this influence and the bias that EAs can bring to resident selection decisions.

Venture Capital in US Medicine: A Briefing for Radiologists

In recent years, there has a been a rapid and significant drop in the number of private practices in US radiology. Many factors have driven this change. One of the most important has been the corporatization of practices. In many cases, this involves third-party financing, one type of which is venture capital. This article will provide an introduction to the venture capital investment model for medical professionals.

The Effect of the COVID-19 Pandemic on Academic Research Gender Disparities in Radiology

Gender disparities have long existed in radiology. The COVID-19 pandemic disrupted research activities worldwide and have impacted gender disparities across medical specialties. This study investigates the effect of the COVID-19 pandemic on gender disparities in radiology academic authorship. A retrospective observational study was conducted using data from 110 843 global and 23 977 US radiology articles. The gender of authors was determined using an automated gender inference tool. Descriptive statistics were applied to explore authorship changes overall globally, in the US as well as across countries and states. Female first-authorship increased globally from 16.9% to 17.6% (p<0.001), and in the US, from 19.0% to 19.6% (p=0.19) in the peri-COVID period. The combined female percentage increased from 19.7% to 20.0% globally (p=0.021), and from 20.2% to 21.1% in the US (p=0.006). Country-level analysis revealed significant increases in female authorship in Colombia, Denmark, Egypt, France, India, and Japan, while New Zealand demonstrated a decrease in female authorship. In the US, Florida, Indiana, Louisiana, Massachusetts, and Ohio experienced increases in female authorship, whereas South Carolina demonstrated a decrease in female authorship. In contrast with other medical fields, the study demonstrates that radiology experienced a slight increase in female authorship in radiology research globally and in the US during the COVID period. While the pandemic may have influenced these findings, further research is needed to establish regional causal relationships and identify best practices for promoting gender equity in radiology research.

Family and Medical Leave Utilization in US Radiology Practices

Family and Medical Leave Utilization in US Radiology Practices

Sociodemographic Variables Reporting in Human Radiology Artificial Intelligence Research

PurposeArtificial intelligence (AI) is rapidly reshaping how radiology is practiced. Its susceptibility to biases, however, is a primary concern as more AI algorithms become available for widespread use. So far, there has been limited evaluation of how sociodemographic variables are reported in radiology AI research. This study aims to evaluate the presence and extent of sociodemographic reporting in human subjects radiology AI original research. MethodsAll human subjects original radiology AI articles published from January to December 2020 in the top six US radiology journals, as determined by impact factor, were reviewed. Reporting of any sociodemographic variables (age, gender, and race or ethnicity) as well as any sociodemographic-based results were extracted. ResultsOf the 160 included articles, 54% reported at least one sociodemographic variable, 53% reported age, 47% gender, and 4% race or ethnicity. Six percent reported any sociodemographic-based results. There was significant variation in reporting of at least one sociodemographic variable by journal, ranging from 33% to 100%. ConclusionsReporting of sociodemographic variables in human subjects original radiology AI research remains poor, putting the results and subsequent algorithms at increased risk of biases.

Leadership: Causing and Curing Burnout in Radiology

Burnout in US radiology has reached crisis proportions. Leaders play critical roles in both causing and preventing burnout. This article will review the current state of the crisis and how leaders can work to stop causing burnout as well as developing proactive strategies for preventing and mitigating burnout.

Artificial Intelligence/Machine Learning Education in Radiology: Multi-institutional Survey of Radiology Residents in the United States

To evaluate radiology residents' perspectives regarding inclusion of artificial intelligence/ machine learning (AI/ML) education in the residency curriculum. An online anonymous survey was sent to 759 residents at 21 US radiology residency programs. Resident demographics, sub-specialty interests, educational background and research experiences, as well as the awareness, availability, and usefulness of various resources for AI/ML education were collected. The survey response rate was 27% (209/759). A total of 74% of respondents were male, 80% were training at large university programs, and only a minority (<20) had formal education or research experience in AI/ML. All four years of training were represented (range: 20%-38%). The majority of the residents agreed or strongly agreed (83%) that AI/ML education should be a part of the radiology residency curriculum and that such education should equip them with the knowledge to troubleshoot an AI tool in practice / determine whether a tool is working as intended (82%). Among the residency programs that offer AI/ML education, the most common resources were lecture series (43%), national informatics courses (28%), and in-house/institutional courses (26%). About 24% of the residents reported no AI/ML educational offerings in their residency curriculum. Hands on AI/ML laboratory (67%) and lecture series (61%) were reported as the most beneficial or effective. The majority of the residents preferred AI/ML education offered as a continuous course spanning the radiology residency (R1 to R4) (76%), followed by mini fellowship during R4 (32%) and as a course during PGY1 (21%). Residents largely favor the inclusion of formal AI/ML education in the radiology residency curriculum, prefer hands-on learning and lectures as learning tools, and prefer a continuous AI/ML course spanning R1-R4.

Assessment of US Radiology Residency Program Websites in the COVID-19 Era.

ObjectiveTo provide an updated evaluation of radiology residency program websites in light of virtual interviewing during the COVID-19 pandemic and encourage programs to improve the quality of their online website presence.MethodsWe evaluated the websites of 197 US radiology residency programs between November and December 2021 for the presence or absence of 30 metrics. The metrics chosen are those considered important by applicants when choosing a program and have been used in other similar papers.ResultsOf the 197 programs, 192 (97.5%) had working websites. The average radiology residency website had 16 of 30 (54%) metrics listed on their websites. Five programs did not have accessible websites and were not included in the analysis. The most comprehensive website had 29 of 30 (97%) of metrics listed and the least comprehensive website had 2 of 30 (7%). There is a statistically significant difference in website comprehensiveness between top 20 and non–top 20 radiology program websites.ConclusionAlthough radiology residency program websites have generally become more comprehensive over time, there is still room for improvement, especially in times of virtual interviews when residency applicants are becoming more and more reliant on program websites to gain essential information about a program. Some key areas to include are diversity and inclusion initiatives, resident wellness, applicant information, program benefits, and showcase of people in the program.

Radiology Practices Employing Nurse Practitioners and Physician Assistants: Characteristics and Trends From 2017 Through 2019.

The number and roles of US nonphysician practitioners (NPPs) have expanded considerably, but little is known about their use by radiology practices. The authors assessed characteristics and trends of radiology practices employing Medicare-recognized NPPs. Using Medicare databases from 2017 through 2019, the authors mapped all nurse practitioners and physician assistants (together "NPPs") to employer groups for which all physicians were radiologists ("radiology practices"). Practices were characterized by size, geography, and radiologist characteristics. Temporal changes were assessed, and NPP employment likelihood was estimated using multivariate logistic regression modeling. As the number of US radiology practices declined by 36.5% (from 2,643 to 1,679) between 2017 and 2019, the number employing NPPs increased by 10.5% (from 228 [8.6%] to 252 [15.0%]). The number of radiologists in NPP-employing practices increased by 10.4% (from 6,596 [35.1%] to 7,282 [40.0%]) as the number of radiology-employed NPPs increased by 17.5% (from 588 to 691). Practices were more likely to employ NPPs when medium (odds ratio [OR], 1.31) or large (OR, 1.25) in size, when urban located (OR, 1.35), and as their percentages of interventional radiologists increased (OR, 5.53 per percentage point) (P < .01 for all). Practices were less likely to employ NPPs as mean radiologist years since completing training increased (OR, 0.99 per year; P < .01). Employment of NPPs by radiology practices has grown considerably in recent years, particularly in larger and urban practices and in those that employ more interventional and early-career radiologists. More work is necessary to better understand how this expanding use of NPPs affects the specialty.

The Radiology Resident Education Research Alliance: The Evolution of a Multi-Institutional Research Cooperative.

The recruitment and training of radiology residents is still driven largely by history and tradition. Radiology educators have shown great interest in innovative practices, but there are barriers to outcomes-based research to drive advancement in the areas of curricular development, innovative teaching methods, constructive resident evaluation, and precise resident selection [1]. Most radiology education research originates from single institutions with small sample sizes that limit both the power to detect small effects and generalizability across the spectrum of US radiology residency programs.

Geographic Trends in Publications and Submissions in Radiology Journals: Decade Report (2010 – 2020)

The Radiology Research Alliance (RRA) of the Association of University Radiologists (AUR) organized a task force to quantify geographic changes in submissions to and publications within radiology journals over time. Twenty journals were selected: 7 US-based general, 4 European-based general, and 9 subspecialty radiology journals. Journals were solicited for submissions and publications based on country of origin from 2010 -2020. Regression models assessed changes over time across countries, and by continent, with an emphasis on the US and China, for each journal category. There were 104,679 publications and 92,446 submissions from 149 countries. Overall, there were significant increases in numbers of publications from Asia (R2=0.66, p <0.01), and specifically, China (R2=0.87, p <0.01). For US journals, there were increased numbers of publications from Asia (R2=0.72, p <0.01) and China (R2=0.98, p <0.01), but decreased numbers from North America (R2=0.41, p=0.03). For European journals, there were increased numbers of publications from Asia (R2=0.79, p <0.01), North America (R2=0.75, p <0.01), and China (R2=0.82, p <0.01). For subspecialty journals, there were increased numbers of publications from North America (R2=0.38, p=0.04) and China (R2=0.73, p <0.01). There was an acute COVID spike in submissions during 2020, with a continuous increase most notable in China (R2=0.96, p <0.01). In the last decade, the numbers of submissions to and publications within general and subspecialty US and European radiology journals have generally increased around the world, with the largest growth in Asia and in particular China.

Chest-Related Imaging Investigations During Multiple Waves of COVID-19 Infection in Hong Kong.

Background: The COVID-19 pandemic has caused significant disruption to healthcare worldwide. In this study, we aim to quantify its impact of chest related radiological procedures over the different waves of local infection in Hong Kong across the territory's public hospitals.Methods: This was an observational study enrolling patients between January 2017 and December 2020. Consecutive population-based chest radiographs, CT, US, and interventional radiology (IR) procedures were obtained public hospitals across Hong Kong.Results: A significant reduction of 10.0% (p < 0.001) in the total number of chest radiographs was observed. Non-significant reduction of 2.5% (p = 0.0989), 39.1% (p = 0.2135), and 1.9% (p = 0.8446) was observed for Chest CT, Chest US, and Chest IR procedures, respectively, in 2020 compared to the projected values.Conclusion: Although, it was anticipated that there would be a significant impact to health services caused by the pandemic, for chest-related investigations in Hong Kong, the impact was not as severe. Quantitative analysis could help with future planning and public health decision making.

Factors Influential in the Selection of Radiology Residents in the Post-Step 1 World: A Discrete Choice Experiment.

Reporting of United States Medical Licensing Examination Step 1 results will transition from a numerical score to a pass or fail result. We sought an objective analysis to determine changes in the relative importance of resident application attributes when numerical Step 1 results are replaced. A discrete choice experiment was designed to model radiology resident selection and determine the relative weights of various application factors when paired with a numerical or pass or fail Step 1 result. Faculty involved in resident selection at 14 US radiology programs chose between hypothetical pairs of applicant profiles between August and November 2020. A conditional logistic regression model assessed the relative weights of the attributes, and odds ratios (ORs) were calculated. There were 212 participants. When a numerical Step 1 score was provided, the most influential attributes were medical school (OR: 2.35, 95% confidence interval [CI]: 2.07-2.67), Black or Hispanic race or ethnicity (OR: 2.04, 95% CI: 1.79-2.38), and Step 1 score (OR: 1.8, 95% CI: 1.69-1.95). When Step 1 was reported as pass, the applicant's medical school grew in influence (OR: 2.78, 95% CI: 2.42-3.18), and there was a significant increase in influence of Step 2 scores (OR: 1.31, 95% CI: 1.23-1.40 versus OR 1.57, 95% CI: 1.46-1.69). There was little change in the relative influence of race or ethnicity, gender, class rank, or clerkship honors. When Step 1 reporting transitions to pass or fail, medical school prestige gains outsized influence and Step 2 scores partly fill the gap left by Step 1 examination as a single metric of decisive importance in application decisions.

A multicenter, hospital-based and non-inferiority study for diagnostic efficacy of automated whole breast ultrasound for breast cancer\xa0in China

This study is the first multi-center non-inferiority study that aims to critically evaluate the effectiveness of HHUS/ABUS in China breast cancer detection. This was a multicenter hospital-based study. Five hospitals participated in this study. Women (30–69 years old) with defined criteria were invited for breast examination by HHUS, ABUS or/and mammography. For BI-RADS category 3, an additional magnetic resonance imaging (MRI) test was provided to distinguish the true negative results from false negative results. For women classified as BI-RADS category 4 or 5, either core aspiration biopsy or surgical biopsy was done to confirm the diagnosis. Between February 2016 and March 2017, 2844 women signed the informed consent form, and 1947 of them involved in final analysis (680 were 30 to 39 years old, 1267 were 40 to 69 years old).For all participants, ABUS sensitivity (91.81%) compared with HHUS sensitivity (94.70%) with non-inferior Z tests, P = 0.015. In the 40–69 age group, non-inferior Z tests showed that ABUS sensitivity (93.01%) was non-inferior to MG sensitivity (86.02%) with P < 0.001 and HHUS sensitivity (95.44%) was non-inferior to MG sensitivity (86.02%) with P < 0.001. Sensitivity of ABUS and HHUS are all superior to that of MG with P < 0.001 by superior test.For all participants, ABUS specificity (92.89%) was non-inferior to HHUS specificity (89.36%) with P < 0.001. Superiority test show that specificity of ABUS was superior to that of HHUS with P < 0.001. In the 40–69 age group, ABUS specificity (92.86%) was non-inferior to MG specificity (91.68%) with P < 0.001 and HHUS specificity (89.55%) was non-inferior to MG specificity (91.68%) with P < 0.001. ABUS is not superior to MG with P = 0.114 by superior test. The sensitivity of ABUS/HHUS is superior to that of MG. The specificity of ABUS/HHUS is non-inferior to that of MG. In China, for an experienced US radiologist, both HHUS and ABUS have better diagnostic efficacy than MG in symptomatic individuals.

Resident Clinician Educator Leadership Pathway Tracks in US Radiology Programs: An ADVICER 2021 Survey Study

Rationale and ObjectivesThe purpose of this manuscript is to elucidate the prevalence, characteristics, outcomes, and perceptions of Alliance of Directors and Vice Chairs of Education in Radiology (ADVICER) members on Clinician Educator Leadership Pathway (CELP) tracks in diagnostic and interventional radiology residency programs in the United States. Materials and MethodsIRB exemption was obtained for this study. A 17-question anonymous survey was administered to the ADVICER members, an Association of University Radiologists (AUR) affiliate group, via email once, and then as a reminder 14 days later. Statistical analysis occurred via built-in analytics of the survey vendor, SurveyMonkey. ResultsThirty five of 72 ADVICER members (48.6%) responded to the survey. 45.7% (16/35) respondents indicated the presence of leadership or specialty pathways in their respective residencies. 28.6% of respondents indicated a CELP (10/35) in their residency programs. 92.0% of respondents endorsed CELP as preparatory for an academic career, and 93.6% endorsed development of a national CELP curriculum. ConclusionThe majority of radiology residency programs do not have a CELP in their training curriculum. ADVICER leaders overwhelmingly support CELP for professional development of the future educational leaders in radiology and endorse the creation of a national CELP curriculum.