American Medical Informatics Association Research Articles

Large language models for biomedicine: foundations, opportunities, challenges, and best practices.

Generative large language models (LLMs) are a subset of transformers-based neural network architecture models. LLMs have successfully leveraged a combination of an increased number of parameters, improvements in computational efficiency, and large pre-training datasets to perform a wide spectrum of natural language processing (NLP) tasks. Using a few examples (few-shot) or no examples (zero-shot) for prompt-tuning has enabled LLMs to achieve state-of-the-art performance in a broad range of NLP applications. This article by the American Medical Informatics Association (AMIA) NLP Working Group characterizes the opportunities, challenges, and best practices for our community to leverage and advance the integration of LLMs in downstream NLP applications effectively. This can be accomplished through a variety of approaches, including augmented prompting, instruction prompt tuning, and reinforcement learning from human feedback (RLHF). Our focus is on making LLMs accessible to the broader biomedical informatics community, including clinicians and researchers who may be unfamiliar with NLP. Additionally, NLP practitioners may gain insight from the described best practices. We focus on 3 broad categories of NLP tasks, namely natural language understanding, natural language inferencing, and natural language generation. We review the emerging trends in prompt tuning, instruction fine-tuning, and evaluation metrics used for LLMs while drawing attention to several issues that impact biomedical NLP applications, including falsehoods in generated text (confabulation/hallucinations), toxicity, and dataset contamination leading to overfitting. We also review potential approaches to address some of these current challenges in LLMs, such as chain of thought prompting, and the phenomena of emergent capabilities observed in LLMs that can be leveraged to address complex NLP challenge in biomedical applications.

Perspectives on the role of industry in informatics research and authorship.

Advances in informatics research come from academic, nonprofit, and for-profit industry organizations, and from academic-industry partnerships. While scientific studies of commercial products may offer critical lessons for the field, manuscripts authored by industry scientists are sometimes categorically rejected. We review historical context, community perceptions, and guidelines on informatics authorship. We convened an expert panel at the American Medical Informatics Association 2022 Annual Symposium to explore the role of industry in informatics research and authorship with community input. The panel summarized session themes and prepared recommendations. Authorship for informatics research, regardless of affiliation, should be determined by International Committee of Medical Journal Editors uniform requirements for authorship. All authors meeting criteria should be included, and categorical rejection based on author affiliation is unethical. Informatics research should be evaluated based on its scientific rigor; all sources of bias and conflicts of interest should be addressed through disclosure and, when possible, methodological mitigation.

A Skills Assessment Pathways-Based Program Assessment Approach in Multidisciplinary Graduate Health Informatics.

Multidisciplinary graduate education programs are hard to assess because of interdependent competencies. Students in these programs come with diverse disciplinary undergraduate degrees, and it is critical to identify knowledge gaps among these diverse learner groups to provide support to fill these gaps. Health Informatics (HI) is a multidisciplinary field in which health, technology, and social science knowledge are foundational to building HI competencies. In 2017, the American Medical Informatics Association identified ten functional domains in which HI competencies are divided. Using pre/post-semester knowledge assessment surveys of graduate students (n=60) between August 2021 to May 2022 in one of the largest graduate HI programs in the United States, we identified courses (n=9) across the curriculum that help build HI-specific competencies. Using statistical analysis, we identified three skills pathways by correlating knowledge gained with course learning objectives and used this to modify the curriculum over four semesters. These skills pathways are connected through one or two courses, where students can choose electives or, in some instances, course modules or assignments that link the skills pathways. Moreover, there is a statistically significant difference in how students gain these skills depending on their prior training, even though they take the same set of courses. Gender and other demographics did not show statistical differences in skills gained. Additionally, we found that research assistantships and internships/practicums provide additional skills not covered in our HI curriculum. Our program assessment methodology and resulting curricular changes might be relevant to HI and other multidisciplinary graduate training programs.

Creation and Evaluation of a Clinical Informatics Match: Initial Findings.

Clinical Informatics (CI) fellowship programs utilize the Electronic Residency Application Service (ERAS) to gather applications but until recently used an American Medical Informatics Association (AMIA) member-developed, simultaneous offer-acceptance process to match fellowship applicants to programs. In 2021, program directors collaborated with the AMIA to develop a new match to improve the process. Describe the results of the first 2 years of the match and address opportunities for improvement. We obtained applicant data for fellowship applicants in 2021 and 2022 from the ERAS and match data for the same years from the AMIA. We analyzed our data using descriptive statistics. There were 159 unique applicants over the 2-year period. Applicants submitted 2,178 applications with a median of 10 per applicant (interquartile range [IQR] 3-20). One hundred and four applicants (65.4%) participated in the match and ranked a median of seven programs (2-12). Forty-two programs in 2021 and 47 programs in 2022 offered a combined total 153 positions in the match. Participating programs ranked a median of eight applicants per year (IQR 5-11). Of participating applicants, 95 (91.3%) successfully matched and of those 66 (69.5%) received their top choice. Thirty-two programs (76.2%) matched at least one candidate in 2021 and 33 programs (70.2%) matched at least one candidate in 2022. In both years, 24 programs filled all available slots (57.1% in 2021 and 51.1% in 2022). Applicants were extremely successful in the new match, which successfully addressed most of the challenges of the simultaneous offer-acceptance process identified by program directors. However, applicant attrition resulted in a quarter of programs going unmatched. Although many programs still filled slots outside the match, fellowship slots may remain unfilled while the CI practice pathway remains open.

Mapping the delineation of practice to the AMIA foundational domains for applied health informatics.

This article reports on the alignment between the foundational domains and the delineation of practice (DoP) for health informatics, both developed by the American Medical Informatics Association (AMIA). Whereas the foundational domains guide graduate-level curriculum development and accreditation assessment, providing an educational pathway to the minimum competencies needed as a health informatician, the DoP defines the domains, tasks, knowledge, and skills that a professional needs to competently perform in the discipline of health informatics. The purpose of this article is to determine whether the foundational domains need modification to better reflect applied practice. Using an iterative process and through individual and collective approaches, the foundational domains and the DoP statements were analyzed for alignment and eventual harmonization. Tables and Sankey plot diagrams were used to detail and illustrate the resulting alignment. We were able to map all the individual DoP knowledge statements and tasks to the AMIA foundational domains, but the statements within a single DoP domain did not all map to the same foundational domain. Even though the AMIA foundational domains and DoP domains are not in perfect alignment, the DoP provides good examples of specific health informatics competencies for most of the foundational domains. There are, however, limited DoP knowledge statements and tasks mapping to foundational domain 6-Social and Behavioral Aspects of Health. Both the foundational domains and the DoP were developed independently, several years apart, and for different purposes. The mapping analyses reveal similarities and differences between the practice experience and the curricular needs of health informaticians. The overall alignment of both domains may be explained by the fact that both describe the current and/or future health informatics professional. One can think of the foundational domains as representing the broad foci for educational programs for health informaticians and, hence, they are appropriately the focus of organizations that accredit these programs.

Identifying research fronts in NLP applications in library and information science using meta-analysis approaches

PurposeThe purpose of this study is to identify the research fronts by analysing highly cited core papers adjusted with the age of a paper in library and information science (LIS) where natural language processing (NLP) is being applied significantly.Design/methodology/approachBy excavating international databases, 3,087 core papers that received at least 5% of the total citations have been identified. By calculating the average mean years of these core papers, and total citations received, a CPT (citation/publication/time) value was calculated in all 20 fronts to understand how a front is relatively receiving greater attention among peers within a course of time. One theme article has been finally identified from each of these 20 fronts.FindingsBidirectional encoder representations from transformers with CPT value 1.608 followed by sentiment analysis with CPT 1.292 received highest attention in NLP research. Columbia University New York, in terms of University, Journal of the American Medical Informatics Association, in terms of journals, USA followed by People Republic of China, in terms of country and Xu, H., University of Texas, in terms of author are the top in these fronts. It is identified that the NLP applications boost the performance of digital libraries and automated library systems in the digital environment.Practical implicationsAny research fronts that are identified in the findings of this paper may be used as a base for researchers who intended to perform extensive research on NLP.Originality/valueTo the best of the authors’ knowledge, the methodology adopted in this paper is the first of its kind where meta-analysis approach has been used for understanding the research fronts in sub field like NLP for a broad domain like LIS.

Global Health Informatics: the state of research and lessons learned.

The Journal of the American Medical Informatics Association (JAMIA) focus issue on Global Health Informatics (GHI) examines how health informatics can address global health issues. The selected papers discuss how informatics can improve health outcomes, especially in low- and middle-income countries (LMICs). The articles in this issue come from a broad range of researchers, practitioners, and policymakers, provide an overview of GHI initiatives and encourage researchers and practitioners to share ideas and best practices. GHI aims to harness digital health and informatics knowledge and tools to improve the care of individuals and populations in LMIC through achieving these 5 aims: effectiveness, efficiencies, patient satisfaction, provider well-being, and equity. A central consideration is the global north-south divide and the sociotechnical determinants of and impacts on the digital health maturity of LMIC. This issue seeks to document and understand how health informatics approaches have been undertaken in multiple LMIC scenarios and environments and how they can demonstrate the scientific impact of informatics-based best practices applied to COVID-19 and beyond.

Electronic health records in nursing from 2000 to 2020: A bibliometric analysis.

Electronic health records (EHR) is the longitudinal data generated by patients in medical institutions and recorded by electronic medical information systems in the form of digital, which is also the most widespread application of big data in medicine. The purpose of this study was to explore the application of electronic health records in the field of nursing and determine the current research status and hotspots. A bibliometric analysis of electronic health records in nursing was undertaken from 2000 to 2020. The literature comes from Web of Science Core Collection database. We used CiteSpace (version 5.7 R5; Drexel University), which is a Java-based software that especially visualized collaborative networks and research topics. A total of 2616 publications were included in the study. We found that publications increased year by year. The Journal of American Medical Informatics Association (n = 921) is the most cited. The United States (n = 1,738) has the most publications in this field. University Penn (n = 63) is the institution with the most publications. There is no influential cooperation network among the authors, of which Bates, David W (n = 12) have the largest number of publications. The relevant publications also focus on the fields of health care science and services, and medical informatics. In keywords, EHR, long-term care, mobile application, inpatient falls, and advance care planning has been researching hotspots in recent years. With the popularization of information systems, the publications of EHR in the nursing field have increased year by year. This study provides the basic structure, potential cooperation, and research trends of EHR in the field of nursing from 2000 to 2020, and provides a reference for nurses to effectively use EHR to help clinical work or scientific researchers explore the potential significances of EHR.

Reflections on the Documentation Burden Reduction AMIA Plenary Session through the Lens of 25 × 5.

Documentation burden, defined as the excessive effort expended on healthcare documentation, is associated with a number of adverse outcomes, including clinician burnout, reduced quality of medical care, and disruption of clinical data contained in the electronic health record.[1] With the growing concern for the wellness of the clinical workforce, documentation burden is receiving national attention. The American Medical Informatics Association (AMIA) has taken the lead by establishing the 25 x 5 Task Force ("Task Force") in December 2021, which aims to reduce clinician documentation burden to 25% of the current state in the coming 5 years. [2] Aligned with the timing of the Task Force launch, the AMIA Clinical Informatics Conference (CIC) 2022 co-chairs, Rosemary Kennedy (Connect America) and Paul Fu (City of Hope), conceptualized an opening plenary panel in a 'fireside chat' format focused on clinical documentation burden.[3] In this editorial, the authors describe the panel discussion, identify key themes from the panel, and offer recommendations to address documentation burden. The proceedings of the AMIA CIC 2022 Fireside Chat serve as an opportunity to acknowledge those who are engaged and passionate about addressing documentation burden from the vantage point of different stakeholders and institutions.

STORIES from the FRONT LINES

Inside Precision MedicineVol. 9, No. S1 Spotlight SupplementSTORIES from the FRONT LINESNephi WaltonMD MS FACMG FAMIANephi WaltonMD MS FACMG FAMIADr Nephi Walton completed his MD and masters in biomedical informatics at the University of Utah School of Medicine. He completed a combined residency in pediatrics and genetics at Washington University in St Louis. He is boarded in both clinical genetics and clinical informatics. At Geisinger, he successfully completed a pilot integration of genomics data into the EPIC EHR system for both pharmacogenomics and CDC tier one genetic conditions. He currently serves as the associate medical director of Intermountain Precision Genomics where he co-leads the HerediGene genomic sequencing return of results program and runs the Intermountain precision genomics WGS clinic. He also serves as the associate medical director of Intermountain's sequencing laboratory. He is the chair of genomics and translational bioinformatics for the American Medical Informatics Association and has presented at several meetings on translating the use of genomics into general medical practice, something he is actively pursuing at Intermountain Healthcare.Search for more papers by this authorPublished Online:9 Jan 2023https://doi.org/10.1089/ipm.09.S1.05AboutSectionsView articleView Full TextPDF/EPUB Permissions & CitationsPermissionsDownload CitationsTrack CitationsAdd to favorites Back To Publication ShareShare onFacebookTwitterLinked InRedditEmail View articleFiguresReferencesRelatedDetails Volume 9Issue S1Dec 2022 InformationCopyright © GEN PublishingTo cite this article:Nephi WaltonMD MS FACMG FAMIA.STORIES from the FRONT LINES.Inside Precision Medicine.Dec 2022.20-21.http://doi.org/10.1089/ipm.09.S1.05Published in Volume: 9 Issue S1: January 9, 2023PDF download

Unlocking the eHealth professionals’ career pathways: A case of Gulf Cooperation Council countries

Unlocking the eHealth professionals’ career pathways: A case of Gulf Cooperation Council countries

Clinical Laboratory Informatics and Analytics: Challenges and Opportunities.

The American Medical Informatics Association defines biomedical and health informatics as the “science of how to use data, information, and knowledge to improve human health and the delivery of health care services.” More specifically within laboratory medicine, informatics and data analytics use multiple sources of data to improve all aspects of the clinical laboratory, from workflow and personnel to result interpretation. With increasing healthcare information complexity, integration and interoperability issues have become readily apparent between health information systems, bringing to the forefront questions about the validity of data exchange and basic data access. Most data generated within the clinical laboratory are of high quality, well annotated, and structured discretely, however turning these data into useful and actionable information can be a difficult data analytics bridge for many to cross. Instrument and laboratory information system (LIS) vendors are beginning to aid in the creation of generalized reports for common laboratories questions; however, this still falls short of the potential of the clinical laboratory to bring more actionable information to hospital leadership, clinicians, and patients. Collaboration among informaticians, information technology (IT) professionals, and the laboratorians is critical to ensure our health information systems can utilize and report laboratory data clinically, in addition to providing interoperable data streams for furthering research, education, and innovation in healthcare.

Meeting the information and communication needs of health disparate populations.

Meeting the information and communication needs of health disparate populations in a culturally congruent and health literate manner is foundational to advancing health equity, health status, and health-related quality of life. The Journal of the American Medical Informatics Association has published seminal papers on this topic over the last 15 years.1–10 Papers have addressed important factors related to meeting information and communication needs including health literacy, numeracy, and graph literacy1,2,4; described innovative solutions for meeting information and communication needs3–10; and conducted comparative evaluations of visual strategies.6–9 In this editorial, I briefly summarize 5 papers that contribute to the literature on these topics. In 2 papers, the authors report user studies of data visualizations of longitudinal data, that is, systolic blood pressure values11 and patient-reported outcomes (PROs) that include measures of health and graph literacy.12 A review paper focuses on the natural language processing literature for automatic simplification of existing biomedical text for consumers and identifies strategies for addressing challenges to progress in the field.13 As the foundation for public health dashboards that effectively communicate with a variety of audiences including the lay public, Ansari and Martin14 describe formative work for the creation of a heuristic evaluation checklist for public health dashboards. In the fifth paper, Valdez et al15 highlight the disability community as a health disparate population and offer a set of guidelines for effective engagement to create digital health technologies that more fully meet the information and communication needs of all disabled individuals.

A Model Curriculum for an Emergency Medicine Residency Rotation in Clinical Informatics.

This curriculum is designed for emergency medicine residents at all levels of training. The curriculum covers basic foundations in clinical informatics for improving patient care and outcomes, utilizing data, and leading improvements in emergency medicine. The curriculum is designed for a four-week rotation. The American College of Graduate Medical Education (ACGME) mandated that all Emergency Medicine (EM) residents receive specific training in the use of information technology.1,2 To our knowledge, a clinical informatics curriculum for EM residents does not exist. We propose the following standardized and reproducible educational curriculum for EM residents. The aim of this curriculum is to teach informatics skills to emergency physicians to improve patient care and outcomes, utilize data, and develop projects to lead change.3 These goals will be achieved by providing a foundational informatics elective for EM residents that follows the delineation of practice for Clinical Informatics outlined by the American Medical Informatics Association (AMIA) and the American Board of Preventive Medicine (ABPM).4-6. The educational strategies used in this curriculum include asynchronous learning via books, papers, videos, and websites. Residents attend administrative sessions (meetings), develop a project proposal, and participate in small group discussions.The rotation emphasizes the basic concepts surrounding clinical informatics with an emphasis on improving care delivery and outcomes, information systems, data governance and analytics, as well as leadership and professionalism. The course focuses on the practical application of these concepts, including implementation, clinical decision support, workflow analysis, privacy and security, information technology across the patient care continuum, health information exchange, data analytics, and leading change through stakeholder engagement. An initial version of the curriculum was introduced to two separate institutions and was completed by three rotating resident physicians and one rotating resident pharmacist. A brief course evaluation as well as qualitative feedback was solicited from elective participants by the course director, via email following the completion of the course, regarding the effectiveness of the course content. Learner feedback was used to influence the development of this complete curriculum. The curriculum was graded by learners on a 5-point Likert scale (1=strongly disagree, 5 = strongly agree). The mean response to, "This course was a valuable use of my elective time," was 5 (sd=0). The mean response to, "I achieved the learning objectives," and "This rotation helped me understand Clinical Informatics," were both 4.75 (sd=0.5). Overall, participants reported that the content was effective for achieving the learning objectives. During initial implementation, we found that the preliminary asynchronous learning component worked less effectively than we anticipated due to a lower volume of content. In response to this, as well as resident feedback, we added significantly more educational content.In conclusion, this model curriculum provides a structured process for an informatics rotation for the emergency medicine resident that utilizes the core competencies established by the governing bodies of the clinical informatics specialty and ACGME. Clinical informatics key concepts, including definitions, fundamental terminology, history, policy and regulations, ethical considerations, clinical decision support, health information systems, data governance and analytics, process improvement, stakeholder engagement and change management.

Persisting symptoms after concussion: Time for a paradigm shift.

Persisting symptoms after concussion: Time for a paradigm shift.

Identification of the most important external features of highly cited scholarly papers through 3 (i.e., Ridge, Lasso, and Boruta) feature selection data mining methods

Highly cited papers are influenced by external factors that are not directly related to the document's intrinsic quality. In this study, 50 characteristics for measuring the performance of 68 highly cited papers, from the Journal of The American Medical Informatics Association indexed in Web of Science (WOS), from 2009 to 2019 were investigated. In the first step, a Pearson correlation analysis is performed to eliminate variables with zero or weak correlation with the target (“dependent”) variable (number of citations in WOS). Consequently, 32 variables are selected for the next step. By applying the Ridge technique, 13 features show a positive effect on the number of citations. Using three different algorithms, i.e., Ridge, Lasso, and Boruta, 6 factors appear to be the most relevant ones. The "Number of citations by international researchers", "Journal self-citations in citing documents”, and "Authors' self-citations in citing documents”, are recognized as the most important features by all three methods here used. The "First author's scientific age”, "Open-access paper”, and "Number of first author's citations in WOS" are identified as the important features of highly cited papers by only two methods, Ridge and Lasso. Notice that we use specific machine learning algorithms as feature selection methods (Ridge, Lasso, and Boruta) to identify the most important features of highly cited papers, tools that had not previously been used for this purpose. In conclusion, we re-emphasize the performance resulting from such algorithms. Moreover, we do not advise authors to seek to increase the citations of their articles by manipulating the identified performance features. Indeed, ethical rules regarding these characteristics must be strictly obeyed.

Assessing perceived effectiveness of career development efforts led by the women in American Medical Informatics Association Initiative.

We sought to ascertain perceived factors affecting women's career development efforts in the American Medical Informatics Association (AMIA) and to provide recommendations for improvements. Data were collected using a 27-item survey administered via the AMIA newsletter and other social channels. Survey questions comprised 3 demographics, 15 Likert-scale, and 9 open-ended items. Likert-scale responses were summarized across respondent ages, career stages, and career domains, and open-ended responses were thematically analyzed. We received survey responses from 109 AMIA women members. Our findings demonstrate that AMIA had made strides in promoting career development, and the most effective AMIA efforts included social events (83%), panel discussions (80%), and scientific sessions (79%). However, despite these efforts, women members perceived that gender-specific challenges persisted within AMIA, and recognized the need for increased networking opportunities (96%), raising awareness of gender-specific challenges (95%), and encouraging gender proportional representation in leadership (92%). International and national biomedical informatics professional communities have put forth efforts to address gender-specific issues in career development. Yet, our study identified that some of these, including the deep-rooted gender power hierarchy and bias, are still perceived as profound in AMIA. Even though existing career development efforts for women are highly effective, important perceived gender-specific career development issues require further attention and investigation to improve existing AMIA activities.

Selecting venues for AMIA events and conferences: guiding ethical principles.

A discussion and debate on the American Medical Informatics Association's (AMIA) Ethical, Legal, and Social Issues (ELSI) Working Group listserv in 2021 raised important issues related to a forthcoming conference in Texas. Texas had recently enacted a restrictive abortion law and restricted voting rights. Several AMIA members advocated for a boycott of the state and the scheduled conference. The discussion led the AMIA Board of Directors to request that the organization's Ethics Committee provide general guidance for principle-based venue selection. This document recommends overarching principles for the venue selection for future AMIA events and conferences. Discussions by the AMIA Board, the Ethics Committee, and the ELSI Working Group informed these recommendations, and this document on guiding principles was approved by the AMIA Board of Directors in April 2022.

Donald A.B. Lindberg, pioneer in biomedical and health informatics: His involvement in creating professional organizations.

Among the many contributions of Donald A.B. Lindberg was his work on behalf of a variety or professional organizations in the field of biomedical and health informatics. These began during his early days at the University of Missouri and continued throughout his 30 years at the National Library of Medicine. This chapter summarizes that work, which occurred both through his personal efforts and through the impact of the NLM under his leadership. Examples include his role in the development of organizations themselves (e.g., the International Medical Informatics Association, the American College of Medical Informatics, and the American Medical Informatics Association) and also his contributions to the professional scientific meetings that have advanced the field (e.g., the Symposium on Computer Applications in Medical Care, MEDINFO, and the AMIA Annual Symposium).

A tribute to Karen Greenwood and her contributions to the American Medical Informatics Association

After 25 years of service to the American Medical Informatics Association (AMIA), Ms Karen Greenwood, the Executive Vice President and Chief Operating Officer, is leaving the organization. In this perspective, we reflect on her accomplishments and her effect on the organization and the field of informatics nationally and globally. We also express our appreciation and gratitude for Ms Greenwood's role at AMIA.