Translational Science Award Research Articles

Assessing Retrieval-Augmented Large Language Model Performance in Emergency Department ICD-10-CM Coding Compared to Human Coders.

Accurate medical coding is essential for clinical and administrative purposes but complicated, time-consuming, and biased. This study compares Retrieval-Augmented Generation (RAG)-enhanced LLMs to provider-assigned codes in producing ICD-10-CM codes from emergency department (ED) clinical records. Retrospective cohort study using 500 ED visits randomly selected from the Mount Sinai Health System between January and April 2024. The RAG system integrated past 1,038,066 ED visits data (2021-2023) into the LLMs' predictions to improve coding accuracy. Nine commercial and open-source LLMs were evaluated. The primary outcome was a head-to-head comparison of the ICD-10-CM codes generated by the RAG-enhanced LLMs and those assigned by the original providers. A panel of four physicians and two LLMs blindly reviewed the codes, comparing the RAG-enhanced LLM and provider-assigned codes on accuracy and specificity. RAG-enhanced LLMs demonstrated superior performance to provider coders in both the accuracy and specificity of code assignments. In a targeted evaluation of 200 cases where discrepancies existed between GPT-4 and provider-assigned codes, human reviewers favored GPT-4 for accuracy in 447 instances, compared to 277 instances where providers' codes were preferred (p<0.001). Similarly, GPT-4 was selected for its superior specificity in 509 cases, whereas human coders were preferred in only 181 cases (p<0.001). Smaller open-access models, such as Llama-3.1-70B, also demonstrated substantial scalability when enhanced with RAG, with 218 instances of accuracy preference compared to 90 for providers' codes. Furthermore, across all models, the exact match rate between LLM-generated and provider-assigned codes significantly improved following RAG integration, with Qwen-2-7B increasing from 0.8% to 17.6% and Gemma-2-9b-it improving from 7.2% to 26.4%. RAG-enhanced LLMs improve medical coding accuracy in EDs, suggesting clinical workflow applications. These findings show that generative AI can improve clinical outcomes and reduce administrative burdens. This work was supported in part through the computational and data resources and staff expertise provided by Scientific Computing and Data at the Icahn School of Medicine at Mount Sinai and supported by the Clinical and Translational Science Awards (CTSA) grant UL1TR004419 from the National Center for Advancing Translational Sciences. Research reported in this publication was also supported by the Office of Research Infrastructure of the National Institutes of Health under award number S10OD026880 and S10OD030463. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The funders played no role in study design, data collection, analysis and interpretation of data, or the writing of this manuscript. A study showed AI models with retrieval-augmented generation outperformed human doctors in ED diagnostic coding accuracy and specificity. Even smaller AI models perform favorably when using RAG. This suggests potential for reducing administrative burden in healthcare, improving coding efficiency, and enhancing clinical documentation.

The intersection of Interprofessional Education and One Health: A qualitative study in human and veterinary medical institutions

Interprofessional Education (IPE) and One Health are two common and overlapping frameworks for teaching collaborative practice. IPE is common at human medical institutions, while One Health is more common in graduate and veterinary programs. The connection between IPE and One Health is still being explored both in scholarship and in real-world professional settings. This prospective, qualitative research study examines the intersection of IPE and One Health at institutions that are members of the Clinical and Translational Science Award (CTSA) One Health Alliance (COHA). COHA consists of veterinary schools partnered with medical institutions through the National Institutes of Health CTSA funding mechanism with the specific goal of advancing the understanding of diseases shared by humans and animals. Twenty-four interviews were conducted with professionals across eight professions. Subjects noted that some of the biggest barriers to IPE education were awareness, accessibility, efficacy, and implementation beyond the classroom. Competency across multiple institutions and a consistent, validated evaluation tool were noted to be lacking. Interviews highlighted a lack of a shared mental model for IPE and One Health across the medical professions, major hurdles for implementation in professional curricula, and a disconnection between bridging IPE and One Health to the workforce and global challenges. Future work in this area may be focused on assessing the IPE and One Health offerings beyond COHA institutions, giving a more holistic understanding on how IPE and One Health are being deployed. One Health can be operationalized through the adoption of IPE principles and practices into curriculum. This research is critical to educate others on current applications, role, and definitions of One Health and IPE. The ultimate goal of this work is to help cultivate transdisciplinary leaders in the human and animal medicine who will have the skills to solve systemic problems.

Understanding enterprise data warehouses to support clinical and translational research: impact, sustainability, demand management, and accessibility.

Healthcare organizations, including Clinical and Translational Science Awards (CTSA) hubs funded by the National Institutes of Health, seek to enable secondary use of electronic health record (EHR) data through an enterprise data warehouse for research (EDW4R), but optimal approaches are unknown. In this qualitative study, our goal was to understand EDW4R impact, sustainability, demand management, and accessibility. We engaged a convenience sample of informatics leaders from CTSA hubs (n = 21) for semi-structured interviews and completed a directed content analysis of interview transcripts. EDW4R have created institutional capacity for single- and multi-center studies, democratized access to EHR data for investigators from multiple disciplines, and enabled the learning health system. Bibliometrics have been challenging due to investigator non-compliance, but one hub's requirement to link all study protocols with funding records enabled quantifying an EDW4R's multi-million dollar impact. Sustainability of EDW4R has relied on multiple funding sources with a general shift away from the CTSA grant toward institutional and industry support. To address EDW4R demand, institutions have expanded staff, used different governance approaches, and provided investigator self-service tools. EDW4R accessibility can benefit from improved tools incorporating user-centered design, increased data literacy among scientists, expansion of informaticians in the workforce, and growth of team science. As investigator demand for EDW4R has increased, approaches to tracking impact, ensuring sustainability, and improving accessibility of EDW4R resources have varied. This study adds to understanding of how informatics leaders seek to support investigators using EDW4R across the CTSA consortium and potentially elsewhere.

Title: Influence of the NASA SPRINT exercise protocol on myonuclear and satellite cell content of the vastus lateralis and soleus during 70 days of bedrest

Objectives: The purpose of this study was to assess the effcacy of the NASA SPRINT exercise protocol in preserving myonuclear and satellite cell number of the quadriceps (vastus lateralis) and triceps surae (soleus) muscles following 70 days of bedrest (i.e., simulated microgravity). These two muscles were the focus of this study because they both atrophy significantly with microgravity exposure in the absence of countermeasures. Hypotheses: The SPRINT exercise protocol will preserve myonuclear number (MNN), myonuclear domain (MND), and satellite cell number (SCN) during 70 days of bedrest. Methods: Subjects were randomized into three groups: bedrest only (BR; n=8, 37±3 y), bedrest with endurance and resistance exercise (BRE; n=9, 34±2 y) and bedrest with endurance and resistance exercise and low dose testosterone injections (BRE+T; n=8, 33±3 y). Muscle biopsies were taken before and at the end of bedrest from the vastus lateralis and soleus, and were sectioned, immunofluorescently stained, and analyzed for MNN (myonuclei/fiber), MND (μm2/myonuclei), and SCN (satellite cells/fiber) in slow (myosin heavy chain (MHC) I) and fast (MHC IIa) muscle fibers. SCN was only determined in MHC I fibers for the soleus. Results: MNN, MND, and SCN in MHC I and IIa fibers of the vastus lateralis and soleus were unchanged (p&lt;0.05) from pre- to post-bedrest within BR, BRE, and BRE+T. However, there was a main effect for time (p&lt;0.05) for a decrease in MND for vastus lateralis MHC I fibers (-8%), as well as soleus MHC I (-18%) and IIa (-18%) fibers. Also, there was a trend for a main effect for time (P=0.059) for an increase (+37%) in SCN for vastus lateralis MHC IIa fibers, which was driven primarily by the exercise groups (BR: -4%, BRE: +36%, BRE+T: +67%). Conclusions: Myonuclear number appears to be unaltered during 70 days of simulated microgravity with or without countermeasures exercise. As a result, any changes in myonuclear domain correspond with alterations in muscle mass. Satellite cell number also appears to be relatively refractory to change, but may be influenced in fast muscle fibers under the combined dynamic cellular control mechanisms of exercise and atrophy encountered during relatively long-duration simulated microgravity. Funding Source: This research was supported by the National Aeronautics and Space Administration, NNJ06HF59G; the Institute for Translational Sciences at the University of Texas Medical Branch, supported in part by a Clinical and Translational Science Award (UR1TR000071) from the National Center for Advancing Translational Sciences, National Institutes of Health; the NASA Flight Analogs Project; and the Human Bioenergetic Program at Ball State University. This is the full abstract presented at the American Physiology Summit 2024 meeting and is only available in HTML format. There are no additional versions or additional content available for this abstract. Physiology was not involved in the peer review process.

Fiber type specific myonuclear and satellite cell content comparisons between the vastus lateralis and soleus muscles in humans

Objectives: The purpose of this study was to compare fiber type specific myonuclear and satellite cell number of the vastus lateralis and soleus muscles in humans. Hypotheses: Myonuclear number (MNN), myonuclear domain (MND), and satellite cell number (SCN) would be similar within each fiber type between the vastus lateralis and soleus. Methods: Muscle biopsies from the vastus lateralis and soleus of 24 individuals (34±7y, 24-55y) were sectioned, immunofluorescently stained, and analyzed for MNN, MND, and SCN in slow (myosin heavy chain (MHC) I) and fast (MHC IIa) muscle fibers. SCN was determined only in MHC I fibers for the soleus. MNN and MND were compared for 24 individuals, while SCN was compared for 19 individuals. Results: MNN was similar in MHC I (3.32±0.81 vs. 3.66±0.65 myonuclei/fiber, P=0.067) and IIa (3.63±0.82 vs. 3.83±0.68 myonuclei/fiber, P&gt;0.05) fibers of the vastus lateralis and soleus. Similarly, MND was not different in MHC I (1649±305 vs. 1673±340 μm2/myonuclei, P&gt;0.05) and IIa (1705±313 vs. 1819±394 μm2/myonuclei, P&gt;0.05) fibers of the vastus lateralis and soleus. However, SCN was 62% higher (p&lt;0.05) in MHC I fibers of the soleus (0.127±0.039 satellite cells/fiber) compared with MHC I fibers of the vastus lateralis (0.083±0.024 satellite cells/fiber). Conclusions: The literature is void of comparisons of myonuclei and satellite numbers across different human muscles. These initial results suggest myonuclear number and the area “governed” by each myonuclei (i.e., the myonuclear domain) appears to be similar within the two main fiber types found in the vastus lateralis and soleus muscles. Interestingly, within the same slow fiber type (i.e., MHC I fibers) across these two muscles, it appears the soleus is programmed to have a substantially higher satellite cell content. The basis for this intrinsic difference in slow myofiber satellite profile may be related to the differences in daily myocellular activity between the vastus lateralis and soleus. This research was supported by the National Aeronautics and Space Administration, NNJ06HF59G; the Institute for Translational Sciences at the University of Texas Medical Branch, supported in part by a Clinical and Translational Science Award (UR1TR000071) from the National Center for Advancing Translational Sciences, National Institutes of Health; the NASA Flight Analogs Project; and the Human Bioenergetic Program at Ball State University. This is the full abstract presented at the American Physiology Summit 2024 meeting and is only available in HTML format. There are no additional versions or additional content available for this abstract. Physiology was not involved in the peer review process.

Metabolic syndrome reduces cerebral blood flow more in women than men

One in three adults in the United States suffers from metabolic syndrome (MetSyn), a state of insulin resistance associated with reduced cerebral blood flow (CBF) and increased risk of stroke and neurologic diseases. Notably, females are at greater risk for cardiovascular and metabolic complications derived from insulin resistance. Still, it remains unclear whether the reduction in CBF seen in MetSyn is sex-specific or uniform between various brain regions. Additionally, fasting triglyceride (TG) levels are stronger predictors of cognitive disease in females than in males. However, little data exist examining the relationship between TG and CBF in females with MetSyn. Therefore, we aimed to examine the interaction between sex and MetSyn on global and regional CBF in young adults and to explore the relationship between fasting TG levels and CBF. We predict that males and females with MetSyn will have lower global and regional CBF compared to healthy controls of the same sex, but to a greater extent in females with MetSyn. Additionally, we predict fasting TG levels will have stronger relationships with CBF in females than in males with MetSyn. Twenty-five healthy controls (CON; 7 females; 24 ± 4 y) and 13 individuals with MetSyn (5 females; 27 ± 8 y) completed this study. Subjects were unmedicated, and females were studied on days 1-5 of their menstrual cycle. A metabolic panel, blood pressure, and waist circumference were collected in a fasted state. Magnetic resonance imaging (MRI, 3 Tesla) with arterial spin labeling was used to quantify global and regional CBF. Two-way (Group x Sex) linear models with Shaffer’s correction and Tukey adjusted post hocs were used to determine differences in CBF. In the MetSyn groups, Pearson correlation coeffcients were assessed between fasting TG levels and each global and regional CBF value. Global CBF was greater in female CON compared to all groups (p ≤ 0.01), and male CON had greater CBF than male MetSyn (p = 0.03), whereas there was no difference between male and female MetSyn (p = 0.43). Regional analyses resulted in the same pattern of between-group differences in the parietal lobe (p = 0.03), the precuneus (p = 0.01), and the frontal lobe (p = 0.01), outside of a nonsignificant difference between male CON and male MetSyn in the frontal lobe (p = 0.12). No interactions existed for the temporal lobe, occipital lobe, or thalamus (p ≥ 0.06). In females with MetSyn, fasting TG levels were moderately negatively correlated with CBF (r = -0.65), whereas a moderate positive relationship existed in males with MetSyn (r = 0.71). These data suggest that MetSyn reduces CBF to a greater extent in females than males, abolishing sex-dependent differences in CBF present in healthy young adults. While female sex is protective against cardiovascular disease in healthy young adults, this is not the case in individuals with MetSyn. Additionally, males and females with MetSyn had opposing relationships between fasting TG levels and CBF. These findings may help explain the greater cardiovascular risk in females with insulin resistance. American Diabetes Association (ADA1-16-ICTS-099 and ADA 1-12-IN-39); Clinical and Translational Science Award program (UL1TR002373); Wisconsin National Primate Research Center (P51OD011106). This is the full abstract presented at the American Physiology Summit 2024 meeting and is only available in HTML format. There are no additional versions or additional content available for this abstract. Physiology was not involved in the peer review process.

The uses of 3-dimensional printing technology in orthodontic offices in North America

The purpose of this study was to examine the use of orthodontic 3-dimensional (3D) printing technology in North America and to understand why orthodontists are, or are not, incorporating 3D printing technology in their practices. A survey questionnaire was delivered on a secure online platform, RedCap (Case Western Reserve University Clinical and Translational Science Award; no. UL1TR002548). The survey consisted of 14-34 items with branching logic. The association between participant demographics and in-house 3D printing was assessed using a chi-square test of independence. A total of 518 responses were recorded. The highest number of responses came from respondents in the 36-45-year age group. Most of the respondents were practice owners; 46.9% had 3D printers in their office. Chi-square tests of independence were performed on the data to see which associations existed. The strongest statistical associations with using an in-house 3D printer are seen with patient load, practice type, years since residency, and orthodontist's position. Approximately 75% of orthodontists use 3D printing technology in some capacity in North America. Major factors that influenced orthodontists to incorporate 3D printing technology into their office were self-interest and research. Major factors that have prevented orthodontists from not incorporating 3D printing technology into their office were space for equipment/ventilation and digital workflow training deficit. Orthodontists use their 3D printers mostly to make plastic retainers from printed models. The strongest associations with using in-house 3D printers are seen in patient load, practice type, years since residency, and orthodontist position. Increasing patient load and being in private practice increases the likelihood of having a 3D printer.

Gut microbiome correlates of recurrent urinary tract infection: a longitudinal, multi-center study

Urinary tract infections (UTI) affect approximately 250 million people annually worldwide. Patients often experience a cycle of antimicrobial treatment and recurrent UTI (rUTI) that is thought to be facilitated by a gut reservoir of uropathogenic Escherichia coli (UPEC). 125 patients with UTI caused by an antibiotic-resistant organism (ARO) were enrolled from July 2016 to May 2019 in a longitudinal, multi-center cohort study. Multivariate statistical models were used to assess the relationship between uropathogen colonization and recurrent UTI (rUTI), controlling for clinical characteristics. 644 stool samples and 895 UPEC isolates were interrogated for taxonomic composition, antimicrobial resistance genes, and phenotypic resistance. Cohort UTI gut microbiome profiles were compared against published healthy and UTI reference microbiomes, as well as assessed within-cohort for timepoint- and recurrence-specific differences. Risk of rUTI was not independently associated with clinical characteristics. The UTI gut microbiome was distinct from healthy reference microbiomes in both taxonomic composition and antimicrobial resistance gene (ARG) burden, with 11 differentially abundant taxa at the genus level. rUTI and non-rUTI gut microbiomes in the cohort did not generally differ, but gut microbiomes from urinary tract colonized patients were elevated in E.coli abundance 7-14 days post-antimicrobial treatment. Corresponding UPEC gut isolates from urinary tract colonizing lineages showed elevated phenotypic resistance against 11 of 23 tested drugs compared to non-colonizing lineages. The gut microbiome is implicated in UPEC urinary tract colonization during rUTI, serving as an ARG-enriched reservoir for UPEC. UPEC can asymptomatically colonize the gut and urinary tract, and post-antimicrobial blooms of gut E.coli among urinary tract colonized patients suggest that cross-habitat migration of UPEC is an important mechanism of rUTI. Thus, treatment duration and UPEC populations in both the urinary and gastrointestinal tract should be considered in treating rUTI and developing novel therapeutics. This work was supported in part by awards from the U.S. Centers for Disease Control and Prevention Epicenter Prevention Program (grant U54CK000482; principal investigator, V.J.F.); to J.H.K. from the Longer Life Foundation (an RGA/Washington University partnership), the National Center for Advancing Translational Sciences (grants KL2TR002346 and UL1TR002345), and the National Institute of Allergy and Infectious Diseases (NIAID) (grant K23A1137321) of the National Institutes of Health (NIH); and to G.D. from NIAID (grant R01AI123394) and the Eunice Kennedy Shriver National Institute of Child Health and Human Development (grant R01HD092414) of NIH. R.T.'s research was funded by the Deutsche Forschungsgemeinschaft (DFG; German Research Foundation; grant 402733540). REDCap is Supported by Clinical and Translational Science Award (CTSA) Grant UL1 TR002345 and Siteman Comprehensive Cancer Center and NCI Cancer Center Support Grant P30 CA091842. The content is solely the responsibility of the authors and does not necessarily represent the official views of the funding agencies.

Altered transcriptomic immune responses of maintenance hemodialysis patients to the COVID-19 mRNA vaccine.

End-stage renal disease (ESRD) patients experience immune compromise characterized by complex alterations of both innate and adaptive immunity, and results in higher susceptibility to infection and lower response to vaccination. This immune compromise, coupled with greater risk of exposure to infectious disease at hemodialysis (HD) centers, underscores the need for examination of the immune response to the COVID-19 mRNA-based vaccines. The immune response to the COVID-19 BNT162b2 mRNA vaccine was assessed in 20 HD patients and cohort-matched controls. RNA sequencing of peripheral blood mononuclear cells was performed longitudinally before and after each vaccination dose for a total of six time points per subject. Anti-spike antibody levels were quantified prior to the first vaccination dose (V1D0) and 7 d after the second dose (V2D7) using anti-spike IgG titers and antibody neutralization assays. Anti-spike IgG titers were additionally quantified 6 mo after initial vaccination. Clinical history and lab values in HD patients were obtained to identify predictors of vaccination response. Transcriptomic analyses demonstrated differing time courses of immune responses, with prolonged myeloid cell activity in HD at 1 wk after the first vaccination dose. HD also demonstrated decreased metabolic activity and decreased antigen presentation compared to controls after the second vaccination dose. Anti-spike IgG titers and neutralizing function were substantially elevated in both controls and HD at V2D7, with a small but significant reduction in titers in HD groups (p<0.05). Anti-spike IgG remained elevated above baseline at 6 mo in both subject groups. Anti-spike IgG titers at V2D7 were highly predictive of 6-month titer levels. Transcriptomic biomarkers after the second vaccination dose and clinical biomarkers including ferritin levels were found to be predictive of antibody development. Overall, we demonstrate differing time courses of immune responses to the BTN162b2 mRNA COVID-19 vaccination in maintenance HD subjects comparable to healthy controls and identify transcriptomic and clinical predictors of anti-spike IgG titers in HD. Analyzing vaccination as an in vivo perturbation, our results warrant further characterization of the immune dysregulation of ESRD. F30HD102093, F30HL151182, T32HL144909, R01HL138628. This research has been funded by the University of Illinois at Chicago Center for Clinical and Translational Science (CCTS) award UL1TR002003.

531 Transforming a Pilot Grant Program to Advance Clinical &amp; Translational Science

OBJECTIVES/GOALS: A new mandate for Clinical &amp; Translational Science Award (CTSA) Programs is for pilot grant funding to support clinical and translational science (CTS) projects that study challenges in the translational research pipeline. This pivot requires new structures and supports to help investigators design and implement high-quality CTS projects. METHODS/STUDY POPULATION: The Michigan Institute for Clinical &amp; Health Research (MICHR) at the University of Michigan (U-M) has launched two rounds of pilot funding since March 2023. Faculty and staff across U-M’s three campuses, community members, and those at collaborating institutions and hospitals were eligible to apply. New pre-award supports included a CTS project framework; a recorded webinar that educated about CTS and the funding opportunity; office hours to provide tailored project feedback; a letter of intent to screen for alignment with CTS; and reviewer training for academic and community reviewers. Funded projects operate like 'mini cooperative agreements”, with MICHR experts partnering with awardees to refine evaluation plans, prepare work products, advise on dissemination, and navigate emergent challenges. RESULTS/ANTICIPATED RESULTS: The first round of funding was launched in the absence of pre-award supports; ten applications we received from faculty proposing translational research rather than CTS. We quickly re-released the FOA, expanding eligibility to staff. We received nine applications, ultimately funding four staff and one faculty studying operational challenges in translation and helping them create robust evaluation plans. We piloted the pre-award supports in our second round, with 40 individuals viewing our webinar and 11 attending office hours. Those who watched the webinar before attending office hours better understood how to embed CTS questions within their programs of research. We recently received 19 letters of intent, addressing both operational and scientific challenges, with 16 eligible to submit applications. DISCUSSION/SIGNIFICANCE: Education and personalized feedback seem to elicit a higher yield of CTS projects. Staff are already adept at solving operational challenges, so the pre-award supports were most critical for faculty accustomed to writing traditional translational research proposals. Staff have most benefited from guidance in evaluation and dissemination.

154 Leveraging Implementation Science Competencies to Establish a D&amp;I Science Core

OBJECTIVES/GOALS: Clinical and Translational Science Award (CTSA) hubs are launching D&amp;I Science cores to provide resources and services to promote the translation of clinical evidence into practice. We developed a D&amp;I Science Core strategic plan reflecting our research community’s needs by assessing Implementation Science (IS) competencies. METHODS/STUDY POPULATION: The Tufts CTSI D&amp;I Science Core was launched in early 2023. To design services that meet research community needs, we conducted a survey and key informant interviews based on Padek etal.’s list of Implementation Science (IS) competencies. The competencies are organized into four domains (Definition, Background, and Rationale; Theory and Approaches; Design &amp; Analysis; and Practice-Based Considerations) and categorized by expertise level (beginner, intermediate, advanced). Participants who had attended or expressed interest in a D&amp;I interest group were asked via an email survey to rate their level of confidence in completing selected IS-related research activities, about their experience with IS research or practice, and the types of resources, services and training they desired. RESULTS/ANTICIPATED RESULTS: Twenty researchers (20/65, 31%) submitted survey responses and six researchers participated in in-depth interviews. Survey respondents felt most confident in engaging stakeholders in IS research and least confident selecting a model or framework for a study. Results suggest that researcher capacity building is needed to: • Understand IS models and frameworks and their approaches, strengths, and limitations • Select and use models and frameworks in studies • Assemble IS teams and prepare grant proposals Suggestions for resources, services, and training, include: • Customized education to address diverse needs, knowledge levels, and learning styles • Promotion of D&amp;I Core consultations and grant support services • Sharing of successful proposals to help researchers learn how to apply IS methods DISCUSSION/SIGNIFICANCE: A strategic workplan for the D&amp;I Science Core was developed and implemented to address the findings. Initial emphasis is on developing easily accessible resources and timely consultations for investigators new to IS needing to apply these methods in current grant proposals, while also providing training resources for deeper skill building.

559 Ideation Jams: Catalyzing Interdisciplinary Teams to Maximize Research Impact

OBJECTIVES/GOALS: There is broad recognition that interdisciplinary research teams are uniquely suited to address complex research questions. At the Michigan Institute for Clinical &amp; Health Research, we recognized a significant gap in support services at the University of Michigan for coordinating interdisciplinary teams to advance translational research. METHODS/STUDY POPULATION: The initial team science challenge we tackled was how to bring together cross-disciplinary groups, for the first time, to engage meaningfully and collaboratively with a ‘wicked’ problem of interest and create a shared vision. To address this, we developed Ideation Jams, which are facilitated experiences that help new groups build community, identify and prioritize research opportunities, determine how individual interests and other potential partners align with opportunities, and commit to next steps that will advance collaborative efforts. Ideation Jams leverage the methods and mindsets of design thinking, including divergence and convergence; making information visual; amplifying diversity; horizontal distribution of responsibility; and bias towards action. RESULTS/ANTICIPATED RESULTS: We have facilitated 11 Ideation Jams with 255 participants, including faculty, staff, health practitioners, and community members, who brought diverse expertise and insight to the research problems. Participant feedback has been overwhelmingly positive, with Ideation Jams fostering shared vision and innovation, and positively impacting various measures related to team performance. Participants have reported that Ideation Jams catalyzed various outcomes, including submission and award of grants, the introduction of new and specialized clinical offerings, and development of an interdisciplinary research agenda for their field of interest. Most recently, we trained representatives from five Clinical &amp; Translational Science Award hubs to implement Ideation Jams at their universities. DISCUSSION/SIGNIFICANCE: Ideation Jams are ideal for mobilizing new groups around complex research problems, moving them from blue-sky thinking to action planning in three hours. Ideation Jams will be integrated into a suite of facilitated experiences, trainings, and grant development services to provide iterative support as teams advance their research priorities.

555 Design Lab Methodology Supports Innovation in Clinical Trials

OBJECTIVES/GOALS: Since 2017, we have used the Design Lab methodology to support investigators taking innovative approaches to clinical effectiveness trial design. To date we have held 12 Design Labs and this year we are creating a handbook that will support dissemination of this approach across the Clinical and Translational Science Award consortium. METHODS/STUDY POPULATION: The Clinical Trial Design Lab brings together a multi-stakeholder group to consider innovative and impactful clinical trial designs. An investigative team is selected from a competitive pool of applicants, after which expert-led consultations support the investigator team to think about evidence generation in the context of the full treatment development pathway. Teams map the stakeholders at each step of this pathway (e.g. clinicians, patients, researchers, funders, industry experts, policy experts, regulatory experts, payers) and consider innovative design solutions. These consultations prepare investigators for an event that involves all stakeholders in a structured and facilitated discussion about trial designs that generate the best evidence and increase potential for health impact. RESULTS/ANTICIPATED RESULTS: The result of our work will be a set of Design Lab principles, a handbook with templates that support stakeholder mapping and structured discussions, and educational resources to accompany the handbook. The work is supported by a literature review that characterizes the multi-component processes included in the Design Lab, situates them within the larger context of team science interventions, and lays groundwork for the development of process metrics and impact evaluation criteria to assess the Design Lab method. In this poster presentation, we will share our multi-component broadly engaged team science approach, provide a brief outline of the principles and educational resources, and include an early version of the evaluation criteria. DISCUSSION/SIGNIFICANCE: Broadly engaged team science supports innovative thinking about study design and is especially important in the development of clinical trials. We have grown the Design Lab program of work over the past seven years and are now able to characterize our team science methodology and support others to use this approach to innovate for health impact.

99 Using LGBTQ+ Community Expertise to Co-Develop Inclusive Sexual Orientation and Gender Identity (SOGI) Screening for Research Studies

OBJECTIVES/GOALS: To promote diverse research engagement and address health disparities by creating an inclusive tool to collect sexual orientation and gender identity (SOGI) data from potential participants #_msoanchor_1 METHODS/STUDY POPULATION: The Penn State Community Health Equity &amp; Engagement in Research (CHEER) team, part of our Clinical and Translational Science Institute (CTSI), developed inclusive screening guidance to collect SOGI data from potential research participants to fill an identified gap in the literature. Guidance was developed through an iterative feedback process, leveraging expertise from local, regional, and national organizations, healthcare systems, and leaders throughout Clinical &amp; Translational Science Award hubs. By eliciting expert feedback, CHEER co-developed a comprehensive SOGI data collection form, filling an important gap of inclusivity in the consenting process. Training of this new tool was delivered to CHEER’s far-reaching listserv researchers (internal and external) and community partners. RESULTS/ANTICIPATED RESULTS: Feedback collected from our LGBTQ+ expert partners resulted in a total of five inclusive SOGI screening questions; two ‘Gender Identity’ questions, one ‘Sexual Orientation’ question, and two ‘Sex’ questions, with “prefer not to answer” and “another option not listed” provided. The goal of this effort is to equip research teams with a tool that integrates SOGI characteristics that may be particularly important to determine study eligibility respective to a person’s identity or orientation. Additionally, collecting SOGI data in an inclusive way may increase trust worthiness in research from potential research participants, particularly among the LGBTQ+ community, who have been underrepresented yet experience several inequities and disparities across multiple health outcomes. DISCUSSION/SIGNIFICANCE: CHEER’s goal is to reduce health disparities in underrepresented populations, including the LGBTQ+ community, by promoting inclusivity and engagement in research. Developing a community-driven screening that addresses the unique needs of the LGBTQ+ community successfully bridges a gap in equity across all research participants.

Implementation and Evaluation of a National Well-Being Curriculum for KL2 Scholars.

Introduction: The study aimed to pilot test a well-being curriculum for KL2 scholars to be used across the Clinical and Translational Science Award consortium. Methods: Between November 2022, and May 2023, 36 KL2 scholars from 25 hubs participated in the program. The General Well-Being Index for U.S. Workers and the Patient Reported Outcomes Measurement Information System (PROMIS-29) were completed by scholars before and after the program. Results: Postparticipation, there was a trend of improvement in the domains of well-being, sleep, anxiety, and fatigue. Conclusion: Implementing a virtual synchronous well-being curriculum allowed the scholars to connect across the consortium and improve their well-being.

Development of TRACER: A Translational Research Accomplishments Cataloguerfor Clinical and Translational Science Award hub activity tracking, evaluation, and decision-making.

Organizations supporting translational research and translational science, including Clinical and Translational Science Award (CTSA) hubs, provide a diverse and often changing array of resources, support, and services to a myriad of researchers and research efforts. While a wide-ranging scope of programs is essential to the advancement of translational research and science, it also complicates a systematic and unified process for tracking activities, studying research processes, and examining impact. To overcome these challenges, the Duke University School of Medicine's CTSA hub created a data platform, Translational Research Accomplishment Cataloguer (TRACER), that provides capacity to enhance strategic decision-making, impact assessment, and equitable resource distribution. This article reviews TRACER development processes, provides an overview of the TRACER platform, addresses challenges in the development process, and describes avenues for addressing or overcoming these challenges. TRACER development allowed our hub to conceptually identify key processes and goals within programs and linkages between programs, and it sets the stage for advancing evidence-based improvement across our hub. This platform development provides key insight into facilitators that can inform other initiatives seeking to collect and align organizational data for strategic decision-making and impact assessment. TRACER or similar platforms are additionally well positioned to advance the study of translational science.

Translation in action: Influence, collaboration, and evolution of COVID-19 research with Clinical and Translational Science Awards consortium support.

The National Institutes of Health (NIH)'s Clinical and Translational Science Awards (CTSA) consortium aims to accelerate translational processes that move discoveries from bench to bedside. The coronavirus disease 2019 (COVID-19) pandemic presented unmatched challenges and applications for CTSA hubs nationwide. Our study used bibliometrics to assess features of COVID-19 publications supported by the national CTSA program to characterize the consortium's response to the pandemic. Our goal was to understand relative scientific influence, collaboration across hubs, and trends in research emphasis over time. We identified publications from NIH's curated iSearch COVID-19 Publication Portfolio from February 2020 to February 2023; 3234 peer-reviewed articles relevant to COVID-19 cited a CTSA grant. All 66 CTSA hubs were represented, with large-size and longstanding hubs contributing more publications. Most publications cited UL1 grants, 457 cited KL2/TL1 training grants, and 164 cited multiple hub grants. Compared to a random sample of non-CTSA-supported COVID-19 publications, the CTSA portfolio exhibited greater clinical relevance, more human research, and higher altmetric and citation influence. Results were similar for multi-hub publications involving networked initiatives like multi-site clinical trials or the National COVID-19 Cohort Collaborative. Shifts from molecular/cellular-oriented research toward human-oriented research over time were evident, demonstrating translation in action. Results illuminate how the CTSA consortium confronted the pandemic through high-quality projects oriented toward human research, working across hubs on high-value collaborations, advancing along the translational spectrum over time. Findings validate CTSA hubs as critical support structures during health emergencies.

A national unmet needs assessment for CTSA-affiliated electronic health record data networks: A customer discovery approach.

The expansion of electronic health record (EHR) data networks over the last two decades has significantly improved the accessibility and processes around data sharing. However, there lies a gap in meeting the needs of Clinical and Translational Science Award (CTSA) hubs, particularly related to real-world data (RWD) and real-world evidence (RWE). We adopted a mixed-methods approach to construct a comprehensive needs assessment that included: (1) A Landscape Context analysis to understand the competitive environment; and (2) Customer Discovery to identify stakeholders and the value proposition related to EHR data networks. Methods included surveys, interviews, and a focus group. Thirty-two CTSA institutions contributed data for analysis. Fifty-four interviews and one focus group were conducted. The synthesis of our findings pivots around five emergent themes: (1) CTSA segmentation needs vary according to resources; (2) Team science is key for success; (3) Quality of data generates trust in the network; (4) Capacity building is defined differently by researcher career stage and CTSA existing resources; and (5) Researchers' unmet needs. Based on the results, EHR data networks like ENACT that would like to meet the expectations of academic research centers within the CTSA consortium need to consider filling the gaps identified by our study: foster team science, improve workforce capacity, achieve data governance trust and efficiency of operation, and aid Learning Health Systems with validating, applying, and scaling the evidence to support quality improvement and high-value care. These findings align with the NIH NCATS Strategic Plan for Data Science.

A landscape assessment of CTSA evaluators and their work in the CTSA consortium, 2021 survey findings.

This article presents a landscape assessment of the findings from the 2021 Clinical and Translational Science Award (CTSA) Evaluators Survey. This survey was the most recent iteration of a well established, national, peer-led systematic snapshot of the CTSA evaluators, their skillsets, listed evaluation resources, preferred methods, and identified best practices. Three questions guided our study: who are the CTSA evaluators, what competencies do they share and how is their work used within hubs. We describe our survey process (logistics of development, deployment, and differences in historical context with prior instruments); and present its main findings. We provide specific recommendations for evaluation practice in two main categories (National vs Group-level) including, among others, the need for a national, strategic plan for evaluation as well as enhanced mentoring and training of the next generation of evaluators. Although based on the challenges and opportunities currently within the CTSA Consortium, takeaways from this study constitute important lessons with potential for application in other large evaluation consortia. To our knowledge, this is the first time 2021 survey findings are disseminated widely, to increase transparency of the CTSA evaluators' work and to motivate conversations within hub and beyond, as to how best to leverage existent evaluative capacity.

Developing a shared understanding of translational science within CTSA hubs through facilitated retreats: A case study.

Translation of critical and broadly impactful health advancements is stymied by insufficient scientific scrutiny of barriers and roadblocks in the process. The Clinical & Translational Science Award (CTSA) funding opportunity announcement released in July 2021 makes clear the distinction between translational research and translational science (TS) and urges a shift from the former to the latter. This represents a significant shift in the overall scientific direction of the CTSA program and necessitates corresponding shifts in CTSA hub operations. To better support TS, the Team Science Core of the Duke CTSA hub designed and facilitated a virtual retreat for hub personnel that (1) enabled organizational learning about TS and (2) identified anticipated challenges and opportunities. A post-retreat survey was utilized to assess the degree to which the retreat met its stated goals. Our survey received a 62% response rate; 100% of respondents would recommend the session to others. Respondents also reported gains in all areas assessed, with evidence for greater understanding of TS and increased perspective of the value and relevance of TS. In this paper, we provide a roadmap for designing and implementing facilitated TS retreats, which we argue is a key step in TS capacity building through workforce development.