Deep Learning Language Model Research Articles

Determining key residues of engineered scFv antibody variants with improved MMP-9 binding using deep sequencing and machine learning

Given the crucial role of specific matrix metalloproteinases (MMPs) in the extracellular matrix, an imbalance in the regulation of activation of matrix metalloproteinase-9 (MMP-9) zymogen and inhibition of the enzyme can result in various diseases, such as cancer, neurodegenerative, and gynecological diseases. Thus, developing novel therapeutics that target MMP-9 with single-chain antibody fragments (scFvs) is a promising approach. We used fluorescent-activated cell sorting (FACS) to screen a synthetic scFv antibody library displayed on yeast for enhanced binding to MMP-9. The screened scFv mutants demonstrated improved binding to MMP-9 compared to the natural inhibitor of MMPs, tissue inhibitor of metalloproteinases (TIMPs). To identify the molecular determinants of these engineered scFv variants that affect binding to MMP-9, we used next-generation DNA sequencing and computational protein structure analysis. Additionally, a deep-learning language model was trained on the screened scFv library of variants to predict the binding affinities of scFv variants based on their CDR-H3 sequences.

Automated Identification of HeartFailureWith Reduced Ejection Fraction Using Deep Learning-Based Natural Language Processing.

The lack of automated tools for measuring care quality limits the implementation of a national program to assess guideline-directed care in heart failure with reduced ejection fraction (HFrEF). The authors aimed to automate the identification of patients with HFrEF at hospital discharge, an opportunity to evaluate and improve the quality of care. The authors developed a novel deep-learning language model for identifying patients with HFrEF from discharge summaries of hospitalizations with heart failure at Yale New Haven Hospital during 2015 to 2019. HFrEF was defined by left ventricular ejection fraction<40% on antecedent echocardiography. The authors externally validated the model at Northwestern Medicine, community hospitals of Yale, and the MIMIC-III (Medical Information Mart for Intensive Care III) database. A total of 13,251 notes from 5,392 unique individuals (age 73 ± 14 years, 48% women), including 2,487 patients with HFrEF (46.1%), were used for model development (train/held-out: 70%/30%). The model achieved an area under receiver-operating characteristic curve (AUROC) of 0.97 and area under precision recall curve (AUPRC) of 0.97 in detecting HFrEF on the held-out set. The model had high performance in identifying HFrEF with AUROC=0.94 and AUPRC=0.91 on 19,242 notes from Northwestern Medicine, AUROC=0.95 and AUPRC=0.96 on 139 manually abstracted notes from Yale community hospitals, and AUROC=0.91 and AUPRC=0.92 on 146 manually reviewed notes from MIMIC-III. Model-based predictions of HFrEF corresponded to a net reclassification improvement of 60.2 ± 1.9% compared with diagnosis codes (P< 0.001). The authors developed a language model that identifies HFrEF from clinical notes with high precision and accuracy, representing a key element in automating quality assessment for individuals with HFrEF.

Elucidating key determinants of engineered scFv antibody in MMP-9 binding using high throughput screening and machine learning.

An imbalance in matrix metalloproteinase-9 (MMP-9) regulation can lead to numerous diseases, including neurological disorders, cancer, and pre-term labor. Engineering single-chain antibody fragments (scFvs) Targeting MMP-9 to develop novel therapeutics for such diseases is desirable. We screened a synthetic scFv antibody library displayed on the yeast surface for binding improvement to MMP-9 using FACS (fluorescent-activated cell sorting). The scFv antibody clones isolated after FACS showed improvement in binding to MMP-9 compared to the endogenous inhibitor. To understand molecular determinants of binding between engineered scFv antibody variants and MMP-9, next-generation DNA sequencing, and computational protein structure analysis were used. Additionally, a deep-learning language model was trained on the synthetic library to predict the binding of scFv variants using their CDR-H3 sequences.

The application of natural language processing for the extraction of mechanistic information in toxicology.

To study the ways in which compounds can induce adverse effects, toxicologists have been constructing Adverse Outcome Pathways (AOPs). An AOP can be considered as a pragmatic tool to capture and visualize mechanisms underlying different types of toxicity inflicted by any kind of stressor, and describes the interactions between key entities that lead to the adverse outcome on multiple biological levels of organization. The construction or optimization of an AOP is a labor intensive process, which currently depends on the manual search, collection, reviewing and synthesis of available scientific literature. This process could however be largely facilitated using Natural Language Processing (NLP) to extract information contained in scientific literature in a systematic, objective, and rapid manner that would lead to greater accuracy and reproducibility. This would support researchers to invest their expertise in the substantive assessment of the AOPs by replacing the time spent on evidence gathering by a critical review of the data extracted by NLP. As case examples, we selected two frequent adversities observed in the liver: namely, cholestasis and steatosis denoting accumulation of bile and lipid, respectively. We used deep learning language models to recognize entities of interest in text and establish causal relationships between them. We demonstrate how an NLP pipeline combining Named Entity Recognition and a simple rules-based relationship extraction model helps screen compounds related to liver adversities in the literature, but also extract mechanistic information for how such adversities develop, from the molecular to the organismal level. Finally, we provide some perspectives opened by the recent progress in Large Language Models and how these could be used in the future. We propose this work brings two main contributions: 1) a proof-of-concept that NLP can support the extraction of information from text for modern toxicology and 2) a template open-source model for recognition of toxicological entities and extraction of their relationships. All resources are openly accessible via GitHub (https://github.com/ontox-project/en-tox).

#let’s-discuss: Analyzing Students’ Use of Emoji when Interacting with Course Readings

AbstractEmoji are commonly used in social media to convey affects, emotions, and attitudes. While popular in social media, their use in educational contexts has been sparsely studied even though emoji can be a natural way for students to express what they are feeling about the learning material. This paper studies how students use instructor-selected emoji when relating to and engaging with educational content. We use an online platform for collaborative annotations where discussions are embedded into the readings for the course. We also make it possible for students to use 11 unique emoji-hashtag pairings to express their thoughts and feelings about the readings and the ongoing discussion. We provide an empirical analysis of the usage of these emoji-hashtag pairs by over 1,800 students enrolled in different offerings of an introductory biology course from multiple academic terms. We also introduce a heat map, which allows the instructional team to visualize the distribution and types of emoji used by students in different parts of the reading material. To evaluate the heat map, we conducted a user study with five instructors/TAs. We found that instructors/TAs use the heat map as a tool for identifying textbook sections that students find difficult and/or interesting and plan to use it to help them design the online content for future classes. Finally, we introduce a computational analysis for predicting emoji/hashtag pairs based on the content of a given student post. We use pre-trained deep learning language models (BERT) to predict the emoji attached to a student’s post and then study the extent to which this model generated in an introductory biology course can be generalized to predict student emoji usage in other courses.

Automated annotation of disease subtypes

BackgroundDistinguishing diseases into distinct subtypes is crucial for study and effective treatment strategies. The Open Targets Platform (OT) integrates biomedical, genetic, and biochemical datasets to empower disease ontologies, classifications, and potential gene targets. Nevertheless, many disease annotations are incomplete, requiring laborious expert medical input. This challenge is especially pronounced for rare and orphan diseases, where resources are scarce. MethodsWe present a machine learning approach to identifying diseases with potential subtypes, using the approximately 23,000 diseases documented in OT. We derive novel features for predicting diseases with subtypes using direct evidence. Machine learning models were applied to analyze feature importance and evaluate predictive performance for discovering both known and novel disease subtypes. ResultsOur model achieves a high (89.4%) ROC AUC (Area Under the Receiver Operating Characteristic Curve) in identifying known disease subtypes. We integrated pre-trained deep-learning language models and showed their benefits. Moreover, we identify 515 disease candidates predicted to possess previously unannotated subtypes. ConclusionsOur models can partition diseases into distinct subtypes. This methodology enables a robust, scalable approach for improving knowledge-based annotations and a comprehensive assessment of disease ontology tiers. Our candidates are attractive targets for further study and personalized medicine, potentially aiding in the unveiling of new therapeutic indications for sought-after targets.

Toward understanding the role of genomic repeat elements in neurodegenerative diseases.

Neurodegenerative diseases cause great medical and economic burdens for both patients and society; however, the complex molecular mechanisms thereof are not yet well understood. With the development of high-coverage sequencing technology, researchers have started to notice that genomic repeat regions, previously neglected in search of disease culprits, are active contributors to multiple neurodegenerative diseases. In this review, we describe the association between repeat element variants and multiple degenerative diseases through genome-wide association studies and targeted sequencing. We discuss the identification of disease-relevant repeat element variants, further powered by the advancement of long-read sequencing technologies and their related tools, and summarize recent findings in the molecular mechanisms of repeat element variants in brain degeneration, such as those causing transcriptional silencing or RNA-mediated gain of toxic function. Furthermore, we describe how in silico predictions using innovative computational models, such as deep learning language models, could enhance and accelerate our understanding of the functional impact of repeat element variants. Finally, we discuss future directions to advance current findings for a better understanding of neurodegenerative diseases and the clinical applications of genomic repeat elements.

DTT: An Example-Driven Tabular Transformer for Joinability by Leveraging Large Language Models

Many organizations rely on data from government and third-party sources, and those sources rarely follow the same data formatting. This introduces challenges in integrating data from multiple sources or aligning external sources with internal databases. Commercial database systems do not offer adequate support for integrating data from heterogeneous sources, and manual integration is both time-consuming and inefficient. State-of-the-art data integration approaches that rely on similarity functions and textual transformations often fail to handle challenging cases where multiple mappings are required, or the mappings go beyond simple textual transformations. In this paper, we study the potentials of deep neural models for transforming tables for joinability. In particular, we cast the problem as a prediction task and develop a framework that leverages large deep-learning language models to transform tabular data from a source formatting to a desired target representation. Our framework can efficiently learn the patterns for mapping a source formatting into an expected target using just a few examples, which can then be used for tasks such as table joining, filling in missing values, and error detection. Compared to state-of-the-art mapping and joining approaches, our framework delivers noticeably more accurate and scalable performance on both real-world and synthetic datasets. Our experimental evaluation also shows that the performance of the proposed framework using our fine-tuned model is at par or better than large language models such as GPT-3, despite the significant difference in size, and that using large language models within our framework improves their performance.

Application of GPT-4 foundation model for risk prediction and stratification in colorectal cancer.

219 Background: Deep Learning Language Models (LLMs) show significant potential in analyzing complex data, particularly in oncology, where they can predict and stratify patient risks, detect non-linear interactions, and weigh multiple variables. This study applies LLMs to stage II/III localized colon cancer patients to assess their effectiveness in identifying risk subgroups. Methods: In this retrospective study, we examined a cohort of stage II and III localized colorectal cancer patients diagnosed from September 2016 to December 2022 at La Paz University Hospital. Utilizing the GPT-4 API beta version, patients were stratified into high and low-risk groups for relapse based on various risk parameters outlined in the table. Notably, adjuvant treatment data was excluded from the input. The model was prompted with the question, "Given the following risk parameters for localized colorectal cancer {param}, guess whether the patient is in low risk or high risk for relapse," and the model provided dichotomic responses categorized as either 1 (high risk) or 0 (low risk). Results: Patient characteristics are depicted in the table. During a median follow-up of 23.68 months, 63 recurrence events and 73 deaths were observed. The GPT-4 model stratified patients into low (37.9%) and high-risk (62.1%) groups, with the survival curves showing a significant difference in outcomes between the groups. Low-risk patients had a relapse-free rate of 93% at 36 months vs 63% in the high-risk group. In univariate analysis, the high-risk variable had a hazard ratio (HR) of 4.26 (95% CI: 2.03-8.95; p &lt; 0.005). When stratified by adjuvant treatment, no significant difference in relapse probability was observed for those receiving therapy. However, without adjuvant therapy, high-risk patients had an HR of 12.41 (95% CI: 3.72-41.97; p &lt; 0.005) for relapse. Conclusions: GPT-4 demonstrated its potential in predicting oncology outcomes by accurately understanding colorectal cancer risk factors. To fully harness the benefits of LLMs in the future of oncology, they must be fine-tuned on extensive oncological datasets to improve patient risk stratification and treatment selection, ultimately leading to better patient outcomes.[Table: see text]

Classification of helical polymers with deep-learning language models

Many macromolecules in biological systems exist in the form of helical polymers. However, the inherent polymorphism and heterogeneity of samples complicate the reconstruction of helical polymers from cryo-EM images. Currently, available 2D classification methods are effective at separating particles of interest from contaminants, but they do not effectively differentiate between polymorphs, resulting in heterogeneity in the 2D classes. As such, it is crucial to develop a method that can computationally divide a dataset of polymorphic helical structures into homogenous subsets. In this work, we utilized deep-learning language models to embed the filaments as vectors in hyperspace and group them into clusters. Tests with both simulated and experimental datasets have demonstrated that our method – HLM (Helical classification with Language Model) can effectively distinguish different types of filaments, in the presence of many contaminants and low signal-to-noise ratios. We also demonstrate that HLM can isolate homogeneous subsets of particles from a publicly available dataset, resulting in the discovery of a previously unreported filament variant with an extra density around the tau filaments.

Abstract 16207: Deep Learning-Based Natural Language Processing of Discharge Summaries for Automated Identification of Heart Failure With Reduced Ejection Fraction

Background: Implementation of a national program to improve guideline-directed care in heart failure with reduced ejection fraction (HFrEF) has been limited by the lack of automated tools to measure care quality. A key challenge for this approach is to identify patients with HFrEF at hospital discharge, an opportunity to evaluate and improve care. Methods: We developed a novel deep-learning language model to detect patients with HFrEF from discharge summaries using a semi-supervised learning framework. Hospitalizations with heart failure (HF) at Yale New Haven Hospital between 2015-2019 were labeled as HFrEF if left ventricular ejection fraction was under 40% on antecedent echocardiography. The model was internally validated with model-based net reclassification improvement (NRI) assessed against chart-based diagnosis codes. We externally validated the model on discharge summaries from hospitalizations with HF at Northwestern Medicine, community hospitals of Yale Health, and MIMIC-III database, confirmed with chart abstraction. Results: A total of 13,251 notes from 5,392 individuals (age 73 ± 14 years, 48% female), including 2,487 (46.1%) patients with HFrEF were used for model development (train/validation/test: 70/15/15%). The model achieved an AUROC 0.97 and AUPRC 0.97 in detecting HFrEF on the test set. In external validation, the model had high performance in identifying HFrEF with AUROC 0.91-0.95 and AUPRC 0.91-0.96 among 19,242 discharge summaries at Northwestern Medicine, 139 notes from Yale community hospitals, and 146 notes from MIMIC-III. Model predictions corresponded to an NRI of 28 ± 7% compared to chart diagnosis (p&lt; 0.001) and identified significant gaps in the use of guideline-directed therapies (Figure). Conclusions: We developed and validated an NLP model that automates the identification of HFrEF from clinical notes with high precision and accuracy, representing a key element to automate quality-of-care measurements in HFrEF.

Bayesian Surprise Predicts Human Event Segmentation in Story Listening.

Event segmentation theory posits that people segment continuous experience into discrete events and that event boundaries occur when there are large transient increases in prediction error. Here, we set out to test this theory in the context of story listening, by using a deep learning language model (GPT-2) to compute the predicted probability distribution of the next word, at each point in the story. For three stories, we used the probability distributions generated by GPT-2 to compute the time series of prediction error. We also asked participants to listen to these stories while marking event boundaries. We used regression models to relate the GPT-2 measures to the human segmentation data. We found that event boundaries are associated with transient increases in Bayesian surprise but not with a simpler measure of prediction error (surprisal) that tracks, for each word in the story, how strongly that word was predicted at the previous time point. These results support the hypothesis that prediction error serves as a control mechanism governing event segmentation and point to important differences between operational definitions of predictionerror.

A Novel Deep Learning Language Model with Hybrid-GFX Embedding and Hyperband Search for Opinion Analysis

A Novel Deep Learning Language Model with Hybrid-GFX Embedding and Hyperband Search for Opinion Analysis

AI Trust Framework and Maturity Model: Improving Security, Ethics and Trust in AI

The following article develops an AI Trust Framework and Maturity Model (AI-TFMM) to improve trust in AI technologies used by Autonomous Human Machine Teams Systems (A-HMT-S). The framework establishes a methodology to improve quantification of trust in AI technologies. Key areas of exploration include security, privacy, explainability, transparency and other requirements for AI technologies to be ethical in their development and application. A maturity model framework approach to measuring trust is applied to improve gaps in quantifying trust and associated metrics of evaluation. Finding the right balance between performance, governance and ethics also raises several critical questions on AI technology and trust. Research examines methods needed to develop an AI-TFMM and validates it against a popular AI technology (Chat GPT). OpenAI's GPT, which stands for "Generative Pre-training Transformer," is a deep learning language model that can generate human-like text by predicting the next word in a sequence based on a given prompt. ChatGPT is a version of GPT that is tailored for conversation and dialogue, and it has been trained on a dataset of human conversations to generate responses that are coherent and relevant to the context. The article concludes with results and conclusions from testing the AI Trust Framework and Maturity Model (AI-TFMM) applied to AI technology. Based on these findings, this paper highlights gaps that could be filled with future research to improve the accuracy, efficacy, application, and methodology of the AI-TFMM.

Composition Based Oxidation State Prediction of Materials Using Deep Learning Language Models.

Oxidation states (OS) are the charges on atoms due to electrons gained or lost upon applying an ionic approximation to their bonds. As a fundamental property, OS has been widely used in charge-neutrality verification, crystal structure determination, and reaction estimation. Currently, only heuristic rules exist for guessing the oxidation states of a given compound with many exceptions. Recent work has developed machine learning models based on heuristic structural features for predicting the oxidation states of metal ions. However, composition-based oxidation state prediction still remains elusive so far, which has significant implications for the discovery of new materials for which the structures have not been determined. This work proposes a novel deep learning-based BERT transformer language model BERTOS for predicting the oxidation states for all elements of inorganic compounds given only their chemical composition. This model achieves 96.82% accuracy for all-element oxidation states prediction benchmarked on the cleaned ICSD dataset and achieves 97.61% accuracy for oxide materials. It is also demonstrated how it can be used to conduct large-scale screening of hypothetical material compositions for materials discovery.

Leveraging pre-trained language models for mining microbiome-disease relationships

BackgroundThe growing recognition of the microbiome’s impact on human health and well-being has prompted extensive research into discovering the links between microbiome dysbiosis and disease (healthy) states. However, this valuable information is scattered in unstructured form within biomedical literature. The structured extraction and qualification of microbe-disease interactions are important. In parallel, recent advancements in deep-learning-based natural language processing algorithms have revolutionized language-related tasks such as ours. This study aims to leverage state-of-the-art deep-learning language models to extract microbe-disease relationships from biomedical literature.ResultsIn this study, we first evaluate multiple pre-trained large language models within a zero-shot or few-shot learning context. In this setting, the models performed poorly out of the box, emphasizing the need for domain-specific fine-tuning of these language models. Subsequently, we fine-tune multiple language models (specifically, GPT-3, BioGPT, BioMedLM, BERT, BioMegatron, PubMedBERT, BioClinicalBERT, and BioLinkBERT) using labeled training data and evaluate their performance. Our experimental results demonstrate the state-of-the-art performance of these fine-tuned models ( specifically GPT-3, BioMedLM, and BioLinkBERT), achieving an average F1 score, precision, and recall of over >0.8 compared to the previous best of 0.74.ConclusionOverall, this study establishes that pre-trained language models excel as transfer learners when fine-tuned with domain and problem-specific data, enabling them to achieve state-of-the-art results even with limited training data for extracting microbiome-disease interactions from scientific publications.

O-089 Using ChatGPT to answer patient questions about fertility: the quality of information generated by a deep learning language model

Abstract Study question What is the quality of information provided by ChatGPT when using common patient questions as prompts? Summary answer Overall, the quality of the information generated by ChatGPT was high with little evidence of commercial bias. What is known already People seeking fertility-related information rely on internet sources when deciding on reproductive planning and assisted conception. The quality of information within the commercial landscape of infertility treatment is poor. ChatGPT, a variant of Generative Pre-trained Transformer v3 (GPT-3), is a language model that uses deep learning to generate human-like text. Given prompts, it generates answers by predicting the next word in the sequence based on patterns learned from training data. The training data for GPT-3 is not curated, but a snapshot of the Web, which includes all kinds of information, including biases that may exist within sources. Study design, size, duration Ten common patient questions were used as prompts. Three questions related to fertility awareness (impact of female/ male age on fertility and fertile window in the menstrual cycle), one to the chance of success with IVF, one to elective egg freezing, one to the benefits of add-ons, one to PCOS and pregnancy, one to choosing a fertility clinic, and one to how many IVF cycles should be attempted. Participants/materials, setting, methods Two experts independently scored the quality of the information generated by the ChatGPT using a scoring matrix with a range of 0 to 7 where higher scores indicate higher quality. Text was rated against humanistic answers based on how well it corresponded (0-3), evidence of commercial bias or controversial claims (no = 1, yes=0), use of accurate proportions/ statistics and whether it was stated that medical advice should be sought (yes=1, no = 0). Main results and the role of chance The scores returned by the two experts were closely aligned with only one point difference for one of the answers. This discrepancy was resolved through discussion. While none of the answers received the maximum score of 7, 6/10 scored 5 or more and 3 received a score of 3-4. Only one answer, the answer to the question about the benefits of add-ons, scored less than 3. This was also the only question where the response had evidence of commercial bias and one of only two that made claims that could be considered controversial. Limitations, reasons for caution The scoring method used in this study has not been validated and is exploratory in nature as this area of evaluation is emerging. However, the use of expert evaluation is common when assessing the performance of machine learning models and often used to fine-tune their parameters and improve their performance. Wider implications of the findings It is known that people seeking fertility-related information rely heavily on online sources such as clinic websites, consumer advocacy organisations, patient support groups and social media. Our findings suggests that ChatGPT may be a useful tool for patients seeking factual and unbiased information regarding fertility and fertility treatment. Trial registration number Not applicable

Ensemble of deep learning language models to support the creation of living systematic reviews for the COVID-19 literature

BackgroundThe COVID-19 pandemic has led to an unprecedented amount of scientific publications, growing at a pace never seen before. Multiple living systematic reviews have been developed to assist professionals with up-to-date and trustworthy health information, but it is increasingly challenging for systematic reviewers to keep up with the evidence in electronic databases. We aimed to investigate deep learning-based machine learning algorithms to classify COVID-19-related publications to help scale up the epidemiological curation process.MethodsIn this retrospective study, five different pre-trained deep learning-based language models were fine-tuned on a dataset of 6365 publications manually classified into two classes, three subclasses, and 22 sub-subclasses relevant for epidemiological triage purposes. In a k-fold cross-validation setting, each standalone model was assessed on a classification task and compared against an ensemble, which takes the standalone model predictions as input and uses different strategies to infer the optimal article class. A ranking task was also considered, in which the model outputs a ranked list of sub-subclasses associated with the article.ResultsThe ensemble model significantly outperformed the standalone classifiers, achieving a F1-score of 89.2 at the class level of the classification task. The difference between the standalone and ensemble models increases at the sub-subclass level, where the ensemble reaches a micro F1-score of 70% against 67% for the best-performing standalone model. For the ranking task, the ensemble obtained the highest recall@3, with a performance of 89%. Using an unanimity voting rule, the ensemble can provide predictions with higher confidence on a subset of the data, achieving detection of original papers with a F1-score up to 97% on a subset of 80% of the collection instead of 93% on the whole dataset.ConclusionThis study shows the potential of using deep learning language models to perform triage of COVID-19 references efficiently and support epidemiological curation and review. The ensemble consistently and significantly outperforms any standalone model. Fine-tuning the voting strategy thresholds is an interesting alternative to annotate a subset with higher predictive confidence.

Unveiling the Thematic Landscape of Generative Pre-trained Transformer (GPT) Through Bibliometric Analysis

Introduction: the Generative Pre-trained Transformer (GPT) is a deep learning language model architecture developed by OpenAI. Aim: to describe the knowledge networks (both at the theoretical and country levels) of the Generative Pre-trained Transformer (GPT) as an emerging technology. Results: 222 Documents were identified, of which 69 were articles, 50 were conference papers, 36 were editorials, 29 were notes, 19 were letters, 14 were reviews, 3 were conference reviews, and 2 were short surveys. In terms of the number of documents per year, 2 were found in 2019, 10 in 2020, 22 in 2021, 44 in 2022, and 144 in 2023. The year-on-year growth rate was over 100 % in all years. The subject area with the highest number of documents was Computer Science with 90 documents. The most productive countries in relation to GPT were the United States with 60 documents, followed by China with 19, the United Kingdom with 18, India with 15, and Australia with 12. Co-occurrence illustrated the centrality of Artificial Intelligence, Natural Language Processing, Deep Learning, and the term Human around ChatGPT and GPT. Conclusions: this bibliometric study aimed to describe the knowledge networks of the Generative Pre-trained Transformer (GPT) as an emerging technology. Although only 222 documents were found, this study revealed a high level of international scientific collaboration in the field. The results suggest that GPT is a highly relevant technology with a wide range of potential applications in natural language processing, artificial intelligence, and deep learning. Moreover, the study was able to qualitatively characterize the main thematic areas surrounding GPT, including its applications in chatbots, text generation, machine translation, sentiment analysis, and more.

Learning to Read and Write in the Language of Proteins

Learning to Read and Write in the Language of Proteins