Pre-trained Transformer Research Articles

Multimodal Affective Communication Analysis: Fusing Speech Emotion and Text Sentiment Using Machine Learning

Affective communication, encompassing verbal and non-verbal cues, is crucial for understanding human interactions. This study introduces a novel framework for enhancing emotional understanding by fusing speech emotion recognition (SER) and sentiment analysis (SA). We leverage diverse features and both classical and deep learning models, including Gaussian naive Bayes (GNB), support vector machines (SVMs), random forests (RFs), multilayer perceptron (MLP), and a 1D convolutional neural network (1D-CNN), to accurately discern and categorize emotions in speech. We further extract text sentiment from speech-to-text conversion, analyzing it using pre-trained models like bidirectional encoder representations from transformers (BERT), generative pre-trained transformer 2 (GPT-2), and logistic regression (LR). To improve individual model performance for both SER and SA, we employ an extended dynamic Bayesian mixture model (DBMM) ensemble classifier. Our most significant contribution is the development of a novel two-layered DBMM (2L-DBMM) for multimodal fusion. This model effectively integrates speech emotion and text sentiment, enabling the classification of more nuanced, second-level emotional states. Evaluating our framework on the EmoUERJ (Portuguese) and ESD (English) datasets, the extended DBMM achieves accuracy rates of 96% and 98% for SER, 85% and 95% for SA, and 96% and 98% for combined emotion classification using the 2L-DBMM, respectively. Our findings demonstrate the superior performance of the extended DBMM for individual modalities compared to individual classifiers and the 2L-DBMM for merging different modalities, highlighting the value of ensemble methods and multimodal fusion in affective communication analysis. The results underscore the potential of our approach in enhancing emotional understanding with broad applications in fields like mental health assessment, human–robot interaction, and cross-cultural communication.

Towards python program repair with generative pre-trained transformer (GPT-3.5)

ChatGPT has a great potential using a simple prompt design, which means further studies can be done to investigate the effect of different prompt designs. Complex software often contains hidden bugs in its source code. Recent study suggests that OpenAIs ChatGPT can perform numerous operations including code-to-code operations like code completion, translation, repair, and summarization, along with language-to-code operations such as code explanation and search. ChatGPTs dialogue capability can assist in generating more accurate bug fixes. However, it sometimes offers solutions without seeking further information, which can mislead users. To address this issue and enhance user experience, we conducted three design iterations to develop D-bugger a system enabling programmers to fix bugs more effectively. We conducted a survey to gauge the need for refinement, designed a low-fidelity prototype with key features, and then created a high-fidelity prototype evaluated by Python users. Our work aims to enhance the debugging process and user engagement.

Improving biomedical entity linking for complex entity mentions with LLM-based text simplification.

Large amounts of important medical information are captured in free-text documents in biomedical research and within healthcare systems, which can be made accessible through natural language processing (NLP). A key component in most biomedical NLP pipelines is entity linking, i.e. grounding textual mentions of named entities to a reference of medical concepts, usually derived from a terminology system, such as the Systematized Nomenclature of Medicine Clinical Terms. However, complex entity mentions, spanning multiple tokens, are notoriously hard to normalize due to the difficulty of finding appropriate candidate concepts. In this work, we propose an approach to preprocess such mentions for candidate generation, building upon recent advances in text simplification with generative large language models. We evaluate the feasibility of our method in the context of the entity linking track of the BioCreative VIII SympTEMIST shared task. We find that instructing the latest Generative Pre-trained Transformer model with a few-shot prompt for text simplification results in mention spans that are easier to normalize. Thus, we can improve recall during candidate generation by 2.9percentage points compared to our baseline system, which achieved the best score in the original shared task evaluation. Furthermore, we show that this improvement in recall can be fully translated into top-1 accuracy through careful initialization of a subsequent reranking model. Our best system achieves an accuracy of 63.6% on the SympTEMIST test set. The proposed approach has been integrated into the open-source xMEN toolkit, which is available online via https://github.com/hpi-dhc/xmen.

Assessing the Ability of a Large Language Model to Score Free-Text Medical Student Clinical Notes: Quantitative Study.

Teaching medical students the skills required to acquire, interpret, apply, and communicate clinical information is an integral part of medical education. A crucial aspect of this process involves providing students with feedback regarding the quality of their free-text clinical notes. The goal of this study was to assess the ability of ChatGPT 3.5, a large language model, to score medical students' free-text history and physical notes. This is a single-institution, retrospective study. Standardized patients learned a prespecified clinical case and, acting as the patient, interacted with medical students. Each student wrote a free-text history and physical note of their interaction. The students' notes were scored independently by the standardized patients and ChatGPT using a prespecified scoring rubric that consisted of 85 case elements. The measure of accuracy was percent correct. The study population consisted of 168 first-year medical students. There was a total of 14,280 scores. The ChatGPT incorrect scoring rate was 1.0%, and the standardized patient incorrect scoring rate was 7.2%. The ChatGPT error rate was 86%, lower than the standardized patient error rate. The ChatGPT mean incorrect scoring rate of 12 (SD 11) was significantly lower than the standardized patient mean incorrect scoring rate of 85 (SD 74; P=.002). ChatGPT demonstrated a significantly lower error rate compared to standardized patients. This is the first study to assess the ability of a generative pretrained transformer (GPT) program to score medical students' standardized patient-based free-text clinical notes. It is expected that, in the near future, large language models will provide real-time feedback to practicing physicians regarding their free-text notes. GPT artificial intelligence programs represent an important advance in medical education and medical practice.

PIM GPT a hybrid process in memory accelerator for autoregressive transformers

Decoder-only Transformer models such as Generative Pre-trained Transformers (GPT) have demonstrated exceptional performance in text generation by autoregressively predicting the next token. However, the efficiency of running GPT on current hardware systems is bounded by low compute-to-memory-ratio and high memory access. In this work, we propose a Process-in-memory (PIM) GPT accelerator, PIM-GPT, which achieves end-to-end acceleration of GPT inference with high performance and high energy efficiency. PIM-GPT leverages DRAM-based PIM designs for executing multiply-accumulate (MAC) operations directly in the DRAM chips, eliminating the need to move matrix data off-chip. Non-linear functions and data communication are supported by an application specific integrated chip (ASIC). At the software level, mapping schemes are designed to maximize data locality and computation parallelism. Overall, PIM-GPT achieves 41 − 137 × , 631 − 1074 × speedup and 123 − 383 × , 320 − 602 × energy efficiency over GPU and CPU baseline on 8 GPT models with up to 1.4 billion parameters.

ChatGPT-4 Consistency in Interpreting Laryngeal Clinical Images of Common Lesions and Disorders.

To investigate the consistency ofChatbot Generative Pretrained Transformer(ChatGPT)-4 in the analysis of clinical pictures of common laryngological conditions. Prospective uncontrolled study. Multicenter study. Patient history and clinical videolaryngostroboscopic images were presented to ChatGPT-4 for differential diagnoses, management, and treatment(s). ChatGPT-4 responses were assessed by 3blinded laryngologists with the artificial intelligence performance instrument (AIPI). The complexity of cases and the consistency between practitioners and ChatGPT-4 for interpreting clinical images were evaluated with a 5-point Likert Scale. The intraclass correlation coefficient (ICC) was used to measure the strength of interrater agreement. Forty patients with a mean complexity score of 2.60 ± 1.15. were included. The mean consistency score for ChatGPT-4 image interpretation was 2.46 ± 1.42. ChatGPT-4 perfectly analyzed the clinical images in 6 cases (15%; 5/5), while the consistency between GPT-4 and judges was high in 5 cases (12.5%; 4/5). Judges reported an ICC of 0.965 for the consistency score (P = .001). ChatGPT-4 erroneously documented vocal fold irregularity (mass or lesion), glottic insufficiency, and vocal cord paralysis in 21 (52.5%), 2 (0.05%), and 5 (12.5%) cases, respectively. ChatGPT-4 and practitioners indicated 153 and 63 additional examinations, respectively (P = .001). The ChatGPT-4 primary diagnosis was correct in 20.0% to 25.0% of cases. The clinical image consistency score was significantly associated with the AIPI score (rs = 0.830; P = .001). The ChatGPT-4 is more efficient in primary diagnosis, rather than in the image analysis, selecting the most adequate additional examinations and treatments.

Evaluating ChatGPT's Performance in the EU*US eHealth Work Foundational Curriculum Using the HITCOMP Self-Assessment Quiz.

This study investigated the performance of OpenAI's Chat Generative Pre-trained Transformer (ChatGPT) in responding to the EU*US eHealth Work Foundational Curriculum. This curriculum, a collaborative effort between European and U.S. institutions, provides an extensive framework for eHealth learning. The assessment involved 321 questions from the online Health Information Technology Competencies (HITCOMP) self-assessment quiz. Using GPT-3.5 model, the study presented each question three times to assess ChatGPT's consistency. Findings revealed an accuracy of 70.7%, indicating a reasonable grasp of eHealth topics, although performance was uneven across the 21 modules. These results underscore ChatGPT's potential in health information technology education and highlight the need for further model enhancements to fully encompass eHealth competencies.

ChatGPT sitting for FRCS Urology examination: Will artificial intelligence get certified?

Introduction: The use of artificial intelligence (AI), including neural networks and deep learning, has rapidly changed many industries. One recent application of AI that has gained the attention of physicians and the general public alike is ChatGPT, a chatbot developed and released by OpenAI Inc. in November2022. We challenged ChatGPT to sit the Fellowship of the Royal College of Surgeons (FRCS) Urology examination, both parts, to assess its performance and determine its eligibility for the title. Methods: We prepared a set of 100 Multiple-Choice Questions (MCQs) for Part 1 and 8 viva scenarios for Part 2. We choose to use ChatGPT 3.5 that is the free version available online, and responses were documented during the simulation. Results: In Part 1 MCQs, Chat Generative Pre-trained Transformer (ChatGPT) managed to answer some questions correctly with an overall score of 35%. However, in Part 2 Viva, its performance improved to an extent, demonstrating proficiency by offering detailed explanations and reasoning, achieving average score in some stations. ChatGPT’s performance raised some concerns on answering questions. Conclusion: AI is definitely an asset that can be used in our daily practice, yet it is crucial to take into consideration the concerns about the accuracy of data provided, including AI hallucination. ChatGPT failed to get the FRCS Urology title this time. Still in the early stages of development, further research is necessary to comprehensively understand their limitations and capabilities. Level of evidence: Not applicable

ChatGPT Research in Healthcare is Increasing Dramatically: A Bibliometric Analysis Based on VOSviewer.

The ChatGPT (Chat Generative Pretrained Transformer), an AI-based chatbot, has garnered significant attention in healthcare. We methodically extracted two significant English databases (PubMed and Web of Science) to examine the current state of research and potential future directions for ChatGPT in the healthcare field. Following screening, 921 publications were included in the analysis, demonstrating a sharp rise in the previous year and a peak number of publications released in October 2023. Meanwhile, we obtained 88 items and categorized them into 10 clusters, of which artificial intelligence, ChatGPT, and humans ranked among the top three in terms of frequency.

Caution Regarding ChatGPT's Appropriateness and Reliability Regarding Surgery for Wrist Arthritis.

Chat Generative Pre-Trained Transformer (ChatGPT), an artificial intelligence (AI) program, is widely used for information compilation. This study sought to analyze the quality and consistency of the information generated by ChatGPT regarding common procedures for wrist arthritis. 32 standardized questions regarding wrist osteoarthritis and related procedures (4-corner-fusion [4CF], proximal row carpectomy [PRC], resurfacing capitate pyrocarbon implant, wrist denervation, and total wrist arthrodesis and arthroplasty) were presented to the ChatGPT-3.5 interface 3 separate times, without feedback. ChatGPT's answers were evaluated for medical accuracy by 3 reviewers and rated as "appropriate," "appropriate but incomplete," or "inappropriate." Ratings were then converted to numerical values to calculate an intraclass correlation coefficient (ICC). A DISCERN score was used to assess quality, and Flesch-Kincade Grade Level and Flesch Reading Ease Score for readability. 75% of the responses were deemed "appropriate," with 23 questions receiving unanimous appropriate ratings across all responses. The ICC was 0.97 (95% CI [0.46, 0.98]), indicating excellent reliability. DISCERN score was 60 (good). The Flesch-Kincaid Grade Level was 14.6 ± 1.9, and the Flesch Reading Ease Score was 25.3 ± 6.7, implying a college reading level. The information that ChatGPT provided for PRC and total wrist arthrodesis and arthroplasty, appeared to be more reliable than for 4CF and denervation. ChatGPT's reliability and accuracy of information varied across procedures, possibly due to unknown and diverse sources. Furthermore, while some answers were factually correct, many provided generic information across differing questions, limiting usefulness. ChatGPT must be used cautiously and the limitations understood.

The diagnostic and triage accuracy of the GPT-3 artificial intelligence model: an observational study

Artificial intelligence (AI) applications in health care have been effective in many areas of medicine, but they are often trained for a single task using labelled data, making deployment and generalisability challenging. How well a general-purpose AI language model performs diagnosis and triage relative to physicians and laypeople is not well understood. We compared the predictive accuracy of Generative Pre-trained Transformer 3 (GPT-3)'s diagnostic and triage ability for 48 validated synthetic case vignettes (<50 words; sixth-grade reading level or below) of both common (eg, viral illness) and severe (eg, heart attack) conditions to a nationally representative sample of 5000 lay people from the USA who could use the internet to find the correct options and 21 practising physicians at Harvard Medical School. There were 12 vignettes for each of four triage categories: emergent, within one day, within 1 week, and self-care. The correct diagnosis and triage category (ie, ground truth) for each vignette was determined by two general internists at Harvard Medical School. For each vignette, human respondents and GPT-3 were prompted to list diagnoses in order of likelihood, and the vignette was marked as correct if the ground-truth diagnosis was in the top three of the listed diagnoses. For triage accuracy, we examined whether the human respondents' and GPT-3's selected triage was exactly correct according to the four triage categories, or matched a dichotomised triage variable (emergent or within 1 day vs within 1 week or self-care). We estimated GPT-3's diagnostic and triage confidence on a given vignette using a modified bootstrap resampling procedure, and examined how well calibrated GPT-3's confidence was by computing calibration curves and Brier scores. We also performed subgroup analysis by case acuity, and an error analysis for triage advice to characterise how its advice might affect patients using this tool to decide if they should seek medical care immediately. Among all cases, GPT-3 replied with the correct diagnosis in its top three for 88% (42/48, 95% CI 75-94) of cases, compared with 54% (2700/5000, 53-55) for lay individuals (p<0.0001) and 96% (637/666, 94-97) for physicians (p=0·012). GPT-3 triaged 70% correct (34/48, 57-82) versus 74% (3706/5000, 73-75; p=0.60) for lay individuals and 91% (608/666, 89-93%; p<0.0001) for physicians. As measured by the Brier score, GPT-3 confidence in its top prediction was reasonably well calibrated for diagnosis (Brier score=0·18) and triage (Brier score=0·22). We observed an inverse relationship between case acuity and GPT-3 accuracy (p<0·0001) with a fitted trend line of -8·33% decrease in accuracy for every level of increase in case acuity. For triage error analysis, GPT-3 deprioritised truly emergent cases in seven instances. A general-purpose AI language model without any content-specific training could perform diagnosis at levels close to, but below, physicians and better than lay individuals. We found that GPT-3's performance was inferior to physicians for triage, sometimes by a large margin, and its performance was closer to that of lay individuals. Although the diagnostic performance of GPT-3 was comparable to physicians, it was significantly better than a typical person using a search engine. The National Heart, Lung, and Blood Institute.

Is GPT-4 Conscious?

GPT-4 (Generative Pre-Trained Transformer 4) is often heralded as a leading commercial AI offering, sparking debates over its potential as a steppingstone toward Artificial General Intelligence. But does it possess consciousness? This paper investigates this key question using the nine qualitative measurements of the Building Blocks theory. GPT-4’s design, architecture, and implementation are compared to each of the building blocks of consciousness to determine whether it has achieved the requisite milestones to be classified as conscious or, if not, how close to consciousness GPT-4 is. Our assessment is that, while GPT-4 in its native configuration is not currently conscious, current technological research and development is sufficient to modify GPT-4 to have all the building blocks of consciousness. Consequently, we argue that the emergence of a conscious AI model is plausible in the near term. The paper concludes with a comprehensive discussion of the ethical implications and societal ramifications of engineering conscious AI entities.

Hidden flaws behind expert-level accuracy of multimodal GPT-4 vision in medicine

Recent studies indicate that Generative Pre-trained Transformer 4 with Vision (GPT-4V) outperforms human physicians in medical challenge tasks. However, these evaluations primarily focused on the accuracy of multi-choice questions alone. Our study extends the current scope by conducting a comprehensive analysis of GPT-4V’s rationales of image comprehension, recall of medical knowledge, and step-by-step multimodal reasoning when solving New England Journal of Medicine (NEJM) Image Challenges—an imaging quiz designed to test the knowledge and diagnostic capabilities of medical professionals. Evaluation results confirmed that GPT-4V performs comparatively to human physicians regarding multi-choice accuracy (81.6% vs. 77.8%). GPT-4V also performs well in cases where physicians incorrectly answer, with over 78% accuracy. However, we discovered that GPT-4V frequently presents flawed rationales in cases where it makes the correct final choices (35.5%), most prominent in image comprehension (27.2%). Regardless of GPT-4V’s high accuracy in multi-choice questions, our findings emphasize the necessity for further in-depth evaluations of its rationales before integrating such multimodal AI models into clinical workflows.

Authentic assessment in medical education: exploring AI integration and student-as-partners collaboration.

Traditional assessments often lack flexibility, personalized feedback, real-world applicability, and the ability to measure skills beyond rote memorization. These may not adequately accommodate diverse learning styles and preferences, nor do they always foster critical thinking or creativity. The inclusion of Artificial Intelligence (AI), especially Generative Pre-trained Transformers, in medical education marks a significant shift, offering both exciting opportunities and notable challenges for authentic assessment practices. Various fields, including anatomy, physiology, pharmacy, dentistry, and pathology, are anticipated to employ the metaverse for authentic assessments increasingly. This innovative approach will likely enable students to engage in immersive, project-based learning experiences, facilitating interdisciplinary collaboration and providing a platform for real-world application of knowledge and skills. This commentary paper explores how AI, authentic assessment, and Student-as-Partners (SaP) methodologies can work together to reshape assessment practices in medical education. The paper provides practical insights into effectively utilizing AI tools to create authentic assessments, offering educators actionable guidance to enhance their teaching practices. It also addresses the challenges and ethical considerations inherent in implementing AI-driven assessments, emphasizing the need for responsible and inclusive practices within medical education. Advocating for a collaborative approach between AI and SaP methodologies, the commentary proposes a robust plan to ensure ethical use while upholding academic integrity. Through navigating emerging assessment paradigms and promoting genuine evaluation of medical knowledge and proficiency, this collaborative effort aims to elevate the quality of medical education and better prepare learners for the complexities of clinical practice.

Evaluation of the Diagnostic Accuracy of GPT-4 in Five Thousand Rare Disease Cases.

Large language models (LLM) have shown great promise in supporting differential diagnosis, but 23 available published studies on the diagnostic accuracy evaluated small cohorts (number of cases, 30-422, mean 104) and have evaluated LLM responses subjectively by manual curation (23/23 studies). The performance of LLMs for rare disease diagnosis has not been evaluated systematically. Here, we perform a rigorous and large-scale analysis of the performance of a GPT-4 in prioritizing candidate diagnoses, using the largest-ever cohort of rare disease patients. Our computational study used 5267 computational case reports from previously published data. Each case was formatted as a Global Alliance for Genomics and Health (GA4GH) phenopacket, in which clinical anomalies were represented as Human Phenotype Ontology (HPO) terms. We developed software to generate prompts from each phenopacket. Prompts were sent to Generative Pre-trained Transformer 4 (GPT-4), and the rank of the correct diagnosis, if present in the response, was recorded. The mean reciprocal rank of the correct diagnosis was 0.24 (with the reciprocal of the MRR corresponding to a rank of 4.2), and the correct diagnosis was placed in rank 1 in 19.2% of the cases, in the first 3 ranks in 28.6%, and in the first 10 ranks in 32.5%. Our study is the largest to be reported to date and provides a realistic estimate of the performance of GPT-4 in rare disease medicine.

Towards efficient AutoML: a pipeline synthesis approach leveraging pre-trained transformers for multimodal data

This paper introduces an Automated Machine Learning (AutoML) framework specifically designed to efficiently synthesize end-to-end multimodal machine learning pipelines. Traditional reliance on the computationally demanding Neural Architecture Search is minimized through the strategic integration of pre-trained transformer models. This innovative approach enables the effective unification of diverse data modalities into high-dimensional embeddings, streamlining the pipeline development process. We leverage an advanced Bayesian Optimization strategy, informed by meta-learning, to facilitate the warm-starting of the pipeline synthesis, thereby enhancing computational efficiency. Our methodology demonstrates its potential to create advanced and custom multimodal pipelines within limited computational resources. Extensive testing across 23 varied multimodal datasets indicates the promise and utility of our framework in diverse scenarios. The results contribute to the ongoing efforts in the AutoML field, suggesting new possibilities for efficiently handling complex multimodal data. This research represents a step towards developing more efficient and versatile tools in multimodal machine learning pipeline development, acknowledging the collaborative and ever-evolving nature of this field.

Fine-Grained Sentiment Classification Using Generative Pretrained Transformer

Social media platforms have seen a significant increase in the number of users and content in recent years. Owing to the increased usage of these platforms, incidents of teasing, provocation—both positive and negative—and harassment, and community attacks have increased tremendously. There is an urgent need to automatically identify such content or tweets that can hamper the well-being of an individual or society. Analyzing social media messages from Twitter and Facebook has become the focus of sentiment analysis in recent years, which formerly focused on online product evaluations. Sentiment analysis is used in a wide range of fields besides product reviews, including harassment, stock markets, elections, disasters, and software engineering. After the tweets have been preprocessed, the extracted features are categorized using classifiers like decision trees, logistic regression, multinomial nave Bayes, support vector machines, random forests, and Bernoulli nave Bayes, as well as deep learning techniques like recurrent neural network (RNN) models, long short-term memory (LSTM) models, bidirectional long short-term memory (BiLSTM) models, and convolutional neural network (CNN) model for sentiment analysis. In this paper, different techniques are compared to classify Twitter tweets into three categories: “positive,” “negative,” and “neutral.” We proposed a novel data-balancing technique for text classification. A text classification technique is proposed for analyzing textual data using the Generative Pretrained Transformer model owing to its contextual understanding and more realistic data generation capability. Comparative analysis of different Machine learning and Deep learning models are performed with and without data balancing. The experiments show that the accuracy and F1-measure of the Twitter sentiment classification classifier are improved. The proposed ensemble has outperformed and achieved an accuracy of 90%, precision of 88%, and 81% F1 score.

Examining ChatGPT adoption among educators in higher educational institutions using extended UTAUT model

PurposeBased on the Unified Theory of Acceptance and Use of Technology (UTAUT) model, this study aims to explore the factors influencing educators’ acceptance and utilization of chat generative pretrained transformer (ChatGPT) in the context of higher educational institutions. This study additionally examines the moderating influence of trust on the association between intention and adoption of ChatGPT.Design/methodology/approachA structured questionnaire was disseminated to 1,214 educators following the purposive sampling method. The hypothesized relationships between the extended UTAUT model constructs and ChatGPT adoption were examined using structural equation modeling.FindingsThe results of this study indicate positive impact of ChatGPT adoption and its use behavior (UB) among educators. Performance expectancy, effort expectancy, hedonic motivation, facilitating conditions and habit have a positive impact on behavior intention to use this tool among educators in higher education institutions, However, the educators intend to use ChatGPT in a personal capacity and are negatively influenced by the social groups. Moreover, behavior intention to adopt ChatGPT has a larger effect on the UB due to the higher involvement of trust.Practical implicationsThis research adds to the literature on technology adoption by emphasizing the significance of trust in the adoption of emerging technologies, such as big language models. This study’s findings have implications for ChatGPT technology developers, legislators and academics considering the implementation of ChatGPT for teaching and research.Originality/valueThis research investigates ChatGPT adoption among higher education educators using the UTAUT model. It emphasizes the role of trust in adoption, highlights key adoption factors and reveals a paradox: educators are personally inclined to use ChatGPT but hesitant due to negative social influence. This study's novelty lies in its focus on trust and its implications for developers, legislators and educators, offering valuable insights for integrating ChatGPT in education.

Information extraction from medical case reports using OpenAI InstructGPT

Background and objectiveResearchers commonly use automated solutions such as Natural Language Processing (NLP) systems to extract clinical information from large volumes of unstructured data. However, clinical text's poor semantic structure and domain-specific vocabulary can make it challenging to develop a one-size-fits-all solution. Large Language Models (LLMs), such as OpenAI's Generative Pre-Trained Transformer 3 (GPT-3), offer a promising solution for capturing and standardizing unstructured clinical information. This study evaluated the performance of InstructGPT, a family of models derived from LLM GPT-3, to extract relevant patient information from medical case reports and discussed the advantages and disadvantages of LLMs versus dedicated NLP methods. MethodsIn this paper, 208 articles related to case reports of foreign body injuries in children were identified by searching PubMed, Scopus, and Web of Science. A reviewer manually extracted information on sex, age, the object that caused the injury, and the injured body part for each patient to build a gold standard to compare the performance of InstructGPT. ResultsInstructGPT achieved high accuracy in classifying the sex, age, object and body part involved in the injury, with 94%, 82%, 94% and 89%, respectively. When excluding articles for which InstructGPT could not retrieve any information, the accuracy for determining the child's sex and age improved to 97%, and the accuracy for identifying the injured body part improved to 93%. InstructGPT was also able to extract information from non-English language articles. ConclusionsThe study highlights that LLMs have the potential to eliminate the necessity for task-specific training (zero-shot extraction), allowing the retrieval of clinical information from unstructured natural language text, particularly from published scientific literature like case reports, by directly utilizing the PDF file of the article without any pre-processing and without requiring any technical expertise in NLP or Machine Learning. The diverse nature of the corpus, which includes articles written in languages other than English, some of which contain a wide range of clinical details while others lack information, adds to the strength of the study.

The judge, the AI, and the Crown: a collusive network

ABSTRACT The article examines the potential implications of ChatGPT ChatGPT (Generative Pre-trained Transformer) [Alec Radford and others, ‘Improving Language Understanding by Generative Pre-Training’ Amazon Web Services (2018) <https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf> accessed 28 May 2024; When this article was submitted, only ChatGPT-3.5 and ChatGPT-4 was available. However, during the revision stage, ChatGPT-4o (also known as GPT-4 Omnus) was introduced to the market, as announced on 13 May 2024.] in judicial decision-making process. This is especially salient given that judicial decisions are made within an interactive ritual chain (Bergman Blix) rather than an ivory tower. This cutting-edge natural language processing model leverages deep learning techniques to potentially aid judiciary members in formulating legal judgements. Therefore, this article assesses the capabilities, limitations, and potential applications of ChatGPT, aiming to evaluate the model's feasibility as a collaborative contributor to legal judgements. In particular, the article examines the utilisation of ChatGPT as an adjunctive tool, supporting human judges in their decision-making processes and, consequently, establishing to what extent artificial intelligence (AI) is used as a tool or collaborator in judicial decision-making. This determines authorship and, accordingly, ownership of judgements.