Generative Pre-trained Transformer Research Articles

Use of chatgpt-4 to explain kawasaki disease to parents and paediatric trainees

Abstract Background Paediatric trainees and parents of Kawasaki disease (KD) patients may face challenges when managing the disease due to insufficient accessibility to KD information. Chat Generative Pre-Trained Transformer (ChatGPT) based on large language models may help explain KD information to paediatric trainees and patient parents. Purpose This study aims to determine the usefulness of Chat Generative Pre- Trained Transformer -4 (ChatGPT-4) in explaining Kawasaki Disease (KD) and its management to parents and paediatric trainees managing KD patients. Methods We created 2 sets of clinical scenarios. In the first set, ChatGPT-4 was instructed to respond to 10 questions based on enquiries from parents of KD patients. Responses were scored in terms of "factual accuracy", "coherence", "comprehensiveness", and "humaneness" by 3 paediatric cardiologists using Likert scale of 0-10. Readability were calculated using Flesch reading-ease test. In the second set, ChatGPT-4 was instructed to respond to 8 KD-related questions based on enquiries from paediatric trainees. Responses are graded based on "relevance", "reliability" and "comprehensiveness" using Likert scale of 0-10 by 3 paediatric cardiologists independently. Reviewers would determine whether major advice from chatGPT-4 would be adopted in clinical judgement. Results For parent-targeted responses, ChatGPT-4 achieved the highest scores in ‘humaneness’ (median 9.00, IQR 8.00 to 9.00) and ‘coherence’ (median 8.00, IQR 7.00 to 8.00). Inaccurate information regarding disease prognosis, actions as well as prescription of medications and surgery is found in 80% of scenarios. Missing information regarding long-term coronary complications, antiplatelet management and cardiac assessments is found in all 10 scenarios. Mean readability of parent-targeted responses is 71.70 ± 6.26, a readability level easily understood by 12-year-olds. For paediatrician-targeted responses, ChatGPT-4 achieved the highest scores in ‘relevance’ (median 9.50, IQR 7.25 to 9.0). Inaccurate information regarding coronary interventions, patient education and immunization recommendations is spotted in 37.5% of the scenarios. Missing information regarding patient education and stress imaging is found in 25% of the scenarios. All reviewers would adopt ChatGPT-4’s advice in 87.5% of the scenarios. Conclusions ChatGPT-4 has significant limitations in accuracy and lacks salient information when providing KD recommendations for parents and paediatric trainees.Performance in parent-targeted questionsPerformance in trainee-targeted question

From recorded to AI‐generated instructional videos: A comparison of learning performance and experience

AbstractGenerative AI (GAI) and AI‐generated content (AIGC) have been increasingly involved in our work and daily life, providing a new learning experience for students. This study examines whether AI‐generated instructional videos (AIIV) can facilitate learning as effectively as traditional recorded videos (RV). We propose an instructional video generation pipeline that includes customized GPT (Generative Pre‐trained Transformer), text‐to‐speech and lip synthesis techniques to generate videos from slides and a clip or a photo of a human instructor. Seventy‐six students were randomly assigned to learn English words using either AIIV or RV, with performance assessed by retention, transfer and subjective measures from cognitive, emotional, motivational and social perspectives. The findings indicate that the AIIV group performed as well as the RV group in facilitating learning, with AIIV showing higher retention but no significant differences in transfer. RV was found to offer a stronger sense of social presence. Although other subjective measures were similar between the two groups, AIIV was perceived as slightly less favourable. However, the AIIV was still found to be moderately to highly attractive, addressing concerns related to the uncanny valley effect. This research demonstrates that AIGC can be an effective tool for learning, offering valuable implications for the use of GAI in educational settings. Practitioner notesWhat is already known about this topic Instructional videos, especially those featuring a teacher's presence, have been widely used in second language learning to facilitate learning. Producing instructional videos is costly and burdensome. Generative AI has great potential for generating educational content. What this paper adds An AI‐generated instructional video (including generated lecture text, voice and appearance) demonstrated greater improvement in students' retention performance in English word learning than a traditional recorded video. Students perceived no significant differences between the AI‐generated instructional video and recorded video in satisfaction, motivation, trust, cognitive load, emotions and parasocial interaction dimensions, although the AI‐generated instructional video group reported slightly lower values. Despite AI‐generated instructional video eliciting a significantly lower value of social presence than recorded video, it led to a reduction in cognitive load and better performance. Implications for practice and/or policy We recommend using the AI‐generated instructional video in both physical and online classes for its positive effects on both learning achievement and learning experience. The findings indicate the equivalence principle in AI‐generated content, highlighting that the appearance, voice and lecture text generated by current AI technology have reached a certain level of quality.

Assessing the need for coronary angiography in high-risk acute coronary syndrome patients using artificial intelligence and computed tomography

Abstract Background/Introduction The use of coronary computed tomography angiography (CCTA) in managing high-risk acute coronary syndrome (ACS) patients is increasingly common. Yet, its role remains unvalidated in this context, posing challenges to traditional clinical decision-making algorithms. Physicians, accustomed to a hierarchical and structured approach involving symptom assessment, electrocardiograms, and biomarkers for deciding on the necessity of invasive coronary angiography (ICA), now face dilemmas when CCTA results challenge established diagnostic pathways and contradict clinical decisions based on usual criteria. Meanwhile, the potential of artificial intelligence (AI) which processes information in a way fundamentally different from human clinical reasoning, has been shown to aid decision-making in cardiovascular medicine. Yet, it has not been tested in this acute setting. Purpose This study aimed to evaluate the efficacy of the Chat Generative Pre-trained Transformer (ChatGPT) in guiding the need for ICA in high-risk ACS patients based on both standard clinical data and CCTA findings. Methods This investigation is a sub-study of a larger research project where high-risk ACS patients (all having a theoretical guideline-based indication for ICA) underwent CCTA prior to ICA. ChatGPT analyzed clinical vignettes containing patient data, electrocardiograms, troponin levels, and CCTA results to determine the necessity of ICA. The AI's recommendations were then compared to actual ICA findings to assess its decision-making accuracy. Results Among 86 patients with a clear indication for ICA, ChatGPT recommended against it for 19 patients, 16 of whom indeed had no significant findings at ICA (Figure). For 67 patients, ChatGPT advised proceeding with ICA, and a significant lesion was confirmed in 58 of them. Consequently, ChatGPT's overall accuracy stood at 86%, with a sensitivity of 95% and a specificity of 64%. The model's negative predictive value was 84%, and its positive predictive value was 87%. Conclusion These preliminary data suggest that ChatGPT can effectively assist in guiding the need for ICA among high-risk ACS patients, potentially reducing unnecessary procedures but further clinical studies to refine AI's role in clinical decision-making are required.ChatGPT's decision as per ICA's results

Applications of ChatGPT in Heart Failure Prevention, Diagnosis, Management, and Research: A Narrative Review.

Heart failure (HF) is a leading cause of mortality, morbidity, and financial burden worldwide. The emergence of advanced artificial intelligence (AI) technologies, particularly Generative Pre-trained Transformer (GPT) systems, presents new opportunities to enhance HF management. In this review, we identified and examined existing studies on the use of ChatGPT in HF care by searching multiple medical databases (PubMed, Google Scholar, Medline, and Scopus). We assessed the role of ChatGPT in HF prevention, diagnosis, and management, focusing on its influence on clinical decision-making and patient education. However, ChatGPT faces limited training data, inherent biases, and ethical issues that hinder its widespread clinical adoption. We review these limitations and highlight the need for improved training approaches, greater model transparency, and robust regulatory compliance. Additionally, we explore the effectiveness of ChatGPT in managing HF, particularly in reducing hospital readmissions and improving patient outcomes with customized treatment plans while addressing social determinants of health (SDoH). In this review, we aim to provide healthcare professionals and policymakers with an in-depth understanding of ChatGPT's potential and constraints within the realm of HF care.

A comparison of drug information question responses by a drug information center and by ChatGPT.

In an effort to expedite the publication of articles, AJHP is posting manuscripts online as soon as possible after acceptance. Accepted manuscripts have been peer-reviewed and copyedited, but are posted online before technical formatting and author proofing. These manuscripts are not the final version of record and will be replaced with the final article (formatted per AJHP style and proofed by the authors) at a later time. A study was conducted to assess the accuracy and ability of Chat Generative Pre-trained Transformer (ChatGPT) to systematically respond to drug information inquiries relative to responses of a drug information center (DIC). Ten drug information questions answered by the DIC in 2022 or 2023 were selected for analysis. Three pharmacists created new ChatGPT accounts and submitted each question to ChatGPT at the same time. Each question was submitted twice to identify consistency in responses. Two days later, the same process was conducted by a fourth pharmacist. Phase 1 of data analysis consisted of a drug information pharmacist assessing all 84 ChatGPT responses for accuracy relative to the DIC responses. In phase 2, 10 ChatGPT responses were selected to be assessed by 3 blinded reviewers. Reviewers utilized an 8-question predetermined rubric to evaluate the ChatGPT and DIC responses. When comparing the ChatGPT responses (n = 84) to the DIC responses, ChatGPT had an overall accuracy rate of 50%. Accuracy across the different question types varied. In regards to the overall blinded score, ChatGPT responses scored higher than the responses by the DIC according to the rubric (overall scores of 67.5% and 55.0%, respectively). The DIC responses scored higher in the categories of references mentioned and references identified. Responses generated by ChatGPT have been found to be better than those created by a DIC in clarity and readability; however, the accuracy of ChatGPT responses was lacking. ChatGPT responses to drug information questions would need to be carefully reviewed for accuracy and completeness.

TECRR: a benchmark dataset of radiological reports for BI-RADS classification with machine learning, deep learning, and large language model baselines

BackgroundRecently, machine learning (ML), deep learning (DL), and natural language processing (NLP) have provided promising results in the free-form radiological reports’ classification in the respective medical domain. In order to classify radiological reports properly, a high-quality annotated and curated dataset is required. Currently, no publicly available breast imaging-based radiological dataset exists for the classification of Breast Imaging Reporting and Data System (BI-RADS) categories and breast density scores, as characterized by the American College of Radiology (ACR). To tackle this problem, we construct and annotate a breast imaging-based radiological reports dataset and its benchmark results. The dataset was originally in Spanish. Board-certified radiologists collected and annotated it according to the BI-RADS lexicon and categories at the Breast Radiology department, TecSalud Hospitals Monterrey, Mexico. Initially, it was translated into English language using Google Translate. Afterwards, it was preprocessed by removing duplicates and missing values. After preprocessing, the final dataset consists of 5046 unique reports from 5046 patients with an average age of 53 years and 100% women. Furthermore, we used word-level NLP-based embedding techniques, term frequency-inverse document frequency (TF-IDF) and word2vec to extract semantic and syntactic information. We also compared the performance of ML, DL and large language models (LLMs) classifiers for BI-RADS category classification.ResultsThe final breast imaging-based radiological reports dataset contains 5046 unique reports. We compared K-Nearest Neighbour (KNN), Support Vector Machine (SVM), Naive Bayes (NB), Random Forest (RF), Adaptive Boosting (AdaBoost), Gradient-Boosting (GB), Extreme Gradient Boosting (XGB), Long Short-Term Memory (LSTM), Bidirectional Encoder Representations from Transformers (BERT) and Biomedical Generative Pre-trained Transformer (BioGPT) classifiers. It is observed that the BioGPT classifier with preprocessed data performed 6% better with a mean sensitivity of 0.60 (95% confidence interval (CI), 0.391-0.812) compared to the second best performing classifier BERT, which achieved mean sensitivity of 0.54 (95% CI, 0.477-0.607).ConclusionIn this work, we propose a curated and annotated benchmark dataset that can be used for BI-RADS and breast density category classification. We also provide baseline results of most ML, DL and LLMs models for BI-RADS classification that can be used as a starting point for future investigation. The main objective of this investigation is to provide a repository for the investigators who wish to enter the field to push the boundaries further.

Performance of ChatGPT and Dental Students on Concepts of Periodontal Surgery.

As a large language model, chat generative pretrained transformer (ChatGPT) has provided a valuable tool for various medical scenarios with its interactive dialogue-based interface. However, there is a lack of studies on ChatGPT's effectiveness in handling dental tasks. This study aimed to compare the knowledge and comprehension abilities of ChatGPT-3.5/4 with that of dental students about periodontal surgery. A total of 134 dental students participated in this study. We designed a questionnaire consisting of four questions about the inclination for ChatGPT, 25 multiple-choice, and one open-ended question. As the comparison of ChatGPT-3.5 and 4, the question about the inclination was removed, and the rest was the same. The response time of ChatGPT-3.5 and 4 as well as the comparison of ChatGPT-3.5 and 4' performances with dental students were measured. Regarding students' feedback on the open-ended question, we also compared the outcomes of ChatGPT-4' and teacher's review. On average, ChatGPT-3.5 and 4 required 3.63 ± 1.18 s (95% confidence interval [CI], 3.14, 4.11) and 12.49 ± 7.29 s (95% CI, 9.48, 15.50), respectively (p < 0.001) for each multiple-choice question. For these 25 questions, the accuracy was 21.51 ± 2.72, 14 and 20 for students, ChatGPT-3.5 and 4, respectively. Furthermore, the outcomes of ChatGPT-4's review were consistent with that of teacher's review. For dental examinations related to periodontal surgery, ChatGPT's accuracy was not yet comparable to that of the students. Nevertheless, ChatGPT shows promise in assisting students with the curriculum and helping practitioners with clinical letters and reviews of students' textual descriptions.

The Problem of AI Hallucination and How to Solve It

AI in education is a topic that has been researched for the last 70 years. However, the last two years have seen very significant changes. These changes relate to the introduction of OpenAI's ChatGPT chatbot in November 2022. The GPT (Generative Pre-trained Transformer) language model has dramatically influenced how the public approaches artificial intelligence. For many, generative language models have become synonymous with AI and have come uncritically viewed as a universal source of answers to most questions. However, it soon became apparent that even generative language models had their limits. Among the main problems that emerged was hallucination (providing answers containing false or misleading information), which is expected in all language models. The main problem of hallucination is that this information is difficult to distinguish from other information, and AI language models are very persuasive in presenting it. The risks of this phenomenon are much more substantial when using language modules to support learning, where the learner cannot distinguish correct information from incorrect information. The proposed paper focuses on the area of AI hallucination in mathematics education. It will first show how AI chatbots hallucinate in mathematics and then present one possible solution to counter this hallucination. The presented solution was created for the AI chatbot Edu-AI and designed to tutor students in mathematics. Usually, the problem is approached so that the system verifies the correctness of the output offered by the chatbot. Within the Edu-AI, checking responses is not implemented, but checking inputs is. If an input containing a factual query is recorded, it is redirected, and the answer is traced to authorised knowledge sources and study materials. If a relevant answer cannot be traced in these sources, a redirect to a natural person who will address the question is offered. In addition to describing the technical solution, the article includes concrete examples of how the system works. This solution has been developed for the educational domain but applies to all domains where users must be provided with relevant information.

Generative AI Models for the Protein Scaffold Filling Problem.

De novo protein sequencing is an important problem in proteomics, playing a crucial role in understanding protein functions, drug discovery, design and evolutionary studies, etc. Top-down and bottom-up tandem mass spectrometry are popular approaches used in the field of mass spectrometry to analyze and sequence proteins. However, these approaches often produce incomplete protein sequences with gaps, namely scaffolds. The protein scaffold filling problem refers to filling the missing amino acids in the gaps of a scaffold to infer the complete protein sequence. In this article, we tackle the protein scaffold filling problem based on generative AI techniques, such as convolutional denoising autoencoder, transformer, and generative pretrained transformer (GPT) models, to complete the protein sequences and compare our results with recently developed convolutional long short-term memory-based sequence model. We evaluate the model performance both on a real dataset and generated datasets. All proposed models show outstanding prediction accuracy. Notably, the GPT-2 model achieves 100% gap-filling accuracy and 100% full sequence accuracy on the MabCampth protein scaffold, which outperforms the other models.

Large language models for structured reporting in radiology: past, present, and future.

Structured reporting (SR) has long been a goal in radiology to standardize and improve the quality of radiology reports. Despite evidence that SR reduces errors, enhances comprehensiveness, and increases adherence to guidelines, its widespread adoption has been limited. Recently, large language models (LLMs) have emerged as a promising solution to automate and facilitate SR. Therefore, this narrative review aims to provide an overview of LLMs for SR in radiology and beyond. We found that the current literature on LLMs for SR is limited, comprising ten studies on the generative pre-trained transformer (GPT)-3.5 (n = 5) and/or GPT-4 (n = 8), while two studies additionally examined the performance of Perplexity and Bing Chat or IT5. All studies reported promising results and acknowledged the potential of LLMs for SR, with six out of ten studies demonstrating the feasibility of multilingual applications. Building upon these findings, we discuss limitations, regulatory challenges, and further applications of LLMs in radiology report processing, encompassing four main areas: documentation, translation and summarization, clinical evaluation, and data mining. In conclusion, this review underscores the transformative potential of LLMs to improve efficiency and accuracy in SR and radiology report processing. KEY POINTS: Question How can LLMs help make SR in radiology more ubiquitous? Findings Current literature leveraging LLMs for SR is sparse but shows promising results, including the feasibility of multilingual applications. Clinical relevance LLMs have the potential to transform radiology report processing and enable the widespread adoption of SR. However, their future role in clinical practice depends on overcoming current limitations and regulatory challenges, including opaque algorithms and training data.

Creating a biomedical knowledge base by addressing GPT inaccurate responses and benchmarking context.

We created GNQA, a generative pre-trained transformer (GPT) knowledge base driven by a performant retrieval augmented generation (RAG) with a focus on aging, dementia, Alzheimer's and diabetes. We uploaded a corpus of three thousand peer reviewed publications on these topics into the RAG. To address concerns about inaccurate responses and GPT 'hallucinations', we implemented a context provenance tracking mechanism that enables researchers to validate responses against the original material and to get references to the original papers. To assess the effectiveness of contextual information we collected evaluations and feedback from both domain expert users and 'citizen scientists' on the relevance of GPT responses. A key innovation of our study is automated evaluation by way of a RAG assessment system (RAGAS). RAGAS combines human expert assessment with AI-driven evaluation to measure the effectiveness of RAG systems. When evaluating the responses to their questions, human respondents give a "thumbs-up" 76% of the time. Meanwhile, RAGAS scores 90% on answer relevance on questions posed by experts. And when GPT-generates questions, RAGAS scores 74% on answer relevance. With RAGAS we created a benchmark that can be used to continuously assess the performance of our knowledge base. Full GNQA functionality is embedded in the free GeneNetwork.org web service, an open-source system containing over 25 years of experimental data on model organisms and human. The code developed for this study is published under a free and open-source software license at https://git.genenetwork.org/gn-ai/tree/README.md .

A Bibliometric Review of Large Language Models Research from 2017 to 2023

Large language models (LLMs), such as OpenAI's Generative Pre-trained Transformer (GPT), are a class of language models that have demonstrated outstanding performance across a range of natural language processing (NLP) tasks. LLMs have become a highly sought-after research area because of their ability to generate human-like language and their potential to revolutionize science and technology. In this study, we conduct bibliometric and discourse analyses of scholarly literature on LLMs. Synthesizing over 5,000 publications, this article serves as a roadmap for researchers, practitioners, and policymakers to navigate the current landscape of LLMs research. We present the research trends from 2017 to early 2023, identifying patterns in research paradigms and collaborations. We start with analyzing the core algorithm developments and NLP tasks that are fundamental in LLMs research. We then investigate the applications of LLMs in various fields and domains, including medicine, engineering, social science, and humanities. Our review also reveals the dynamic, fast-paced evolution of LLMs research. Overall, this article offers valuable insights into the current state, impact, and potential of LLMs research and its applications.

Revolution or risk?-Assessing the potential and challenges of GPT-4V in radiologic image interpretation.

ChatGPT-4 Vision (GPT-4V) is a state-of-the-art multimodal large language model (LLM) that may be queried using images. We aimed to evaluate the tool's diagnostic performance when autonomously assessing clinical imaging studies. A total of 206 imaging studies (i.e., radiography (n = 60), CT (n = 60), MRI (n = 60), and angiography (n = 26)) with unequivocal findings and established reference diagnoses from the radiologic practice of a large university hospital were accessed. Readings were performed uncontextualized, with only the image provided, and contextualized, with additional clinical and demographic information. Responses were assessed along multiple diagnostic dimensions and analyzed using appropriate statistical tests. With its pronounced propensity to favor context over image information, the tool's diagnostic accuracy improved from 8.3% (uncontextualized) to 29.1% (contextualized, first diagnosis correct) and 63.6% (contextualized, correct diagnosis among differential diagnoses) (p ≤ 0.001, Cochran's Q test). Diagnostic accuracy declined by up to 30% when 20 images were re-read after 30 and 90 days and seemed unrelated to the tool's self-reported confidence (Spearman's ρ = 0.117 (p = 0.776)). While the described imaging findings matched the suggested diagnoses in 92.7%, indicating valid diagnostic reasoning, the tool fabricated 258 imaging findings in 412 responses and misidentified imaging modalities or anatomic regions in 65 images. GPT-4V, in its current form, cannot reliably interpret radiologic images. Its tendency to disregard the image, fabricate findings, and misidentify details, especially without clinical context, may misguide healthcare providers and put patients at risk. Question Can Generative Pre-trained Transformer 4 Vision (GPT-4V) interpret radiologic images-with and without clinical context? Findings GPT-4V performed poorly, demonstrating diagnostic accuracy rates of 8% (uncontextualized), 29% (contextualized, most likely diagnosis correct), and 64% (contextualized, correct diagnosis among differential diagnoses). Clinical relevance The utility of commercial multimodal large language models, such as GPT-4V, in radiologic practice is limited. Without clinical context, diagnostic errors and fabricated findings may compromise patient safety and misguide clinical decision-making. These models must be further refined to be beneficial.

GPT-4-based AI agents-the new expert system for detection of antimicrobial resistance mechanisms?

The study titled "GPT-4-based AI agents-the new expert system for detection of antimicrobial resistance mechanisms?" is critically important as it explores the integration of advanced artificial intelligence (AI) technologies, like generative pre-trained transformer (GPT)-4, into the field of laboratory medicine, specifically in the diagnostics of antimicrobial resistance (AMR). With the growing challenge of AMR, there is a pressing need for innovative solutions that can enhance diagnostic accuracy and efficiency. This research assesses the capability of AI to support the existing two-step confirmatory process recommended by the European Committee on Antimicrobial Susceptibility Testing for detecting beta-lactamases in Gram-negative bacteria. By potentially speeding up and improving the precision of initial screenings, AI could reduce the time to appropriate treatment interventions. Furthermore, this study is vital for validating the reliability and safety of AI tools in clinical settings, ensuring they meet stringent regulatory standards before they can be broadly implemented. This could herald a significant shift in how laboratory diagnostics are performed, ultimately leading to better patient outcomes.

Automatic Detection and Extraction of Key Resources from Tables in Biomedical Papers.

Tables are useful information artifacts that allow for easy detection of data "missingness" by humans and have been implemented by several publishers to improve the amount of information present for key resources and reagents such as antibodies, cell lines, and other tools that constitute the inputs to a study. To improve the availability of these tables in the broader biomedical literature, we introduced four pipelines for key resource table extraction from biomedical documents in PDF format. Our approach reconstructs key resource tables using image level table detection and structure detection generated table boundary, column (and row) bounding box information together with PDF text alignment. To remedy row over-segmentation resulting from overflowing table cell contents, we introduced a language modeling (LM) based row merging solution where a character-level generative pre-trained transformer (GPT) model was pre-trained on more than 11 million scientific table contents from PubMed Central Open Access Subset (PMC OAS). All introduced pipelines significantly outperformed GROBID baseline while our Table LM based row merging based pipeline, significantly outperformed all other pipelines including our OCR based pipeline.

A transformer-based approach to Nigerian Pidgin text generation

AbstractThis paper describes the development of a transformer-based text generation model for Nigerian Pidgin also known as Naijá, a popular language in West Africa. Despite its wide use, Nigerian Pidgin remains under-resourced, particularly in areas related to text generation and natural language processing. These difficulties are primarily due to technological constraints rather than the language’s fundamental attributes. There is currently a demand for Nigerian Pidgin-specific solutions because it is used in everyday communication and has a unique linguistic blend. This paper aims to close this gap by exploring the application of state-of-the-art transformer technology to develop a text generation model for Nigerian Pidgin. This work uses the public Afriberta-corpus dataset to optimize the Generative Pre-trained Transformer (GPT-2) model across a sizeable dataset. The performance evaluators, BLEU and Perplexity metrics provide a detailed breakdown of the model’s text quality and predictive accuracy. Despite the difficulties caused by a limited amount of training data, preliminary evaluations show that the model can generate coherent Nigerian Pidgin text. The performance evaluation yielded perplexity scores of 43.56 for variable target reference length and 43.26 for fixed text length. BLEU scores of 0.15 for fixed max length and 0.56 for variable reference target length. This highlights the quality of generated text and the significant improvement when the generated text length is aligned with the reference target. Our work was benchmarked against African American Vernacular (AAVE) revealing that BLEU scores for AAVE are significantly lower than those for Standard American English, with BLEU given as 0.26. Our Nigerian Pidgin model, with a BLEU score of 0.56, shows a better performance. However, both results suggest that both dialects are challenging for language models. Leveraging the pre-trained transformer-based language model and evaluation metrics, we showcase the model’s capacity for coherent Nigerian Pidgin text generation. For future research, the research work can serve as a good foundation for advancement and progress in the Nigerian Pidgin language generation and other low-resource languages.

An exploratory study on the potential of ChatGPT as an AI-assisted diagnostic tool for visceral leishmaniasis

Visceral leishmaniasis (VL) is a severe parasitic disease that poses significant diagnostic challenges due to its complex presentation and the necessity for comprehensive diagnostic methods. This exploratory study investigates the potential of Chat Generative Pre-trained Transformer (ChatGPT)/GPT-4, an artificial intelligence (AI) chatbot, in assisting the diagnostic process for VL. We evaluated the diagnostic accuracy of ChatGPT/GPT-4 in generating differential diagnosis lists for eight clinical vignette cases of VL, authored by a Brazilian infectious disease doctor. Our findings reveal that ChatGPT/GPT-4 included VL in the top five differential diagnoses in 75% of the cases (95% confidence interval [CI]: 40.1 &amp;ndash; 93.7%) and identified VL as the top diagnosis in 50% of the cases (95% CI: 30.3 &amp;ndash; 86.5%). These results underscore the high potential of ChatGPT/GPT-4 as an AI-assisted diagnostic tool, which is capable of providing accurate differential diagnoses and assisting healthcare professionals in resource-limited settings. The study highlights the broader applicability of AI chatbots in medical diagnostics, not only for common conditions but also for specialized and less prevalent diseases like VL. By integrating AI tools into the diagnostic workflow, healthcare providers can enhance their diagnostic accuracy and efficiency, ultimately improving patient outcomes. This research contributes to the growing body of evidence supporting the utility of AI in healthcare and underscores the need for further studies to validate these findings across larger and more diverse clinical scenarios.

Evaluation of a Large Language Model on the American Academy of Pediatrics' PREP Emergency Medicine Question Bank

Background Large language models (LLMs), including ChatGPT (Chat Generative Pretrained Transformer), a popular, publicly available LLM, represent an important innovation in the application of artificial intelligence. These systems generate relevant content by identifying patterns in large text datasets based on user input across various topics. We sought to evaluate the performance of ChatGPT in practice test questions designed to assess knowledge competency for pediatric emergency medicine (PEM). Methods We evaluated the performance of ChatGPT for PEM board certification using a popular question bank used for board certification in PEM published between 2022 and 2024. Clinicians assessed performance of ChatGPT by inputting prompts and recording the software's responses, asking each question over 3 separate iterations. We calculated correct answer percentages (defined as correct in at least 2/3 iterations) and assessed for agreement between the iterations using Fleiss' κ. Results We included 215 questions over the 3 study years. ChatGPT responded correctly to 161 of PREP EM questions over 3 years (74.5%; 95% confidence interval, 68.5%–80.5%), which was similar within each study year (75.0%, 71.8%, and 77.8% for study years 2022, 2023, and 2024, respectively). Among correct responses, most were answered correctly on all 3 iterations (137/161, 85.1%). Performance varied by topic, with the highest scores in research and medical specialties and lower in procedures and toxicology. Fleiss' κ across the 3 iterations was 0.71, indicating substantial agreement. Conclusion ChatGPT provided correct answers to PEM responses in three-quarters of cases, over the recommended minimum of 65% provided by the question publisher for passing. Responses by ChatGPT included detailed explanations, suggesting potential use for medical education. We identified limitations in specific topics and image interpretation. These results demonstrate opportunities for LLMs to enhance both the education and clinical practice of PEM.

Fostering Tech Innovation

This study aimed to interpret the value of integration of TRIZ (Teoriya Resheniya Izobretatelskih Zadach; Theory of Inventive Problem Solving) and ChatGPT(Chat Generative Pre-trained Transformer) to enhance technological and entrepreneurial problem-solving and decision-making. TRIZ offers a structured approach to innovation, while ChatGPT excels in generating diverse and innovative responses through advanced natural language processing. From the combination of them, we tried to discover synergies that promote innovation in highly competitive business areas. Through the analysis of case studies, including "Imperfect Waterproof Zipper" and "Drilling a Hole in a Thin-Walled Tube", we discovered that this integration not only aligns with actual problem-solving outcomes but also enhances the quality of solutions, particularly benefiting developers with limited TRIZ knowledge. We identified that leveraging ChatGPT enables developers and entrepreneurs to approach challenges with enhanced creativity, yielding practical and innovative solutions through these case studies. Our approach, focusing on real-world applications, demonstrates the study's contribution by providing a novel strategy for combining structured problem-solving with AI capabilities. The primary motivation behind this research was to ascertain whether AI can amplify the problem-solving framework of TRIZ, thereby extending its utility beyond traditional domains. The findings underline the importance of AI in creative problem-solving, suggesting that even those unfamiliar with TRIZ can apply its principles effectively with the aid of ChatGPT. This research adds to the existing knowledge by showcasing how AI can be a powerful ally in the creative process, offering new avenues for problem-solving and strategic decision-making. In conclusion, our study demonstrates that the collaboration between TRIZ and ChatGPT not only elevates creativity but also equips developers and entrepreneurs with competitive strategies, emphasizing the role of AI in driving forward human innovation and creativity.

Trustworthiness, Value, Danger, and Readability of ChatGPT-Generated Responses to Health Questions Related to Pulmonary Arterial Hypertension.

To enhance outcomes for patients with pulmonary arterial hypertension (PAH), comprehensive and individualized therapy is needed. A large language model called Generative Pre-trained Transformer (ChatGPT) has the ability to provide expert yet patient-friendly care. We wanted to determine how well ChatGPT could accurately and consistently respond to inquiries on knowledge and management for PAH. When 20 PAH patients were diagnosed, they were asked what concerns they had about PAH and what they had researched online.In the evaluation, it was determined that patients frequently searched the Internet for answers to eight queries. These eight queries were posed to ChatGPT, and their responses were recorded. Ten experts in the field of PAH assessed the trustworthiness, value, and hazard of the answers generated by the ChatGPT. According to evaluations conducted by experts, the ChatGPT-generated responses were deemed trustworthy with an average score of 8.4 (7.7-9.2) and valuable with an average score of 7.9 (7.4-8.2). Based on the statistical analysis, it can be inferred that most professionals believed that the utilization of prompts provided by ChatGPT did not present a substantial risk, with a mean of 2.1 (1.7-2.5).The answers were assessed for readability using two different indicators, namely the Flesch-Kincaid Grade Level (FKGL) and the Simple Measure of Gobbledygook (SMOG). The average FKGL value was determined to be 13.52 ± 2.40, indicating a "difficult" level of readability. ChatGPT provides reliable PAH-related information, but it is important to seek professional medical advice before making any decisions regarding PAH. ChatGPT can only provide general information and support, but a qualified healthcare provider can offer tailored recommendations.