Human Annotators Research Articles

Segment-anything embedding for pixel-level road damage extraction using high-resolution satellite images

When a strong earthquake occurs, roads are the lifelines of rescue. The rapid development of high-resolution satellite imaging platforms has made the application of remote sensing technology for road damage identification possible. Over the years, road damage identification has required a significant amount of manual involvement, making it difficult to meet the needs of rapid post-disaster response. The automatic recognition of road damage using satellite images has always been difficult. Damaged areas appear in the satellite images with blurry boundaries, versatile sizes, and uneven spatial distributions. With the aim of automatic pixel-level road damage identification, we introduce the first road damage dataset, CAU-RoadDamage, which includes high-resolution satellite images and pixel-level human annotations. Moreover, we propose the application of a pre-trained vision foundation model for the first time to automatically identify road damage. Low-rank adaptation technology is used to fine-tune the foundation model on the satellite images, and two-way attention is used to integrate the foundation model with domain specialist model components. The proposed segmentation model is compared to multiple state-of-the-art methods on the CAU-RoadDamage dataset. Our approach achieves the highest F1 of 76.09%, which is notably higher than that of the other models. The experimental results demonstrate the feasibility of pixel-level road damage recognition and the applicability of vision foundation models for downstream remote sensing tasks. The CAU-RoadDamage dataset will be made publicly available at https://github.com/CAU-HE/RoadDamageExtraction.

A versatile automated pipeline for quantifying virus infectivity by label-free light microscopy and artificial intelligence

Virus infectivity is traditionally determined by endpoint titration in cell cultures, and requires complex processing steps and human annotation. Here we developed an artificial intelligence (AI)-powered automated framework for ready detection of virus-induced cytopathic effect (DVICE). DVICE uses the convolutional neural network EfficientNet-B0 and transmitted light microscopy images of infected cell cultures, including coronavirus, influenza virus, rhinovirus, herpes simplex virus, vaccinia virus, and adenovirus. DVICE robustly measures virus-induced cytopathic effects (CPE), as shown by class activation mapping. Leave-one-out cross-validation in different cell types demonstrates high accuracy for different viruses, including SARS-CoV-2 in human saliva. Strikingly, DVICE exhibits virus class specificity, as shown with adenovirus, herpesvirus, rhinovirus, vaccinia virus, and SARS-CoV-2. In sum, DVICE provides unbiased infectivity scores of infectious agents causing CPE, and can be adapted to laboratory diagnostics, drug screening, serum neutralization or clinical samples.

Integrating Widespread Coral Reef Monitoring Tools for Managing both Area and Point Annotations

Abstract. Large-area image acquisition techniques are essential in underwater investigations: high-resolution 3D image-based reconstructions have improved coral reef monitoring by enabling novel seascape ecological analysis. Artificial intelligence (AI) offers methods for significantly accelerating image data interpretation, such as automatically recognizing, enumerating, and measuring organisms. However, the rapid proliferation of these technological achievements has led to a relative lack of standardization of methods. Remarkably, there are notable differences in procedures for generating human and AI annotations, and there is also a scarcity of publicly available datasets and shared machine-learning models. The lack of standard procedures makes it challenging to compare and reproduce scientific findings. One way to overcome this problem is to make the most used platforms by coral reef scientists interoperable so that the analyses can all be exported into a common format. This paper introduces functionality to promote interoperability between three popular open-source software tools dedicated to the digital study of coral reefs: TagLab, CoralNet, and Viscore. As users of each platform may have different analysis pipelines, we discuss several workflows for managing and processing point and area annotations, improving collaboration among these tools. Our work sets the foundation for a more seamless ecosystem that maintains the established investigation procedures of various laboratories but allows for easier result sharing.

Diverse and tailored image generation for zero-shot multi-label classification

Recently, zero-shot multi-label classification has garnered considerable attention owing to its capacity to predict unseen labels without human annotations. Nevertheless, prevailing approaches often use seen classes as imperfect proxies for unseen classes, resulting in suboptimal performance. Drawing inspiration from the success of text-to-image generation models in producing realistic images, we propose an innovative solution: generating synthetic data to construct a training set explicitly tailored for proxyless training of unseen labels. Our approach introduces a novel image generation framework that produces multi-label synthetic images of unseen classes for classifier training. To enhance the diversity of the generated images, we leverage a pretrained large language model to generate diverse prompts. Employing a pretrained multimodal contrastive language-image pretraining (CLIP) model as a discriminator, we assessed whether the generated images accurately represented the target classes. This enables automatic filtering of inaccurately generated images and preserves classifier accuracy. To refine the text prompts for more precise and effective multi-label object generation, we introduced a CLIP score-based discriminative loss to fine-tune the text encoder in the diffusion model. In addition, to enhance the visual features of the target task while maintaining the generalization of the original features and mitigating catastrophic forgetting resulting from fine-tuning the entire visual encoder, we propose a feature fusion module inspired by transformer attention mechanisms. This module aids in capturing the global dependencies between multiple objects more effectively. Extensive experimental results validated the effectiveness of our approach, demonstrating significant improvements over state-of-the-art methods. The code is available at https://github.com/TAKELAMAG/Diff-ZS-MLC.

Unsupervised 3D Object Segmentation of Point Clouds by Geometry Consistency.

In this paper, we study the problem of 3D object segmentation from raw point clouds. Unlike existing methods which usually require a large amount of human annotations for full supervision, we propose the first unsupervised method, called OGC, to simultaneously identify multiple 3D objects in a single forward pass, without needing any type of human annotations. The key to our approach is to fully leverage the dynamic motion patterns over sequential point clouds as supervision signals to automatically discover rigid objects. Our method consists of three major components, 1) the object segmentation network to directly estimate multi-object masks from a single point cloud frame,2)the auxiliary self-supervised scene flow estimator,and 3)our core object geometry consistency component. By carefully designing a series of loss functions, we effectively take into account the multi-object rigid consistency and the object shape invariance in both temporal and spatial scales. This allows our method to truly discover the object geometry even in the absence of annotations. We extensively evaluate our method on five datasets, demonstrating the superior performance for object part instance segmentation and general object segmentation in both indoor and the challenging outdoor scenarios. Our code and data are available at https://github.com/vLAR-group/OGC.

Study on the Domain Adaption of Korean Speech Act using Daily Conversation Dataset and Petition Corpus

In Korean, quantitative speech act studies have usually been conducted on single utterances with unspecified sources. In this study, we annotate sentences from the National Institute of Korean Language's Messenger Corpus and the National Petition Corpus, as well as example sentences from an academic paper on contemporary Korean vlogging, and check the discrepancy between human annotation and model prediction. In particular, for sentences with differences in locutionary and illocutionary forces, we analyze the causes of errors to see if stylistic features used in a particular domain affect the correct inference of speech act. Through this, we see the necessity to build and analyze a balanced corpus in various text domains, taking into account cases with different usage roles, e.g., messenger conversations belonging to private conversations and petition corpus/vlogging script that have an unspecified audience.

Synthetic Optical Coherence Tomography Angiographs for Detailed Retinal Vessel Segmentation Without Human Annotations.

Optical coherence tomography angiography (OCTA) is a non-invasive imaging modality that can acquire high-resolution volumes of the retinal vasculature and aid the diagnosis of ocular, neurological and cardiac diseases. Segmenting the visible blood vessels is a common first step when extracting quantitative biomarkers from these images. Classical segmentation algorithms based on thresholding are strongly affected by image artifacts and limited signal-to-noise ratio. The use of modern, deep learning-based segmentation methods has been inhibited by a lack of large datasets with detailed annotations of the blood vessels. To address this issue, recent work has employed transfer learning, where a segmentation network is trained on synthetic OCTA images and is then applied to real data. However, the previously proposed simulations fail to faithfully model the retinal vasculature and do not provide effective domain adaptation. Because of this, current methods are unable to fully segment the retinal vasculature, in particular the smallest capillaries. In this work, we present a lightweight simulation of the retinal vascular network based on space colonization for faster and more realistic OCTA synthesis. We then introduce three contrast adaptation pipelines to decrease the domain gap between real and artificial images. We demonstrate the superior segmentation performance of our approach in extensive quantitative and qualitative experiments on three public datasets that compare our method to traditional computer vision algorithms and supervised training using human annotations. Finally, we make our entire pipeline publicly available, including the source code, pretrained models, and a large dataset of synthetic OCTA images.

Deep learning-based classification of erosion, synovitis and osteitis in hand MRI of patients with inflammatory arthritis

ObjectivesTo train, test and validate the performance of a convolutional neural network (CNN)-based approach for the automated assessment of bone erosions, osteitis and synovitis in hand MRI of patients with...

Natural language processing for automated breast cancer recurrence detection and classification in computed tomography reports.

e13591 Background: Relapse is a major concern for oncologists and breast cancer survivors that necessitates additional treatment and often leads to mortality. Cancer registries routinely track cancer mortality, but few monitor for relapse because of logistical challenges and prohibitive costs. In this context, Natural Language Processing (NLP) is a promising tool. Merging artificial intelligence with linguistics, NLP can rapidly analyze vast volumes of text in electronic health records. This capability of NLP is particularly valuable for Computed Tomography (CT) scans used in breast cancer care. CT scans are routinely used to characterize breast cancer progression and are described in transcribed dictations by radiologists. We aimed to apply NLP to these text reports to identify breast cancer relapses. Objective: To automate breast cancer relapse detection and classification in CT text reports using NLP. Methods: We analyzed 1,445 CT text reports from patients diagnosed with breast cancer between January 1, 2005, and December 31, 2014. These reports underwent manual review by trained human annotators. Text was annotated to identify terminology defining local, regional, and distant breast cancer relapses. Annotated reports were partitioned into a training-validation set (90% cohort) and a test set (10% cohort) for NLP model development. Results: In our dataset of 1,445 CT text reports, 72 (5.0%) were classified as local relapse, 97 (6.7%) as regional relapse, and 743 (51.4%) as distant relapse. The performance of our NLP model using the training-validation dataset can be summarized by the following metrics and 95% confidence intervals: 94% (±3.2) accuracy for detection and 96% (±2.9) accuracy for classification. The performance of our NLP model was confirmed using the test dataset, with 90% (±4.5) accuracy for detection and 91% (±6.3) accuracy for classification. For reference, all metrics are outlined (Table). Conclusions: Our model for identifying regional and distant relapses in CT reports had excellent performance, but had lower sensitivity for local relapses, posing a risk of false negatives. Automating the identification and classification of breast cancer relapses, if used retrospectively, can enhance cancer registry data about patient outcomes and, if used prospectively, holds the potential for enhancing patient care. [Table: see text]

Saliency infused dialogue response generation: Improving task oriented text generation using feature attribution

Challenges persist in dialogue scenarios, particularly in multi-turn dialogues where response generation often disregards contextual information beyond the last user utterance, resulting in fluent yet inadequate responses. This paper addresses these issues by identifying and resolving common shortcomings in base model responses during response generation and proposes methods to enhance response quality in unannotated dialogue settings. Our approach involves augmenting information from multiple sources, including keywords, salient features, and knowledge graph triples. We compare the effectiveness of these methods against both the base model and human annotation, which includes dialogue acts and entities. Our findings demonstrate that appending extracted tokens significantly enhances response quality compared to annotated information. In task-oriented dialogue, models perform best when infused with saliency and knowledge graph triples, as shown in the MultiWOZ dataset. Conversely, focusing solely on saliency yields better results for open-domain dialogue, as demonstrated with the DailyDialog dataset. For contextual relevance, the information infusion could also approach the performance of the LLama2 model with only a tenth of the available parameters.

A sentiment corpus for the cryptocurrency financial domain: the CryptoLin corpus

AbstractThe objective of this paper is to describe Cryptocurrency Linguo (CryptoLin), a novel corpus containing 2683 cryptocurrency-related news articles covering more than a three-year period. CryptoLin was human-annotated with discrete values representing negative, neutral, and positive news respectively. Eighty-three people participated in the annotation process; each news title was randomly assigned and blindly annotated by three human annotators, one in each different cohort, followed by a consensus mechanism using simple voting. The selection of the annotators was intentionally made using three cohorts with students from a very diverse set of nationalities and educational backgrounds to minimize bias as much as possible. In case one of the annotators was in total disagreement with the other two (e.g., one negative vs two positive or one positive vs two negative), we considered this minority report and defaulted the labeling to neutral. Fleiss’s Kappa, Krippendorff’s Alpha, and Gwet’s AC1 inter-rater reliability coefficients demonstrate CryptoLin’s acceptable quality of inter-annotator agreement. The dataset also includes a text span with the three manual label annotations for further auditing of the annotation mechanism. To further assess the quality of the labeling and the usefulness of CryptoLin dataset, it incorporates four pretrained Sentiment Analysis models: Vader, Textblob, Flair, and FinBERT. Vader and FinBERT demonstrate reasonable performance in the CryptoLin dataset, indicating that the data was not annotated randomly and is therefore useful for further research1. FinBERT (negative) presents the best performance, indicating an advantage of being trained with financial news. Both the CryptoLin dataset and the Jupyter Notebook with the analysis, for reproducibility, are available at the project’s Github. Overall, CryptoLin aims to complement the current knowledge by providing a novel and publicly available Gadi and Ángel Sicilia (Cryptolin dataset and python jupyter notebooks reproducibility codes, 2022) cryptocurrency sentiment corpus and fostering research on the topic of cryptocurrency sentiment analysis and potential applications in behavioral science. This can be useful for businesses and policymakers who want to understand how cryptocurrencies are being used and how they might be regulated. Finally, the rules for selecting and assigning annotators make CryptoLin unique and interesting for new research in annotator selection, assignment, and biases.

SNIPPET: A Framework for Subjective Evaluation of Visual Explanations Applied to DeepFake Detection

Explainable Artificial Intelligence (XAI) attempts to help humans understand machine learning decisions better and has been identified as a critical component towards increasing the trustworthiness of complex black-box systems, such as deep neural networks (DNNs). In this paper, we propose a generic and comprehensive framework named SNIPPET and create a user interface for the subjective evaluation of visual explanations, focusing on finding human-friendly explanations. SNIPPET considers human-centered evaluation tasks and incorporates the collection of human annotations. These annotations can serve as valuable feedback to validate the qualitative results obtained from the subjective assessment tasks. Moreover, we consider different user background categories during the evaluation process to ensure diverse perspectives and comprehensive evaluation. We demonstrate SNIPPET on a DeepFake face dataset. Distinguishing real from fake faces is a non-trivial task even for humans, that depends on rather subtle features, making it a challenging use case. Using SNIPPET, we evaluate four popular XAI methods which provide visual explanations: Gradient-weighted Class Activation Mapping (GradCAM), Layer-wise Relevance Propagation (LRP), attention rollout (rollout), and Transformer Attribution (TA). Based on our experimental results, we observe preference variations among different user categories. We find that most people are more favorable to the explanations of rollout. Moreover, when it comes to XAI-assisted understanding, those who have no or lack relevant background knowledge often consider that visual explanations are insufficient to help them understand. We open-source our framework for continued data collection and annotation at https://github.com/XAI-SubjEvaluation/SNIPPET.

Multiple Image Distortion DNN Modeling Individual Subject Quality Assessment

A recent research direction is focused on training Deep Neural Networks (DNNs) to replicate individual subject assessments of media quality. These DNNs are referred to as Artificial Intelligence-based Observers (AIOs). An AIO is designed to simulate, in real-time, the quality ratings of a specific individual, enabling an automatic quality assessment that accounts for subjects characteristics and preferences. Training AIOs is a promising but challenging research area due to the greater noise in individual raw opinion scores compared to the Mean Opinion Score (MOS). Effective learning from noisy labels necessitates the training of complex models on large-scale datasets. Unfortunately, this is challenging for AIOs as the media quality assessment community lacks extensive datasets that include individual opinion scores. To address the complexity of the task, we first created a dataset comprising two million samples, with synthetic labels derived from human annotation. We then trained a customized network for image quality assessment, named Multi-Distortion ResNet50 (MDResNet50), on this dataset. The weights of the MDResNet50 were subsequently utilized to initialize the learning process of each AIO, thereby avoiding the need to train a complex model from scratch on a small-scale dataset with raw individual opinion scores. Computational experiments show that our approach significantly advances the state-of-the-art in the AIO research. In particular: i) we demonstrate through a simulation the ability of AIOs to mimic two well-known behavioral characteristics of a subject, i.e.,bias and inconsistency, when scoring the media quality; ii) we train and release DNN-based AIOs that, compared to the state-of-the-art, exhibit a higher performance with a statistical significance in assessing multiple image distortions; iii) we train AIOs that more accurately mimic the sensitivity of real subjects to noise and color saturation, and also better predict the opinion score distribution compared to the state-of-the-art AIOs.

ProtVar: mapping and contextualizing human missense variation.

Genomic variation can impact normal biological function in complex ways and so understanding variant effects requires a broad range of data to be coherently assimilated. Whilst the volume of human variant data and relevant annotations has increased, the corresponding increase in the breadth of participating fields, standards and versioning mean that moving between genomic, coding, protein and structure positions is increasingly complex. In turn this makes investigating variants in diverse formats and assimilating annotations from different resources challenging. ProtVar addresses these issues to facilitate the contextualization and interpretation of human missense variation with unparalleled flexibility and ease of accessibility for use by the broadest range of researchers. By precalculating all possible variants in the human proteome it offers near instantaneous mapping between all relevant data types. It also combines data and analyses from a plethora of resources to bring together genomic, protein sequence and function annotations as well as structural insights and predictions to better understand the likely effect of missense variation in humans. It is offered as an intuitive web server https://www.ebi.ac.uk/protvar where data can be explored and downloaded, and can be accessed programmatically via an API.

Limitations of Large Language Models in Propaganda Detection Task

Propaganda in the digital era is often associated with online news. In this study, we focused on the use of large language models and their detection of propaganda techniques in the electronic press to investigate whether it is a noteworthy replacement for human annotators. We prepared prompts for generative pre-trained transformer models to find spans in news articles where propaganda techniques appear and name them. Our study was divided into three experiments on different datasets—two based on an annotated SemEval2020 Task 11 corpora and one on an unannotated subset of the Polish Online News Corpus, which we claim to be an even bigger challenge as an example of an under-resourced language. Reproduction of the results of the first experiment resulted in a higher recall of 64.53% than the original run, and the highest precision of 81.82% was achieved for gpt-4-1106-preview CoT. None of our attempts outperformed the baseline F1 score. One of the attempts with gpt-4-0125-preview on original SemEval2020 Task 11 achieved an almost 20% F1 score, but it was below the baseline, which oscillated around 50%. Part of our work that was dedicated to Polish articles showed that gpt-4-0125-preview had a 74% accuracy in the binary detection of propaganda techniques and 69% in propaganda technique classification. The results for SemEval2020 show that the outputs of generative models tend to be unpredictable and are hardly reproducible for propaganda detection. For the time being, these are unreliable methods for this task, but we believe they can help to generate more training data.

Transformative User Credibility Assessment on Twitter: A RNN based Heuristic Approach

Objectives: To construct a comprehensive weighted multi-dimensional model to assess the impact of influence score of Twitter users, considering the credibility based on user profile, their tweets and social interactions aiming to empower users in distinguishing fake news or misinformation. Methods: The credibility evaluation is formulated based on text analysis, user account attributes, and user social engagement. We've gathered around 100,000 tweets from 100 users using Tweepy API over a six-month duration for the purpose of evaluating credibility. The collected tweets spanned diverse professions namely politics, entertainment, business, science, sports, and trending topics. We chose to utilize a self-devised deep active learning model to classify and label the unlabelled data instead of engaging in time-consuming human annotation for the tweets we gathered. Findings: The obtained accuracy for influence score evaluation for Recurrent Neural Network, Random Forest, Naïve Bayes, Decision Tree, and Support Vector Machine are 89.03%, 79.10%, 81.59%, 73.06% and 79.45% respectively. Upon reviewing and analysing the outcomes, RNN surpassed all other models achieving an exceptional accuracy of 89.03%. Novelty: Employing a weighted multi-dimensional framework, it systematically evaluates the influence score by considering the credibility of both users and tweets within the context of Twitter. Weighted features are instrumental in capturing the relative importance of different elements, leading to a more refined and context-aware decision-making process. In contrast to earlier research, which predominantly centred on the credibility of individual tweets, our research work shifts the focus to a broader perspective, encompassing the credibility of users, their tweets and their overall social influence. By incorporating user influence score, the framework not only empower users in discerning fake news or mis-information but also elevates their ability to gauge the reliability of information, offering a nuanced approach to news credibility analysis. Keywords: Active Learning, Credibility Score, User Influence, Twitter, Machine Learning, Recurrent Neural Network

Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answering

Abstract Instruction-following models are attractive alternatives to fine-tuned approaches for question answering (QA). By simply prepending relevant documents and an instruction to their input, these models can be adapted to various information domains and tasks without additional training. However, these models tend to produce verbose responses with supplementary information, which makes traditional QA metrics like exact match (EM) and F1 unreliable for accurately quantifying model performance. In this work, we evaluate instruction-following models along two fronts: 1) how well they satisfy user’s information need (correctness), and 2) whether they disseminate information supported by the provided knowledge (faithfulness). Guided by human evaluation and analysis, we highlight the shortcomings of traditional metrics for both correctness and faithfulness and propose simple token-overlap metrics that correlate highly with human judgments. Our analysis reveals that for correctness, instruction-following models perform comparably to models specifically fine-tuned for that task. However, they struggle to accurately judge the relevance of the provided knowledge and often hallucinate in their responses. We hope our work encourages more holistic evaluation of instruction-following models for QA. Our code and human annotation data is available at https://github.com/McGill-NLP/instruct-qa.

An Artificial Intelligence model for implant segmentation on periapical radiographs.

To segment dental implants on PA radiographs using a Deep Learning (DL) algorithm. To compare the performance of the algorithm relative to ground truth determined by the human annotator. Three hundred PA radiographs were retrieved from the radiographic database and consequently annotated to label implants as well as teeth on the LabelMe annotation software. The dataset was augmented to increase the number of images in the training data and a total of 1294 images were used to train, validate and test the DL algorithm. An untrained U-net was downloaded and trained on the annotated dataset to allow detection of implants using polygons on PA radiographs. A total of one hundred and thirty unseen images were run through the trained U-net to determine its ability to segment implants on PA radiographs. The performance metrics are as follows: accuracy of 93.8%, precision of 90%, recall of 83%, F-1 score of 86%, Intersection over Union of 86.4% and loss = 21%. The trained DL algorithm segmented implants on PA radiographs with high performance similar to that of the humans who labelled the images forming the ground truth.

Using Large Language Models to Support Content Analysis: A Case Study of ChatGPT for Adverse Event Detection.

This study explores the potential of using large language models to assist content analysis by conducting a case study to identify adverse events (AEs) in social media posts. The case study compares ChatGPT's performance with human annotators' in detecting AEs associated with delta-8-tetrahydrocannabinol, a cannabis-derived product. Using the identical instructions given to human annotators, ChatGPT closely approximated human results, with a high degree of agreement noted: 94.4% (9436/10,000) for any AE detection (Fleiss κ=0.95) and 99.3% (9931/10,000) for serious AEs (κ=0.96). These findings suggest that ChatGPT has the potential to replicate human annotation accurately and efficiently. The study recognizes possible limitations, including concerns about the generalizability due to ChatGPT's training data, and prompts further research with different models, data sources, and content analysis tasks. The study highlights the promise of large language models for enhancing the efficiency of biomedical research.

A critical assessment of using ChatGPT for extracting structured data from clinical notes.

Existing natural language processing (NLP) methods to convert free-text clinical notes into structured data often require problem-specific annotations and model training. This study aims to evaluate ChatGPT's capacity to extract information from free-text medical notes efficiently and comprehensively. We developed a large language model (LLM)-based workflow, utilizing systems engineering methodology and spiral "prompt engineering" process, leveraging OpenAI's API for batch querying ChatGPT. We evaluated the effectiveness of this method using a dataset of more than 1000 lung cancer pathology reports and a dataset of 191 pediatric osteosarcoma pathology reports, comparing the ChatGPT-3.5 (gpt-3.5-turbo-16k) outputs with expert-curated structured data. ChatGPT-3.5 demonstrated the ability to extract pathological classifications with an overall accuracy of 89%, in lung cancer dataset, outperforming the performance of two traditional NLP methods. The performance is influenced by the design of the instructive prompt. Our case analysis shows that most misclassifications were due to the lack of highly specialized pathology terminology, and erroneous interpretation of TNM staging rules. Reproducibility shows the relatively stable performance of ChatGPT-3.5 over time. In pediatric osteosarcoma dataset, ChatGPT-3.5 accurately classified both grades and margin status with accuracy of 98.6% and 100% respectively. Our study shows the feasibility of using ChatGPT to process large volumes of clinical notes for structured information extraction without requiring extensive task-specific human annotation and model training. The results underscore the potential role of LLMs in transforming unstructured healthcare data into structured formats, thereby supporting research and aiding clinical decision-making.