Bidirectional Encoder Representations Research Articles

A Transparent Pipeline for Identifying Sexism in Social Media: Combining Explainability with Model Prediction

In this study, we present a new approach that combines multiple Bidirectional Encoder Representations from Transformers (BERT) architectures with a Convolutional Neural Network (CNN) framework designed for sexism detection in text at a granular level. Our method relies on the analysis and identification of the most important terms contributing to sexist content using Shapley Additive Explanations (SHAP) values. This approach involves defining a range of Sexism Scores based on both model predictions and explainability, moving beyond binary classification to provide a deeper understanding of the sexism-detection process. Additionally, it enables us to identify specific parts of a sentence and their respective contributions to this range, which can be valuable for decision makers and future research. In conclusion, this study introduces an innovative method for enhancing the clarity of large language models (LLMs), which is particularly relevant in sensitive domains such as sexism detection. The incorporation of explainability into the model represents a significant advancement in this field. The objective of our study is to bridge the gap between advanced technology and human comprehension by providing a framework for creating AI models that are both efficient and transparent. This approach could serve as a pipeline for future studies to incorporate explainability into language models.

An optimized BERT fine-tuned model using an artificial bee colony algorithm for automatic essay score prediction

Background The Automatic Essay Score (AES) prediction system is essential in education applications. The AES system uses various textural and grammatical features to investigate the exact score value for AES. The derived features are processed by various linear regressions and classifiers that require the learning pattern to improve the overall score. Issues Moreover, the classifiers face catastrophic forgetting problems, which maximizes computation complexity and reduce prediction accuracy. The forgetting problem can be resolved using the freezing mechanism; however, the mechanism can cause prediction errors. Method Therefore, this research proposes an optimized Bi-directional Encoder Representation from Transformation (BERT) by applying the Artificial Bee Colony algorithm (ABC) and Fine-Tuned Model (ABC-BERT-FTM) to solve the forgetting problem, which leads to higher prediction accuracy. Therefore, the ABC algorithm reduces the forgetting problem by selecting optimized network parameters. Results Two AES datasets, ASAP and ETS, were used to evaluate the performance of the optimized BERT of the AES system, and a high accuracy of up to 98.5% was achieved. Thus, based on the result, we can conclude that optimizing the BERT with a suitable meta-heuristic algorithm, such as the ABC algorithm, can resolve the forgetting problem, eventually increasing the AES system’s prediction accuracy.

Advancements in Word Embeddings: A Comprehensive Survey and Analysis

In recent years, the field of Natural Language Processing (NLP) has seen significant growth in the study of word representation, with word embeddings proving valuable for various NLP tasks by providing representations that encapsulate prior knowledge. We reviewed word embedding models, their applications, cross-lingual embeddings, model analyses, and techniques for model compression. We offered insights into the evolving landscape of word representations in NLP, focusing on the models and algorithms used to estimate word embeddings and their analysis strategies. To address this, we conducted a detailed examination and categorization of these evaluations and models, highlighting their significant strengths and weaknesses. We discussed a prevalent method of representing text data to capture semantics, emphasizing how different techniques can be effectively applied to interpret text data. Unlike traditional word representations, such as Word to Vector (word2vec), newer contextual embeddings, like Bidirectional Encoder Representations from Transformers (BERT) and Embeddings from Language Models (ELMo), have pushed the boundaries by capturing the use of words through diverse contexts and encoding information transfer across different languages. These embeddings leverage context to represent words, leading to innovative applications in various NLP tasks.

Cross-Task Rumor Detection: Model Optimization Based on Model Transfer Learning and Graph Convolutional Neural Networks (GCNs)

With the widespread adoption of social media, the rapid dissemination of rumors poses a severe threat to public perception and social stability, emerging as a major challenge confronting society. Hence, the development of efficient and accurate rumor detection models has become an urgent need. Given the challenges of rumor detection tasks, including data scarcity, feature complexity, and difficulties in cross-task knowledge transfer, this paper proposes a BERT–GCN–Transfer Learning model, an integrated rumor detection model that combines BERT (Bidirectional Encoder Representations from Transformers), Graph Convolutional Networks (GCNs), and transfer learning techniques. By harnessing BERT’s robust text representation capabilities, the GCN’s feature extraction prowess on graph-structured data, and the advantage of transfer learning in cross-task knowledge sharing, the model achieves effective rumor detection on social media platforms. Experimental results indicate that this model achieves accuracies of 0.878 and 0.892 on the Twitter15 and Twitter16 datasets, respectively, significantly enhancing the accuracy of rumor detection compared to baseline models. Moreover, it greatly improves the efficiency of model training.

A BERT-based with fuzzy logic sentimental classifier for sarcasm detection

This study explores the challenging task of sarcasm detection in text, a crucial aspect of sentiment analysis in Natural Language Processing (NLP). Sarcasm, characterized by irony and nuanced language, often complicates the interpretation of emotional tone in text. To address this, the study employs a hybrid model that integrates BERT (Bidirectional Encoder Representations from Transformers) with fuzzy logic. BERT's deep semantic understanding is combined with the sequential processing of LSTM and the flexible decision-making capabilities of fuzzy logic. The model's performance is validated using multiple datasets, demonstrating a significant improvement in accuracy, particularly in detecting subtle and context-dependent sarcasm. This research contributes to the advancement of sentiment analysis, offering a robust framework for handling complex linguistic expressions in various NLP applications.

Empowering geoportals HCI with task-oriented chatbots through NLP and deep transfer learning

ABSTRACT In the past ten years, chatbot development has matured to become one of the most well-distinguished outcomes of artificial intelligence. Despite some criticism, Bing AI, ChatGPT and other natural language processing (NLP) products of similar nature are becoming more popular. The creation of chatbots can close several gaps in geographic information retrieval as well. This research introduces and successfully implements, for the first time, a model for integrating task-oriented chatbots into geoportals, with the goal of easing user requests, improving access to geospatial services, and fostering human-computer interactions (HCI). Additionally, it presents a novel recommendation solution for matching the most appropriate volunteer to the user’s geospatial needs based on expertise similarity, semantic similarity, and community feedback. The three categories of finding map services, discovering geoprocessing services, and volunteer expert recommendations were shown to be the most significant geoportal bot intents. Depending on the requirement, each intent additionally includes various entities such as time, place, description, skill, etc. The notion of deep transfer learning (DTL) was then put into practice by customizing a pre-trained BERT (Bidirectional Encoder Representations from Transformers) model for our particular aim and creating a task-oriented conversational agent. According to the results, effective intent classification and entity recognition in the geospatial domain could arise from this approach. We performed the training process with 200 sample data, 20% of which were utilized in a stratified manner for testing, and we obtained f1-scores of at least 0.75. Finally, a pilot Geoportal Chabot that combines crowdsourcing and conversational agents’ approaches was put into use and tested with success. In keeping with SDI technical purposes, the system might direct users to common geospatial web services, namely WMS and WPS, in addition to including natural language understanding (NLU) and natural language generation (NLG) capabilities. Result of user-centered evaluation indicated that the integration of a chatbot significantly reduces the average time required to access geospatial data and processing services by more than 50%. Notably, this effect is even more pronounced when locating an expert, with a fivefold decrease in the time required. Finally, overall user satisfaction rose from 86% to 94%.

Machine learning and natural language processing to assess the emotional impact of influencers' mental health content on Instagram.

This study aims to examine, through artificial intelligence, specifically machine learning, the emotional impact generated by disclosures about mental health on social media. In contrast to previous research, which primarily focused on identifying psychopathologies, our study investigates the emotional response to mental health-related content on Instagram, particularly content created by influencers/celebrities. This platform, especially favored by the youth, is the stage where these influencers exert significant social impact, and where their analysis holds strong relevance. Analyzing mental health with machine learning techniques on Instagram is unprecedented, as all existing research has primarily focused on Twitter. This research involves creating a new corpus labelled with responses to mental health posts made by influencers/celebrities on Instagram, categorized by emotions such as love/admiration, anger/contempt/mockery, gratitude, identification/empathy, and sadness. The study is complemented by modelling a set of machine learning algorithms to efficiently detect the emotions arising when faced with these mental health disclosures on Instagram, using the previous corpus. Results have shown that machine learning algorithms can effectively detect such emotional responses. Traditional techniques, such as Random Forest, showed decent performance with low computational loads (around 50%), while deep learning and Bidirectional Encoder Representation from Transformers (BERT) algorithms achieved very good results. In particular, the BERT models reached accuracy levels between 86-90%, and the deep learning model achieved 72% accuracy. These results are satisfactory, considering that predicting emotions, especially in social networks, is challenging due to factors such as the subjectivity of emotion interpretation, the variability of emotions between individuals, and the interpretation of emotions in different cultures and communities. This cross-cutting research between mental health and artificial intelligence allows us to understand the emotional impact generated by mental health content on social networks, especially content generated by influential celebrities among young people. The application of machine learning allows us to understand the emotional reactions of society to messages related to mental health, which is highly innovative and socially relevant given the importance of the phenomenon in societies. In fact, the proposed algorithms' high accuracy (86-90%) in social contexts like mental health, where detecting negative emotions is crucial, presents a promising research avenue. Achieving such levels of accuracy is highly valuable due to the significant implications of false positives or false negatives in this social context.

Advancing rheumatology with natural language processing: insights and prospects from a systematic review.

Natural language processing (NLP) and large language models (LLMs) have emerged as powerful tools in healthcare, offering advanced methods for analysing unstructured clinical texts. This systematic review aims to evaluate the current applications of NLP and LLMs in rheumatology, focusing on their potential to improve disease detection, diagnosis and patient management. We screened seven databases. We included original research articles that evaluated the performance of NLP models in rheumatology. Data extraction and risk of bias assessment were performed independently by two reviewers, following Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. The Quality Assessment Tool for Observational Cohort and Cross-Sectional Studies was used to evaluate the risk of bias. Of 1491 articles initially identified, 35 studies met the inclusion criteria. These studies utilized various data types, including electronic medical records and clinical notes, and employed models like Bidirectional Encoder Representations from Transformers and Generative Pre-trained Transformers. High accuracy was observed in detecting conditions such as RA, SpAs and gout. The use of NLP also showed promise in managing diseases and predicting flares. NLP showed significant potential in enhancing rheumatology by improving diagnostic accuracy and personalizing patient care. While applications in detecting diseases like RA and gout are well developed, further research is needed to extend these technologies to rarer and more complex clinical conditions. Overcoming current limitations through targeted research is essential for fully realizing NLP's potential in clinical practice.

Development of a Natural Language Processing (NLP) model to automatically extract clinical data from electronic health records: results from an Italian comprehensive stroke center

IntroductionData collection often relies on time-consuming manual inputs, with a vast amount of information embedded in unstructured texts such as patients’ medical records and clinical notes. Our study aims to develop a pipeline that combines active learning (AL) and NLP techniques to enhance data extraction in an acute ischemic stroke cohort. Materials and methodsConsecutive acute ischemic stroke patients who received reperfusion therapies at IRCCS Humanitas Research Hospital were included. The Italian NLP Bidirectional Encoder Representations from Transformers (BERT) model was trained with AL to automatically extract clinical variables from electronic health text. Simulated active learning performances were evaluated on a set of labels representing patients’ comorbidities, comparing Bayesian Uncertainty Sampling by Disagreement (BALD) and random text selection. Prognostic models predicting patients’ functional outcomes using Gradient Boosting were trained on manually labelled and semi-automatically extracted data and their performance was compared. ResultsThe active learning process initially showed null performance until around 20% of texts were labelled, possibly due to root layers freezing in the BERT model, yet overall, active learning improves model learning efficiency across most comorbidities. Prognostic modelling showed no significant difference in performance between models trained on manually labelled versus semi-automatically extracted data, indicating effective prediction capabilities in both settings. ConclusionsWe developed an efficient language model to automate the extraction of clinical data from Italian unstructured health texts in a cohort of ischemic stroke patients. In a preliminary analysis, we demonstrated its potential applicability for enhancing prediction model accuracy.

Cascaded Adaptive Dilated Temporal Convolution Network-Based Efficient Sentiment Analysis Model from Social Media Posts

Sentimental analysis is one of the most complicated tasks in text mining. It is the process of converting unstructured text to structural text to identify the meaningful pattern of data. Different kinds of sentiments in social media posts are classified into positive and negative. Natural Language Processing (NLP) plays an imperative task in examining sentiments on social media. Sentimental analysis techniques frequently use text data to help to monitor customer feedback about products and understand the needs of customers in the field of business. The text data are gathered from social media comments that lead to several drawbacks such as spelling errors, noise, and unstructured data, and some of the data are in abbreviation form in which abbreviations are hard to understand from the text data. Therefore, the innovative sentiment analysis method is implemented to identify the kinds of sentiments from the collected social media posts. At first, the text data are assembled from the social media posts. The collected text data are subjected to text preprocessing techniques to improve the data quality. NLP techniques such as a bag of n-grams, glove embedding, and Bidirectional Encoder Representations from Transformers (BERT) are applied to separate the features from the text data. The removed features from these techniques are given to the fusion of the optimal feature stage, where the weight optimization takes place via the Mutated Random Parameter-based Serval Optimization Algorithm (MRP-SOA). The attained optimal weighted fused features contain the most relevant information, and it is fed to the classifier for analyzing the sentiments. For analyzing the sentiments, the Cascaded Adaptive Dilated Temporal Convolutional Network (CADTCN) is utilized, and the parameters from ADTCN are optimized with the same to get higher classification outcomes. The analysis outcomes are compared with the traditional sentiment analysis models in terms of various metrics to show the success of the advanced design.

Exploring Players’ Perspectives: A Comprehensive Topic Modeling Case Study on Elden Ring

Game reviews heavily influence public perception. User feedback is crucial for developers, offering valuable insights to enhance game quality. In this research, Metacritic game reviews for Elden Ring were analyzed for topic modeling using Latent Dirichlet Allocation (LDA), Bidirectional Encoder Representations from Transformers (BERT), and a hybrid model combining both to identify effective methods for extracting underlying themes in player feedback. We analyzed and interpreted these models’ outputs to learn the game reviews. We aimed to identify the differences, similarities, and variations between the three to determine which provided more valuable and instructive information. Our findings indicate that each method successfully identified keywords with some similarities in identified words. The LDA model had the highest silhouette score, indicating the most distinct clustering. The LDA-BERT model had a 1% higher coherence score than LDA, indicating more meaningful topics.

An advanced learning approach for detecting sarcasm in social media posts: Theory and solutions

AbstractObjectiveUsers of social media platforms such as Facebook, Instagram, and Twitter can view and share their daily life events through text, photographs, or videos. These platforms receive many sarcastic posts daily because there were fewer limits on what could be posted. The presence of multiple languages and media types in a single post makes it harder to identify sarcastic messages on the current platform than on posts written solely in English.MethodsThis study provides both the theory and solutions about sarcastic post detection on social platforms. Hindi–English code‐mixed data were used to train and test the automated models for sarcasm detection. The models in this study were constructed using traditional machine learning, deep neural networks, LSTM (long short‐term memory), CNN (convolutional neural network), and the combinations of BERT (Bidirectional Encoder Representations from Transformers) with LSTM.ResultsThe experimental results confirm that in the Hindi–English code‐mixed data set, the CNN, LSTM, and BERT‐LSTM ensemble perform best for sarcasm detection. The proposed model achieved an accuracy of 96.29 percent and outperformed by 2.29 percent compared to the existing models.ConclusionThe performance of the proposed system strengthens the code‐mixed sarcastic post detection on social platforms. The model will help filter not only English but also Hindi‐English code‐mixed sarcastic posts on social platforms.

Do firms listen to the ESG voices of minority investors? Evidence from China

This study examines whether and how the expression of ESG(Environmental, Social, and Governance) concerns by minority investors in their communication with firms affects firms' ESG performance. We employ the bidirectional encoder representation from transformer (BERT) model to identify ESG statements in questions asked by minority investors and construct sentence-level ESG concern indicators. The findings indicate that the more ESG-related questions investors ask in their interactions with firms, the better the firms' future ESG performance. This effect is more pronounced when investors' ESG-related questions are more negative, when managers show greater willingness to learn from investor interactions, when listed firms emphasize investor relations management, and when listed firms are held by ESG funds. Additional analyses reveal that pressure from the market and regulation may explain why listed companies listen to the ESG voice of minority investors. In addition, we find that minority investor voices do not lead to opportunistic greenwashing when driving improvements in firms' ESG performance. Overall, our findings suggest that minority investors promote corporate sustainability through active voice.

On the development and validation of large language model-based classifiers for identifying social determinants of health

The assessment of social determinants of health (SDoH) within healthcare systems is crucial for comprehensive patient care and addressing health disparities. Current challenges arise from the limited inclusion of structured SDoH information within electronic health record (EHR) systems, often due to the lack of standardized diagnosis codes. This study delves into the transformative potential of large language models (LLM) to overcome these challenges. LLM-based classifiers-using Bidirectional Encoder Representations from Transformers (BERT) and A Robustly Optimized BERT Pretraining Approach (RoBERTa)-were developed for SDoH concepts, including homelessness, food insecurity, and domestic violence, using synthetic training datasets generated by generative pre-trained transformers combined with authentic clinical notes. Models were then validated on separate datasets: Medical Information Mart for Intensive Care-III and our institutional EHR data. When training the model with a combination of synthetic and authentic notes, validation on our institutional dataset yielded an area under the receiver operating characteristics curve of 0.78 for detecting homelessness, 0.72 for detecting food insecurity, and 0.83 for detecting domestic violence. This study underscores the potential of LLMs in extracting SDoH information from clinical text. Automated detection of SDoH may be instrumental for healthcare providers in identifying at-risk patients, guiding targeted interventions, and contributing to population health initiatives aimed at mitigating disparities.

SigBERT: vibration-based steel frame structural damage detection through fine-tuning BERT

PurposeThis article introduces SigBERT, a novel approach that fine-tunes bidirectional encoder representations from transformers (BERT) for the purpose of distinguishing between intact and impaired structures by analyzing vibration signals. Structural health monitoring (SHM) systems are crucial for identifying and locating damage in civil engineering structures. The proposed method aims to improve upon existing methods in terms of cost-effectiveness, accuracy and operational reliability.Design/methodology/approachSigBERT employs a fine-tuning process on the BERT model, leveraging its capabilities to effectively analyze time-series data from vibration signals to detect structural damage. This study compares SigBERT's performance with baseline models to demonstrate its superior accuracy and efficiency.FindingsThe experimental results, obtained through the Qatar University grandstand simulator, show that SigBERT outperforms existing models in terms of damage detection accuracy. The method is capable of handling environmental fluctuations and offers high reliability for non-destructive monitoring of structural health. The study mentions the quantifiable results of the study, such as achieving a 99% accuracy rate and an F-1 score of 0.99, to underline the effectiveness of the proposed model.Originality/valueSigBERT presents a significant advancement in SHM by integrating deep learning with a robust transformer model. The method offers improved performance in both computational efficiency and diagnostic accuracy, making it suitable for real-world operational environments.

Enhancing Literature Review Efficiency: A Case Study on Using Fine-Tuned BERT for Classifying Focused Ultrasound-Related Articles

Over the past decade, focused ultrasound (FUS) has emerged as a promising therapeutic modality for various medical conditions. However, the exponential growth in the published literature on FUS therapies has made the literature review process increasingly time-consuming, inefficient, and error-prone. Machine learning approaches offer a promising solution to address these challenges. Therefore, the purpose of our study is to (1) explore and compare machine learning techniques for the text classification of scientific abstracts, and (2) integrate these machine learning techniques into the conventional literature review process. A classified dataset of 3588 scientific abstracts related and unrelated to FUS therapies sourced from the PubMed database was used to train various traditional machine learning and deep learning models. The fine-tuned Bio-ClinicalBERT (Bidirectional Encoder Representations from Transformers) model, which we named FusBERT, had comparatively optimal performance metrics with an accuracy of 0.91, a precision of 0.85, a recall of 0.99, and an F1 of 0.91. FusBERT was then successfully integrated into the literature review process. Ultimately, the integration of this model into the literature review pipeline will reduce the number of irrelevant manuscripts that the clinical team must screen, facilitating efficient access to emerging findings in the field.

A Domain-Specific Lexicon for Improving Emergency Management in Gas Pipeline Networks through Knowledge Fusing

Emergencies in gas pipeline networks can lead to significant loss of life and property, necessitating extensive professional knowledge for effective response and management. Effective emergency response depends on specialized knowledge, which can be captured efficiently through domain-specific lexicons. The goal of this research is to develop a specialized lexicon that integrates domain-specific knowledge to improve emergency management in gas pipeline networks. The process starts with an enhanced version of Term Frequency–Inverse Document Frequency (TF-IDF), a statistical method used in information retrieval, combined with filtering logic to extract candidate words from investigation reports. Simultaneously, we fine tune the Chinese Bidirectional Encoder Representations from Transformers (BERT) model, a state-of-the-art language model, with domain-specific data to enhance semantic capture and integrate domain knowledge. Next, words with similar meanings are identified through word similarity analysis based on standard terminology and risk inventories, facilitating lexicon expansion. Finally, the domain-specific lexicon is formed by amalgamating these words. Validation shows that this method, which integrates domain knowledge, outperforms models that lack such integration. The resulting lexicon not only assigns domain-specific weights to terms but also deeply embeds domain knowledge, offering robust support for cause analysis and emergency management in gas pipeline networks.

Unveiling the hidden patterns: A novel semantic deep learning approach to fake news detection on social media

The rise of social media as a source of news consumption has led to the spread of fake news, posing serious consequences for both individuals and society. The detection and prevention of fake news are essential, and previous research has shown that incorporating news content along with its associated headlines and user comments can improve detection performance. However, the semantic relationships between these elements have not been fully explored. This paper proposes a novel approach that models the relationships between news bodies and associated headlines/user comments using deep learning techniques, such as fine-tuned Bidirectional Encoder Representations from Transformers (BERT) and cross-level cross-modality attention sub-networks. In our proposed model, we utilize two different configurations of BERT: pool-based representation, which provides a representation of the entire document, and sequence representation, which represents each token within the document (i.e., at the word and text levels). The approach also encodes user-posting behavioural features and fuses the output of these components to detect fake news using a classification layer. Our experiments on benchmark datasets demonstrate the superiority of the proposed method over existing state-of-the-art (SOTA) approaches, highlighting the importance of utilizing semantic relationships for improved fake news detection (FND). These findings have significant implications for combating the spread of fake news and protecting society from its negative effects.

Fine-grained vulnerability detection for medical sensor systems

The Internet of Things (IoT) has revolutionized the healthcare system by connecting medical sensors to the internet, while also posing challenges to the security of medical sensor networks (MSN). Given the extreme sensitivity of medical data, any vulnerability may result in data breaches and misuse, impacting patient safety and privacy. Therefore, safeguarding MSN security is critical. As medical sensor devices rely on smart healthcare software systems for data management and communication, precisely detecting system code vulnerabilities is essential to ensuring network security. Effective software vulnerability detection targets two key objectives: (i) achieving high accuracy and (ii) directly identifying vulnerable code lines for developers to fix. To address these challenges, we introduce Vulcoder, a novel vulnerability-oriented, encoder-driven model based on the Bidirectional Encoder Representations from Transformers (BERT) architecture. We propose a one-to-one mapping function to capture code semantics through abstract syntax trees (AST). Combined with multi-head attention, Vulcoder achieves precise function- and line-level detection of software vulnerabilities in MSN. This accelerates the vulnerability remediation process, thereby strengthening network security. Experimental results on various datasets demonstrate that Vulcoder outperforms previous models in identifying vulnerabilities within MSN. Specifically, it achieves a 1%–419% improvement in function-level prediction F1 scores and a 12.5%–380% increase in line-level localization precision. Therefore, Vulcoder helps enhance security defenses and safeguard patient privacy in MSN, facilitating the development of smart healthcare.

CACER: Clinical concept Annotations for Cancer Events and Relations.

Clinical notes contain unstructured representations of patient histories, including the relationships between medical problems and prescription drugs. To investigate the relationship between cancer drugs and their associated symptom burden, we extract structured, semantic representations of medical problem and drug information from the clinical narratives of oncology notes. We present Clinical concept Annotations for Cancer Events and Relations (CACER), a novel corpus with fine-grained annotations for over 48000 medical problems and drug events and 10000 drug-problem and problem-problem relations. Leveraging CACER, we develop and evaluate transformer-based information extraction models such as Bidirectional Encoder Representations from Transformers (BERT), Fine-tuned Language Net Text-To-Text Transfer Transformer (Flan-T5), Large Language Model Meta AI (Llama3), and Generative Pre-trained Transformers-4 (GPT-4) using fine-tuning and in-context learning (ICL). In event extraction, the fine-tuned BERT and Llama3 models achieved the highest performance at 88.2-88.0 F1, which is comparable to the inter-annotator agreement (IAA) of 88.4 F1. In relation extraction, the fine-tuned BERT, Flan-T5, and Llama3 achieved the highest performance at 61.8-65.3 F1. GPT-4 with ICL achieved the worst performance across both tasks. The fine-tuned models significantly outperformed GPT-4 in ICL, highlighting the importance of annotated training data and model optimization. Furthermore, the BERT models performed similarly to Llama3. For our task, large language models offer no performance advantage over the smaller BERT models. We introduce CACER, a novel corpus with fine-grained annotations for medical problems, drugs, and their relationships in clinical narratives of oncology notes. State-of-the-art transformer models achieved performance comparable to IAA for several extraction tasks.