Event Extraction Research Articles

MultiADE: A Multi-domain benchmark for Adverse Drug Event extraction

Objective:Active adverse event surveillance monitors Adverse Drug Events (ADE) from different data sources, such as electronic health records, medical literature, social media and search engine logs. Over the years, many datasets have been created, and shared tasks have been organised to facilitate active adverse event surveillance. However, most—if not all—datasets or shared tasks focus on extracting ADEs from a particular type of text. Domain generalisation—the ability of a machine learning model to perform well on new, unseen domains (text types)—is under-explored. Given the rapid advancements in natural language processing, one unanswered question is how far we are from having a single ADE extraction model that is effective on various types of text, such as scientific literature and social media posts. Methods:We contribute to answering this question by building a multi-domain benchmark for adverse drug event extraction, which we named MultiADE. The new benchmark comprises several existing datasets sampled from different text types and our newly created dataset—CADECv2, which is an extension of CADEC (Karimi et al., 2015), covering online posts regarding more diverse drugs than CADEC. Our new dataset is carefully annotated by human annotators following detailed annotation guidelines. Conclusion:Our benchmark results show that the generalisation of the trained models is far from perfect, making it infeasible to be deployed to process different types of text. In addition, although intermediate transfer learning is a promising approach to utilising existing resources, further investigation is needed on methods of domain adaptation, particularly cost-effective methods to select useful training instances.The newly created CADECv2 and the scripts for building the benchmark are publicly available at CSIRO’s Data Portal (https://data.csiro.au/collection/csiro:62387). These resources enable the research community to further information extraction, leading to more effective active adverse drug event surveillance.

MaskDGNets: Masked-attention guided dynamic graph aggregation network for event extraction.

Considering that the traditional deep learning event extraction method ignores the correlation between word features and sequence information, it cannot fully explore the hidden associations between events and events and between events and primary attributes. To solve these problems, we developed a new framework for event extraction called the masked attention-guided dynamic graph aggregation network. On the one hand, to obtain effective word representation and sequence representation, an interaction and complementary relationship are established between word vectors and character vectors. At the same time, a squeeze layer is introduced in the bidirectional independent recurrent unit to model the sentence sequence from both positive and negative directions while retaining the local spatial details to the maximum extent and establishing practical long-term dependencies and rich global context representations. On the other hand, the designed masked attention mechanism can effectively balance the word vector features and sequence semantics and refine these features. The designed dynamic graph aggregation module establishes effective connections between events and events, and between events and essential attributes, strengthens the interactivity and association between them, and realizes feature transfer and aggregation on graph nodes in the neighborhood through dynamic strategies to improve the performance of event extraction. We designed a reconstructed weighted loss function to supervise and adjust each module individually to ensure the optimal feature representation. Finally, the proposed MaskDGNets framework is evaluated on two baseline datasets, DuEE and CCKS2020. It demonstrates its robustness and event extraction performance, with F1 of 81.443% and 87.382%, respectively.

Event type induction using latent variables with hierarchical relationship analysis

The conventional approach to event extraction requires predefined event types and their corresponding annotations to train event extractors. However, these prerequisites are often difficult to satisfy in real-world applications. To automatically induct event types, most work has been devoted to clustering event triggers, where a cluster of event triggers is represented as an event type. Some works use trigger semantics, while others use co-occurrence relationships to cluster triggers. However, the clustering results of event triggers obtained by the above work are not sufficiently detailed in describing event types, making it difficult to accurately determine the corresponding event types manually. This paper proposes an open-domain event type induction framework that automatically discovers a set of event types from a given corpus. Unlike previous work on event trigger clustering, this paper takes into consideration the hierarchical relationship of event types to partition the event trigger clusters into event mains and subtypes. The framework employs a latent variable-based neural generation module and a semantic-based clustering module, the former of which obtains event trigger clusters representing the main types of events by jointly projecting the co-occurrence and semantic information of event triggers into a latent space for event type latent variable mining, and the latter of which further divides these event trigger clusters into event subtypes based on semantic information. Finally, experiment results show that, compared with the benchmark model, the ETGen-Clus can improve event type quality scores of 6.23% and 3.11% on the two datasets, respectively.

A Novel Technique for the Extraction of Dynamic Events in Extreme Ultraviolet Solar Images

High-spatial-resolution images of the solar corona acquired in the extreme ultraviolet (EUV), most notably with the Atmospheric Imaging Assembly (AIA) instrument on the Solar Dynamics Observatory (SDO) reveal the abundance of dynamic events which range from flaring bright points and jets to erupting prominences and coronal mass ejections (CMEs). In this work we present novel techniques to extract such dynamic events from the more steady background corona using 17.1 nm SDO-AIA images. The techniques presented here treat any time series of coronal images as a matrix that can be decomposed into two matrices representing the background and the dynamic component, respectively. The latter has the properties of a so-called sparse matrix, and the proposed methods are classified as methods based on sparse representations. The proposed methods are the median-filter method, the principal component pursuit, and the dynamic-mode decomposition, all of which include data pre-processing using the noise-adaptive fuzzy equalization method. The study reveals that the median-filter method and the dynamic-mode decomposition enhance all motions in the time series and produce similar results. On the other hand, the principal component pursuit enables the clear differentiation of CMEs from the background corona, thus providing a valuable tool for the characterization of their acceleration profiles in the low corona as seen in the EUV.

Bioregulatory event extraction using large language models: a case study of rice literature

The extraction of biological regulation events has been a key focus in the field of biomedical nature language processing (BioNLP). However, existing methods often encounter challenges such as cascading errors in text mining pipelines and limitations in topic coverage from the selected corpus. Fortunately, the emergence of large language models (LLMs) presents a potential solution due to their robust semantic understanding and extensive knowledge base. To explore this potential, our project at the Biomedical Linked Annotation Hackathon 8 (BLAH 8) investigates the feasibility of using LLMs to extract biological regulation events. Our findings, based on the analysis of rice literature, demonstrate the promising performance of LLMs in this task, while also highlighting several concerns that must be addressed in future LLM-based application in low-resource topic.

Aircraft human‐machine interaction assistant design: A novel multimodal data processing and application framework

AbstractDuring aircraft operations, pilots rely on human‐machine interaction platforms to access essential information services. However, the development of a highly usable aerial assistant necessitates the incorporation of two interaction modes: active‐command and passive‐response modes, along with three input modes: voice inputs, situation inputs, and plan inputs. This research focuses on the design of an aircraft human‐machine interaction assistant (AHMIA), which serves as a multimodal data processing and application framework for human‐to‐machine interaction in a fully voice‐controlled manner. For the voice mode, a finetuned FunASR model is employed, leveraging private aeronautical datasets to enable specific aeronautical speech recognition. For the situation mode, a hierarchical situation events extraction model is proposed, facilitating the utilization of high‐level situational features. For the plan mode, a multi‐formations double‐code network plan diagram with a timeline is utilized to effectively represent plan information. Notably, to bridge the gap between human language and machine language, a hierarchical knowledge engine named process‐event‐condition‐order‐skill (PECOS) is introduced. PECOS provides three distinct products: the PECOS model, the PECOS state chart, and the PECOS knowledge description. Simulation results within the air confrontation scenario demonstrate that AHMIA enables active‐command and passive‐response interactions with pilots, thereby enhancing the overall interaction modality.

Political-RAG: using generative AI to extract political information from media content

ABSTRACT In the digital era, media content is crucial for political analysis, providing valuable insights through news articles, social media posts, speeches, and reports. Natural Language Processing (NLP) has transformed Political Information Extraction (IE), automating tasks such as event extraction and sentiment analysis. Traditional NLP methods, while effective, are often task-specific and require specialized expertise. In contrast, Large Language Models (LLMs) powered by Generative Artificial Intelligence (GenAI) offer a more integrated solution. However, domain-specific challenges persist, which led to the development of the Retrieval-Augmented Generation (RAG) framework. RAG enhances LLMs by incorporating external data retrieval, addressing issues related to data availability. To demonstrate RAG’s capabilities, we introduce the Political-RAG system, designed to extract political event information from media content, including Twitter data and news articles. Initially developed for event extraction, the Political-RAG system lays the foundation for developing various complex Political IE tasks. These include detecting hate speech, analyzing conflicts, assessing political bias, and evaluating social trends, sentiment, and opinions.

SIAT: Document-level Event Extraction via Spatiality-Augmented Interaction Model with Adaptive Thresholding

Document-level event extraction endeavors to automatically extract structural events from a given document. Many existing approaches focus on modeling entity interactions and decoding these interactions into events, assigning each entity as an event argument. However, these approaches encounter two primary limitations: they exclusively capture semantic dependencies to model entity interactions, overlooking the indication of the spatial distribution features of entities; they decode interactions imprecisely with a hard binary-classification boundary, potentially failing to calibrate micro differences in interactions. To overcome these limitations, we introduce a novel approach termed the S patiality-augmented I nteraction Model with A daptive T hresholding (SIAT). Our method addresses the first limitation by calculating the relative position encoding of entities to represent spatial interaction features. These features are then integrated with multi-granularity semantic interactions, enhancing the modeling of entity interactions for each entity pair. Furthermore, we introduce an adaptive event decoding mechanism, which establishes a more flexible decision boundary for different entity interactions. Additionally, an adaptive loss function for threshold learning is designed to further refine the model. Experimental results demonstrate that our proposed method achieves competitive performance compared to state-of-the-art methods on two public event extraction datasets while maintaining considerable training efficiency.

Agent-DA: Enhancing low-resource event extraction with collaborative multi-agent data augmentation

Low-resource event extraction presents a significant challenge in real-world applications, particularly in domains like pharmaceuticals, military and law, where data is frequently insufficient. Data augmentation, as a direct method for expanding samples, is considered an effective solution. However, existing data augmentation methods often suffer from text fluency issues and label hallucination. To address these challenges, we propose a framework called Agent-DA, which leverages multi-agent collaboration for event extraction data augmentation. Specifically, Agent-DA follows a three-step process: data generation by the large language model, collaborative filtering by both the large language model and small language model to discriminate easy samples, and the use of an adjudicator to identify hard samples. Through iterative and selective augmentation, our method significantly enhances both the quantity and quality of event samples, improving text fluency and label consistency. Extensive experiments on the ACE2005-EN and ACE2005-EN+ datasets demonstrate the effectiveness of Agent-DA, with F1-score improvements ranging from 0.15% to 16.18% in trigger classification and from 2.2% to 15.67% in argument classification.

Enhancing public opinion monitoring for social hot events with a time series neural network-based logic map

A method for constructing a commercial map of social hotspots based on time-series neural networks is proposed. It explores map construction through three stages: atomic event graph building, fusion, and public opinion analysis. The neural network captures event dynamics and public opinion evolution. An unsupervised learning approach trains the model to explore deep event connections. A time map reflects event trends and opinion changes. A sentence-level event extraction model provides rich information. An empirical mode decomposition method analyzes IMF data to understand intrinsic patterns. Time-series neural networks classify and reconstruct event graphs, constructing a knowledge graph. Experiments show this method reduces storage costs, simplifies prediction, and exhibits robustness. It offers new insights for commercial mapping and contributes to public opinion monitoring.

Business insights using RAG–LLMs: a review and case study

ABSTRACT As organizations increasingly rely on diverse data sources like invoices and surveys, efficient Information Extraction (IE) is crucial. Natural Language Processing (NLP) enhances IE through tasks such as Named Entity Recognition (NER), Relation Extraction (RE), Event Extraction (EE), Term Extraction (TE), and Topic Modeling (TM). However, implementing these methods requires significant expertise, which smaller organizations often lack. Large Language Models (LLMs), powered by Generative Artificial Intelligence (GenAI), can address this by performing multiple IE tasks without extensive development costs. However, LLMs may struggle with domain-specific accuracy. Integrating Retrieval-Augmented Generation (RAG) with LLMs improves precision by incorporating external data. Despite the potential, research on RAG-LLM applications in the business domain is limited. This article reviews Business IE systems, explores RAG-LLM applications across disciplines, and presents a case study demonstrating how RAG-LLMs can enhance business insights, offering scalable, cost-effective solutions.

A comprehensive review of existing corpora and methods for creating annotated corpora for event extraction tasks

Abstract Purpose The purpose of this study is to serve as a comprehensive review of the existing annotated corpora. This review study aims to provide information on the existing annotated corpora for event extraction, which are limited but essential for training and improving the existing event extraction algorithms. In addition to the primary goal of this study, it provides guidelines for preparing an annotated corpus and suggests suitable tools for the annotation task. Design/methodology/approach This study employs an analytical approach to examine available corpus that is suitable for event extraction tasks. It offers an in-depth analysis of existing event extraction corpora and provides systematic guidelines for researchers to develop accurate, high-quality corpora. This ensures the reliability of the created corpus and its suitability for training machine learning algorithms. Findings Our exploration reveals a scarcity of annotated corpora for event extraction tasks. In particular, the English corpora are mainly focused on the biomedical and general domains. Despite the issue of annotated corpora scarcity, there are several high-quality corpora available and widely used as benchmark datasets. However, access to some of these corpora might be limited owing to closed-access policies or discontinued maintenance after being initially released, rendering them inaccessible owing to broken links. Therefore, this study documents the available corpora for event extraction tasks. Research limitations Our study focuses only on well-known corpora available in English and Chinese. Nevertheless, this study places a strong emphasis on the English corpora due to its status as a global lingua franca, making it widely understood compared to other languages. Practical implications We genuinely believe that this study provides valuable knowledge that can serve as a guiding framework for preparing and accurately annotating events from text corpora. It provides comprehensive guidelines for researchers to improve the quality of corpus annotations, especially for event extraction tasks across various domains. Originality/value This study comprehensively compiled information on the existing annotated corpora for event extraction tasks and provided preparation guidelines.

Multi-Grained Gradual Inference Model for Multimedia Event Extraction

Multi-Grained Gradual Inference Model for Multimedia Event Extraction

Toward Long Form Audio-Visual Video Understanding

We live in a world filled with never-ending streams of multimodal information. As a more natural recording of the real scenario, long form audio-visual videos (LFAVs) are expected as an important bridge for better exploring and understanding the world. In this article, we propose the multisensory temporal event localization task in long form videos and strive to tackle the associated challenges. To facilitate this study, we first collect a large-scale LFAV dataset with 5,175 videos and an average video length of 210 seconds. Each collected video is elaborately annotated with diversified modality-aware events, in a long-range temporal sequence. We then propose an event-centric framework for localizing multisensory events as well as understanding their relations in long form videos. It includes three phases in different levels: snippet prediction phase to learn snippet features, event extraction phase to extract event-level features, and event interaction phase to study event relations. Experiments demonstrate that the proposed method, utilizing the new LFAV dataset, exhibits considerable effectiveness in localizing multiple modality-aware events within long form videos. We hope that our newly collected dataset and novel approach serve as a cornerstone for furthering research in the realm of LFAV understanding. Project page: https://gewu-lab.github.io/LFAV/ .

AssocKD: An Association-Aware Knowledge Distillation Method for Document-Level Event Argument Extraction

Event argument extraction is a crucial subtask of event extraction, which aims at extracting arguments that correspond to argument roles when given event types. The majority of current document-level event argument extraction works focus on extracting information for only one event at a time without considering the association among events; this is known as document-level single-event extraction. However, the interrelationship among arguments can yield mutual gains in their extraction. Therefore, in this paper, we propose AssocKD, an Association-aware Knowledge Distillation Method for Document-level Event Argument Extraction, which enables the enhancement of document-level multi-event extraction with event association knowledge. Firstly, we introduce an association-aware training task to extract unknown arguments with the given privileged knowledge of relevant arguments, obtaining an association-aware model that can construct both intra-event and inter-event relationships. Secondly, we adopt multi-teacher knowledge distillation to transfer such event association knowledge from the association-aware teacher models to the event argument extraction student model. Our proposed method, AssocKD, is capable of explicitly modeling and efficiently leveraging event association to enhance the extraction of multi-event arguments at the document level. We conduct experiments on RAMS and WIKIEVENTS datasets and observe a significant improvement, thus demonstrating the effectiveness of our method.

OGSS: An Ontology-Guided and Scheduled-Sampling Approach for Overlapping Event Extraction

Event extraction is a complex and challenging task in the field of information extraction. It aims to identify event types, triggers, and argument information from the text. In recent years, overlapping event extraction has attracted the attention of researchers because of its higher challenge and practicability, and some work has carried out in-depth research on overlapping event extraction and achieved remarkable results. But these works (1) ignore the role of ontology knowledge in event extraction; (2) use the same semantic encoding for multi-stage models, lacking consideration for the independent characteristics of extraction tasks such as event types, triggers, and arguments; and (3) face issues in the training process of multi-stage models, such as error cascading and slow convergence. To address the above issues, we propose an ontology-guided and scheduled-sampling approach for overlapping event extraction, termed as OGSS. First, we design a symmetric matrix for event ontology knowledge representation and integrate it into the semantic encoding process, infusing ontology knowledge into event extraction. Second, for extraction targets such as event types, triggers, and arguments, we process the semantic encoding according to the characteristics of each extraction target, obtaining semantic representations tailored for each subtask. Finally, we view multi-stage predictions as sequential outputs of a joint model, using a scheduled sampling strategy between subtasks to effectively mitigate the cascading propagation of errors during training and accelerate model convergence. We conduct extensive experiments on the FewFc event extraction benchmark dataset. The results show that OGSS achieves significant improvements in overlapping event extraction tasks compared to previous methods.

Design and Evaluation of Real-Time Data Storage and Signal Processing in a Long-Range Distributed Acoustic Sensing (DAS) Using Cloud-Based Services.

In cloud-based Distributed Acoustic Sensing (DAS) sensor data management, we are confronted with two primary challenges. First, the development of efficient storage mechanisms capable of handling the enormous volume of data generated by these sensors poses a challenge. To solve this issue, we propose a method to address the issue of handling the large amount of data involved in DAS by designing and implementing a pipeline system to efficiently send the big data to DynamoDB in order to fully use the low latency of the DynamoDB data storage system for a benchmark DAS scheme for performing continuous monitoring over a 100 km range at a meter-scale spatial resolution. We employ the DynamoDB functionality of Amazon Web Services (AWS), which allows highly expandable storage capacity with latency of access of a few tens of milliseconds. The different stages of DAS data handling are performed in a pipeline, and the scheme is optimized for high overall throughput with reduced latency suitable for concurrent, real-time event extraction as well as the minimal storage of raw and intermediate data. In addition, the scalability of the DynamoDB-based data storage scheme is evaluated for linear and nonlinear variations of number of batches of access and a wide range of data sample sizes corresponding to sensing ranges of 1-110 km. The results show latencies of 40 ms per batch of access with low standard deviations of a few milliseconds, and latency per sample decreases for increasing the sample size, paving the way toward the development of scalable, cloud-based data storage services integrating additional post-processing for more precise feature extraction. The technique greatly simplifies DAS data handling in key application areas requiring continuous, large-scale measurement schemes. In addition, the processing of raw traces in a long-distance DAS for real-time monitoring requires the careful design of computational resources to guarantee requisite dynamic performance. Now, we will focus on the design of a system for the performance evaluation of cloud computing systems for diverse computations on DAS data. This system is aimed at unveiling valuable insights into performance metrics and operational efficiencies of computations on the data in the cloud, which will provide a deeper understanding of the system's performance, identify potential bottlenecks, and suggest areas for improvement. To achieve this, we employ the CloudSim framework. The analysis reveals that the virtual machine (VM) performance decreases significantly the processing time with more capable VMs, influenced by Processing Elements (PEs) and Million Instructions Per Second (MIPS). The results also reflect that, although a larger number of computations is required as the fiber length increases, with the subsequent increase in processing time, the overall speed of computation is still suitable for continuous real-time monitoring. We also see that VMs with lower performance in terms of processing speed and number of CPUs have more inconsistent processing times compared to those with higher performance, while not incurring significantly higher prices. Additionally, the impact of VM parameters on computation time is explored, highlighting the importance of resource optimization in the DAS system design for efficient performance. The study also observes a notable trend in processing time, showing a significant decrease for every additional 50,000 columns processed as the length of the fiber increases. This finding underscores the efficiency gains achieved with larger computational loads, indicating improved system performance and capacity utilization as the DAS system processes more extensive datasets.

EADRE: Event-type Aware Dynamic Representation of Entities in Document-level Event Extraction

EADRE: Event-type Aware Dynamic Representation of Entities in Document-level Event Extraction

CACER: Clinical concept Annotations for Cancer Events and Relations.

Clinical notes contain unstructured representations of patient histories, including the relationships between medical problems and prescription drugs. To investigate the relationship between cancer drugs and their associated symptom burden, we extract structured, semantic representations of medical problem and drug information from the clinical narratives of oncology notes. We present Clinical concept Annotations for Cancer Events and Relations (CACER), a novel corpus with fine-grained annotations for over 48000 medical problems and drug events and 10000 drug-problem and problem-problem relations. Leveraging CACER, we develop and evaluate transformer-based information extraction models such as Bidirectional Encoder Representations from Transformers (BERT), Fine-tuned Language Net Text-To-Text Transfer Transformer (Flan-T5), Large Language Model Meta AI (Llama3), and Generative Pre-trained Transformers-4 (GPT-4) using fine-tuning and in-context learning (ICL). In event extraction, the fine-tuned BERT and Llama3 models achieved the highest performance at 88.2-88.0 F1, which is comparable to the inter-annotator agreement (IAA) of 88.4 F1. In relation extraction, the fine-tuned BERT, Flan-T5, and Llama3 achieved the highest performance at 61.8-65.3 F1. GPT-4 with ICL achieved the worst performance across both tasks. The fine-tuned models significantly outperformed GPT-4 in ICL, highlighting the importance of annotated training data and model optimization. Furthermore, the BERT models performed similarly to Llama3. For our task, large language models offer no performance advantage over the smaller BERT models. We introduce CACER, a novel corpus with fine-grained annotations for medical problems, drugs, and their relationships in clinical narratives of oncology notes. State-of-the-art transformer models achieved performance comparable to IAA for several extraction tasks.

Real-Time Extraction of News Events Based on BERT Model

Abstract For the large number of news reports generated every day, in order to obtain effective information from these unstructured news text data more efficiently. In this paper, we study the real-time crawling of news data from news websites through crawling techniques and propose a BERT model-based approach to extract events from news long text. In this study, NetEase news website is selected as an example for real-time extraction to crawl the news data of this website. BERT model as a pre-trained model based on two-way encoded representation of transformer performs well on natural language understanding and natural language generation tasks. In this study, we will fine-tune the training based on BERT model on news corpus related dataset and perform sequence annotation through CRF layer to finally complete the event extraction task. In this paper, the DUEE dataset is chosen to train the model, and the experiments show that the overall performance of the BERT model is better than other network models. Finally, the model of this paper is further optimised, using the ALBERT and RoBERTa models improved on the basis of the BERT model, experiments were conducted, the results show that both models are improved compared to the BERT model, the ALBERT model algorithm performs the best, the model algorithm's F1 value is 1% higher than that of BERT. The results show that the performance is optimised.