Automatic Information Extraction Research Articles

Enhancing Knowledge Graph Consistency through Open Large Language Models: A Case Study

High-quality knowledge graphs (KGs) play a crucial role in many applications. However, KGs created by automated information extraction systems can suffer from erroneous extractions or be inconsistent with provenance/source text. It is important to identify and correct such problems. In this paper, we study leveraging the emergent reasoning capabilities of large language models (LLMs) to detect inconsistencies between extracted facts and their provenance. With a focus on ``open'' LLMs that can be run and trained locally, we find that few-shot approaches can yield an absolute performance gain of 2.5-3.4% over the state-of-the-art method with only 9% of training data. We examine the LLM architectures' effect and show that Decoder-Only models underperform Encoder-Decoder approaches. We also explore how model size impacts performance and counterintuitively find that larger models do not result in consistent performance gains. Our detailed analyses suggest that while LLMs can improve KG consistency, the different LLM models learn different aspects of KG consistency and are sensitive to the number of entities involved.

An automated information extraction system from the knowledge graph based annual financial reports

This article presents a semantic web-based solution for extracting the relevant information automatically from the annual financial reports of the banks/financial institutions and presenting this information in a queryable form through a knowledge graph. The information in these reports is significantly desired by various stakeholders for making key investment decisions. However, this information is available in an unstructured format making it much more complex and challenging to understand and query manually or even through digital systems. Another challenge that makes the understanding of information more complex is the variation of terminologies among financial reports of different banks or financial institutions. The solution presented in this article signifies an ontological approach to solving the standardization problems of the terminologies in this domain. It further addresses the issue of semantic differences to extract relevant data sharing common semantics. Such semantics are then incorporated by implementing their representation as a Knowledge Graph to make the information understandable and queryable. Our results highlight the usage of Knowledge Graph in search engines, recommender systems and question-answering (Q-A) systems. This financial knowledge graph can also be used to serve the task of financial storytelling. The proposed solution is implemented and tested on the datasets of various banks and the results are presented through answers to competency questions evaluated on precision and recall measures.

Classification of Image using Convolutional Neural Network (CNN)

Abstract: Computer vision is concerned with the automatic extraction, analysis, and understanding of useful information from a single image or a sequence of images. We have used Convolutional Neural Networks (CNN) in automatic image classification systems. In most cases, we utilize the features from the top layer of the CNN for classification; however, those features may not contain enough useful information to predict an image correctly. In some cases, features from the lower layer carry more discriminative power than those from the top. Therefore, applying features from a specific layer only to classification seems to be a process that does not utilize learned CNN’s potential discriminant power to its full extent. Because of this property we are in need of fusion of features from multiple layers. We want to create a model with multiple layers that will be able to recognize and classify the images. We want to complete our model by using the concepts of Convolutional Neural Network and CIFAR-10 dataset. Moreover, we will show how MatConvNet can be used to implement our model with CPU training as well as less training time. The objective of our work is to learn and practically apply the concepts of Convolutional Neural Network.

Methods of Automatic Analysis of Information Presentation Dynamics in Texts Based on Adaptable Dictionaries of Scientific Terms

The paper proposes a method for determining the dynamics of aspectual content of scientific texts. The problems arising in the automatic processing of scientific texts are discussed, approaches to the creation of a combined method of aspect-oriented analysis of scientific texts are proposed. The results of experiments of aspect analysis of scientific publications in the field of mathematics are given. The algorithm of aspect analysis is built on the basis of processing both the semantic domain graph formed as a result of automatic extraction of information from the text of the mathematical encyclopedia and the weighted semantic graph of the publication. Weighting functions for calculating the dependence of links between professional terms of the semantic domain graph are proposed. During the processing the text of a scientific publication, the semantic domain graph is transformed into the semantic graph of the publication, the two-phase processing algorithm of which forms semantic dependencies between text fragments.

Comparing generative and extractive approaches to information extraction from abstracts describing randomized clinical trials

BackgroundSystematic reviews of Randomized Controlled Trials (RCTs) are an important part of the evidence-based medicine paradigm. However, the creation of such systematic reviews by clinical experts is costly as well as time-consuming, and results can get quickly outdated after publication. Most RCTs are structured based on the Patient, Intervention, Comparison, Outcomes (PICO) framework and there exist many approaches which aim to extract PICO elements automatically. The automatic extraction of PICO information from RCTs has the potential to significantly speed up the creation process of systematic reviews and this way also benefit the field of evidence-based medicine.ResultsPrevious work has addressed the extraction of PICO elements as the task of identifying relevant text spans or sentences, but without populating a structured representation of a trial. In contrast, in this work, we treat PICO elements as structured templates with slots to do justice to the complex nature of the information they represent. We present two different approaches to extract this structured information from the abstracts of RCTs. The first approach is an extractive approach based on our previous work that is extended to capture full document representations as well as by a clustering step to infer the number of instances of each template type. The second approach is a generative approach based on a seq2seq model that encodes the abstract describing the RCT and uses a decoder to infer a structured representation of a trial including its arms, treatments, endpoints and outcomes. Both approaches are evaluated with different base models on a manually annotated dataset consisting of RCT abstracts on an existing dataset comprising 211 annotated clinical trial abstracts for Type 2 Diabetes and Glaucoma. For both diseases, the extractive approach (with flan-t5-base) reached the best F1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$F_1$$\\end{document} score, i.e. 0.547 (±0.006\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\pm 0.006$$\\end{document}) for type 2 diabetes and 0.636 (±0.006\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\pm 0.006$$\\end{document}) for glaucoma. Generally, the F1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$F_1$$\\end{document} scores were higher for glaucoma than for type 2 diabetes and the standard deviation was higher for the generative approach.ConclusionIn our experiments, both approaches show promising performance extracting structured PICO information from RCTs, especially considering that most related work focuses on the far easier task of predicting less structured objects. In our experimental results, the extractive approach performs best in both cases, although the lead is greater for glaucoma than for type 2 diabetes. For future work, it remains to be investigated how the base model size affects the performance of both approaches in comparison. Although the extractive approach currently leaves more room for direct improvements, the generative approach might benefit from larger models.

Customer Feedback Analysis Using Text Mining

Complexity surrounding the holistic nature of customer experience has made measuring customer perceptions of interactive service experiences challenging. At the same time, advances in technology and changes in methods for collecting explicit customer feedback are generating increasing volumes of unstructured textual data, making it difficult for managers to analyze and interpret this information. Consequently, text mining, a method enabling automatic extraction of information from textual data, is gaining in popularity. However, this method has performed below expectations in terms of depth of analysis of customer experience feedback and accuracy. In this study, we advance linguistics-based text mining modeling to inform the process of developing an improved framework. The proposed framework incorporates important elements of customer experience, service methodologies and theories such as co-creation processes, interactions and context. This more holistic approach for analyzing feedback facilitates a deeper analysis of customer feedback experiences, by encompassing three value creation elements: activities, resources, and context (ARC). Empirical results show that the ARC framework facilitates the development of a text mining model for analysis of customer textual feedback that enables companies to assess the impact of interactive service processes on customer experiences. The proposed text mining model shows high accuracy levels and provides flexibility through training. As such, it can evolve to account for changing contexts over time and be deployed across different (service) business domains; we term it an “open learning” model. The ability to timely assess customer experience feedback represents a pre-requisite for successful co-creation processes in a service environment.

Path-BigBird: An AI-Driven Transformer Approach to Classification of Cancer Pathology Reports.

Surgical pathology reports are critical for cancer diagnosis and management. To accurately extract information about tumor characteristics from pathology reports in near real time, we explore the impact of using domain-specific transformer models that understand cancer pathology reports. We built a pathology transformer model, Path-BigBird, by using 2.7 million pathology reports from six SEER cancer registries. We then compare different variations of Path-BigBird with two less computationally intensive methods: Hierarchical Self-Attention Network (HiSAN) classification model and an off-the-shelf clinical transformer model (Clinical BigBird). We use five pathology information extraction tasks for evaluation: site, subsite, laterality, histology, and behavior. Model performance is evaluated by using macro and micro F1 scores. We found that Path-BigBird and Clinical BigBird outperformed the HiSAN in all tasks. Clinical BigBird performed better on the site and laterality tasks. Versions of the Path-BigBird model performed best on the two most difficult tasks: subsite (micro F1 score of 72.53, macro F1 score of 35.76) and histology (micro F1 score of 80.96, macro F1 score of 37.94). The largest performance gains over the HiSAN model were for histology, for which a Path-BigBird model increased the micro F1 score by 1.44 points and the macro F1 score by 3.55 points. Overall, the results suggest that a Path-BigBird model with a vocabulary derived from well-curated and deidentified data is the best-performing model. The Path-BigBird pathology transformer model improves automated information extraction from pathology reports. Although Path-BigBird outperforms Clinical BigBird and HiSAN, these less computationally expensive models still have utility when resources are constrained.

Synthetic Data for Sentinel-2 Semantic Segmentation

Satellite observations provide critical data for a myriad of applications, but automated information extraction from such vast datasets remains challenging. While artificial intelligence (AI), particularly deep learning methods, offers promising solutions for land cover classification, it often requires massive amounts of accurate, error-free annotations. This paper introduces a novel approach to generate a segmentation task dataset with minimal human intervention, thus significantly reducing annotation time and potential human errors. ‘Samples’ extracted from actual imagery were utilized to construct synthetic composite images, representing 10 segmentation classes. A DeepResUNet was solely trained on this synthesized dataset, eliminating the need for further fine-tuning. Preliminary findings demonstrate impressive generalization abilities on real data across various regions of Quebec. We endeavored to conduct a quantitative assessment without reliance on manually annotated data, and the results appear to be comparable, if not superior, to models trained on genuine datasets.

Natural Language Processing for Radiation Oncology: Personalizing Treatment Pathways.

Natural language processing (NLP), a technology that translates human language into machine-readable data, is revolutionizing numerous sectors, including cancer care. This review outlines the evolution of NLP and its potential for crafting personalized treatment pathways for cancer patients. Leveraging NLP's ability to transform unstructured medical data into structured learnable formats, researchers can tap into the potential of big data for clinical and research applications. Significant advancements in NLP have spurred interest in developing tools that automate information extraction from clinical text, potentially transforming medical research and clinical practices in radiation oncology. Applications discussed include symptom and toxicity monitoring, identification of social determinants of health, improving patient-physician communication, patient education, and predictive modeling. However, several challenges impede the full realization of NLP's benefits, such as privacy and security concerns, biases in NLP models, and the interpretability and generalizability of these models. Overcoming these challenges necessitates a collaborative effort between computer scientists and the radiation oncology community. This paper serves as a comprehensive guide to understanding the intricacies of NLP algorithms, their performance assessment, past research contributions, and the future of NLP in radiation oncology research and clinics.

Parsing of Research Documents into XML Using Formal Grammars

Automatic information extraction of content and style format in paged documents is challenging. It requires the conversion of the original document into a granular level of details for which every document section and content is identifiable. This functionality or tool does not exist for any academic research document yet. In this paper, we present an automated process of parsing research paper documents into XML files using a formal method approach of context-free grammars (CFGs) and regular expressions (REGEXs) definable of a standard template. We created a tool for the algorithms to parse these documents into tree-like structures organized as XML files named research_XML (RX) parser. The RX tool performed the extraction of syntactic structure and semantic information of the document’s contents into XML files. These XML output files are lightweight, analyzable, query-able, and web interoperable. The RX tool has a success rate of 91% when evaluated on fifty varying research documents of 160 average pages and 8,004 total pages. The tool and test data are accessible on GitHub repo. The novelty of our process is specific to applying formal techniques for information extraction in structured multipaged documents and academic research documents thus advancing the research in automatic information extraction.

Simultaneous, vision-based fish instance segmentation, species classification and size regression.

Overexploitation of fisheries is a worldwide problem, which is leading to a large loss of diversity, and affects human communities indirectly through the loss of traditional jobs, cultural heritage, etc. To address this issue, governments have started accumulating data on fishing activities, to determine biomass extraction rates, and fisheries status. However, these data are often estimated from small samplings, which can lead to partially inaccurate assessments. Fishing can also benefit of the digitization process that many industries are undergoing. Wholesale fish markets, where vessels disembark, can be the point of contact to retrieve valuable information on biomass extraction rates, and can do so automatically. Fine-grained knowledge about the fish species, quantities, sizes, etc. that are caught can be therefore very valuable to all stakeholders, and particularly decision-makers regarding fisheries conservation, sustainable, and long-term exploitation. In this regard, this article presents a full workflow for fish instance segmentation, species classification, and size estimation from uncalibrated images of fish trays at the fish market, in order to automate information extraction that can be helpful in such scenarios. Our results on fish instance segmentation and species classification show an overall mean average precision (mAP) at 50% intersection-over-union (IoU) of 70.42%, while fish size estimation shows a mean average error (MAE) of only 1.27 cm.

ANALYSIS OF USER COSTS BASED ON PAYMENTS FOR RECOMMENDATIONS IN MOBILE APPLICATIONS

The innovation in payment methods has become one of the breakthroughs in the ways of consumption of goods and services. The system provided by mobile payments is also an important platform for promoting online business. Personalization of the recommendation system is an important part of the business, which is based on personal preferences, interests and purchasing behavior. User data collected from previous purchase histories will be further processed and used to offer similar or interesting products and information based on certain machine learning algorithms for them. This article introduces cost analysis of a mobile application user based on artificial intelligence to enable online platforms to create a business model for recommendation systems inside the application. Machine learning is one of the tools with highly automated information extraction from large amounts of data. In this paper, we will consider machine learning in forecasting and analysis.

A multiscale feature fusion enhanced CNN with the multiscale channel attention mechanism for efficient landslide detection (MS2LandsNet) using medium-resolution remote sensing data

ABSTRACT Deep learning (DL) models have been widely used for remote sensing-based landslide mapping due to their impressive capabilities for automatic information extraction. However, the large volumes of parameters and calculations have compromised the efficiency of DL models in extracting landslides from a large set of RS images. Lightweight convolutional neural networks (CNNs) exhibit promising feature representation abilities with fewer parameters. This study aims to introduce a new lightweight CNN called MS2LandsNet, designed to detect landslides with both high efficiency and accuracy. The MS2LandsNet consists of three down-sampling stages embedded with multi-scale feature fusion (MFF), aiming to decrease parameters while aggregating contextual features. Additionally, we incorporate multi-scale channel attention (MSCA) into MFF to improve performance. According to experimental results on three landslip datasets, MS2LandsNet obtains the highest F1 score of 85.90% and the highest IoU of 75.28%. Notably, MS2LandsNet accomplishes the resuts with the fewest parameters and the fastest inference speed, outperforming seven classical semantic segmentation models and three lightweight CNNs. The proposed lightweight model holds potential for application on a cloud computing platform for larger-scale landslide mapping tasks in future work.

Automatic information extraction from Employment Tribunal judgements using large language models

Automatic information extraction from Employment Tribunal judgements using large language models

KEY TECHNOLOGIES FOR 1:10,000 RAPID MAPPING BASED ON GF-7 SATELLITE

Abstract. In order to solve the problems of low efficiency, high cost and long cycle of the traditional 1:10000 basic geographic information data production method (aerial photography combined with field mapping), this paper proposes a 1:10000 rapid mapping method based on GF-7 satellite data. It solves the key technical difficulties of automatic information recognition, extraction, integration and mapping of water body, building, road, vegetation, contour, and other elements. And an automatic software system of “image -> patch -> data -> map” is developed, which realizes the rapid production of 1:10000 basic geographic information data products and improves the ability of high-precision basic geographic information resource construction.

Deep learning uncertainty quantification for clinical text classification

Introduction:Machine learning algorithms are expected to work side-by-side with humans in decision-making pipelines. Thus, the ability of classifiers to make reliable decisions is of paramount importance. Deep neural networks (DNNs) represent the state-of-the-art models to address real-world classification. Although the strength of activation in DNNs is often correlated with the network’s confidence, in-depth analyses are needed to establish whether they are well calibrated. Method:In this paper, we demonstrate the use of DNN-based classification tools to benefit cancer registries by automating information extraction of disease at diagnosis and at surgery from electronic text pathology reports from the US National Cancer Institute (NCI) Surveillance, Epidemiology, and End Results (SEER) population-based cancer registries. In particular, we introduce multiple methods for selective classification to achieve a target level of accuracy on multiple classification tasks while minimizing the rejection amount—that is, the number of electronic pathology reports for which the model’s predictions are unreliable. We evaluate the proposed methods by comparing our approach with the current in-house deep learning-based abstaining classifier. Results:Overall, all the proposed selective classification methods effectively allow for achieving the targeted level of accuracy or higher in a trade-off analysis aimed to minimize the rejection rate. On in-distribution validation and holdout test data, with all the proposed methods, we achieve on all tasks the required target level of accuracy with a lower rejection rate than the deep abstaining classifier (DAC). Interpreting the results for the out-of-distribution test data is more complex; nevertheless, in this case as well, the rejection rate from the best among the proposed methods achieving 97% accuracy or higher is lower than the rejection rate based on the DAC. Conclusions:We show that although both approaches can flag those samples that should be manually reviewed and labeled by human annotators, the newly proposed methods retain a larger fraction and do so without retraining—thus offering a reduced computational cost compared with the in-house deep learning-based abstaining classifier.

DODFMiner: An automated tool for Named Entity Recognition from Official Gazettes

Official gazettes are documents published by governments to publicize their actions, spanning long periods of time and making an important transparency mechanism. These documents have information on laws, contracts, and bidding processes, as well as on civil servants and their careers in public service. Automatic information extraction of these documents may contribute to public transparency, with two tasks being especially useful: the classification of the different segments of these documents, the so called acts; and the Named Entity Recognition (NER) within the acts. The variety of official gazettes and their patterns brings up the necessity of constructing different tools for specific gazettes. In this paper, we propose DODFMiner, a command-line interface tool to classify acts and extract named entities from the Official Gazette of the Federal District. The tool follows a 3-step approach: the pre-processing of the input data; text classification using rule-based systems with regular expressions; and NER with Machine Learning algorithms. It allows users to input JSON files and receive CSV as output, providing information that allows users to track government procurements through years, contracts duration and total amount, among others. We also propose a set of experiments to support the choice of models included in the tool, covering the classification and NER steps. Text classification achieved a mean F1-score of 0.778, while to the NER, we compared 3 different architectures, CRF with a mean F1-score of 0.851, CNN-biLSTM-CRF with 0.787 and CNN-CNN-LSTM with 0.841.

AI-powered decision-making in facilitating insurance claim dispute resolution

AbstractLeveraging Artificial Intelligence (AI) techniques to empower decision-making can promote social welfare by generating significant cost savings and promoting efficient utilization of public resources, besides revolutionizing commercial operations. This study investigates how AI can expedite dispute resolution in road traffic accident (RTA) insurance claims, benefiting all parties involved. Specifically, we devise and implement a disciplined AI-driven approach to derive the cost estimates and inform negotiation decision-making, compared to conventional practices that draw upon official guidance and lawyer experience. We build the investigation on 88 real-life RTA cases and detect an asymptotic relationship between the final judicial cost and the duration of the most severe injury, marked by a notable predicted $${R}^{2}$$ R 2 value of 0.527. Further, we illustrate how various AI-powered toolkits can facilitate information processing and outcome prediction: (1) how regular expression (RegEx) collates precise injury information for subsequent predictive analysis; (2) how alternative natural language processing (NLP) techniques construct predictions directly from narratives. Our proposed RegEx framework enables automated information extraction that accommodates diverse report formats; different NLP methods deliver comparable plausible performance. This research unleashes AI’s untapped potential for social good to reinvent legal-related decision-making processes, support litigation efforts, and aid in the optimization of legal resource consumption.

Automatic information extraction in the AI chip domain using gated interactive attention and probability matrix encoding method

Automatic information extraction in the AI chip domain using gated interactive attention and probability matrix encoding method

NATURAL LANGUAGE PROCESSING BASED THREAT ASSESSMENT

Military intelligence agencies are continually faced with the formidable task of evaluating and mitigating diverse risks in an increasingly complex and dynamic global landscape. NLP has emerged as a powerful tool for enhancing the capabilities of Military Intelligence operations. NLP techniques enable the automatic extraction, analysis, and interpretation of information from vast volumes of unstructured textual data, such as open-source intelligence reports, social media, intercepted communications. By processing and understanding these textual sources, NLP systems can assist Military Intelligence analysts in identifying potential threats, uncovering hidden patterns, and making timely and informed decisions. Threat assessment is required to safeguard security interests and maintain the sovereignty of nation. By the rapidly expanding fields in emerging technologies India can gain that decisive advantage.