MicrobExpert: A mixture-of-experts framework for cross-domain knowledge mining of food–microbe–disease associations from biomedical texts

BackgroundTo train chunkers in recognizing noun phrases and verb phrases in biomedical text, an annotated corpus is required. The creation of gold standard corpora (GSCs), however, is expensive and time-consuming. GSCs therefore tend to be small and to focus on specific subdomains, which limits their usefulness. We investigated the use of a silver standard corpus (SSC) that is automatically generated by combining the outputs of multiple chunking systems. We explored two use scenarios: one in which chunkers are trained on an SSC in a new domain for which a GSC is not available, and one in which chunkers are trained on an available, although small GSC but supplemented with an SSC.ResultsWe have tested the two scenarios using three chunkers, Lingpipe, OpenNLP, and Yamcha, and two different corpora, GENIA and PennBioIE. For the first scenario, we showed that the systems trained for noun-phrase recognition on the SSC in one domain performed 2.7-3.1 percentage points better in terms of F-score than the systems trained on the GSC in another domain, and only 0.2-0.8 percentage points less than when they were trained on a GSC in the same domain as the SSC. When the outputs of the chunkers were combined, the combined system showed little improvement when using the SSC. For the second scenario, the systems trained on a GSC supplemented with an SSC performed considerably better than systems that were trained on the GSC alone, especially when the GSC was small. For example, training the chunkers on a GSC consisting of only 10 abstracts but supplemented with an SSC yielded similar performance as training them on a GSC of 100-250 abstracts. The combined system even performed better than any of the individual chunkers trained on a GSC of 500 abstracts.ConclusionsWe conclude that an SSC can be a viable alternative for or a supplement to a GSC when training chunkers in a biomedical domain. A combined system only shows improvement if the SSC is used to supplement a GSC. Whether the approach is applicable to other systems in a natural-language processing pipeline has to be further investigated.

By extensively mining system data and integrating with artificial intelligence means, knowledge graph can be exploited in various tasks of power communication network, effectively prompting the efficiency and performance of maintenance. One of the pivotal step of the knowledge graph construction is the named entity recognition. Abundant semantic features extracted from corpus can directly improve the accuracy of resulting concepts in knowledge graph. However, existing entity recognition method is mainly based on conventional word embedding technique such as Word2Vec, which still focuses on information within single word. In this paper, we propose to construct knowledge graph with the most recently proposed BERT-BiLSTM-CRF. This model can fully consider contextual information over words and extract more semantic features for further procedures. Our experimental results on realistic maintenance data of power communication networks proved the efficacy of BERT-BiLSTM-CRF model in the construction of knowledge graph. With the assistance of knowledge graph, we build applications for two typical maintenance scenarios, process standardization and fault disposal instruction, respectively. The knowledge graph has shown promising prospect as a novel auxiliary mechanism to power communication networks.

Named entity recognition (NER) is a core component in many natural language processing applications. Most NER systems rely on supervised machine learning methods, which depend on time-consuming and expensive annotations in different languages and domains. This paper presents a method for automatically building silver-standard NER corpora from Chinese Wikipedia. We refine novel and language-dependent features by exploiting the text and structure of Chinese Wikipedia. To reduce tagging errors caused by entity classification, we design four types of heuristic rules based on the characteristics of Chinese Wikipedia and train a supervised NE classifier, and a combined method is used to improve the precision and coverage. Then, we realize type identification of implicit mention by using boundary information of outgoing links. By selecting the sentences related with the domains of test data, we can train better NER models. In the experiments, large-scale NER corpora containing 2.3 million sentences are built from Chinese Wikipedia. The results show the effectiveness of automatically annotated corpora, and the trained NER models achieve the best performance when combining our silver-standard corpora with gold-standard corpora.

Knowledge about the software used in scientific investigations is necessary for different reasons, including provenance of the results, measuring software impact to attribute developers, and bibliometric software citation analysis in general. Additionally, providing information about whether and how the software and the source code are available allows an assessment about the state and role of open source software in science in general. While such analyses can be done manually, large scale analyses require the application of automated methods of information extraction and linking. In this paper, we present SoftwareKG—a knowledge graph that contains information about software mentions from more than 51,000 scientific articles from the social sciences. A silver standard corpus, created by a distant and weak supervision approach, and a gold standard corpus, created by manual annotation, were used to train an LSTM based neural network to identify software mentions in scientific articles. The model achieves a recognition rate of .82 F-score in exact matches. As a result, we identified more than 133,000 software mentions. For entity disambiguation, we used the public domain knowledge base DBpedia. Furthermore, we linked the entities of the knowledge graph to other knowledge bases such as the Microsoft Academic Knowledge Graph, the Software Ontology, and Wikidata. Finally, we illustrate, how SoftwareKG can be used to assess the role of software in the social sciences.

As a key application of technology-enhanced learning, Intelligent Tutoring Systems have long been constrained by bottlenecks such as expert-dependent, costly manual knowledge base construction and difficulties in adapting to unstructured teaching resources. Concurrently, generative large language models face challenges in educational question-answering, including factual inaccuracies and insufficient logical reasoning capabilities. To address these issues, this study proposes a framework for an Intelligent Tutoring System based on the automatic construction of multimodal knowledge graphs and Retrieval-Augmented Generation (RAG). The system integrates technologies such as FFmpeg, Whisper, OCR, and layout analysis to establish a pipeline for the fully automatic extraction and construction of knowledge graphs not only from course videos, but also from textbook PDFs. This process enables the integration of auditory information from videos with visual and textual knowledge from textbooks, building on this foundation, the framework combines graph retrieval and vector retrieval strategies, leveraging the RAG mechanism to drive large language models in generating accurate and explainable question-answering content. Experimental results demonstrate that the proposed system achieves positive feedback in terms of knowledge graph construction, the average accuracy and relevance of intelligent Q&A responses, overall user satisfaction, and system performance. Beyond automation, its core innovation is a cross-modal fusion mechanism that aligns and integrates knowledge from auditory explanations and visual-textual textbook content, thereby creating a unified, instructionally-structured knowledge graph. Thus, this study provides a feasible and innovative path from multimodal resources to intelligent services for Intelligent Tutoring Systems, holding significant practical implications for advancing personalized learning.

BackgroundRare diseases affect millions worldwide but sometimes face limited research focus individually due to low prevalence. Many rare diseases do not have specific International Classification of Diseases, Ninth Edition (ICD-9) and Tenth Edition (ICD-10), codes and therefore cannot be reliably extracted from granular fields like “Diagnosis” and “Problem List” entries, which complicates tasks that require identification of patients with these conditions, including clinical trial recruitment and research efforts. Recent advancements in large language models (LLMs) have shown promise in automating the extraction of medical information, offering the potential to improve medical research, diagnosis, and management. However, most LLMs lack professional medical knowledge, especially concerning specific rare diseases, and cannot effectively manage rare disease data in its various ontological forms, making it unsuitable for these tasks.ObjectiveOur aim is to create an end-to-end system called automated rare disease mining (AutoRD), which automates the extraction of rare disease–related information from medical text, focusing on entities and their relations to other medical concepts, such as signs and symptoms. AutoRD integrates up-to-date ontologies with other structured knowledge and demonstrates superior performance in rare disease extraction tasks. We conducted various experiments to evaluate AutoRD’s performance, aiming to surpass common LLMs and traditional methods.MethodsAutoRD is a pipeline system that involves data preprocessing, entity extraction, relation extraction, entity calibration, and knowledge graph construction. We implemented this system using GPT-4 and medical knowledge graphs developed from the open-source Human Phenotype and Orphanet ontologies, using techniques such as chain-of-thought reasoning and prompt engineering. We quantitatively evaluated our system’s performance in entity extraction, relation extraction, and knowledge graph construction. The experiment used the well-curated dataset RareDis2023, which contains medical literature focused on rare disease entities and their relations, making it an ideal dataset for training and testing our methodology.ResultsOn the RareDis2023 dataset, AutoRD achieved an overall entity extraction F1-score of 56.1% and a relation extraction F1-score of 38.6%, marking a 14.4% improvement over the baseline LLM. Notably, the F1-score for rare disease entity extraction reached 83.5%, indicating high precision and recall in identifying rare disease mentions. These results demonstrate the effectiveness of integrating LLMs with medical ontologies in extracting complex rare disease information.ConclusionsAutoRD is an automated end-to-end system for extracting rare disease information from text to build knowledge graphs, addressing critical limitations of existing LLMs by improving identification of these diseases and connecting them to related clinical features. This work underscores the significant potential of LLMs in transforming health care, particularly in the rare disease domain. By leveraging ontology-enhanced LLMs, AutoRD constructs a robust medical knowledge base that incorporates up-to-date rare disease information, facilitating improved identification of patients and resulting in more inclusive research and trial candidacy efforts.

Knowledge Graph Construction (KGC) from text unlocks information held within unstructured text and is critical to a wide range of downstream applications. General approaches to KGC from text are heavily reliant on the existence of knowledge bases, yet most domains do not even have an external knowledge base readily available. In many situations this results in information loss as a wealth of key information is held within "non-entities". Domain-specific approaches to KGC typically adopt unsupervised pipelines, using carefully crafted linguistic and statistical patterns to extract co-occurred noun phrases as triples, essentially constructing text graphs rather than true knowledge graphs. In this research, for the first time, in the same flavour as Collobert et al.'s seminal work of "Natural language processing (almost) from scratch" in 2011, we propose a Seq2KG model attempting to achieve "Knowledge graph construction (almost) from scratch". An end-to-end Sequence to Knowledge Graph (Seq2KG) neural model jointly learns to generate triples and resolves entity types as a multi-label classification task through deep learning neural networks. In addition, a novel evaluation metric that takes both semantic and structural closeness into account is developed for measuring the performance of triple extraction. We show that our end-to-end Seq2KG model performs on par with a state of the art rule-based system which outperformed other neural models and won the first prize of the first Knowledge Graph Contest in 2019. A new annotation scheme and three high-quality manually annotated datasets are available to help promote this direction of research.

This paper presents an innovative method to construct a reservoir geology knowledge graph (KG) by combining theories in reservoir geology and oil & gas knowledge from experts, which incorporates the recent advances in KG and natural language processing (NLP) technologies. The reservoir geology ontology was built by extracting knowledge from professional books and hydrocarbon dictionaries in the oil & gas industry, which was used to guide the construction of a reservoir geology KG. The triplets of KG was extracted from the well logs, well descriptions, well formations, and well tests data of over 1,500 wells and contains more than 300,000 entities and 600,000 relations. By encoding the entities, relations and attributes in the knowledge graph into vectors, the authors built a knowledge-driven neural formation evaluation model for predicting different formation types. The model was applied to J pilot block in northwest C oilfield and discovered a new pay zone, which corrects the original interpretation of this formation done by experts.The exploration includes the following steps. Firstly the authors established a set of hydrocarbon geological knowledge classification criterion, then constructed a reservoir geological KG and building machine learning models for intelligence formation identification in the base of the aforementioned KG, together with relevant geological parameters, finally applied the models in two oilfield datasets.This work builds the first reservoir geology KG in the oil & gas domain. It is also the first attempt to combine knowledge-driven and data-driven approach for formation evaluation modeling. As the first intelligent model applied in practice, the formation evaluation model achieves significant outcomes in an oilfield, setting a successful precedent for other areas.KeywordsReservoir geologyKnowledge graphKnowledge-powered neural formation evaluation model

BioKGrapher: Initial Evaluation of Automated Knowledge Graph Construction from Biomedical Literature

Purpose: This research project has a single purpose: the construction and evaluation of PKG-LLM, a knowledge graph framework whose application is primarily intended for cognitive neuroscience. It also aims to improve predictions of relationships among neurological entities and improve named entity recognition (NER) and relation extraction (RE) from large neurological datasets. Employing the GPT-4 and expert review, we aim to demonstrate how this framework may outperform traditional models by way of precision, recall, and F1 score, intending to provide key insights into possible future clinical and research applications in the field of neuroscience. Method: In the evaluation of PKG-LLM, there were two different tasks primarily: relation extraction (RE) and named entity recognition (NER). Both tasks processed data and obtained performance metrics, such as precision, recall, and F1-score, using GPT-4. Moreover, there was an integration of an expert review process comprising neurologists and domain experts reviewing those extracted relationships and entities and improving such final performance metrics. Model comparative performance was reported against StrokeKG and Heart Failure KG. On the other hand, PKG-LLM evinced itself to link prediction-in-cognition through metrics such as Mean Rank (MR), Mean Reciprocal Rank (MRR), and Precision at K (P@K). The model was evaluated against other link prediction models, including TransE, RotatE, DistMult, ComplEx, ConvE, and HolmE. Findings: PKG-LLM demonstrated competitive performance in both relation extraction and named entity recognition tasks. In its traditional form, PKG-LLM achieved a precision of 75.45\%, recall of 78.60\%, and F1-score of 76.89\% in relation extraction, which improved to 82.34\%, 85.40\%, and 83.85\% after expert review. In named entity recognition, the traditional model scored 73.42\% precision, 76.30\% recall, and 74.84\% F1-score, improving to 81.55\%, 84.60\%, and 82.99\% after expert review. For link prediction, PKG-LLM achieved an MRR of 0.396, P@1 of 0.385, and P@10 of 0.531, placing it in a competitive range compared to models like TransE, RotatE, and ConvE. Conclusion: This study showed that PKG-LLM mainly outperformed the existing models by adding expert reviews in its application in extraction and named entity recognition tasks. Further, the model's competitive edge in link prediction lends credence to its capability in knowledge graph construction and refinement in the field of cognitive neuroscience as well. PKG-LLM's superiority over existing models and its ability to generate more accurate results with clinical relevance indicates that it is a significant tool to augment neuroscience research and clinical applications. The evaluation process entailed using GPT-4 and expert review. This approach ensures that the resulting knowledge graph is scientifically compelling and practically beneficial in more advanced cognitive neuroscience research.

Automatic knowledge graph (KG) construction is widely used in industry for data integration and access, and there are several approaches to enable (semi-)automatic construction of knowledge graphs. One important approach is to map the raw data to a given knowledge graph schema, often a domain ontology, and construct the entities and properties according to the ontology. However, the existing approaches to construct knowledge graphs are not always efficient enough and the resulting knowledge graphs are not sufficiently application-oriented and user-friendly. The challenge arises from the trade-off: the domain ontology should be knowledge-oriented, to reflect the general domain knowledge rather than data particularities; while a knowledge graph schema should be data-oriented, to cover all data features. If the former is directly used as the knowledge graph schema, this can cause issues like blank nodes created due to classes unmapped to data and deep knowledge graph structures. To this end, we propose a system for ontology reshaping, which generates knowledge graph schemata that fully cover the data while also covers domain knowledge well. We evaluated our approach extensively with a user study and three real manufacturing datasets from Bosch against four baselines, showing promising results.KeywordsSemantic data integrationKnowledge graphOntology reshapingGraph algorithmAutomatic knowledge graph construction

The field of Natural Language Processing (NLP) focuses on developing computational techniques to analyze and extract information from human language. With the exponential growth of unstructured textual data, NLP-based techniques have become essential for extracting valuable insights from this data. However, existing information extraction systems have limitations in terms of extracting valuable information without predefined relations or ontology and storing the extracted knowledge effectively. This Ph.D. thesis aims to enhance open information extraction methods to represent unstructured textual data efficiently and effectively. The first part of the research focuses on Open Information Extraction (OIE) systems and their challenges. Existing OIE methods, including pattern-based and machine learning-based approaches, as well as neural techniques, are analyzed to understand their limitations. A Bidirectional Gated Recurrent Unit (Bi-GRU) OIE model is proposed in Chapter 3, which utilizes contextualized word embeddings to extract relevant triples from unstructured text. Experimental results demonstrate the effectiveness of this model in generating high-quality relation triples. Chapter 4 addresses the lack of labeled data, a common problem in NLP tasks. The research extends the OIE model from Chapter 3 by using learned features to generate relation triples and explores the transferability of these features across different OIE domains and the related task of Relation Extraction (RE). The results show comparable performance with traditional training, indicating the potential of OIE in achieving NLP performance without labeled data. In Chapter 5, the focus shifts to enhancing pre-trained language models for taxonomy classification. Pre-trained language models often struggle with unseen patterns during inference, and the limited size of annotated data poses a challenge. A two-stage fine-tuning procedure, incorporating data augmentation techniques, is proposed to improve the generalizability of pre-trained models. Experimental results demonstrate strong generalizability on unseen data, with an F1 score of 91.25%. Chapter 6 explores the use of OIE for constructing a knowledge graph, specifically in the context of cyber threat intelligence. Open-CyKG, an open cyber threat intelligence knowledge graph framework, is designed using an attention-based neural OIE model and a Named Entity Recognition (NER) model. Refinement and canonicalization techniques are employed to overcome ambiguity and data redundancy during knowledge graph construction. The results show that querying the constructed knowledge graph can be done efficiently, highlighting the support of OIE in knowledge graph development. The proposed components achieve beyond-state-of-the-art results in terms of OIE performance, NER performance, and knowledge graph canonicalization. The research presented in the previous chapters demonstrates significant improvements in the efficiency and effectiveness of open information extraction methods for representing unstructured textual data. These advancements leverage techniques such as data augmentation, multi-stage fine-tuning, and pre-trained language models. The construction of knowledge graphs, enabled by OIE, has the potential to mimic human intelligence and benefit various complex applications, including recommender systems, search engines, and dialog systems.

PurposeThis study aims to extract knowledge of ancient Chinese scientific and technological documents bibliographic summaries (STDBS) and provide the knowledge graph (KG) comprehensively and systematically. By presenting the relationship among content, discipline, and author, this study focuses on providing services for knowledge discovery of ancient Chinese scientific and technological documents.Design/methodology/approachThis study compiles ancient Chinese STDBS and designs a knowledge mining and graph visualization framework. The authors define the summaries' entities, attributes, and relationships for knowledge representation, use deep learning techniques such as BERT-BiLSTM-CRF models and rules for knowledge extraction, unify the representation of entities for knowledge fusion, and use Neo4j and other visualization techniques for KG construction and application. This study presents the generation, distribution, and evolution of ancient Chinese agricultural scientific and technological knowledge in visualization graphs.FindingsThe knowledge mining and graph visualization framework is feasible and effective. The BERT-BiLSTM-CRF model has domain adaptability and accuracy. The knowledge generation of ancient Chinese agricultural scientific and technological documents has distinctive time features. The knowledge distribution is uneven and concentrated, mainly concentrated on C1-Planting and cultivation, C2-Silkworm, and C3-Mulberry and water conservancy. The knowledge evolution is apparent, and differentiation and integration coexist.Originality/valueThis study is the first to visually present the knowledge connotation and association of ancient Chinese STDBS. It solves the problems of the lack of in-depth knowledge mining and connotation visualization of ancient Chinese STDBS.

[Purpose/Significance]: The digitalization of education is an essential path to advancing higher education. The construction of knowledge graphs is a key approach to achieving the digitalization and intelligence of education. [Method/Process]: This paper leverages the rich video resources of existing national first-class undergraduate programs and, based on the teaching orientations of different universities, independently designs customized ontologies and extraction principles. These are then integrated into the LLM knowledge graph builder to ensure the hierarchical structure of the overall course framework. The course video content is transformed into text form, and large language models (LLMS) and word segmentation tools are used for core content extraction, text cleaning, and lexical analysis. The structured text is then converted into SPO (Subject-Predicate-Object) triplets database. [Results/Conclusions]: Finally, the database is imported into the LLM knowledge graph builder, which is pre-configured with extraction rules. It will automatically generate the knowledge graph. After the text is imported into the LLM knowledge graph builder, it will be manually checked to ensure it better meets the actual needs of the students. [Innovation/Limitations]: The research team plans to apply the knowledge graph to train a specialized knowledge-based Q&A assistant. This will support students' understanding and self-assessment of knowledge points in an online learning community. Student feedback will be used to improve and enrich the knowledge graph. Compared to existing methods, this approach better aligns with the constantly evolving digital teaching resources available online, offering more comprehensive and higher-level automation.

Domain knowledge graph has been widely adopted for various domains, e.g., medicine, agriculture and service industry, because it can provide promising functions including intelligent search and personalized recommendation. Knowledge graph is normally composited of a huge number of entities and relations (connect entities), and the utility of knowledge graph largely depends on the richness of these entities/relations. To construct high-value knowledge graph (e.g., informative domain knowledge graph), researchers aim at automatically extracting entities from massive heterogeneous sources, which is often impossible to achieve with pure manual labor.With the blooming of natural language processing (NLP), researchers have proposed Named Entity Recognition (NER) technique to automatically extract entities from raw texts. NER is mostly regarded as a supervised sequence labeling/tagging task; that is, recognizing entities from unseen texts according to the patterns learned from labeled texts. As an essential step for knowledge graph construction, as well as some other NLP tasks, the development of NER is one of the main focuses in both the academia and the industry in recent years. Under this background, this competition targets at exploring novel and insightful NER methods to better capture the entities, especially for the construction of domain knowledge graphs.