Ontology Learning Research Articles

Type-2 Neutrosophic Ontology for Automated Essays Scoring in Cybersecurity Education

Given the growing demand for cybersecurity education, the practice of protecting network and software systems from digital and electronic attacks, investing in educational cybersecurity helps significantly reduce the risk of data breaches and protect against security breaches, and given the urgent need and growing number of students worldwide, it is also a way to connect with and between customers by building trust-based relationships, especially regarding essays. Automated Essay Scoring (AES) is a scalable solution for grading large amounts of essays with multiple uses, making it ideal for cybersecurity certification programs, online courses, and standardized tests. In the field of educational cybersecurity, automated essay scoring poses unique challenges due to specialized terminology, persistent and evolving threats. These automated scoring systems use domain-defined ontologies to grade essays but struggle to manage uncertainties, such as ambiguous language and partially valid arguments, which can influence the accuracy of their scoring. Traditional ontologies often struggle to interpret such uncertainties, leading to inconsistent results. Type 2 neutrosophic clustering (T2NS) as a novel approach introduced in this paper is combined with an automated article scoring system based on the cybersecurity learning ontology to address these challenges. The main steps include extracting concepts relevant to this research area from the articles, formalizing the cybersecurity scoring criteria as ontological rules and extending the ontology using T2NS, as well as defining membership functions to measure uncertainty and inconsistency levels. This evaluation using benchmark datasets of cybersecurity articles shows that this approach significantly enhances the scoring reliability and robustness of the approach compared to the basic AES methods.

Enhancing human phenotype ontology term extraction through synthetic case reports and embedding-based retrieval: A novel approach for improved biomedical data annotation

With the increasing utilization of exome and genome sequencing in clinical and research genetics, accurate and automated extraction of human phenotype ontology (HPO) terms from clinical texts has become imperative. Traditional methods for HPO term extraction, such as PhenoTagger, often face limitations in coverage and precision. In this study, we propose a novel approach that leverages large language models (LLMs) to generate synthetic sentences with clinical context, which were semantically encoded into vector embeddings. These embeddings are linked to HPO terms, creating a robust knowledgebase that facilitates precise information retrieval. Our method circumvents the known issue of LLM hallucinations by storing and querying these embeddings within a true database, ensuring accurate context matching without the need for a predictive model. We evaluated the performance of three different embedding models, all of which demonstrated substantial improvements over PhenoTagger. Top recall (sensitivity), precision (positive-predictive value, PPV), and F1 are 0.64, 0.64, and 0.64, respectively, which were 31%, 10%, and 21% better than PhenoTagger. Furthermore, optimal performance was achieved when we combined the best performing embedding model with PhenoTagger (a.k.a. Fused model), resulting in recall (sensitivity), precision (PPV), and F1 values of 0.7, 0.7, and 0.7, respectively, which are 10%, 10%, and 10% better than the best embedding models. Our findings underscore the potential of this integrated approach to enhance the precision and reliability of HPO term extraction, offering a scalable and effective solution for biomedical data annotation.

Interpreting the structure and results of a data warehouse model using ontology and machine learning techniques

In this paper, we present an intelligent methodology for assisting decision-makers in both understanding the structure of a data warehouse model and making decisions. The support module proposed by our method comprises three operations: (i) transforming a data warehouse model into an ontology, allowing for the display of the different terminology related to a specific domain as well as the different semantic relationships between them, (ii) recommending a series of queries to the decision-maker that enables an understanding of the reasoning logic based on the ontology’s structure, (iii) enriching the different results obtained from some analysis tools through the use of advanced machine learning techniques. The originality of our proposed methodology lies in its ability to influence a decision-maker’s thinking in order to encourage him to take full advantage of the services provided by the data warehouse model. We apply our proposed methodology to an extended data warehouse model that enables the analysis of social media data related to people with personality disorders (PD). The primary goal of this model is to provide decision-makers with suitable services that allow them to make meaningful decisions for people with personality disorders around the world. This task was carried out by analyzing the activities and content of people on social media. In addition, one of the main advantages of this model is the use of various artificial intelligence (AI) and natural language processing (NLP) techniques. Our proposed methodology is implemented and the results achieved are evaluated using both quantitative and qualitative methods.

Dynamic Retrieval Augmented Generation of Ontologies using Artificial Intelligence (DRAGON-AI)

BackgroundOntologies are fundamental components of informatics infrastructure in domains such as biomedical, environmental, and food sciences, representing consensus knowledge in an accurate and computable form. However, their construction and maintenance demand substantial resources and necessitate substantial collaboration between domain experts, curators, and ontology experts.We present Dynamic Retrieval Augmented Generation of Ontologies using AI (DRAGON-AI), an ontology generation method employing Large Language Models (LLMs) and Retrieval Augmented Generation (RAG). DRAGON-AI can generate textual and logical ontology components, drawing from existing knowledge in multiple ontologies and unstructured text sources.ResultsWe assessed performance of DRAGON-AI on de novo term construction across ten diverse ontologies, making use of extensive manual evaluation of results. Our method has high precision for relationship generation, but has slightly lower precision than from logic-based reasoning. Our method is also able to generate definitions deemed acceptable by expert evaluators, but these scored worse than human-authored definitions. Notably, evaluators with the highest level of confidence in a domain were better able to discern flaws in AI-generated definitions. We also demonstrated the ability of DRAGON-AI to incorporate natural language instructions in the form of GitHub issues.ConclusionsThese findings suggest DRAGON-AI's potential to substantially aid the manual ontology construction process. However, our results also underscore the importance of having expert curators and ontology editors drive the ontology generation process.

LLMs4OL 2024 Datasets: Toward Ontology Learning with Large Language Models

Ontology learning (OL) from unstructured data has evolved significantly, with recent advancements integrating large language models (LLMs) to enhance various aspects of the process. The paper introduces the LLMs4OL 2024 datasets, developed to benchmark and advance research in OL using LLMs. The LLMs4OL 2024 dataset as a key component of the LLMs4OL Challenge, targets three primary OL tasks: Term Typing, Taxonomy Discovery, and Non-Taxonomic Relation Extraction. It encompasses seven domains, i.e. lexosemantics and biological functions, offering a comprehensive resource for evaluating LLM-based OL approaches Each task within the dataset is carefully crafted to facilitate both Few-Shot (FS) and Zero-Shot (ZS) evaluation scenarios, allowing for robust assessment of model performance across different knowledge domains to address a critical gap in the field by offering standardized benchmarks for fair comparison for evaluating LLM applications in OL.

Preface for LLMs4OL 2024: The 1st Large Language Models for Ontology Learning Challenge at the 23rd ISWC

We are pleased to present the proceedings of the “1st Large Language Models for Ontology Learning Challenge (LLMs4OL 2024)”, held at the 23rd International Semantic Web Conference (ISWC). This challenge marks a significant a dvancement in utilizing Large Language Models (LLMs) for Ontology Learning (OL)—a key Semantic Web component that facilitates the automatic extraction of structured knowledge from unstructured data. The challenge features three main tasks: Term Typing (identifying categories for terms), Taxonomy Discovery (uncovering hierarchical relationships), and Non-Taxonomic Relation Extraction (identifying other meaningful relationships between terms). Each task is designed to test different facets of ontology construction and to encourage the exploration of innovative techniques. This challenge seeks to foster collaboration, inspire innovation, and expand the capabilities of LLMs in OL. The proceedings include a collection of innovative solutions and insights from global participants, highlighting the crucial role of LLMs in enhancing the web with structured knowledge. We believe the outcomes of this challenge will propel further advancements in OL and its applications on the semantic web. We would like to extend our gratitude to all the participants for their invaluable contributions, which their solutions and dedication have greatly enriched this challenge. Our sincere thanks also go to the conference organizers and committee for their efforts in hosting this event. We are deeply appreciative of the reviewers for their evaluations and feedback. Their reviews have been instrumental in enhancing the quality of the submissions. We would like to specifically acknowledge: Dr. Amin Keshavarzi (Postdoctoral Researcher, L3S & TIB, Germany) Mostafa Rahgouy (Lead Project Coordinator, Auburn University, USA) Sahar Tahmasebi (Doctoral Researcher, TIB, Germany) Milad Molazadeh Oskuee (Lead NLP Researcher, Iran) Aida Usmanova (Doctoral Researcher, Leuphana Universit ¨ at L¨uneburg, Germany) Emetis Niazmand (Doctoral Researcher, TIB, Germany) Lastly, we would like to express our gratitude to ”Xenia Felice van Edig” and ”Karolina Linerová” from the TIB Open Publishing, for their support in bringing these proceedings online. We also would like to acknowledge that the 1st LLMs4OL Challenge @ ISWC 2024 jointly supported by the NFDI4DataScience initiative (DFG, German Research Foundation, Grant ID: 460234259) and the SCINEXT project (BMBF, German Federal Ministry of Education and Research, Grant ID: 01lS22070).

Phoenixes at LLMs4OL 2024 Tasks A, B, and C: Retrieval Augmented Generation for Ontology Learning

Large language models (LLMs) showed great capabilities in ontology learning (OL) where they automatically extract knowledge from text. In this paper, we proposed a Retrieval Augmented Generation (RAG) formulation for three different tasks of ontology learning defined in the LLMs4OL Challenge at ISWC 2024. For task A - term typing - we considered terms as a query and encoded the query through the Query Encoder model for searching through knowledge base embedding of types embeddings obtained through Context Encoder. Next, using Zero-Shot Prompt template we asked LLM to determine what types are appropriate for a given term within the term typing task. Similarly, for Task B, we calculated the similarity matrix using an encoder-based transformer model, and by applying the similarity threshold we considered only similar pairs to query LLM to identify whatever pairs have the "is-a" relation between a given type and in a case of having the relationships which one is "parent" and which one is "child". In final, for Task C -- non-taxonomic relationship extraction -- we combined both approaches for Task A and B, where first using Task B formulation, child-parents are identified then using Task A, we assigned them an appropriate relationship. For the LLMs4OL challenge, we experimented with the proposed framework over 5 subtasks of Task A, all subtasks of Task B, and one subtask of Task C using Mistral-7B LLM.

Silp_nlp at LLMs4OL 2024 Tasks A, B, and C: Ontology Learning through Prompts with LLMs

Our team, silp_nlp, participated in the LLMs4OL Challenge at ISWC 2024, engaging in all three tasks focused on ontology generation. The tasks include predicting the type of a given term, extracting a hierarchical taxonomy between two terms, and extracting non-taxonomy relations between two terms. To accomplish these tasks, we used machine learning models such as random forest, logistic regression and generative models for the first task and generative models such as llama-3-8b-instruct, mistral 8*7b and GPT-4o-mini for the second and third tasks. Our results showed that generative models performed better for certain domains, such as subtasks A6 and B2. However, for other domains, the prompt-based technique failed to generate promising results. Our team achieved first place in six subtasks and second place in five subtasks, demonstrating our expertise in ontology generation.

LLMs4OL 2024 Overview: The 1st Large Language Models for Ontology Learning Challenge

This paper outlines the LLMs4OL 2024, the first edition of the Large Language Models for Ontology Learning Challenge. LLMs4OL is a community development initiative collocated with the 23rd International Semantic Web Conference (ISWC) to explore the potential of Large Language Models (LLMs) in Ontology Learning (OL), a vital process for enhancing the web with structured knowledge to improve interoperability. By leveraging LLMs, the challenge aims to advance understanding and innovation in OL, aligning with the goals of the Semantic Web to create a more intelligent and user-friendly web. In this paper, we give an overview of the 2024 edition of the LLMs4OL challenge and summarize the contributions.

DaSeLab at LLMs4OL 2024 Task A: Towards Term Typing in Ontology Learning

The report presents the evaluation results of our approach in the LLM4OL Challenge, where we fine-tuned GPT-3.5 for Task A (Term Typing) across three different datasets. Our approach demonstrated consistent and robust performance during few- shot testing, achieving top rankings in several datasets and sub-datasets, proving the potential of fine-tuning LLMs for ontology creation tasks.

The Ghost at LLMs4OL 2024 Task A: Prompt-Tuning-Based Large Language Models for Term Typing

The LLMs4OL Challenge @ ISWC 2024 aims to explore the intersection of Large Language Models (LLMs) and Ontology Learning (OL) through three main tasks: 1) Term Typing, 2) Taxonomy Discovery and 3) Non-Taxonomic Relation Extraction. In this paper, we present our system's design for the term typing task. Our approach utilizes automatic prompt generation using soft prompts to enhance term typing accuracy and efficiency. We conducted experiments on several datasets, including WordNet, UMLS, GeoNames, NCI, MEDCIN, and SNOMEDCT_US. Our approach outperformed the baselines on most datasets, except for GeoNames, where it faced challenges due to the complexity and specificity of this domain, resulting in substantially lower scores. Additionally, we report the overall results of our approach in this challenge, which highlight its promise while also indicating areas for further improvement.

RWTH-DBIS at LLMs4OL 2024 Tasks A and B

The increasing capabilities of Large Language Models (LLMs) have opened new opportunities for enhancing Ontology Learning (OL), a process crucial for structuring domain knowledge in a machine-readable format. This paper reports on the participation of the RWTH-DBIS team in the LLMs4OL Challenge at ISWC 2024, addressing two primary tasks: term typing and taxonomy discovery. We used LLaMA-3-8B and GPT-3.5-Turbo models to find the performance gaps between open-source and commercial LLMs. For open-source LLMs, our methods included domain-specific continual training, fine-tuning, and knowledge-enhanced prompt-tuning. These approaches were evaluated on the benchmark datasets from the challenge, i.e., GeoNames, UMLS, Schema.org, and the Gene Ontology (GO), among others. The results indicate that domain-specific continual training followed by task-specific fine-tuning enhances the performance of open-source LLMs in these tasks. However, performance gaps remain when compared to commercial LLMs. Additionally, the developed prompting strategies demonstrate substantial utility. This research highlights the potential of LLMs to automate and improve the OL process, offering insights into effective methodologies for future developments in this field.

DSTI at LLMs4OL 2024 Task A: Intrinsic Versus Extrinsic Knowledge for Type Classification

We introduce semantic towers, an extrinsic knowledge representation method, and compare it to intrinsic knowledge in large language models for ontology learning. Our experiments show a trade-off between performance and semantic grounding for extrinsic knowledge compared to a fine-tuned model's intrinsic knowledge. We report our findings on the Large Language Models for Ontology Learning (LLMs4OL) 2024 challenge.

TSOTSALearning at LLMs4OL Tasks A and B : Combining Rules to Large Language Model for Ontology Learning

This paper presents our contribution to the Large Language Model For Ontology Learning (LLMs4OL) challenge hosted by ISWC conference. The challenge involves extracting and classifying various ontological components from multiple datasets. The organizers of the challenge provided us with the train set and the test set. Our goal consists of determining in which conditions foundation models such as BERT can be used for ontologies learning. To achieve this goal, we conducted a series of experiments on various datasets. Initially, GPT-4 was tested on the wordnet dataset, achieving an F1-score of 0.9264. Subsequently, we performed additional experiments on the same dataset using BERT. These experiments demonstrated that by combining BERT with rule-based methods, we achieved an F1-score of 0.9938, surpassing GPT-4 and securing the first place for term typing on the Wordnet dataset.

A Hybrid Approach to Ontology Construction for the Badini Kurdish Language

Semantic ontologies have been widely utilized as crucial tools within natural language processing, underpinning applications such as knowledge extraction, question answering, machine translation, text comprehension, information retrieval, and text summarization. While the Kurdish language, a low-resource language, has been the subject of some ontological research in other dialects, a semantic web ontology for the Badini dialect remains conspicuously absent. This paper addresses this gap by presenting a methodology for constructing and utilizing a semantic web ontology for the Badini dialect of the Kurdish language. A Badini annotated corpus (UOZBDN) was created and manually annotated with part-of-speech (POS) tags. Subsequently, an HMM-based POS tagger model was developed using the UOZBDN corpus and applied to annotate additional text for ontology extraction. Ontology extraction was performed by employing predefined rules to identify nouns and verbs from the model-annotated corpus and subsequently forming semantic predicates. Robust methodologies were adopted for ontology development, resulting in a high degree of precision. The POS tagging model attained an accuracy of 95.04% when applied to the UOZBDN corpus. Furthermore, a manual evaluation conducted by Badini Kurdish language experts yielded a 97.42% accuracy rate for the extracted ontology.

Fuzzy deep visual panic detection

Deep visual data analysis from social network has become an increasingly important area of research. In fact, this form of assessment makes it viable to recognize new information on social users which incorporates emotions. In order to recognize users’ emotions and other latent attributes, most of the existing approaches have used textual data and have obtained accurate results. However, little attention has been paid to visual data that have become increasingly popular in recent years.This work describes how to develop a conceptual representation model for social network analysis and social emotion analysis based on the integration of fuzzy logic and ontological modeling. The primary aim is to create an ontology that can extract new information about a social user’s mood, which can be classified as panic, no-panic, or neutral. Fuzzy logic is necessary to deal with subjective data, as users share imprecise and vague data on their social profiles. Fuzzy logic has been observed as a successful method to capture the expression of emotions due to the fuzzy nature of emotions and the ambiguous definitions of emotion words. The proposed work investigate the role of fuzzy logic in social network analysis. This study simulate a fuzzy deep system integrated with ontology for classifying social visual data (shared images) into panic, no-panic or neutral classes in order to determine the social users’ stress intensity. The Social distancing and the huge amount of shared data in Tunisia were calculated to demonstrate this classification. The experiments performed in this paper aim to create not only a novel annotated visual database named visual panic database, but also a new semantic model for modeling users’ profiles, in social network, based on the combination between ontology and deep learning techniques. In this context, we will work in the future on the combination between a user’s visual and textual data in order to improve the performance of this recognition. The proposed fuzzy system reflected the viral proliferation in stressed users and achieved an accuracy of 87%.

Review on integration of ontology and deep learning in cultural heritage image retrieval

Image retrieval methods are currently developing towards big data processing. The literature review is focused on image big data extraction with cultural heritage domain as training and testing datasets. The development of image retrieval process starts from content-based using machine algorithms, deep learning to ontology-based. Image recognition research with cultural heritage domain is conducted because of the importance of preserving and appreciating cultural heritage, in this case, cultural heritage images such as Indonesian Batik are discussed. Batik motif images are Indonesian cultural heritage that has thousands of motifs that are grouped into many classes with a non-linear hyperplane. The problem is focused on processing big data that has many classes. Currently research is evolving into knowledge-based image retrieval using ontologies due to semantic gap constraints. The results of this literature study can be the basis for developing research on the application of appropriate deep learning algorithms so as to utilize the hierarchy of classes and subclasses of image ontologies with cultural heritage domains.

Quantum ontology de-naturalized: What we can't learn from quantum mechanics

Philosophers of science commonly connect ontology and science, stating that these disciplines maintain a two-way relationship: on the one hand, we can extract ontology from scientific theories; on the other hand, ontology provides the realistic content of our scientific theories. In this article, we will critically examine the process of naturalizing ontology, i.e., confining the work of ontologists merely to the task of pointing out which entities certain theories commit themselves to. We will use non-relativistic quantum mechanics as a case study. We begin by distinguishing two roles for ontology: the first would be characterized by cataloging existing entities according to quantum mechanics; the second would be characterized by establishing more general ontological categories in which existing entities must be classified. We argue that only the first step is available for a naturalistic approach; the second step not being open for determination or anchoring in science. Finally, we also argue that metaphysics is still a step beyond ontology, not contained in either of the two tasks of ontology, being thus even farther from science.

Precise Molecular Subtyping Reveals Heterogeneity of Lung Adenocarcinoma Based on DNA Methylation

Background: Due to the high heterogeneity of lung adenocarcinoma (LUAD), which restricts the effectiveness of therapy, precise molecular subgrouping of LUAD is of great significance. Clinical research has demonstrated the significant potential of DNA methylation as a classification indicator for human malignancies. Methods: WGML framework (which was developed based on weighted gene correlation network analysis (WGCNA), Gene Ontology (GO), and machine learning) was developed to precisely subgroup molecular subtypes of LUAD. This framework included two parts: the WG algorithm and the machine learning part. The WG algorithm part was an original algorithm used to obtain a crucial module, which was characterized by weighted correlation network analysis, functional annotation, and mathematical algorithms. The machine learning part utilized the Boruta algorithm, random forest algorithm, and Gradient Boosting Regression Tree algorithm to select feature genes. Then, based on the results of the WGML framework, subtypes were computed by the hierarchical clustering algorithm. A series of analyses, including dimensionality reduction methods, survival analysis, clinical stage analysis, immune infiltration analysis, tumor environment analysis, immune checkpoints analysis, TIDE analysis, CYT analysis, somatic mutation analysis, and drug sensitivity analysis, were utilized to demonstrate the effectiveness of subgrouping. GEO datasets were used to externally validate the results. Meanwhile, another subgrouping method of LUAD from another study was employed to compare with the WGML framework. Result: By importing DNA methylation data into the WGML framework, nine genes were obtained to further subgroup LUAD. Three subtypes, the Carcinogenesis subtype, Immune-infiltration subtype, and Chemoresistance subtype, were identified. The dimensionality reduction method exhibited great distinctness between subtypes. A series of analyses were employed to exhibit the difference among the three subtypes and to demonstrate the accuracy of the definition of subtypes. Besides, the WGML framework was compared with a LUAD subgrouping method from another research, which demonstrated that WGML had better efficiency for subgrouping LUAD. Conclusion: This study provides a novel LUAD subgrouping framework named WGML for the accurate subgrouping of lung adenocarcinoma. result: By importing DNA methylation data into WGML framework, nine genes were obtained to further subgroup LUAD. Three subtypes, Carcinogenesis subtype, Immune-infiltration subtype, and Chemoresistance subtype were identified. Dimensionality reduction method exhibited great distinctness between subtypes. A series analyses were employed to exhibit the difference of three subtypes and to demonstrate accuracy of definition of subtypes. Besides, WGML framework was compared with a LUAD subgrouping method from another research, which demonstrated WGML had better efficiency for subgrouping LUAD.

From specialized knowledge frames to linguistically based ontologies

This paper explains conceptual modeling within the framework of Frame-Based Terminology (Faber, 2012; 2015; 2022), as applied to EcoLexicon (ecolexicon.ugr.es), a specialized knowledge base on the environment (León-Araúz, Reimerink &, Faber, 2019; Faber & León-Araúz, 2021). It describes how a frame-based terminological resource is currently being restructured and reengineered as an initial step towards its formalization and subsequent transformation into an ontology. It also explains how the information in EcoLexicon can be integrated in environmental ontologies such as ENVO (Buttigieg, Morrison, Smith, Mungall & Lewis, 2013; Buttigieg, Pafilis, Lewis, Schildhauer, Walls & Mungall, 2016), particularly at the bottom tiers of the Ontology Learning Layer Cake (Cimiano, 2006; Cimiano, Maedche, Staab & Volker, 2009). The assumption is that frames, as a conceptual modeling tool, and information extracted from corpora can be used to represent the conceptual structure of a specialized domain.