Knowledge Representation Research Articles

Continual learning and its industrial applications: A selective review

AbstractIn many industrial applications, datasets are often obtained in a sequence associated with a series of similar but different tasks. To model these datasets, a machine‐learning algorithm, which performed well on the previous task, may not have as strong a performance on the current task. When the architecture of the algorithm is trained to adapt to new tasks, often the whole architecture needs to be revised and the old knowledge of modeling can be forgotten. Efforts to make the algorithm work for all the relevant tasks can cost large computational resources and data storage. Continual learning, also called lifelong learning or continual lifelong learning, refers to the concept that these algorithms have the ability to continually learn without forgetting the information obtained from previous task. In this work, we provide a broad view of continual learning techniques and their industrial applications. Our focus will be on reviewing the current methodologies and existing applications, and identifying a gap between the current methodology and the modern industrial needs.This article is categorized under: Technologies > Artificial Intelligence Fundamental Concepts of Data and Knowledge > Knowledge Representation Application Areas > Business and Industry

Effective knowledge representation and utilization for sustainable collaborative learning across heterogeneous systems

AbstractThe increasingly decentralized and private nature of data in our digital society has motivated the development of collaborative intelligent systems that enable knowledge aggregation among data owners. However, collaborative learning has only been investigated in simple settings. For example, clients are often assumed to train solution models de novo, disregarding all prior expertise. The learned model is typically represented in task‐specific forms that are not generalizable to unseen, emerging scenarios. Finally, a universal model representation is enforced among collaborators, ignoring their local compute constraints or input representations. These limitations hampers the practicality of prior collaborative systems in learning scenarios with limited task data that demand constant knowledge adaptation and transfer across information silos, tasks, and learning models, as well as the utilization of prior solution expertise. Furthermore, prior collaborative learning frameworks are not sustainable on a macro scale where participants desire fairness allocation of benefits (e.g., access to the combined model) based on their costs of participation (e.g., overhead of model sharing and training synchronization, risk of information breaches, etc.). This necessitates a new perspective of collaborative learning where the server not only aggregates but also conducts valuation of the participant's contribution, and distribute aggregated information to individuals in commensurate to their contribution. To substantiate the above vision, we propose a new research agenda on developing effective and sustainable collaborative learning frameworks across heterogeneous systems, featuring three novel computational capabilities on knowledge organization: model expression, comprehension, and valuation.

The Unified Phenotype Ontology (uPheno): A framework for cross-species integrative phenomics.

Phenotypic data are critical for understanding biological mechanisms and consequences of genomic variation, and are pivotal for clinical use cases such as disease diagnostics and treatment development. For over a century, vast quantities of phenotype data have been collected in many different contexts covering a variety of organisms. The emerging field of phenomics focuses on integrating and interpreting these data to inform biological hypotheses. A major impediment in phenomics is the wide range of distinct and disconnected approaches to recording the observable characteristics of an organism. Phenotype data are collected and curated using free text, single terms or combinations of terms, using multiple vocabularies, terminologies, or ontologies. Integrating these heterogeneous and often siloed data enables the application of biological knowledge both within and across species. Existing integration efforts are typically limited to mappings between pairs of terminologies; a generic knowledge representation that captures the full range of cross-species phenomics data is much needed. We have developed the Unified Phenotype Ontology (uPheno) framework, a community effort to provide an integration layer over domain-specific phenotype ontologies, as a single, unified, logical representation. uPheno comprises (1) a system for consistent computational definition of phenotype terms using ontology design patterns, maintained as a community library; (2) a hierarchical vocabulary of species-neutral phenotype terms under which their species-specific counterparts are grouped; and (3) mapping tables between species-specific ontologies. This harmonized representation supports use cases such as cross-species integration of genotype-phenotype associations from different organisms and cross-species informed variant prioritization.

Simona: a comprehensive R package for semantic similarity analysis on bio-ontologies

BackgroundBio-ontologies are keys in structuring complex biological information for effective data integration and knowledge representation. Semantic similarity analysis on bio-ontologies quantitatively assesses the degree of similarity between biological concepts based on the semantics encoded in ontologies. It plays an important role in structured and meaningful interpretations and integration of complex data from multiple biological domains.ResultsWe present simona, a novel R package for semantic similarity analysis on general bio-ontologies. Simona implements infrastructures for ontology analysis by offering efficient data structures, fast ontology traversal methods, and elegant visualizations. Moreover, it provides a robust toolbox supporting over 70 methods for semantic similarity analysis. With simona, we conducted a benchmark against current semantic similarity methods. The results demonstrate methods are clustered based on their mathematical methodologies, thus guiding researchers in the selection of appropriate methods. Additionally, we explored annotation-based versus topology-based methods, revealing that semantic similarities solely based on ontology topology can efficiently reveal semantic similarity structures, facilitating analysis on less-studied organisms and other ontologies.ConclusionsSimona offers a versatile interface and efficient implementation for processing, visualization, and semantic similarity analysis on bio-ontologies. We believe that simona will serve as a robust tool for uncovering relationships and enhancing the interoperability of biological knowledge systems.

OGSS: An Ontology-Guided and Scheduled-Sampling Approach for Overlapping Event Extraction

Event extraction is a complex and challenging task in the field of information extraction. It aims to identify event types, triggers, and argument information from the text. In recent years, overlapping event extraction has attracted the attention of researchers because of its higher challenge and practicability, and some work has carried out in-depth research on overlapping event extraction and achieved remarkable results. But these works (1) ignore the role of ontology knowledge in event extraction; (2) use the same semantic encoding for multi-stage models, lacking consideration for the independent characteristics of extraction tasks such as event types, triggers, and arguments; and (3) face issues in the training process of multi-stage models, such as error cascading and slow convergence. To address the above issues, we propose an ontology-guided and scheduled-sampling approach for overlapping event extraction, termed as OGSS. First, we design a symmetric matrix for event ontology knowledge representation and integrate it into the semantic encoding process, infusing ontology knowledge into event extraction. Second, for extraction targets such as event types, triggers, and arguments, we process the semantic encoding according to the characteristics of each extraction target, obtaining semantic representations tailored for each subtask. Finally, we view multi-stage predictions as sequential outputs of a joint model, using a scheduled sampling strategy between subtasks to effectively mitigate the cascading propagation of errors during training and accelerate model convergence. We conduct extensive experiments on the FewFc event extraction benchmark dataset. The results show that OGSS achieves significant improvements in overlapping event extraction tasks compared to previous methods.

You’re doing it wrong: the governance of motherhood through mommy blogs

ABSTRACT Mommy blog communities have been established by mothers who publish their stories online with the stated purpose to empower themselves and their peers by offering a seemingly unfiltered representation of motherhood. While this benefit has been felt by many readers, there are invisible governing discourses that have prevented such blogs from becoming authentic spaces of resistance to the hegemonic ideals of motherhood. To understand the governing forces placed upon mothers through “mommy blogs,” we explored the question: How is motherhood shaped through narratives (re)presented in mommy blogs? Critical discourse analysis, informed by a governmentality perspective, was used to analyze mommy blog entries to illuminate representations of power and knowledge within the blog entries. The findings suggest that a governmentality perspective is useful in understanding how mommy blogs enact power over mothers using mechanisms of governance through 1) sameness and imperfection, 2) perceived self-compassion, 3) advice and support, and 4) the scientification of motherhood. As more people turn to online communities for support with the experience of mothering, it is important to garner a critical understanding of how mommy blogs continue to influence contemporary experiences of motherhood.

ORPP—An Ontology for Skill-Based Robotic Process Planning in Agile Manufacturing

Ontology plays a significant role in AI (Artificial Intelligence) and robotics by providing structured data, reasoning, action understanding, context awareness, knowledge transfer, and semantic learning. The structured framework created by the ontology for knowledge representation is crucial for enabling intelligent behavior in robots. This paper provides a state-of-the-art analysis on the existing ontology approaches and at the same time consolidates the terms in the robotic task planning domain. The major gap identified in the literature is the need to bridge higher-level robotic process management and lower-level robotic control. This gap makes it difficult for operators/non-robotic experts to integrate robots into their production processes as well as evaluate key performance indicators (KPI) of the processes. To fill the gap, the authors propose an ontology for skill-based robotics process planning (ORPP). ORPP not only provides a standardization in the robotic process planning in the agile manufacturing domain but also enables non-robotic experts to design and plan their production processes using an intuitive Process-Task-Skill-Primitive structure to control low-level robotic actions. On the performance level, this structure provides traceability of the KPIs down to the robot control level.

Distinguishing word identity and sequence context in DNA language models

Transformer-based large language models (LLMs) are very suited for biological sequence data, because of analogies to natural language. Complex relationships can be learned, because a concept of "words" can be generated through tokenization. Training the models with masked token prediction, they learn both token sequence identity and larger sequence context. We developed methodology to interrogate model learning, which is both relevant for the interpretability of the model and to evaluate its potential for specific tasks. We used DNABERT, a DNA language model trained on the human genome with overlapping k-mers as tokens. To gain insight into the model′s learning, we interrogated how the model performs predictions, extracted token embeddings, and defined a fine-tuning benchmarking task to predict the next tokens of different sizes without overlaps. This task evaluates foundation models without interrogating specific genome biology, it does not depend on tokenization strategies, vocabulary size, the dictionary, or the number of training parameters. Lastly, there is no leakage of information from token identity into the prediction task, which makes it particularly useful to evaluate the learning of sequence context. We discovered that the model with overlapping k-mers struggles to learn larger sequence context. Instead, the learned embeddings largely represent token sequence. Still, good performance is achieved for genome-biology-inspired fine-tuning tasks. Models with overlapping tokens may be used for tasks where a larger sequence context is of less relevance, but the token sequence directly represents the desired learning features. This emphasizes the need to interrogate knowledge representation in biological LLMs.

Problem-based learning as a teaching strategy for the foundations of knowledge representation systems: The experience with cellular automata

Problem-based learning as a teaching strategy for the foundations of knowledge representation systems: The experience with cellular automata

Enhanced generalization and specialization of brain representations of semantic knowledge in healthy aging

Aging is often associated with a decrease in cognitive capacities. However, semantic memory appears relatively well preserved in healthy aging. Both behavioral and neuroimaging studies support the view that changes in brain networks contribute to this preservation of semantic cognition. However, little is known about the role of healthy aging in the brain representation of semantic categories. Here we used pattern classification analyses and computational models to examine the neural representations of living and non-living word concepts. The results demonstrate that brain representations of animacy in healthy aging exhibit increased similarity across categories, even across different task contexts. This pattern of results aligns with the neural dedifferentiation hypothesis that proposes that aging is associated with decreased specificity in brain activity patterns and less efficient neural resource allocation. However, the loss in neural specificity for different categories was accompanied by increased dissimilarity of item-based conceptual representations within each category. Taken together, the age-related patterns of increased generalization and specialization in the brain representations of semantic knowledge may reflect a compensatory mechanism that enables a more efficient coding scheme characterized by both compression and sparsity, thereby helping to optimize the limited neural resources and maintain semantic processing in the healthy aging brain.

The role of an ontology-based knowledge backbone in a circular factory

Abstract In a circular factory, new products are produced reusing parts from used products, as well as newly manufactured parts. The production system consists of disassembly, testing as well as assembly steps. Due to the unforeseeable conditions of the used parts, the complexity of such a circular factory is challenging. This paper contributes a concept of an ontology-based knowledge backbone to master the challenges of such a circular factory. The concept addresses the representation of knowledge especially taking into account uncertainty, how to design queries and means to detect similarities and analogies. Furthermore, the role of research data management with automatized workflows as a supplier for FAIR data is elaborated.

Formalization of Elements and Control Schemes in Intelligent System Models

The purpose of the work is to formalize the concepts and principles that form the basis of the mathematical model for controlling the life cycles of intelligent systems. The controlling elements formal description is part of a universal model of such systems. This model part is important for implementing complex intelligent systems with a large amount and diversity of knowledge flows and knowledge processing schemes performed simultaneously. The other two areas of this model are associated with a unified logical-algebraic description of the concept of knowledge representation formalism, as well as concepts related to the architecture of intelligent systems. Formalization of ideas about control in intelligent systems is carried out according to the principle of a hierarchy of controlling elements and processes. The elements of the mathematical model that implements this principle correspond to event-controlled automata with infinite memory and associated algorithms for the automata simulation in specific systems. The objectives of the research include the development of formats for describing the hierarchy of diagrams and assemblies of control elements in a universal model of an intelligent system, and diagrams of algorithms for the functioning of intelligent systems based on the processing of model descriptions. Models of intelligent system control elements hierarchy and execution algorithms for the sets of activated control elements for the implementation time of life cycles of intelligent systems are described by extending the special ontological structures. These structures implement customizations of universal control model

Internet of Conscious Things: Ontology-Based Social Capabilities for Smart Objects

Emerging distributed intelligence paradigms for the Internet of Things (IoT) call for flexible and dynamic reconfiguration of elementary services, resources and devices. In order to achieve such capability, this paper faces complex interoperability and autonomous decision problems by proposing a thorough framework based on the integration of the Semantic Web of Things (SWoT) and Social Internet of Things (SIoT) paradigms. SWoT enables low-power knowledge representation and autonomous reasoning at the edge of the network through carefully optimized inference services and engines. This layer provides service/resource management and discovery primitives for a decentralized collaborative social protocol in the IoT, based on the Linked Data Notifications(LDN) over Linked Data Platform on Constrained Application Protocol (LDP-CoAP). The creation and evolution of friend and follower relationships between pairs of devices is regulated by means of novel dynamic models assessing trust as a usefulness reputation score. The close SWoT-SIoT integration overcomes the functional limitations of existing proposals, which focus on either social device or semantic resource management only. A smart mobility case study on Plug-in Electric Vehicles (PEVs) illustrates the benefits of the proposal in pervasive collaborative scenarios, while experiments show the computational sustainability of the dynamic relationship management approach.

The Role of Cognitive Schema Theory in the Translation from Arabic into English

This study investigates the role of schema theory in translation and aims to gain knowledge that may progress AI-related domains through the simulation of cognition and AI. Schema theory hypothesizes that translators use mental frameworks (schemata) to organize and interpret information. In the context of translation, schemata play a crucial role in knowledge representation, affecting limitations made during the translation process. A cognitive approach is employed and qualitative methods are used in the analysis. A corpus is collected from the translation of a SL Arabic novel (Frankenstein in Baghdad by A. Saadawi) into TL English, covering domains and language pairs. The translation process is analyzed to identify cognitive patterns employed by the translator as they apply schemata to achieve cross-cultural renditions. Conclusions reveal that schema activation significantly impacts translation choices, influencing how the translator interprets and conveys meaning across languages. Cognitive patterns shed light on how cultural and linguistic factors influence the transfer of information between SL and TL. Additionally, the study uncovers variations in schema utilization across different translation tasks and the adaptability and flexibility of cognitive processes in response to varying contexts and linguistic challenges. By deepening our understanding of schema theory, this research contributes to the design, training, and assessment of AI algorithms. It also provides valuable insights into the cognitive mechanisms underlying successful cross-linguistic communication and offers practical implications for translators engaged in intercultural exchange. Finally, this paper recommends that AI systems can drive advancements in MT and other AI-related fields of prior knowledge to produce accuracy and more closely with native speakers by using schema theory concepts.

Inferential knowledge and epistemic dimensions

Abstract Knowledge representation is one way to exploit expertise in a given domain by logical means. But, what kind of knowledge does one acquire from an inference (or inference on a query result over a knowledge base)? Such a question may appear awkward since the answer seems so obvious: from an inference, one simply acquires knowledge. This is undoubtedly the case when only one type of knowledge (for instance, expert knowledge) is involved in an inference. What if several types of knowledge are involved? What type of knowledge can one deduce from a plurality of knowledge types? I claim that reasoning with different knowledge concepts requires a fine-grained representation of knowledge in which every knowledge type finds a singular expression in order to avoid some epistemic equivocity associated with a coarse-grained representation of knowledge. In the first part of the paper, I revisit the Muddy Children Puzzle, which usually serves to illustrate common knowledge in dynamic epistemic logic. I try to show that this problem also shows some sort of epistemic equivocity between concepts of knowledge and, consequently, that the problem calls for some epistemological refinements concerning the representation of the types of knowledge at play in an inference. In the second part, I address this issue from a semantic point of view, and I develop a fragment of epistemic logic capable of providing a solution to the problem of epistemic equivocity.

Developing a Logistics Ontology for Natural Language Processing

The business area requires the presence of Supply Chain services, to respond to the constantly evolving requirements. Supply chains are dynamic networks that include a continuous flow of processes, starting with the supplier and ending with the customer. For effective supply chain management, a conceptual understanding of the knowledge underlying this field is of utmost importance. On the other hand, ontologies are considered one of the most appropriate ways of representing knowledge and valuable tools for decision-making situations. The purpose of this paper is to present an ontology created to identify the common concepts of supply chain management systems, with the ultimate goal of supporting and promoting Natural Language Processing technologies through the representation of terminological knowledge for this specific field.

Template-Based Contrastive Distillation Pretraining for Math Word Problem Solving.

Since math word problem (MWP) solving aims to transform natural language problem description into executable solution equations, an MWP solver needs to not only comprehend the real-world narrative described in the problem text but also identify the relationships among the quantifiers and variables implied in the problem and maps them into a reasonable solution equation logic. Recently, although deep learning models have made great progress in MWPs, they ignore the grounding equation logic implied by the problem text. Besides, as we all know, pretrained language models (PLM) have a wealth of knowledge and high-quality semantic representations, which may help solve MWPs, but they have not been explored in the MWP-solving task. To harvest the equation logic and real-world knowledge, we propose a template-based contrastive distillation pretraining (TCDP) approach based on a PLM-based encoder to incorporate mathematical logic knowledge by multiview contrastive learning while retaining rich real-world knowledge and high-quality semantic representation via knowledge distillation. We named the pretrained PLM-based encoder by our approach as MathEncoder. Specifically, the mathematical logic is first summarized by clustering the symbolic solution templates among MWPs and then injected into the deployed PLM-based encoder by conducting supervised contrastive learning based on the symbolic solution templates, which can represent the underlying solving logic in the problems. Meanwhile, the rich knowledge and high-quality semantic representation are retained by distilling them from a well-trained PLM-based teacher encoder into our MathEncoder. To validate the effectiveness of our pretrained MathEncoder, we construct a new solver named MathSolver by replacing the GRU-based encoder with our pretrained MathEncoder in GTS, which is a state-of-the-art MWP solver. The experimental results demonstrate that our method can carry a solver's understanding ability of MWPs to a new stage by outperforming existing state-of-the-art methods on two widely adopted benchmarks Math23K and CM17K. Code will be available at https://github.com/QinJinghui/tcdp.

Interval Methods in Knowledge Representation

Interval Methods in Knowledge Representation

Modeling and supporting adaptive Complex Data-Intensive Web Systems via XML and the O-O paradigm: The OO-XAHM model

The data model is a critical component of an Adaptive Web System (AWS). The major goals of such a data model are describing the application domain of the AWS and capturing data about the user in order to support the “adaptation effect”. There have been many proposals for data models, principally based on knowledge representation, machine learning, logic and reasoning, and, recently, ontologies. These models are focused on the implementation of the core layer of AWS, that is realizing the adaptation of contents and presentations of the system, but sometimes they are poor with respect to the application domain design. In this paper, we present an extension of the state-of-the-art XML Adaptive Hypermedia Model (XAHM), Object-Oriented XAHM (OO-XAHM) that supports the application domain modeling using an object-oriented approach. We also provide the formal definition of the model, its description via Unified Modeling Language (UML), and its implementation using XML Schema. Finally, we provide a complete case study that focuses the attention on the well-known Italian archaeological site Pompeii.

Three-way concept lattice based on Boolean formal context

The theory of formal concept analysis(FCA) is an important mathematical method for knowledge representation and knowledge discovery. The Boolean formal context is proposed by introducing Boolean matrices and logical operations into FCA. Based on the concept lattice of the Boolean formal context and the column vector(row-vector)-oriented concept lattice of the Boolean formal context, this paper proposes the column vector(row vector)-induced three-way concept lattice of the Boolean formal context and the column vector(row vector)-induced three-way column vector(row vector)-oriented concept lattice of the Boolean formal context, and proves their rationality. Then, the isomorphism between the three-way concept lattice of the Boolean formal context and the three-way concept lattice of the general formal context is proved.