Narrative Knowledge Representation Language Research Articles

Cultural heritage digital twin: modeling and representing the visual narrative in Leonardo Da Vinci’s Mona Lisa

In this paper, Artificial Intelligence/Knowledge Representation methods are used for the digital modeling of cultural heritage elements. Accordingly, the new concept of digital cultural heritage twin is presented as composed of a physical component and an immaterial component of the cultural entity. The former concerns the physical aspects, i.e. style, name of the artist, execution time, dimension, etc. The latter represents the emotional and intangible aspects transmitted by the entity, i.e. emotions, thoughts, opinions. In order to digitally model the physical and immaterial components of the twin, the Narrative Knowledge Representation Language has been formally introduced and described. It is particularly suitable for representing the immaterial aspects of the cultural entity, as it is capable of modeling in a simple but rigorous and efficient way complex situations and events, behaviours, attitudes, etc. As an experiment, NKRL has been adopted for representing some of the most relevant intangible items of the visual narrative underlying the hidden painting that lies beneath the Mona Lisa (La Gioconda) image painted by Leonardo Da Vinci on the same poplar panel. Real-time application of the resulting knowledge base opens up novel possibilities for the development of virtual objects, chatbots and expert systems, as well as the definition of semantic search platforms related to cultural heritage.

Spatio-Temporal Convolutional Networks and N-Ary Ontologies for Human Activity-Aware Robotic System

Endowing a companion robot with cognitive abilities to recognize human daily activities, in particular from body skeletons information, is a significant challenge, which needs complex and novel approaches. Recently, most of the proposed approaches exploit the hand-crafted features or the predefined traversal rules techniques to recognize human daily activities from skeleton information, which often lead to the deficit of robustness and generalization. In this work, a novel hybrid framework for human activity-aware robotic system is proposed. In the low-level, a novel Spatio-Temporal Joint based Convolutional Neural Network (STJ-CNN) is proposed to recognize human daily activities in the ambient environments. In the high-level, novel representation and inference services based on Narrative Knowledge Representation Language (NKRL) are proposed to represent and combine the detected human activities with the ambient events, and to infer the semantic context of the detected activity. Empirical experiments on real-world datasets have been conducted, besides an online demonstration created to validate the proposed approach. The final results show that the proposed approach outperforms the baseline models with a significant improvement up to 24% in terms of $F$-score on DAHLIA dataset.

Conceptual Encoding and Advanced Management of Leonardo da Vinci’s Mona Lisa: Preliminary Results

This paper describes a preliminary experiment concerning the use of advanced Artificial Intelligence/Knowledge Representation techniques to improve the present formalization/digitization procedures of Cultural Heritage assets—with reference, in particular, to all types of Cultural Heritage “iconographic” entities. In this context, in agreement with the recent proposal to characterize the digital description of Cultural Heritage items making use of the notion of “Cultural Heritage Digital Twin”, we are mainly concerned with the possibility to consider not only the external, “physical”, aspects of these iconographic items but also the “message” they convey in a more or less explicit way. For our experiment, some aspects of the Mona Lisa painting by Leonardo da Vinci have been formalized, along with their context, making use of NKRL, the “Narrative Knowledge Representation Language”. NKRL is, in reality, both a Knowledge Representation language and a full Computer Science environment, used to represent/manage in an advanced way "narrative" (in the widest meaning of this word) information. The initial results of the experiment are described in the paper, along with some thoughts about their possible interest and developments.

Advanced Knowledge Representation and Reasoning Making Use of an Advanced N-ary Model

We discuss in this paper some aspects of NKRL, the Narrative Knowledge Representation Language. This is a high-level n-ary conceptual tool specially conceived for the representation and management of real world, dynamically characterized entities like situations, events and complex events, actions (e.g., in a robotics context) scripts/scenarios/narratives etc. After having pointed out some shortcomings of the standard ontological solutions for dealing with this sort of information, and having recalled some general characteristics of NKRL (like the addition of an "ontology of events" to the usual "ontology of objects"), we focus on the rules/inferential aspects proper to this language. We introduce, then, the general, formal model of "rule" used in an NKRL context and we show how this can be appropriately adapted to the setup of advanced types of inference operations based, e.g., on "analogical" and "causal" reasoning.

Modelling and Exploiting the Temporal Information Associated with Complex “Narrative” Documents

This paper supplies some details on the solutions adopted in a narrative knowledge representation language (NKRL) context for representing/managing the temporal information included within 'narrative' multi-media documents. They are based on the association of two notions, the 'category' and the 'perspective' of dating. The first allows us to describe the logical implications of the distribution on the time axis of the timestamps associated with an event. The second takes into account the different types of fuzziness with which these timestamps can be captured. From an operational point of view, the two notions have also an important inferential function since they represent the basic building blocks of an advanced indexing system for NKRL knowledge bases. This allows us to reduce to sets of simple comparison operations the computational procedures needed for selecting all the data characterised by temporal information that are congruent with the temporal criteria specified in a given query.

Functional and semantic roles in a high-level knowledge representation language

We describe in this paper a formalization of the notion of “role” that involves a clear separation between two very different sorts of roles. Semantic roles, like student or customer, are seen as (pre-defined) transitory properties that can be associated with (usually animate) entities. From a formal point of view, they can be represented as standard concepts to be placed into a specific branch of a particular ontology; they formalize the static and classificatory aspects of the notion of role. Functional roles must be used, instead, to model those pervasive and dynamic situations corresponding to events, activities, circumstances etc. that are characterized by spatio-temporal references; see, e.g., “John is now acting as a student”. They denote the specific function with respect to the global meaning of an event/situation/activity... that is performed by the entities involved in this event/situation... and formalize the dynamic and relational aspects of the notion of role. A functional role of the subject/agent/actor/protagonist... type is used to associate “John” with the notion of student or customer (semantic roles) during a specific time interval. Formally, functional roles are expressed as primitive symbols like subject, object, source, beneficiary. Semantic and functional roles interact smoothly when they are used to deal with challenging knowledge representation problems like the so-called “counting problem”, or when we need to set-up powerful inference rules whose atoms can directly denote complex situations. In this paper, the differentiation between semantic and functional roles will be illustrated from an narrative knowledge representation language (NKRL) point of view. NKRL is a high-level conceptual tool used for the computer-usable representation and management of the inner meaning of syntactically complex and semantically rich multimedia information. But, as we will see, the importance of this distinction goes well beyond its usefulness in a specific NKRL context. In particular, the use of functional roles is of paramount importance for the set-up of those evolved n-ary forms of knowledge representation that allow us to get rid from the limitations in expressiveness proper to the standard (binary) solutions.

A semantic approach for enhancing assistive services in ubiquitous robotics

The Ambient Intelligence (AmI) technologies have the potential to create intelligent environments with new generation of assistive services, enhanced with ubiquitous robots. These environments have the ability to be anticipatory, responsive and intelligent providers of assistive services anytime and anywhere. These services can assist frail persons effectively in their daily tasks. One of the main challenging research problems in assistive robotics is to endow ubiquitous robots with ability to pro-actively taking on some tasks to help humans in performing complex activities, by participating with them just as other humans do, in normal societies or organizations.In this paper, we propose a collective intelligence framework based on narrative reasoning and natural language processing. In the proposed approach, we propose a hybrid model that bridges together the Narrative Knowledge Representation Language (NKRL), from natural language processing field, and the HARMS (Humans, software Agents, Robots, Machines and Sensors) model, from multi-agent systems engineering field. This model is able to (i) drive the dialogues between humans, robots and smart devices, (ii) understand a complex situation, and (iii) trigger reactive actions, in the ubiquitous environment, according to given contexts.Two scenarios dedicated to the assistance of a frail person in a smart home equipped with a companion robot and smart objects are implemented and discussed for validation purposes of the proposed framework.

A structured and in-depth representation of the semantic content of elementary and complex events

'Events' are understood here, following the standard philosophical acceptation and in agreement with a Davidsonian and neo–Davidsonian approach, as the explicit, 'narrative' accounts of (spatio–temporal constrained) behaviours, states, actions and mutual interactions that concern animate/inanimate, human/non–human, abstract/concrete etc. well–identified 'entities'. Within this 'explicit' and 'symbolic' framework, we describe then how to isolate and represent in a computer–suitable form 'elementary events' inside a stream of narrative information, how to connect together these elementary events to give rise to 'complex events' through, e.g., 'causal' or 'conditional' relationships and how to make use of these formal structures for querying and inference operations. In this context, we will make use of NKRL (the Narrative Knowledge Representation Language), a language/environment expressly specified and implemented for dealing with narratives and temporal information, as formal support for our line of reasoning.

Sentiments analysis at conceptual level making use of the Narrative Knowledge Representation Language

This paper illustrates some of the knowledge representation structures and inference procedures proper to a high-level, fully implemented conceptual language, NKRL (Narrative Knowledge Representation Language). The aim is to show how these tools can be used to deal, in a sentiment analysis/opinion mining context, with some common types of human (and non-human) “behaviors”. These behaviors correspond, in particular, to the concrete, mutual relationships among human and non-human characters that can be expressed under the form of non-fictional and real-time “narratives” (i.e., as logically and temporally structured sequences of “elementary events”).

Conceptual and content-based annotation of (multimedia) documents

This paper focuses on the techniques used in an NKRL environment (NKRL = Narrative Knowledge Representation Language) to deal with a general problem affecting the so-called "semantic/conceptual ann...

Advanced computational reasoning based on the NKRL conceptual model

After having recalled some well-known shortcomings linked with the Semantic Web approach to the creation of (application oriented) systems of “rules” – e.g., limited expressiveness, adoption of an Open World Assumption (OWA) paradigm, absence of variables in the original definition of OWL – this paper examines the technical solutions successfully used for implementing advanced reasoning systems according to the NKRL’s methodology. NKRL (Narrative Knowledge Representation Language) is a conceptual meta-model and a Computer Science environment expressly created to deal, in an ‘intelligent’ and complete way, with complex and content-rich non-fictional ‘narrative’ data sources. These last include corporate memory documents, news stories, normative and legal texts, medical records, surveillance videos, actuality photos for newspapers and magazines, etc. In this context, we will expound first the need for distinguishing between “plain/static” and “structured/dynamic” knowledge and for introducing appropriate (and different) knowledge representation structures for these two types of knowledge. In a structured/dynamic context, we will then show how the introduction of “functional roles” – associated with the possibility of making use of n-ary structures – allows us to build up highly ‘expressive’ rules whose “atoms” can directly represent complex situations, actions, etc. without being restricted to the use of binary clauses. In an NKRL context, “functional roles” are primitive symbols interpreted as “relations” – like “subject”, “object”, “source”, “beneficiary”, etc. – that link a semantic predicate with its arguments within an n-ary conceptual formula. Functional roles contrast then with the “semantic roles” that are equated to ordinary concepts like “student”, to be inserted into the “non-sortal” (no direct instances) branch of a traditional ontology.

Knowledge representation and inference techniques to improve the management of gas and oil facilities

This paper describes an experimental work carried out in the framework of an important European project to create and make use of a wide-ranging knowledge base in the gas/oil domain. In the context of this work, “knowledge base” means a collection of formal statement relating, with a negligible loss of information, the inner content (the ‘meaning’) of “complex events” included in two different “storyboards”. These events – originally presented under the form of unstructured natural language information – concern some general activities proper to the management of gas/oil facilities, like recognizing and monitoring gas leakage alarms in a gas processing plant or triggering the different steps needed to activate a gas turbine. To express this sort of information and to set up the knowledge base, the NKRL (Narrative Knowledge Representation Language) formalism has been used. NKRL is a conceptual meta-model and Computer Science environment expressly created to deal, in an ‘intelligent’ and complete way, with complex and content-rich ‘narrative’ data sources. The final knowledge base has been firstly tested in depth using the standard NKRL querying and information retrieval tools. High-level inference procedures have then been used, both “transformation rules” – unsuccessful queries are ‘transformed’ to produce results that are ‘semantically similar’ to those searched for initially – and “hypothesis rules” – information in the knowledge base is automatically aggregated to supply a sort of ‘causal’ explanation of some retrieved events.

A structured metadata approach for dealing in an 'intelligent' way with complex 'narrative' information

In this paper, we evoke first the ubiquity and the importance of the so-called |non-fictional narrative| information. This consists of streams of logically and chronologically related elementary events that describe the activities or experiences of some (not necessarily human) characters. We show then that the usual knowledge representation and |ontological| techniques have difficulties in finding complete solutions for representing and using this type of information. After that, we supply some details about Narrative Knowledge Representation Language (NKRL); NKRL is both a representation language – based on the use of |structured metadata| – and a querying/inferencing environment, which has been expressly created for an |intelligent| exploitation of (mainly non-fictional) narratives. The paper will be illustrated with some examples concerning recent concrete applications of NKRL.

A Conceptual Methodology for Dealing with Terrorism "Narratives"

This paper concerns the use of in-depth analytical/conceptual techniques pertaining to the Artificial Intelligence domain to deal with narrative information (or “narratives”) in the terrorism- and crime-related areas. More precisely, the authors supply details about NKRL (Narrative Knowledge Representation Language), a representation and querying/inferencing environment especially created for an advanced exploitation of all types of narrative information. This description will be integrated with concrete examples that illustrate the use of NKRL tools in two recent ‘defence’ applications, the first dealing with a corpus of “Southern Philippines terrorism” news stories used in an R&D European project, the second, carried out in collaboration with the French “Délégation Générale pour l’Armement” (DGA, Central Bureau for Armament), which handles news stories about Afghanistan’s war.

Ontologies and reasoning techniques for (legal) intelligent information retrieval systems

An application of Narrative Knowledge Representation Language (NKRL) techniques on (declassified) 'terrorism in Southern Philippines' documents has been carried out in the context of the IST Parmenides project. This paper describes some aspects of this work: it is our belief, in fact, that the Knowledge Representation techniques and the Intelligent Information Retrieval tools used in this experiment can be of some interest also in an 'Ontological Modelling of Legal Events and Legal Reasoning' context.

A conceptual annotation approach to indexing in a Web-based information system

WWW objects are characterized by data with heterogeneous characteristics such as text, images, both still and moving, graphs and sound. One of the most important problems in this context is to represent and query the content of WWW objects. All the specialists have agreed that the possibility of adding to multimedia WWW objects some sort of conceptual annotations describing their information content would greatly contribute to solve the problem of their intelligent indexing and retrieval. In this paper, we propose to associate with the WWW objects not the final conceptual annotation, but a simple Natural Language (NL) caption, in the form of short texts representing a general, neutral description of their informational content. The NL caption is then converted into a conceptual annotation in NKRL (Narrative Knowledge Representation Language) l35,36r, making use of a semi-automatic translation system like those we have implemented in the context of recent European projects.

NKRL, a knowledge representation tool for encoding the ‘meaning’ of complex narrative texts

In this paper, we describe NKRL (Narrative Knowledge Representation Language), a language designed for representing, in a standardized way, the semantic content (the ‘meaning’) of complex narrative texts. After having introduced informally the four ‘components’ (specialized sub-languages) of NKRL, we will describe (some of) the data structures proper to each of them, trying to show that the NKRL coding retains the main informational elements of the original narrative expressions. We will then focus on an important subset of NKRL, the so-called AECS sub-language, showing in particular that the operators of this sub-language can be used to represent some sorts of ‘plural’ expressions.