Definition Of Visual Languages Research Articles

Multimodal interaction systems: information and time features

Multimodal interaction systems combine visual information (involving images, text, sketches and so on) with voice, gestures and other modalities to provide flexible and powerful dialogue approaches, enabling users to choose one or more of the multiple interaction modalities. They break down the barriers in adopting mobile devices for value-added services and the use of integrated multiple input modes enables users to benefit from the natural approach used in human communication. This paper deals with the main features of multimodal interaction and systems, starting from the definition of visual language given in Bottoni et al. (1995) and extending it to multimodality. Modal/multimodal message, interpretation and materialisation functions and multimodal sentence are defined. This paper introduces and formally defines the different classes of cooperation between different modes, introducing the time relationships among the involved modalities and the relationships between chunks of information connected with these modalities.

Syntax Definition with Graphs

Syntax definition of visual languages is required for a precise definition of a visual language; it is a requirement for the automatic creation of tools that process diagrams of the language. This paper describes a lecture together with an exercise which have been given by the author at the SegraVis Summer-School at Schloss Dagstuhl, Germany, in May 2004. The paper presents in a tutorial like way the DiaGen approach of defining visual language syntax with hypergraphs and hypergraph grammars.

Barcelona, Spain, October 7-8, 2002: Graph-Based Tools (GraBaTs 2002)

Barcelona, Spain, October 7-8, 2002: Graph-Based Tools (GraBaTs 2002)

AGG and GENGED: Graph Transformation-Based Specification and Analysis Techniques for Visual Languages

The GENGED concepts and environment allow for the visual definition of visual languages (VLs) and to generate VL-specific visual environments for editing and simulation. The editing features capture either syntax-directed editing and/or free-hand editing. In the latter case, a user-defined diagram has to be analyzed in order to check the correctness of the diagram. In addition, behavioral diagrams can be simulated, i.e. the behavior of situations specified by diagrams can be validated. The specification and analysis of VLs by GENGED is based on algebraic graph transformation concepts realized by the AGG system. In this article we give a brief survey on AGG and GENGED.

A visual environment for visual languages

The visual environment G ENGE D supports the visual definition of visual languages (VLs). Each VL is defined by an alphabet and a grammar. From a specific VL-definition, a VL-specification is generated which is the input of a graphical editor allowing for syntax-directed editing of diagrams over the specified VL. G ENGE D as well as each VL is based on the well-defined concepts of algebraic graph transformation and graphical constraint solving. The underlying formalism is hidden from the user, but it is essential for a formal presentation and manipulation of graphical structures. In this contribution, the G ENGE D concepts and environment are briefly proposed and illustrated by the definition of a simple kind of the well-known statechart language.

Computing with graphs and graph transformations

Many software applications require the construction and manipulation of graphs. In standard programming languages, this is accomplished using low-level mechanisms such as pointer manipulation or array indexing. In contrast, graph productions are a convenient high-level visual notation for coding graph modifications. A graph production replaces one subgraph by another subgraph. Graph productions can define a graph grammar and graph language, or can directly transform an input graph into an output graph. Graph transformation has been applied in many areas, including the definition of visual languages and their tools, the construction of software development environments, the definition of constraint programming algorithms, the modeling of distributed systems, and the construction of neural networks. One application is presented in detail: the interpretation of mathematical notation in scanned document images. The graph models the set of mathematical symbols, and their spatial and logical relationships. This graph is transformed by productions written in the PROGRES language. Copyright © 1999 John Wiley & Sons, Ltd.

Defining visual languages for interactive computing

A novel definition of visual languages allows a uniform approach to satisfying the needs of visual reasoning faced in visual human-computer interaction. The way the machine associates a computational meaning with an image, and conversely, the way it generates an image on the screen from a computation are formally described. A definition of a visual sentence and of a visual language as a set of visual sentences is discussed. A hierarchy of visual languages is derived in relation to the requirements for intelligible, manageable and trustable interaction between humans and computers.

Visual languages their definition and applications in system development

This paper address the issue of the formal definition of visual (graphical) languages, and their applications to system specification and development. The main assertion is that such visual languages can be used for (formal) system specification. And that they would possess the benefits of formal specification languages, as well as the intuitive appeal and ease of use of graphical methods. This paper presents a conceptual and formal model for the definition of visual languages. This model is then used to define an aspect of language support environments, namely a language-directed editing facility.

Simulation of hepatological models: a study in visual interactive exploration of scientific problems

In many different fields of science and technology, visual expressions formed by diagrams, sketches, plots and even images are traditionally used to communicate not only data but also procedures. When these visual expressions are systematically used within a scientific community, bi-dimensional notations often develop which allow the construction of complex messages from sets of primitive icons. This paper discusses how these notations can be translated into visual languages and organized into an interactive environment designed to improve the user's ability to explore scientific problems. To facilitate this translation, the use of Conditional Attributed Rewriting Systems has been extended to visual language definition. The case of a visual language in the programming of a simulation of populations of hepatic cells is studied. A discussion is given of how such a visual language allows the construction of programs through the combination of graphical symbols which are familiar to the physician or which schematize shapes familiar to him in that they resemble structures the observes in real experiments. It is also shown how such a visual approach allows the user to focus on the solution of his problems, avoiding any request for unnecessary precision and most requests for house-keeping data during the interaction.