Augmented Transition Networks Research Articles

Semantic Fusion for Natural Multimodal Interfaces using Concurrent Augmented Transition Networks

Semantic fusion is a central requirement of many multimodal interfaces. Procedural methods like finite-state transducers and augmented transition networks have proven to be beneficial to implement semantic fusion. They are compliant with rapid development cycles that are common for the development of user interfaces, in contrast to machine-learning approaches that require time-costly training and optimization. We identify seven fundamental requirements for the implementation of semantic fusion: Action derivation, continuous feedback, context-sensitivity, temporal relation support, access to the interaction context, as well as the support of chronologically unsorted and probabilistic input. A subsequent analysis reveals, however, that there is currently no solution for fulfilling the latter two requirements. As the main contribution of this article, we thus present the Concurrent Cursor concept to compensate these shortcomings. In addition, we showcase a reference implementation, the Concurrent Augmented Transition Network (cATN), that validates the concept’s feasibility in a series of proof of concept demonstrations as well as through a comparative benchmark. The cATN fulfills all identified requirements and fills the lack amongst previous solutions. It supports the rapid prototyping of multimodal interfaces by means of five concrete traits: Its declarative nature, the recursiveness of the underlying transition network, the network abstraction constructs of its description language, the utilized semantic queries, and an abstraction layer for lexical information. Our reference implementation was and is used in various student projects, theses, as well as master-level courses. It is openly available and showcases that non-experts can effectively implement multimodal interfaces, even for non-trivial applications in mixed and virtual reality.

A Rule Based Cognitive System of Emotion Recognition

AbstractEmotions are an important aspect of human nature and behaviour that helps a person communicate and interact with others in a social environment. The present work is such an attempt towards machine intelligence, to recognize the cause-and-effect of the emotional states of humans. In the present times, the intelligence quotient (IQ) measured earlier is replaced by the emotional intelligence (EI)- a factor that helps assess the performance and efficiency of humans in various arenas like home, office, sports, clubs etc.A Questionnaire has been designed, to be taken up by subjects whose emotional status, need to be understood. The set of answers received would fire the appropriate rule and result in a specific emotion. These set of rules, once designed are static. The concept of embedded emotion and hence of a rule in another rule (inheritance) is used. A set of 13 rules have been discovered depicting the causes of specific emotions and their inclusion in other emotions, thus enabling the design of augmented transition networks (ATN). Java Expert System Shell (JESS) has been used to design this rule-based emotion recognition system and the results are presented in the corresponding results section.

Bug Diagnosis By String Matching

We have developed an entirely new template-automaton-based knowledge database system for an interactive intelligent language tutoring system (ILTS) for Japanese-English translation whereby model translations as well as a taxonomy of bugs extracted from ill formed translations typical of nonnative learners are collected. Unlike conventional rule-based systems whose complicated solution search procedure and labor-intensive processing have led to so-called knowledge engineer bottlenecks of the expert systems, the new dynamic programming-based heaviest common sequence (HCS) matching algorithm is both efficient and robust in which error diagnosis is implemented by selecting, from among many candidates' paths in the system template, a path having an HCS of a highest similarity with a student's free-format translation input. This best matched path to the given ill formed sentence is used to provide contingent feedback messages. We have laid down a theoretical framework for the global HCS matching algorithm which is applied to the dual form of acyclic weighted digraphs by topologically sorting the template automaton structured according to augmented transition networks. An extensive evaluation test of the diagnostic engine has ensured the validity, efficiency, and robustness of the algorithm in providing error-contingent feedback to a wide spectrum of learners even with different educational backgrounds.

Pick‐and‐place application development using voice and visual commands

PurposeThe purpose of this paper is to design an interactive industrial robotic system which can be used to assist a “layperson” in re‐casting a generic pick‐and‐place application. A user can program a pick‐and‐place application simply by pointing to objects in the work area and speaking simple and intuitive natural language commands.Design/methodology/approachThe system was implemented in C# using the EMGU wrapper classes for OpenCV as well as the MS Speech Recognition API. The target language to be recognized was modelled using traditional augmented transition networks which were implemented as XML Grammars. The authors developed an original finger‐pointing algorithm using a unique combination of standard morphological and image processing techniques. Recognized voice commands trigger the vision component to capture what a user is pointing at. If the specified action requires robot movement, the required information is sent to the robot control component of the system, which then transmits the commands to the robot controller for execution.FindingsThe voice portion of the system was tested on the factory floor in a “typical” manufacturing environment, which was right at the maximum allowable average decibel level specified by OSHA. The findings show that a modern/standard MS Speech API voice recognition system can achieve a 100 per cent accuracy of simple commands; although at the noisy levels of 89 decibels on average, every one out of six commands had to be repeated. The vision component was test of 72 test subjects who had no prior knowledge of this work. The system accurately recognized what the test subjects were pointing at 95 per cent of the time within five seconds of hand readjusting.Research limitations/implicationsThe vision component suffers from the “typical” problems: very shiny surfaces can cause problems; very poor contrast between the pointing hand and the background; and occlusions. Currently the system can only handle a limited amount of depth recovery using a spring mounted gripper. A second camera (future work) needs to be incorporated in order to handle large depth variations in the work area.Practical implicationsThis system could have a huge impact on how factory floor workers interact with robotic equipment.Originality/valueThe testing of the voice system on a factory floor, although simple, is very important. It proves the viability of this component of the system and debunks arguments that factories are simply too noisy for current voice technology. The unique finger‐pointing algorithm developed by the authors is also an important contribution to the field. In particular, the manner in which the pointing vector was constructed. Furthermore, very few papers report results of non‐experts using their pointing algorithms. The paper reports concrete results that show the system is intuitive and user friendly to “laypersons”.

Towards a Syntactic Structural Analysis and an Augmented Transition Explanation: A Comparative Study of the Globally Ambiguous Sentences and Garden Path Sentences

Globally ambiguous sentences can be understood in more than one way and any decoding involved can be understandable. Garden Path (GP) sentences possess a special structural feature with a semantic trigger, which calls for a much heavier cognition burden since only one decoding is reasonable. The present paper explores the data structures of globally ambiguous sentences and GP ones. The augmented transition networks (ATNs) are used to deepen the analyses and comparison of globally ambiguous sentences and GP ones. We come to the conclusion that GP sentences, syntactically and cognitively different from globally ambiguous ones, have more complex structures.

Adaptable dialog architecture and runtime engine (AdaRTE): A framework for rapid prototyping of health dialog systems

Spoken dialog systems have been increasingly employed to provide ubiquitous access via telephone to information and services for the non-Internet-connected public. They have been successfully applied in the health care context; however, speech technology requires a considerable development investment. The advent of VoiceXML reduced the proliferation of incompatible dialog formalisms, at the expense of adding even more complexity. This paper introduces a novel architecture for dialogue representation and interpretation, AdaRTE, which allows developers to lay out dialog interactions through a high-level formalism, offering both declarative and procedural features. AdaRTE's aim is to provide a ground for deploying complex and adaptable dialogs whilst allowing experimentation and incremental adoption of innovative speech technologies. It enhances augmented transition networks with dynamic behavior, and drives multiple back-end realizers, including VoiceXML. It has been especially targeted to the health care context, because of the great scale and the need for reducing the barrier to a widespread adoption of dialog systems.

Brain-computer music interface for composition and performance

This paper introduces a new brain-computer interface (BCI) system that uses electroencephalogram (EEG) information to steer generative rules to compose and perform music. It starts by noting the various attempts at the design of BCI systems, including systems for music. Then it presents a short technical introduction to EEG sensing and analysis. Next, it introduces the generative music component of the system, which employs our own adapted version of a machine-learning technique based on ATNs (Augmented Transition Networks) for the computer-replication of musical styles. The system constantly monitors the EEG of the subject and activates generative rules associated with the activity of different frequency bands of the spectrum of the EEG signal. The system also measures the complexity of the EEG signal to modulate the tempo (beat) and dynamics (loudness) of the performance. Subjects can be trained to select between different classes of generative rules to produce original pieces of music.

Modelling the logical structure of books and journals using augmented transition network grammars

This paper presents a grammar for books and journals using augmented transition networks in automated document analysis. The approach takes the structure of layout elements in books and journals to be part of a semiotic system, which therefore can be described using methods developed for the description of other semiotic systems, e.g. languages. It differs from previous research in the domain of document analysis and understanding as it deals in an exhaustive way with rather generic classes of multi‐page printed objects, i.e. books (monographs) and journals. To achieve this aim, abstract relations instead of document specific formatting rules are taken into account.

Formalisms for grammatical knowledge representation

Computational linguists require descriptively powerful, computationally effective formalisms for representing grammatical information or knowledge. A wide variety of formalisms have been employed in natural language processing systems over the past several decades, including simple phrase structure grammars, augmented transition networks, logic grammars and unification-based grammar formalisms. Until fairly recently however, comparatively little attention has been given to the issues which underly good grammar formalism design in computational linguistics.

Computerized interpretation of electromyographic data

A computer program was developed to plan, interpret and report the analysis of electromyographic data. The computer model formalized electrodiagnostic decisions through the use of a computational method known as augmented transition networks (ATN). The diagnoses of entrapment neuropathies, plexopathies, and radiculopathies were incorporated into the computer model. The system analyzed a difficult set of electromyographic (EMG) cases and the computer and physician reached the same conclusion 78% of the time. The system could explain the reason it reached its diagnostic conclusions and report to the physician all the relevant pathophysiologic derangements. The system could also offer advice on how one could strengthen a tentative diagnostic conclusion. While this system is limited in its diagnostic coverage of EMG problems, it may be a useful assistant to the electromyographer by focusing attention on significant pathophysiologic derangements detected on EMG examination.

Co-operation of syntax and semantics in flexive languages

Abstract The idiosyncrasies of the fiexic Slavonic languages, such as the rich case system and complex morphology, entail a free word-order and require a specialized approach when parsed. We claim complex morphology is not the only disadvantage of the language. Augmented transition networks for Slovak are introduced, and their adequacy for Slovak and comparison to English networks are discussed. Further, we introduce briefly our concept of semantic analysis, based on a semantic case system, verb and noun valency frames. The phenomenon of nominalization is investigated. The central theme of this paper is the close communication of syntax and semantics. The sequentially working modules shift misinterpretations and ambiguities from one to another. This leads to combinatorial explosions and the loss of effectiveness. The communication between the syntactic and semantic modules enables the exclusion of some syntactically correct, but semantically nonsensical interpretations in early phases of sentence processi...

Application of expert systems and pattern recognition methodologies to facilities layout planning

Abstract This paper deals with two basic concepts of artificial intelligence (AI), from a facilities layout problem domain perspective. In this work, the facilities layout problem is treated as a multi-objective situation. From conventional multi-objective perspective, the philosophy underlying this work is not a different one. However, the qualitative constraints are handled via a symbolic manipulation structure. The two conceptualizations are: (a) an expert system and (b) a pattern recognition system. In the expert system, the heuristics used are based on the augmented transition networks of natural language processing. In the pattern recognition system, the use of productions rules to capture the expert knowledge is illustrated. For both the systems example problems are given.

YANLI: a powerful natural language front-end tool

An important issue in achieving acceptance of computer systems used by the nonprogramming community is the ability to communicate with these systems in natural language. Often, a great deal of time in the design of any such system is devoted to the natural language front end. An obvious way to simplify this task is to provide a portable natural language front-end tool or facility that is sophisticated enough to allow for a reasonable variety of input; allows modification; and, yet, is easy to use. This paper describes such a tool that is based on augmented transition networks (ATNs). It allows for user input to be in sentence or nonsentence form or both, provides a detailed parse tree that the user can access, and also provides the facility to generate responses and save information. The system provides a set of ATNs or allows the user to construct ATNs using system utilities. The system is written in Franz Lisp and was developed on a DEC VAX 11/780 running the ULTRIX-32 operating system.

Morphologic Analysis of Compound Words

SummaryAlgorithms for morphologic analysis of compound words and related dictionary structures are described’ on the basis of different word models. The algorithms are formulated in Backus-Naur form as well as Augmented Transition Networks. The results of applying the algorithms to the 56,509 different words of the “Systematisierte Nomenklatur der Medizin” (SNOMED) are presented.

Studies in formal semantics: Intensionality, temporality, negation Edited by Franz Guenthner and Christian Rohrer (review)

REVIEWS455 attacks so substantial a fragment of English. The sample grammar in Appendix D is 26 pages long, and covers a great deal of ground. As a theoretical proposal regarding the psychological mechanisms underlying human syntactic recognition , it is not an obvious success and is also (by its own admission) premised on some highly controversial assumptions (namely the alleged grammatical universals of REST).8 My most negative opinion, however, concerns a nonsubstantive matter, namely that the book is outrageously overpriced considering the poor quality of its format. REFERENCES Kaplan, Ronald M. 1972. Augmented transition networks as psychological models of sentence comprehension. Artificial Intelligence 3.77-100. Miller, George A., and Philip N. Johnson-Laird. 1977. Language and perception. Cambridge, MA: Harvard University Press. Ross, John R. 1967. Constraints on variables in syntax. MIT dissertation. Wanner, Eric, and Michael Maratsos. 1979. An ATN approach to comprehension. Linguistic theory and psychological reality, ed. by Morris Halle et al., 119-61. Cambridge, MA: MIT Press. Woods, William. 1970. Transition network grammars for natural language analysis. Comm. Assoc, for Comp. Mach. 13.591-606. [Received 15 July 1981.] 8 One of M's major concerns is to show that some of Chomsky's proposed constraints follow directly from the determinism hypothesis. Assuming that they do follow, and that they are valid, the result would nonetheless be cold comfort to an orthodox Chomskyan, to whom it is an article of faith that the universals in question represent innate predispositions that cannot be explained in terms of such 'functional' or 'performance-related' considerations as limitations on memory. Studies in formal semantics: Intensionality, temporality, negation. Edited by Franz Guenthner and Christian Rohrer. (North-Holland linguistic series, 35.) Amsterdam: North-Holland, 1978. Pp. viii, 265. / 62.50. Reviewed by Leonard M. Faltz, Arizona State University The formal study of natural-language semantics has recently occupied an active and growing group of scholars. G&R's collection represents work in this field carried out in the mid-to-late 70's at Stuttgart and Bonn. Following an introduction (1-10) in which the study of formal semantics is outlined and placed in perspective with regard to the current state of logic and of linguistics, eight papers are presented in three groups: Intensionality (three papers), Temporality (four papers), and Negation (one paper). Readers working in formal semantics will want to know about the sorts of proposals found here. I personally was particularly interested in the articles of the 'Temporality' section, hoping that some insight into the logic of aspect would be obtained by the careful construction of formal systems. As it turned out, after reading the articles, I feel that we have yet to get to the bottom of the matter—that the essential features which control the logic of aspect have still eluded the investigators. There may be inherent limitations to the sort of approach taken here—although deeper investigations may prove this suspicion wrong. 456LANGUAGE, VOLUME 58, NUMBER 2 (1982) Fabrizio Mondadori, 'Interpreting modal semantics' (13-40), is primarily a review of the justifications for representing intensions as functions from possible worlds to extensions, together with a discussion of the applicability of this construction to proper names and natural-kind terms. He emphasizes the fact that modal semantics is a theory of truth conditions, not of language use— and that therefore epistemological considerations should not play a role in justifying the way constructions are set up to represent modal semantics. Franz Guenthner, 'Systems of intensional logic and the semantics of natural languages' (41-74), sketches the four formal semantic systems of Kripke, Âqvist, Bressan, and Montague, and compares the ways that these systems handle variables and constants (e.g., are variables assigned to individuals relative to worlds, or independently of worlds?), quantification, predication (especially intensional predicates), and indexical expressions (such as time-indexed expressions). Aldo Bressan, 'The interpreted modal calculus MC and some of its applications' (75-1 18), tells how the language MC was developed by Bressan to formalize the logic of physical sciences (especially classical particle physics). The article begins with a terse but complete definition of the syntax and model-theoretic semantics of this language. Bressan then shows how a number of modally interesting sentences (e.g. The...

Definite clause grammars for language analysis—A survey of the formalism and a comparison with augmented transition networks

A ciear and powerful formalism for describing languages, both natural and artificial, follows from a method for expressing grammars in logic due to Colmerauer and Kowalski. This formalism, which is a natural extension of context-free grammars, we call “definite clause grammars” (DCGs). A DCG provides not only a description of a language, but also an effective means for analysing strings of that language, since the DCG, as it stands, is an executable program of the programming language Prolog. Using a standard Prolog compiler, the DCG can be compiled into efficient code, making it feasible to implement practical language analysers directly as DCGs. This paper compares DCGs with the successful and widely used augmented transition network (ATN) formalism, and indicates how ATNs can be translated into DCGs. It is argued that DCGs can be at least as efficient as ATNs, whilst the DCG formalism is clearer, more concise and in practice more powerful.

An English language question answering system for a large relational database

By typing requests in English, casual users will be able to obtain explicit answers from a large relational database of aircraft flight and maintenance data using a system called PLANES. The design and implementation of this system is described and illustrated with detailed examples of the operation of system components and examples of overall system operation. The language processing portion of the system uses a number of augmented transition networks, each of which matches phrases with a specific meaning, along with context registers (history keepers) and concept case frames; these are used for judging meaningfulness of questions, generating dialogue for clarifying partially understood questions, and resolving ellipsis and pronoun reference problems. Other system components construct a formal query for the relational database, and optimize the order of searching relations. Methods are discussed for handling vague or complex questions and for providing browsing ability. Also included are discussions of important issues in programming natural language systems for limited domains, and the relationship of this system to others.

Augmented Transition Networks as a design tool for personalized database systems

This paper illustrates the use of Augmented Transition Networks (ATNs) as a design tool for constructing document retrieval systems for those personalized applications which are too small or specialized to attract a commercial vendor. ATNs, which are explained in the context of this application, are used not only to improve the human/computer interface with the retrieval system but also to conceptually organize its structure.

Induction of Augmented Transition Networks*

LAS is a program that acquires augmented transition network (ATN) grammars. It requires as data sentences of the language and semantic network representatives of their meaning. In acquiring the ATN grammars, it induces the word classes of the language, the rules of formation for sentences, and the rules mapping sentences onto meaning. The induced ATN grammar can be used both for sentence generation and sentence comprehension. Critical to the performance of the program are assumptions that it makes about the relation between sentence structure and surface structure (the graph deformation condition), about when word classes may be formed and when ATN networks may be merged, and about the structure of noun phrases. These assumptions seem to be good heuristics which are largely true for natural languages although they would not be true for many nonnatural languages. Provided these assumptions are satisfied LAS seems capable of learning any context‐free language.

Induction of augmented transition networks

LAS is a program that acquires augmented transition network (ATN) grammars. It requires as data sentences of the language and semantic network representatives of their meaning. In acquiring the ATN grammars, it induces the word classes of the language, the rules of formation for sentences, and the rules mapping sentences onto meaning. The induced ATN grammar can be used both for sentence generation and sentence comprehension. Critical to the performance of the program are assumptions that it makes about the relation between sentence structure and surface structure (the graph deformation condition), about when word classes may be formed and when ATN networks may be merged, and about the structure of noun phrases. These assumptions seem to be good heuristics which are largely true for natural languages although they would not be true for many nonnatural languages. Provided these assumptions are satisfied LAS seems capable of learning any context-free language.