Action Syntax Research Articles

Sabre: A Narrative Planner Supporting Intention and Deep Theory of Mind

Sabre is a narrative planner—a centralized, omniscient decision maker that solves a multi-agent storytelling problem. The planner has an author goal it must achieve, but every action taken by an agent must make sense according to that agent's individual intentions and limited, possibly wrong beliefs. This paper describes the implementation of Sabre, which supports a rich action syntax and imposes no arbitrary limit on the depth of theory of mind. We present a search procedure for generating plans that achieve the author goals while ensuring all agent actions are explained, and we report the system's performance on several narrative planning benchmark problems.

The Ontogenesis of Action Syntax

Language and action share similar organizational principles. Both are thought to be hierarchical and recursive in nature. Here we address the relationship between language and action from developmental and neurophysiological perspectives. We discuss three major aspects: The extent of the analogy between language and action; the necessity to extend research on the yet largely neglected aspect of action syntax; the positive contribution of a developmental approach to this topic. We elaborate on the claim that adding an ontogenetic approach will help to obtain a comprehensive picture about both the interplay between language and action and its development, and to answer the question whether the underlying mechanisms of detecting syntactic violations of action sequences are similar to or different from the processing of language syntactic violations.

Syntactical regularities of action sequences in the infant brain: when structure matters.

Throughout life, actions and language are inherent to social interactions. A long-standing research question in cognitive neuroscience concerns the interrelation between verbal and non-verbal forms of social interactions, that is, language and action. Perceiving how actions are performed and why they are performed in a certain way is crucial for the observer to anticipate the actor's goal and to prepare an appropriate response. It is suggested that predicting upcoming events in a given action sequence can be compared to the way we process the language information flow. Goal-directed actions can be sequenced in small units, which are organized according to a hierarchical plan, resembling the hierarchical organization of language. Research on adults suggests that manipulating the action structure (i.e., action syntax) leads to analogous cortical signatures as a similar manipulation of a sentence structure (i.e., language syntax). Whereas in adults language and action knowledge are based on life-time experience, in infants both domains are still developing. The current study examined the neural processing of structural violations of observed goal-directed action sequences in infants at 6-7 months, using event-related potentials (ERPs). Results showed that a structural violation of the action sequence elicited bilateral frontal positivity effects. This suggests that infants capture structural regularities, and it adds a crucial element to the understanding of general syntactic regularities and their violation from an ontogenetic perspective.

Syntactic theory is also a metaphor.

Syntactic theory is also a metaphor.

Hierarchical processing in music, language, and action: Lashley revisited.

Sixty years ago, Karl Lashley suggested that complex action sequences, from simple motor acts to language and music, are a fundamental but neglected aspect of neural function. Lashley demonstrated the inadequacy of then-standard models of associative chaining, positing a more flexible and generalized “syntax of action” necessary to encompass key aspects of language and music. He suggested that hierarchy in language and music builds upon a more basic sequential action system, and provided several concrete hypotheses about the nature of this system. Here, we review a diverse set of modern data concerning musical, linguistic, and other action processing, finding them largely consistent with an updated neuroanatomical version of Lashley's hypotheses. In particular, the lateral premotor cortex, including Broca's area, plays important roles in hierarchical processing in language, music, and at least some action sequences. Although the precise computational function of the lateral prefrontal regions in action syntax remains debated, Lashley's notion—that this cortical region implements a working-memory buffer or stack scannable by posterior and subcortical brain regions—is consistent with considerable experimental data.

Joint Action Syntax in Japanese Martial Arts

Participation in interpersonal competitions, such as fencing or Japanese martial arts, requires players to make instantaneous decisions and execute appropriate motor behaviors in response to various situations. Such actions can be understood as complex phenomena emerging from simple principles. We examined the intentional switching dynamics associated with continuous movement during interpersonal competition in terms of their emergence from a simple syntax. Linear functions on return maps identified two attractors as well as the transitions between them. The effects of skill differences were evident in the second- and third-order state-transition diagrams for these two attractors. Our results suggest that abrupt switching between attractors is related to the diverse continuous movements resulting from quick responses to sudden changes in the environment. This abrupt-switching-quick-response behavior is characterized by a joint action syntax. The resulting hybrid dynamical system is composed of a higher module with discrete dynamics and a lower module with continuous dynamics. Our results suggest that intelligent human behavior and robust autonomy in real-life scenarios are based on this hybrid dynamical system, which connects interpersonal coordination and competition.

The minimalist grammar of action

Language and action have been found to share a common neural basis and in particular a common 'syntax', an analogous hierarchical and compositional organization. While language structure analysis has led to the formulation of different grammatical formalisms and associated discriminative or generative computational models, the structure of action is still elusive and so are the related computational models. However, structuring action has important implications on action learning and generalization, in both human cognition research and computation. In this study, we present a biologically inspired generative grammar of action, which employs the structure-building operations and principles of Chomsky's Minimalist Programme as a reference model. In this grammar, action terminals combine hierarchically into temporal sequences of actions of increasing complexity; the actions are bound with the involved tools and affected objects and are governed by certain goals. We show, how the tool role and the affected-object role of an entity within an action drives the derivation of the action syntax in this grammar and controls recursion, merge and move, the latter being mechanisms that manifest themselves not only in human language, but in human action too.

The syntax of human actions and interactions

Human motion is a natural phenomenon that involves several different aspects in the representational level. Among these aspects, we find the discovery of motor primitives used to build complex motion; the representation of complex actions in terms of these primitives; the generalization of movement concerning different parameters such as target location, speed, and resistance force; the temporal concatenation of motion in a sequence of actions that considers co-articulation; and the parallelization of movement in space that allows the performance of different actions at the same time (e.g., walk and wave). In order to model all important aspects of human motion, we seek a representation that considers these problems in a single framework. This way, we advocate that human motion may be represented as a language. Our Human Activity Language (HAL) consists of kinetology, morphology, and syntax. Kinetology, the phonology of human movement, involves the learning of motor primitives through segmentation and symbolization. Morphology concerns the representation of action words in terms of kinetemes and the discovery of a set of essential coordinated actuators for each action. Syntax is related to the construction of motion sentences using action words in sequence or in parallel. In this paper, we extend HAL syntax to consider human interactions between two subjects. We captured a praxicon, lexicon of human movement, with a number of human interactions such as shake hands, shove, and pass on. We empirically show that human interactions have a particular syntax based on the syntax of individual actions.

Reading Malory Aloud: Syntax, Gender, and Narrative Pace

This essay explores the contrast between the paratactic syntax of narrative action and the hypotactic syntax of speakers reflecting on situations.

Rett syndrome, EEG and the motor cortex as a model for better understanding of attention deficit hyperactivity disorder (ADHD).

EEG data in Rett syndrome (RS) are quite often characterized by spikes over the central region (especially in sleep). Furthermore, there has been a recent observation of a prominent and peculiar theta (4-6/s) rhythm over the central region and vicinity. In view of the clinical phenomena of RS, it is pointed out that clinical and electrical findings are based upon hyperexcitability of the motor cortex (or a state of motor cortex dyscontrol). States of dyscontrol (disinhibition) are caused by deficits in higher cortical functions. It is reasonable to assume a primary dysfunction for the frontal lobe which is also smaller in size than other cerebral regions. The behavioral (motor) disturbance in RS can be easily explained as the expression of disturbed prefrontal/premotor function with special effect on the "syntax of action". It is further theorized that such a state of fronto-motor cortex uncoupling may also exist in attention deficit hyperkinetic disorder (ADHD) even though there is hardly any structural CNS impairment in that condition. A presumed "lazy frontal lobe" is likely to react favorably to the stimulation provided by methylphenidate (a therapeutic effect which should not be viewed as paradoxical).

Coding of Serial Order by Neostriatal Neurons: A “Natural Action” Approach to Movement Sequence

The neostriatum controls behavioral sequencing, or action syntax, as well as simpler aspects of movement. Yet the precise nature of the neostriatums role in sequencing remains unclear. Here we used a "natural action" approach that combined electrophysiological and neuroethological techniques. We identified neostriatal neurons that code the serial order of natural movement sequences of rats. During grooming behavior, rats emit complex but highly predictable species-specific sequences of movements, termed "syntactic chains." Neuronal activity of 41% of cells in the dorsolateral and ventromedial neostriatum coded the sequential pattern of syntactic chains. Only 14% coded simple motor properties of grooming movements. Neurons fired preferentially during syntactic chains compared with similar grooming movements made in different sequential order or to behavioral resting. Sequential coding differed between the dorsolateral and ventromedial neostriatum. Neurons in the dorsolateral site increased firing by 116% during syntactic chains, compared with only a 30% increase by neurons in the ventromedial site, and dorsolateral neurons showed strongest coding of grooming syntax by several additional criteria. These data demonstrate that neostriatal neurons code abstract properties of serial order for natural movement and support the hypothesis that the dorsolateral neostriatum plays a special role in implementing action syntax.

Implementation of Action Sequences by a Neostriatal Site: A Lesion Mapping Study of Grooming Syntax

The neostriatum and its connections control the sequential organization of action ("action syntax") as well as simpler aspects of movement. This study focused on sequential organization of rodent grooming. Grooming syntax provides an opportunity to study how neural systems coordinate natural patterns of serial order. The most stereotyped of these grooming patterns, a "syntactic chain," has a particularly stereotyped order that recurs thousands of times more often than could occur by chance. The purpose of the present study was to identify the crucial site within the striatopallidal system where lesions disrupt the syntax or serial order of syntactic grooming chains without disrupting constituent movements. Small excitotoxin lesions were made using quinolinic acid at bilateral sites within the dorsolateral, dorsomedial, ventrolateral, or ventromedial neostriatum, or in the ventral pallidum or globus pallidus of rats. An objective technique for mapping functional lesions was used to quantify cell death and to map precisely those lesions that disrupted grooming syntax. Our results identified a single site within the anterior dorsolateral neostriatum, slightly more than a cubic millimeter in size (1.3 x 1.0 x 1.0 mm), as crucial to grooming syntax. Damage to this site did not disrupt the ability to emit grooming actions. By contrast, damage to sites in the ventral pallidum and globus pallidus impaired grooming actions but left the sequential organization of grooming syntax intact. Neural circuits within this crucial "action syntax site" seem to implement sequential patterns of behavior as a specific function.

A theory of bisimulation for the π-calculus

We study a new formulation of bisimulation for the π-calculus [MPW92], which we have called open bisimulation ( ∼ ). In contrast with the previously known bisimilarity equivalences, ∼ is preserved by all π-calculus operators, including input prefix. The differences among all these equivalences already appear in the sublanguage without name restrictions: Here the definition of ∼ can be factorised into a “standard” part which, modulo the different syntax of actions, is the CCS bisimulation, and a part specific to the π-calculus, which requires name instantiation. Attractive features of ∼ are: A simple axiomatisation (of the finite terms), with a completeness proof which leads to the construction of minimal canonical representatives for the equivalence classes of ∼; an “efficient” characterisation, based on a modified transition system. This characterisation seems promising for the development of automated-verification tools and also shows the call-by-need flavour of ∼. Although in the paper we stick to the π-calculus, the issues developed may be relevant to value-passing calculi in general.

Ontogeny of Action Syntax in Altricial and Precocial Rodents: Grooming Sequences of Rat and Guinea Pig Pups

Abstract Behavior occurs as coordinated patterns of serial order, with rules of 'action syntax'. Grooming behavior of adult rodents provides several striking examples of action syntax rules. The most stereotyped of these is a 'syntactic chain' pattern, which organizes up to 25 facial strokes and licking movements into a predictable sequence. This pattern previously has been found to be emitted by diverse rodent species from all major suborders: myomorph, sciuromorph, and caviomorph. In this study, we compared the postnatal ontogeny of grooming syntax in two rodent species: rat versus guinea pig. Rats are relatively altricial at birth, whereas guinea pigs are precocial. A day-by-day study of the fine-grain structure of sequential patterns was carried out during the first three weeks after birth, using slow-motion videoanalyses and a choreographic notation system for scoring behavioral grooming sequences. The results showed that substantial action syntax rules exist in guinea pig grooming even on the day of birth. For guinea pigs, postnatal grooming syntax development was limited to minor increments in the strength of the syntactic pattern and in postural control. By contrast, for rat pups, action syntax did not begin to appear until the second postnatal week, and developed gradually into the third week. The development of rodent syntactic patterns in both species appeared to be independent of the maturation of the movements that composed the pattern. Our results indicate that action syntax rules develop as hierarchical entities independent from constituent movements.

State Matrix Representation of Assembly and Robot Planning

SummaryA new approach to represent assembly called state matrix representation and an algorithm for automatic robot assembly planning based on this representation is proposed. The state matrix representation of assembly is configured by considering the inter-relationships of parts and objects involved in the initial and the goal structures. Thanks to this new representation, the planning lgorithm is straightforward and can be easily and efficiently implemented with simple matrix manipulation. Unlike other planning methods, the actions involved in the assembly process are not defined in advance but are generated at planning time. The syntax of actions are designed so that while directly reflecting the semantics of actions, they can be easily manipulated by the planner. Two examples of how to plan an assembly based on this representation are given in the paper.

Food-caching in timber wolves, and the question of rules of action syntax

This report presents data on the sequence of motor operations used by captive timber wolves to cache food. Videotapes were obtained of 151 caching episodes by 8 wolves. The vast majority of these episodes contained 3 distinct phases, each composed of movements unique to that phase. The excavation of the cache site was always done with the forefeet, and burying of the food was always done with the snout. Both the identity of the movements, and the serial order of phases were independent of the sex of the animal, the season in which the observations were made, and the nature of the substrate. A comparison of the temporal sequencing of these actions with the temporal stereotypy seen in rodent motor patterns (e.g. grooming) revealed a striking phenomenological similarity. The factors shaping the temporal sequencing in the two behaviors are, however, probably very different. This is because much, though not all, of the temporal stereotypy in the sequence of movements used by the wolf in caching is constrained by the logistics of the cache operation, while this is not the case for the phases of facial grooming in rodents. The implications of our data for the kinds of behavioral evidence required for ascription of such stereotypy to a central pattern generator are discussed.

Deafferentation does not disrupt natural rules of action syntax

Natural rules of action syntax control the sequential order of grooming and ingestive/aversive actions emitted by rats. Grooming and ingestive actions share a common feature in that all are performed with or directed towards the mouth, tongue, and face. This study examined the role of orofacial somatosensory cues and feedback in the generation of natural action syntax. Bilateral deafferentation of the mandibular and maxillary branches of the trigeminal nerve was used to eliminate tactile sensation from the rostral face and mouth while preserving motor function. Neither the overall degree of sequential stereotypy ( H) of grooming or ingestive sequences, nor the generation of particular natural sequencing rules were affected by trigeminal deafferentation. These natural rules appear to be specified by the brain without need of somatosensory feedback.

Natural syntax rules control action sequence of rats

Knowledge of the principles by which behavioral sequences are generated is essential to progress in our understanding of neural mechanisms. We describe here a set of natural principles or syntax rules that organize the components of grooming and feeding. The behavioral stream of facial grooming or of taste-elicited ingestive/aversive consummatory actions of rats can be viewed as a long series of individual movements linked together to form functional sequences. In order to ascertain the syntax rules that determine how these actions are linked together, many thousands of spontaneous grooming and elicited ingestive/aversive actions were videotaped and scored with a microcomputer. Techniques of information analysis of sequential stereotypy, tabulation of the sequential transitions between single actions and between action groups, and visual inspection for linear action chains, were employed to expose underlying rules of behavioral sequencing. These analyses revealed two global patterns: action perseveration and transitional reciprocation between sequential pairs and triplets, which together account for approximately 75% of all sequential transitions during grooming and ingestion/aversion. The pattern of transitional reciprocation could be divided further into patterns of alternation between individual actions on the one hand, and between perseverating bouts of actions on the other. Global syntax rules applied equally to actions emitted during grooming or during taste-elicited ingestion/aversion. In addition, a specific rule of linear chaining was found to apply only to facial grooming. These natural rules of action syntax provide insight into the sequential structure of behavior, and lend themselves well to analyses of neural mechanisms.

The art of natural graphic man-machine conversation

The design of interactive graphic systems whose aim is good symbiosis between man and machine involves numerous factors. Many of those factors can be judged from the perspective of natural spoken conversation between two people.Guiding rules and principles for design of such systems are presented as a framework for a survey of design techniques for man-machine conversation. Attention is especially focused on ideas of action syntax structuring, logical equivalences among action devices, and avoidance of psychological blocks to communication.

The art of natural graphic man—Machine conversation

The design of interactive graphic systems whose aim is good symbiosis between man and machine involves numerous factors. Many of those factors can be judged from the perspective of natural spoken conversation between two people. Guiding rules and principles for design of such systems are presented as a framework for a survey of design techniques for man-machine conversation. Attention is especially focused on ideas of action syntax structuring, logical equivalences among action devices, and avoidance of psychological blocks to communication.