Complex Action Sequences Research Articles

Slave to the striatal habit (Commentary on Tricomi et al.)

Colloquially, we all know what it means to perform an action by ‘habit’. But scientifically, characterizing behaviors as such is not so straightforward. Indeed, while addictive behaviors are often framed in the context of habit learning (Graybiel, 2008), this remains a controversial issue in the neurobiology of addiction literature (Berridge, 2007; Robbins & Everitt, 2007). For example, does one light up a cigarette as an instinctive response to cues that are generally associated with that motor program (due to prior stimulus-response reinforcement learning)? Or rather, do such behaviors reflect a goal-directed process in which the craving of a particular desired outcome (i.e. the nicotine high) is used to flexibly determine any complex sequence of actions needed to achieve it? Both of these processes likely contribute to habit-like behavior, but teasing them apart (at both cognitive and neurobiological levels) is critical. In animal research, habits are typically operationalized as behaviors that are so ingrained as to be insensitive to their consequent outcomes, as evaluated with a devaluation procedure. Animals are first trained to press a lever to produce a reward. Next, the reward itself is devalued so that it is no longer desirable to them (e.g. by providing free access to the reward so that they are satiated). The animal is then presented with the lever to determine whether they respond in a ‘goal-directed’ manner by reducing lever presses, indicative that their actions are sensitive to the (no longer desired) outcome. However, given sufficient prior training, animals will continue to press the lever ‘by habit’ despite the fact that they do not want the reward. In rats, it is now well established that the dorsolateral striatum is responsible for ingraining stimulus-response associations that lead to habit-like behavior, whereas prefrontal cortical regions are required for updating action-value representations and allowing the animal to exert goal-directed behavior (Yin et al., 2004). By contrast, despite years of research on procedural and reinforcement learning, evidence for veridical habit learning in humans is lacking. In this issue of EJN, Tricomi et al. (2009) investigated this question with functional magnetic resonance imaging (fMRI) and behavioral testing over multiple sessions. They used a variable interval reinforcement schedule known to promote habitual behavior in rodents. Participants were presented with fractal stimulus cues and learned to press one of two different keys as often as they wished. Each cue, if responded to with the correct key and at the appropriate time, would yield a different food reward (corn chips or chocolate bars). After the training session, the devaluation procedure was administered: participants were given free access to one of the two rewards so that they were satiated to that food. In the critical test phase, when presented with the fractal cues, those who had undergone only a single day of training behaved in a goal-directed manner: they responded selectively less to the stimulus associated with the devalued reward. In stark contrast, those who had undergone three days of training continued to respond ‘by habit’ to both cues as if the devaluation procedure had never occurred. What are the brain mechanisms of such habitual responding? Notably, in the imaging analysis, an area within the dorsolateral striatum – specifically the right posterior putamen and the globus pallidus to which it projects – showed greater activation at the end of the three day training procedure than at the beginning. These findings are remarkably consistent with those implicating homologous areas in rodent habitual behavior (Yin et al., 2004). Furthermore, activations in the ventromedial prefrontal cortex (vmPFC), which were previously shown to be sensitive to changes in subjective value following devaluation (Valentin et al., 2007) appeared to ‘ramp up’ on each trial in anticipation of reward outcomes – consistent with an action-outcome goal-directed process. How do brain areas involved in habitual versus goal-directed behavior interact? Whereas basal ganglia activations increased across training, suggestive of a progressive reinforcement learning process, the putative action-outcome representations in vmPFC continued to ramp up in anticipation of reward throughout all training sessions. Thus habitual responding appears not to result from a decrease in the anticipation of reward outcomes across trials, but rather due to strengthening of stimulus-response links. This interpretation is consistent with that embedded within computational models in which habitual responding emerges even as both striatal and prefrontal systems represent their respective estimates of reward likelihood for the action at hand, but with striatum progressively exerting a dominant bias on behavioral control (Daw et al., 2005; Frank & Claus, 2006). In summary, these data provide compelling evidence for common neurobiological substrates of habit learning in humans and rodents. Future studies should investigate the degree to which these processes are altered by clinical disorders including obsessive compulsive disorder and addiction.

Vision, eye movements, and natural behavior

Historically, the principal function of vision has been to provide the information needed to support action. Visually mediated actions rely on three systems: the gaze system responsible for locating and fixating task-relevant objects, the motor system of the limbs to carry out the task, and the visual system to supply information to the other two. All three systems are under the control of a fourth system, the schema system, which specifies the current task and plans the overall sequence of actions. These four systems have separate but interconnected cortical representations. The way these systems interact in time and space is discussed here in relation to two studies of the gaze changes and manipulations made during two ordinary food preparation tasks. The main conclusions are that complex action sequences consist of a succession of individual object-related actions, each of which typically involve a turn toward the object (if needed), followed by fixation and finally manipulation monitored by vision. Gaze often moves on to the next object just before manipulation is complete. Task-irrelevant objects are hardly ever fixated, implying that the control of fixation comes principally from top-down instructions from the schema system, not bottom-up salience. Single fixations have identifiable functions (locating, directing, guiding, and checking) related to the action to be taken. Several variants of the basic object-related action scheme are discussed, including single-action events in ball sports involving only one anticipatory gaze shift, continuous production loops in text and music reading, and storage-action alternation in copying tasks such as portrait sketching.

Learning to imitate novel motion sequences

Many imitative behaviors entail complex sequences of component actions that must be recalled and performed in the proper order. It is well known that imitation of complex actions tends to improve with repeated opportunities to observe and execute the target behavior. But what actually makes this practice-based improvement possible? To address this question, we had subjects view and then reproduce sequences of connected, randomly directed motions of a disc. Even a single repetition of a motion sequence substantially reduced errors in reproduction. Improvement seemed to follow a power law, with accuracy in reproducing each motion segment improving by an amount proportional to the current error for that segment. Analysis of the pauses separating a reproduction's segments suggests that with learning, multiple segments in memory are grouped into more compact representations. To test overt performance's contribution to repetition-based improvement, we compared subjects' performance when they reproduced the stimulus trajectory after each repetition to when they did so only once, after the final repetition. Performance was similar following the final repetition in both conditions, indicating that seeing the model, without actual imitation, was sufficient for learning--even in the absence of an explicit error signal. In another experiment, subjects viewed three presentations of each model, with the second presentation given in forward (start to end) or backward (end to start) order. Performance was significantly better when all three presentations were in the same, consistent order, suggesting that repetition reinforced some temporal aspects of a trajectory as it was being learned, and not merely a better representation of the static shape traced by the motion of the disc. These results provide a first look into explicit learning of sequential, nonverbal material, which is central to many tasks of daily life.

Observing complex action sequences: The role of the fronto-parietal mirror neuron system

A fronto-parietal mirror neuron network in the human brain supports the ability to represent and understand observed actions allowing us to successfully interact with others and our environment. Using functional magnetic resonance imaging (fMRI), we wanted to investigate the response of this network in adults during observation of hierarchically organized action sequences of varying complexity that emerge at different developmental stages. We hypothesized that fronto-parietal systems may play a role in coding the hierarchical structure of object-directed actions. The observation of all action sequences recruited a common bilateral network including the fronto-parietal mirror neuron system and occipito-temporal visual motion areas. Activity in mirror neuron areas varied according to the motoric complexity of the observed actions, but not according to the developmental sequence of action structures, possibly due to the fact that our subjects were all adults. These results suggest that the mirror neuron system provides a fairly accurate simulation process of observed actions, mimicking internally the level of motoric complexity. We also discuss the results in terms of the links between mirror neurons, language development and evolution.

Review of the Windmill Pitch: Biomechanics and Injuries

Objective To review the literature of the biomechanics of the windmill fast-pitch and its implications for injury. This information may be utilized in treating youth windmill pitchers. Data Source A MEDLINE search was conducted to retrieve articles regarding the windmill pitch. Key terms were then taken from the pilot search and used to conduct a systematic search and review of the literature. Results Articles containing information on the windmill pitch and injuries associated with the motion were reviewed. Additional information pertaining to the overhand baseball pitch and overuse injuries in youth were analyzed and synthesized into the body of information. Conclusion A complex sequence of actions is required to successfully perform the windmill pitch. Overuse injuries are common in windmill pitchers. A well-designed conditioning schedule and the regulation of the frequency and volume of pitching in youth fast-pitch may assist with managing injury associated with this activity. Further investigation of specific treatment methods is needed.

Discrete and cyclical units of action in a mixed target pair aiming task

Two experiments addressed the issue of discrete and cyclical units as possible basic units of action that might be used to construct complex actions based on task constraints. The experiments examined the influence of low and high accuracy constraints on the end-effector's motion in rhythmical aiming movements. Both experiments utilized a Fitts-type task under three accuracy constraints: (1) big target pairing-low index of movement difficulty (ID), (2) small target pairing-high ID, and (3) mixed target pairing-one target high ID and the other target low ID. Experiment I was a 1-degree-of-freedom ( df) task that required subjects to crossover the inside edge of targets in a target pair using elbow flexion-extension motions. Experiment II used a 2- df task that required subjects to tap back and forth between targets in a target pair using a hand-held stylus. In both experiments, end-effector motion in the low ID condition was cyclical with the end-effector's motion consistent with a limit-cycle attractor description, while in the high ID condition end-effector motion was discrete and consistent with a fixed-point attractor description. The mixed target pairing produced both discrete and cyclical features in the end-effector's dynamics that suggested a functional linking of discrete and cyclical units of action as the optimal movement solution. Evidence supporting the above statements was found in the kinematic measures of movement time (MT), dwell time, proportion of MT accelerating and decelerating, and in a measure of harmonicity (Guiard 1993, Acta Psychol 82:139-159; Guiard 1997, Hum Mov Sci 16:97-131). Extended practice in the mixed target condition revealed a bias towards cyclical motion with practice. The results demonstrate that discrete and cyclical motion, represented as limit-cycle and fixed-point attractors, are basic units of action that the motor system uses in constructing more complex action sequences. The results are discussed with reference to coordinative structures and the generalized motor program as basic units of action. Issues pertaining to visual feedback processing and movement braking in rapid aiming tasks are also discussed.

Planning and the neurotechnology of social behaviour

The human brain has a remarkable ability to generate plans for sequences of actions that allow human agents to co-operate with and to manipulate the behaviour of others. It is widely claimed that the operations underlying plan developments, behaviour sequencing and inhibition of inappropriate responses to the environment are carried out in the prefrontal cortex. This implies that the prefrontal cortex is a natural system with the capacity to utilise cognitive technology. The present paper argues that social competence is a manifestation of action planning in which other agents feature as plan elements. Accordingly, plans that involve other agents are expected to be more complex than plans which do not. In the light of evidence that negative information makes particularly heavy processing demands, social judgements involving prohibition or unacceptability are expected to create most difficulty for the human action planning system. These assumptions were tested by measuring the ability of patients with prefrontal injuries to detect anomalous action sequences, using a specially constructed Action Acceptability Test. It was hypothesised that if the frontal lobes play a major role in action planning, patients with frontal lobe injuries should show impaired ability to detect faulty action plans, particularly when such plans relate to complex social action sequences, and action sequences involving unacceptable behaviours. The hypotheses were generally supported as frontal-injury patients proved to be worse at detecting both complex social sequences and deviant action sequences than participants with non-frontal injuries and normal control participants. The results of the study are consistent with the view that human social competence results from the cognitive processes associated with action planning and the data also supports the claim that action planning processes are specifically disrupted by damage to the prefrontal cortex. The findings provide some confirmation for the Cognitive Technology perspective, in that action planning does seem to be physically associated with a specific brain area, and including social agents and deviations from acceptability in action plans do seem to be manipulations that operate to make action plans more difficult to process, hence causing more errors in individuals with damage to the prefrontal cortex. The results also provide some encouragement for the belief that new cognitive tools can be constructed that link brain processes to other levels of description, such as social behaviour. The Action Acceptability Test, a prognostic tool developed to predict the social competence of frontal-injury patients, is offered as one such example.

The Handbook of cognitive neuropsychology: what deficits reveal about the human mind

Part 1. Foundations. M. Coltheart, Assumptions and Methods in Cognitive Neuropsychology. O. Selnes, Contributions from the Study of Deficits to Our Understanding of Brain Function: A Historical Overview. Part 2. Objects. M.J. Riddoch, G. Humphreys, Recognizing and Identifying Objects. E. de Haan, Face Recognition. Part 3. Attention. C. Umilta, Mechanisms of Attention. M. Farah, Consciousness. Part 4. Words. A. Hillis, The Organization of Lexical Knowledge. W. Badecker, Allen, Morphology: The Internal Structure of Words. B. Rapp, J. Folk, Reading Words. M. Tainturier, B. Rapp, Spelling Words. L. Nickels, Spoken Word Production. Part 5. Sentences. R. Martin, Understanding Sentences. R. Berndt, Producing Sentences. T. Gollan, J. Kroll, Bilingualism. Part 6. Memory. A. Parkin, Memory Systems. J. Shelton, A. Caramazza, Semantic Categories. D. Schacter, C. Dodson, Constructive Processes in Memory. Part 7. Space, Time, Numerosity and Music. M. Noel, Numerical Cognition. I. Peretz, Musical Cognition. M. McCloskey, Spatial Frames of Reference. J. Mangels, R. Ivry, Temporal Processing. Part 8. Actions and Plans. L. Buxbaum, B. Coslett, Motor Programming and Execution. G. Humphreys, Planning and Executing Complex Action Sequences. Part 9. Future Directions. M. McCloskey, Future Directions.

Incremental Evolution of Complex General Behavior

Several researchers have demonstrated how complex action sequences can be learned through neuroevolution (i.e., evolving neural networks with genetic algorithms). However, complex general behavior such as evading predators or avoiding obstacles, which is not tied to specific environments, turns out to be very difficult to evolve. Often the system discovers mechanical strategies, such as moving back and forth, that help the agent cope but are not very effective, do not appear believable, and do not generalize to new environments. The problem is that a general strategy is too difficult for the evolution system to discover directly. This article proposes an approach wherein such complex general behavior is learned incrementally, by starting with simpler behavior and gradually making the task more challenging and general. The task transitions are implemented through successive stages of Delta coding (i.e., evolving modifications), which allows even converged populations to adapt to the new task. The method is tested in the stochastic, dynamic task of prey capture and is compared with direct evolution. The incremental approach evolves more effective and more general behavior and should also scale up to harder tasks.

Position effects in chunked linear orders

When someone is asked which of two items from a linearly ordered set of items comes first in that order, reaction time decreases with increasing distance between the two items in the given order. This is known as thedistance effect, a very robust finding in studies on linear orders. Surprisingly, investigations onchunked linear orders show unexpected diverse results. For example, reaction times todifferent probes (i. e., probes with items from different chunks) are sometimes longer than those tosame probes (i. e., probes with items from the same chunk). But if subjects' latency depends on the discriminability of chunk labels vs. exact position information within a chunk,different probes should always yield faster order judgments thansame probes. Two experiments investigated whether position effects, including both end-of-list and end-ofchunk effects, can account for the unexpected finding that between-chunk probes are sometimes answered more slowly than within-chunk probes. Moreover, the linea-rorder paradigm was extended to complex action sequences with more than two chunks, using original cooking recipes as materials. The results showed thatchunk-position effects play a crucial role in comparative judgments. Probes from the first and the last chunk were answered faster than those from a middle chunk.Border items (i. e., items next to a chunk boundary) did not produce a significant effect. However, chunking affected subjects' reaction time, as was indicated by the absence of a distance effect fordifferent probes. Besides, action sequences that are ordered in time, as well as concept lists used previously, can be considered linear orders, thus supporting the view that linear orders reflect a basic representation schema of human memory.

Task support in an office system

The tools in current office systems are designed to carry out simple tasks that are common to most offices. For example, tasks such as communication, time management and document production are supported by the electronic mail, calendar and text editor tools. It has been pointed out that this level of task support is only of limited effectiveness in addressing the problem of office productivity [HAMM79,HAMM80]. A more effective system would support higher-level tasks that are directly related to the goals or functions of the office. This type of task often involves decision-making, complex sequences of actions, and interaction with a number of other people. The crucial feature of these tasks is that, unlike the simple tasks mentioned previously, tasks that more closely implement a particular office's functions must be very office-specific. It will not be possible to provide a generic set of tools to directly support all office tasks. However, it should be possible to “customize” an office system by providing a means of describing the high-level tasks and how they can be supported with the existing set of tools.Descriptions of office tasks are usually called office procedures. Although the name implies that the activities described are very routine and structured, office procedures can also be used to describe more unstructured, management-level tasks. Office procedure formalisms have been used to define and analyse offices [ELLI80,BAIL83], and to automate office tasks [ZISM77]. A description of a task using a procedure formalism is subject to change and represents only a standard or typical way of carrying out that task. A system that uses office procedures to support tasks must provide mechanisms to deal with these limitations of procedures [FIKE80].

Task support in an office system

The tools in current office systems are designed to carry out simple tasks that are common to most offices. For example, tasks such as communication, time management and document production are supported by the electronic mail, calendar and text editor tools. It has been pointed out that this level of task support is only of limited effectiveness in addressing the problem of office productivity [HAMM79,HAMM80]. A more effective system would support higher-level tasks that are directly related to the goals or functions of the office. This type of task often involves decision-making, complex sequences of actions, and interaction with a number of other people. The crucial feature of these tasks is that, unlike the simple tasks mentioned previously, tasks that more closely implement a particular office's functions must be very office-specific. It will not be possible to provide a generic set of tools to directly support all office tasks. However, it should be possible to “customize” an office system by providing a means of describing the high-level tasks and how they can be supported with the existing set of tools. Descriptions of office tasks are usually called office procedures. Although the name implies that the activities described are very routine and structured, office procedures can also be used to describe more unstructured, management-level tasks. Office procedure formalisms have been used to define and analyse offices [ELLI80,BAIL83], and to automate office tasks [ZISM77]. A description of a task using a procedure formalism is subject to change and represents only a standard or typical way of carrying out that task. A system that uses office procedures to support tasks must provide mechanisms to deal with these limitations of procedures [FIKE80].

Some psychological relations between action and language structure.

This study investigated the relationship between complex grammatical structures and complex action sequences. A developmental progression of strategies for combining seriated cups identified in an earlier study (Greenfield et al., 1972) was used to demonstrate some psychological consequences of formal parallels between language and action. The role of grammatical complexity and situational structure in language-action relations was explored. The results have implications for understanding the organization and development of complex action, its control by verbal commands, and the basic processes of speech comprehension.