Action Selection Process Research Articles

A new fine-grained evolutionary algorithm based on cellular learning automata

In this paper a new evolutionary algorithm, called the CLA-EC (Cellular Learning Automata Based Evolutionary Computing), is proposed. This algorithm is a combination of evolutionary algorithms and the Cellular Learning Automata (CLA). In the CLA-EC each genome string in the population is assigned to one cell of the CLA, which is equipped with a set of learning automata. Actions selected by the learning automata of a cell determine the genome string for that cell. Based on a local rule, a reinforcement signal vector is generated and given to the set of learning automata residing in the cell. Each learning automaton in the cell updates its internal structure according to a learning algorithm and the received signal vector. The processes of action selection and updating the internal structures of learning automata are repeated until a predetermined criterion is met. To show the efficiency of the proposed model, to solve several optimization problems including real valued function optimization and data clustering problems.

Behavioral and distributed architecture of control for minimalist mobile robots

This paper describes a specific behavioral and reactive architecture of control applied to complex and cooperative tasks. The coordination between elementary behaviors (layers of skills) is insured using both hierarchical and fusion action mechanisms. Indeed, competitive and cooperative mechanisms are gathered giving thus an Hybrid and Hierarchical Action Selection Process Hybrid-HASP in order to perform complex box-pushing tasks by a set of minimalist mobile robots. Specific low-level behaviors, called altruistic behaviors reproducing the simple interaction (attraction/repulsion) of individuals constituting societies of insects are integrated into the architecture of control in order to improve the efficiency of robots coordination. The analysis of a great number of simulations allows us to have convincing statistical results which show among others, the existence of an optimal number of robots to achieve the cooperative task and underline the importance of implicit communications induced by the altruistic behaviors.

An Integrate-and-fire Model of Prefrontal Cortex Neuronal Activity during Performance of Goal-directed Decision Making

The orbital frontal cortex appears to be involved in learning the rules of goal-directed behavior necessary to perform the correct actions based on perception to accomplish different tasks. The activity of orbitofrontal neurons changes dependent upon the specific task or goal involved, but the functional role of this activity in performance of specific tasks has not been fully determined. Here we present a model of prefrontal cortex function using networks of integrate-and-fire neurons arranged in minicolumns. This network model forms associations between representations of sensory input and motor actions, and uses these associations to guide goal-directed behavior. The selection of goal-directed actions involves convergence of the spread of activity from the goal representation with the spread of activity from the current state. This spiking network model provides a biological implementation of the action selection process used in reinforcement learning theory. The spiking activity shows properties similar to recordings of orbitofrontal neurons during task performance.

Cognition, motivation, emotion and action: a dynamic and vulnerable interdependence

A model of the interactive processes of cognition, motivation, emotion and action selection is proposed. Building on earlier theoretical models, it is argued that behaviour is determined by a combination of ‘on-line’ direct controls and ‘off-line’ cognitive controls. These result in different modes of behavioural control exhibiting different behavioural properties. Certain similar changes over phylogeny and ontogeny can be observed in terms of an altered relative weight between these controls. Under natural conditions, adaptation can be served by a change of weight between different levels of control as a function of circumstances. It is suggested that under abnormal conditions, e.g. in certain domestic environments, the change in weight of control can be such as to lead to behavioural pathology. Conversely, by a focus on different levels of control, the model can also give indications of good welfare practice.

Improved QMDPPolicy for Partially Observable Markov Decision Processes in Large Domains: Embedding Exploration Dynamics

Abstract Artificial Intelligence techniques were primarily focused on domains in which at each time the state of the world is known to the system. Such domains can be modeled as a Markov Decision Process (MDP). Action and planning policies for MDPs have been studied extensively and several efficient methods exist. However, in real world problems pieces of information useful for the process of action selection are often missing. The theory of Partially Observable Mazkov Decision Processes (POMDP’s) covers the problem domain in which the full state of the environment is not directly perceivable by the agent. Current algorithms for the exact solution of POMDP’s are only applicable to domains with a small number of states. To cope with more extended state spaces, a number of methods that achieve sub-optimal solutions exist and among these the QI,IDP approach seems to be the best. We introduce a novel technique, called Explorative [Qtilde]P (EQI-IDP) which constitutes an important enhancement of the [Qtilde]P ...

KNOWLEDGE GRANULARITY SPECTRUM, ACTION PYRAMID, AND THE SCALING PROBLEM

In this paper we introduce the concept of knowledge granularity and study the relationship between different knowledge representation schemes and the scaling problem. By scale to a task, we mean that an agent's planning system and knowledge representation scheme are able to generate the range of behaviors required by the task in a timely fashion. Action selection is critical to an agent performing a task in a dynamic, unpredictable environment. Knowledge representation is central to the agent's action selection process. It is important to study how an agent should adapt its methods of representation such that its performance can scale to different task requirements. Here we study the following issues. One is the knowledge granularity problem: to what detail should an agent represent a certain kind of knowledge if a single granularity of representation is to be used. Another is the representation scheme problem: to scale to a given task, should an agent represent its knowledge using a single granularity or a set of hierarchical granularities.

Modular Q-learning based multi-agent cooperation for robot soccer

In a multi-agent system, action selection is important for the cooperation and coordination among agents. As the environment is dynamic and complex, modular Q-learning, which is one of the reinforcement learning schemes, is employed in assigning a proper action to an agent in the multi-agent system. The architecture of modular Q-learning consists of learning modules and a mediator module. The mediator module of the modular Q-learning system selects a proper action for the agent based on the Q-value obtained from each learning module. To obtain better performance, along with the Q-value, the mediator module also considers the state information in the action selection process. A uni-vector field is used for robot navigation. In the robot soccer environment, the effectiveness and applicability of modular Q-learning and the uni-vector field method are verified by real experiments using five micro-robots.