Connectionist Representation Research Articles

Knowledge Representation: Two Kinds Of Emergence

Two different but closely related issues in current cognitive science will be considered in this essay. One is the controversial and extensively discussed question of how connectionist and symbolic representations of knowledge are related to each other. The other concerns the notion of connectionist learning and its relevance for the understanding of the distinction between propositional and nonpropositional knowledge. More specifically, I shall give an overview of a result in Rantala and Vaden (1994) establishing a limiting case correspondence between symbolic and connectionist representations and, on the other hand, study the problem, preliminarily investigated in Rantala (1998), of how propositional knowledge may arise from nonpropositional knowledge. I shall also try to point out that on some more or less plausible assumptions, often made by cognitive scientists, these results may have some significance when we try to comprehend the nature of human knowledge representation. Some of these assumptions are rather hypothethical and debatable for the time being and they will become justified in the future only if there will be more progress in the empirical and theoretical research on the brain and on artificial networks. The assumptions concern, besides some questions of the behavior of neural networks, such things as the relevance of pattern recognition for modelling human cognition, in particular, knowledge acquisition, and the relation between emergence and reduction.

Connectionist modelling in psychology: a localist manifesto.

Over the last decade, fully distributed models have become dominant in connectionist psychological modelling, whereas the virtues of localist models have been underestimated. This target article illustrates some of the benefits of localist modelling. Localist models are characterized by the presence of localist representations rather than the absence of distributed representations. A generalized localist model is proposed that exhibits many of the properties of fully distributed models. It can be applied to a number of problems that are difficult for fully distributed models, and its applicability can be extended through comparisons with a number of classic mathematical models of behaviour. There are reasons why localist models have been underused, though these often misconstrue the localist position. In particular, many conclusions about connectionist representation, based on neuroscientific observation, can be called into question. There are still some problems inherent in the application of fully distributed systems and some inadequacies in proposed solutions to these problems. In the domain of psychological modelling, localist modelling is to be preferred.

Localism as a first step toward symbolic representation

Page argues convincingly for several important properties of localist representations in connectionist models of cognition. I argue that another important property of localist representations is that they serve as the starting point for connectionist representations of symbolic (relational) structures because they express meaningful properties independent of one another and their relations.

Internal reinforcement in a connectionist genetic programming approach

Genetic programming (GP) can learn complex concepts by searching for the target concept through evolution of a population of candidate hypothesis programs. However, unlike some learning techniques, such as Artificial Neural Networks (ANNs), GP does not have a principled procedure for changing parts of a learned structure based on that structure's performance on the training data. GP is missing a clear, locally optimal update procedure, the equivalent of gradient-descent backpropagation for ANNs. This article introduces a new algorithm, “internal reinforcement”, for defining and using performance feedback on program evolution. This internal reinforcement principled mechanism is developed within a new connectionist representation for evolving parameterized programs, namely “neural programming”. We present the algorithms for the generation of credit and blame assignment in the process of learning programs using neural programming and internal reinforcement. The article includes a comprehensive overview of genetic programming and empirical experiments that demonstrate the increased learning rate obtained by using our principled program evolution approach.

Semantic systematicity and context in connectionist networks

A bstract . Fodor and Pylyshyn argued that connectionist models could not be used to exhibit and explain a phenomenon that they termed systematicity, and which they explained by possession of composition syntax and semantics for mental representations and structure sensitivity of mental processes. This inability of connectionist models, they argued, was particularly serious since it meant that these models could not be used as alternative models to classical symbolic models to explain cognition. In this paper, a connectionist model is used to identify some properties which collectively show that connectionist networks supply means for accomplishing a stronger version ofsystematicity than Fodor and Pylyshyn opted for. It is argued that 'context-dependent systematicity' is achievable within a connectionist framework. The arguments put forward rest on a particular formulation of content and context of connectionist representation, firmly and technically based on connectionist primitives in a learning environment. The perspective is motivated by the fundamental differences between the connectionist and classical architectures, in terms of prerequisites, lower-level functionality and inherent constraints. The claim is supported by a set of experiments using a connectionist architecture that demonstrates both an ability of enforcing, what Fodor and Pylyshyn term systematic and nonsystematic processing using a single mechanism, and how novel items can be handled without prior classification. The claim relies on extended learning feedback which enforces representational context dependence.

Dynamic Models of Simple Judgments: II. Properties of a Self-Organizing PAGAN (Parallel, Adaptive, Generalized Accumulator Network) Model for Multi-Choice Tasks

This is the second of two papers comparing connectionist and traditional stochastic latency mechanisms with respect to their ability to account for simple judgments. In the first, we reviewed evidence for a self-regulating accumulator module for two- and three-category discrimination. In this paper, we examine established neural network models that have been applied to predicting response time measures, and discuss their representational and adaptational limitations. We go on to describe and evaluate the network implementation of a Parallel Adaptive Generalized Accumulator Network (PAGAN), based on the interconnection of a number of self-regulating, generalized accumulator modules. The enhancement of PAGAN through the incorporation of distributed connectionist representation is briefly discussed.

The development of a connectionist expert system for compensation of color deviation in offset lithographic printing

The offset lithographic printing process requires the operator to make appropriate and timely on-line adjustments to compensate for color deviations from the desired print. An operator acquires proficiency by working on the same machine over a period of several years; thus he is able to apply adjustments according to its specific characteristics. It was found that this machine-specific knowledge consists of articulated and unarticulated knowledge. A connectionist representation was designed to map the observable variables to the operator's adjustments; while a forward chaining expert system was developed to represent the operator's articulated knowledge. A weight-based conflict resolution technique was constructed to dynamically update the knowledge base. This paper begins by presenting the press characterization problem. Then the development of the system is described. Finally, an analysis of results that cover all possible categories is documented.

Connectionist variable binding

Variable binding has long been a challenge to connectionists. Attempts to perform variable binding using localist and distributed connectionist representations are discussed, and problems inherent in each type of representation are outlined.

The acquisition of lexical and grammatical aspect in Chinese

This study reports three experiments on how children learning Mandarin Chinese comprehend and use aspect markers. These experiments examine the role of lexical aspect in children's acquisition of grammatical aspect. Results provide converging evidence for children's early sensitivity to (1) the association between atelic verbs and the imperfective aspect markers zai, -zhe, and -ne, and (2) the association between telic verbs and the perfective aspect marker -le. Children did not show a sensitivity in their use or understanding of aspect markers to the difference between stative and activity verbs or between semelfactive and activity verbs. These results are consistent with Slobin's (1985) basic child grammar hypothesis that the contrast between process and result is important in children's early acquisition of temporal morphology. In contrast, they are inconsistent with Bickerton's (1981, 1984) language bioprogram hypothesis that the distinctions between state and process and between punctual and nonpunctual are preprogrammed into language learners. We suggest new ways of looking at the results in the light of recent probabilistic hypotheses that emphasize the role of input, prototypes and connectionist representations.

Artificial neural networks for coordination and control: The portability of experiential representations

It is time to locate connectionist representation theory in the new wave of robotics research. The utility of representations developed in artificial neural networks (ANNs) during learning has been demonstrated in cognitive science research since the 1980s. The research reported here puts learned representations to work in a decentered control task, the disembodied arm problem, in which a mobile robot operates an arm fixed to a table to pick up objects. There is no physical linkage between the arm and the robot and so the robot's point of view must be decentered. This is done by developing a modular Artificial Neural Net system in three stages: (i) a classifier net is trained with laser scan data to output transformationally invariant position classes; (ii) an arm net is trained for picking up objects; (iii) an inter net is trained to communicate and coordinate the sensing and acting. The completed system is shown to create new nonsymbolic transformationally invariant representations in order to perform the effective generalization of decentered viewpoints.

Confluent Preorder Parsing of Deterministic Grammars

In this paper, syntactic parsing is discussed in the context of connectionism, a new model, the confluent preorder parser (CPP), is proposed which exemplifies the holistic parsing paradigm. Holistic parsing has the advantage that little knowledge has to be assumed concerning the detailed parsing algorithm. This algorithm is often unkown or debatable, especially when human language understanding is concerned. In the CPP, syntactic parsing is achieved by transforming from the connectionist representation of the sentence to the connectionist representation of the preorder traversal of its parse tree, instead of to the representation of the parse tree itself. As revealed by the simulation experiments, generalization performance is excellent (as high as 90%). Also, the CPP is capable of parsing erroneous sentences and resolving lexical category ambiguities. A systematic study is conducted to explore the range of factors which can affect the effectiveness of the system. The error-recovery capability is especially useful in natural language processing when incomplete or even ungrammatical sentences must be dealt with.

Learning and adaptation in an airborne laser fire controller

A simulated battlefield, containing airborne lasers that shoot ballistic missiles down, provides an excellent test-bed for developing adaptive controllers. An airborne laser fire controller, which can adapt the strategy it uses for target selection, is developed. The approach is to transform a knowledge-based controller into an adaptable connectionist representation, use supervised training to initialize the weights so that the adaptable controller mimics the knowledge-based controller, and then use directed search with simulation-based performance evaluation to continuously adapt the controller behavior to the dynamic environmental conditions. New knowledge can be directly extracted from the automatically discovered controllers. Three directed search methods are characterized for production training, and compared with the better characterized gradient descent methods commonly used for supervised training. Automated discovery of improved controllers is demonstrated, as is automated adaptation of controller behavior to changes in environmental conditions.

(S)RAAM: An Analytical Technique for Fast and Reliable Derivation of Connectionist Symbol Structure Representations

Recursive auto-associative memory (RAAM) has become established in the connectionist literature as a key contribution in the strive to develop connectionist representations of symbol structures. However, RAAMs use the backpropagation algorithm and therefore can be difficult to train and slow to learn. In addition, it is often hard to analyze exactly what a network has learnt and, therefore, it is difficult to state what composition mechanism is used by a RAAM for constructing representations. In this paper, we present an analytical version of RAAM, denoted as simplified RAAM or (S)RAAM. (S)RAAM models a RAAM very closely in that a single constructor matrix is derived which can be applied recursively to construct connectionist representations of symbol structures. The derivation, like RAAM, exhibits a moving target effect because training patterns adjust during learning but, unlike RAAM, the training is very fast. The analytical model allows a clear statement to be made about generalization characteristics and it can be shown that, in practice, the model will converge.

Do Connectionist Representations Earn Their Explanatory Keep?

Abstract: In this paper I assess the explanatory role of internal representations in connectionist models of cognition. Focusing on both the internal‘hidden’units and the connection weights between units, I argue that the standard reasons for viewing these components as representations are inadequate to bestow an explanatorily useful notion of representation. Hence, nothing would be lost from connectionist accounts of cognitive processes if we were to stop viewing the weights and hidden units as internal representations.

Do Connectionist Representations Earn their Explanatory Keep?

Do Connectionist Representations Earn their Explanatory Keep?

From simple associations to systematic reasoning: A connectionist representation of rules, variables and dynamic bindings using temporal synchrony

AbstractHuman agents draw a variety of inferences effortlessly, spontaneously, and with remarkable efficiency – as though these inferences were a reflexive response of their cognitive apparatus. Furthermore, these inferences are drawn with reference to a large body of background knowledge. This remarkable human ability seems paradoxical given the complexity of reasoning reported by researchers in artificial intelligence. It also poses a challenge for cognitive science and computational neuroscience: How can a system of simple and slow neuronlike elements represent a large body of systemic knowledge and perform a range of inferences with such speed? We describe a computational model that takes a step toward addressing the cognitive science challenge and resolving the artificial intelligence paradox. We show how a connectionist network can encode millions of facts and rules involvingn-ary predicates and variables and perform a class of inferences in a few hundred milliseconds. Efficient reasoning requires the rapid representation and propagation of dynamic bindings. Our model (which we refer to as SHRUTI) achieves this by representing (1) dynamic bindings as the synchronous firing of appropriate nodes, (2) rules as interconnection patterns that direct the propagation of rhythmic activity, and (3) long-term facts as temporal pattern-matching subnetworks. The model is consistent with recent neurophysiological evidence that synchronous activity occurs in the brain and may play a representational role in neural information processing. The model also makes specific psychologically significant predictions about the nature of reflexive reasoning. It identifies constraints on the form of rules that may participate in such reasoning and relates the capacity of the working memory underlying reflexive reasoning to biological parameters such as the lowest frequency at which nodes can sustain synchronous oscillations and the coarseness of synchronization.

Useful ideas for exploiting time to engineer representations

Commentary on target article From simple associations to systematic reasoning: a connectionist representation of rules, variables, and dynamic bindings using temporal synchrony, by L. Shastri and V. Ajjangadde, pp. 417-494

Control of a pH Plant Using Connectionist Representations

In this paper the control of a pH plant using connectionist representations is addressed. Four nonlinear control schemes based on connectionist representations of the controlled plant are developed and compared. The results show that it is possible achieve better control than a linear control scheme and there are limitations imposed by the nonlinearity of the plant and the degree of approximation of the model over the differents approaches studied.

SC-net: A hybrid connectionist, symbolic system

This paper describes the SC-net system that has been developed to provide expert systems capability augmented with learning in a hybrid connectionist/symbolic approach. A distributed connectionist representation of cells connected by links is used to represent symbolic knowledge. Rules may be directly encoded in the connectionist network or learned from examples. The learning method is a form of instance-based learning in which some of the individual instances in the training set are encoded by adding structure to the network and others cause modifications to biases in the network. Both continuous and nominal attributes are directly represented in the network structure. A limited form of variables in the form of attribute value bindings on the right-hand side of rules is supported. Relational comparators in the form of cell groups are also supported. Relational comparators and attribute value structures are represented by groups of connected cells in the network. The learning algorithm is presented and methods for providing generalization in an instance-based connectionist environment are presented. Empirical results are presented, which include learning in domains (fevers and gems) that contain uncertainty and the well-known iris, and soybean data sets together with a real world domain for semiconductor wafer fault diagnosis. The generalization ability of the learned network is shown to be good in several domains including iris. The system is shown to compare favorably with a nonneural instance-based learning algorithm IBL.

Expert networks: Paradigmatic conflict, technological rapproachement

A rule-based expert system is demonstrated to have both a symbolic computational network representation and a sub-symbolic connectionist representation. These alternate views enhance the usefulness of the original system by facilitating introduction of connectionist learning methods into the symbolic domain. The connectionist representation learns and stores metaknowledge in highly connected subnetworks and domain knowledge in a sparsely connected expert network superstructure. The total connectivity of the neural network representation approximates that of real neural systems and hence avoids scaling and memory stability problems associated with other connectionist models.