Natural Information Processing Systems Research Articles

Biological evolution and human cognition are analogous information processing systems.

The mechanisms that govern biological evolution and human cognition are analogous, as both follow the same principles of natural information processing systems. In this article, we describe the following five principles that provide an analogy between biological evolution and human cognition: (a) Randomness as Genesis Principle and (b) Borrowing and Reorganizing Principle, which indicate how natural information processing systems obtain information; (c) Narrow Limits of Change Principle and (d) Information Store Principle, which indicate how information is processed and stored; and (e) Environmental Organizing and Linking Principle, which indicate how stored information is used to generate actions appropriate to an environment. In human cognition, these analogs only apply to cognitive processes associated with biologically secondary knowledge, the knowledge typically taught in educational institutions. Based on these five principles, cognitive load theory researchers have provided diverse prescriptions to optimize instructional activities and materials. We conclude by discussing general instructional implications and future research directions based on this analogy.

Evolutionary Perspective on Human Cognitive Architecture in Cognitive Load Theory: a Dynamic, Emerging Principle Approach

Adopting an evolutionary approach to substantiate major characteristics of human cognitive architecture has been one of the major recent developments in cognitive load theory. According to this approach, human cognitive architecture is a natural information processing system which can be described by five general principles. This paper attempts to (1) identify the scope of applicability of these principles in natural information processing systems of different levels of complexity, (2) reconcile the coexistence of implicit (primary) and controlled (secondary) processes within the same human cognitive architecture, and (3) incorporate motivational factors into the evolutionary approach to human cognitive architecture. The paper suggests two principal modifications to the traditional formulation of the evolutionary approach. Firstly, natural information processing systems are viewed as dynamically evolving systems with new principles added with increasing levels of complexity of the systems. Secondly, a new (the explicit intention to learn) principle is added at the level of human cognition. This sixth principle is expected to address (1) the emergence of controlled mechanisms dealing with biologically secondary information as expressed by conscious processing in working memory and (2) the role of learner motivation in such processes from an evolutionary perspective. The paper concludes with discussion of theoretical and practical instructional implications of the proposed modifications.

Classification at the accuracy limit: facing the problem of data ambiguity

Data classification, the process of analyzing data and organizing it into categories or clusters, is a fundamental computing task of natural and artificial information processing systems. Both supervised classification and unsupervised clustering work best when the input vectors are distributed over the data space in a highly non-uniform way. These tasks become however challenging in weakly structured data sets, where a significant fraction of data points is located in between the regions of high point density. We derive the theoretical limit for classification accuracy that arises from this overlap of data categories. By using a surrogate data generation model with adjustable statistical properties, we show that sufficiently powerful classifiers based on completely different principles, such as perceptrons and Bayesian models, all perform at this universal accuracy limit under ideal training conditions. Remarkably, the accuracy limit is not affected by certain non-linear transformations of the data, even if these transformations are non-reversible and drastically reduce the information content of the input data. We further compare the data embeddings that emerge by supervised and unsupervised training, using the MNIST data set and human EEG recordings during sleep. We find for MNIST that categories are significantly separated not only after supervised training with back-propagation, but also after unsupervised dimensionality reduction. A qualitatively similar cluster enhancement by unsupervised compression is observed for the EEG sleep data, but with a very small overall degree of cluster separation. We conclude that the handwritten letters in MNIST can be considered as ’natural kinds’, whereas EEG sleep recordings are a relatively weakly structured data set, so that unsupervised clustering will not necessarily re-cover the human-defined sleep stages.

Reflex inhibition. Global manifestation of the local mechanism

The paper considers the issues of training and retraining in self-organizing natural information processing systems and shows the reasons for the rule “The one who comes first gets all” and the possibility of creating a technical device, which is an artificial intelligence carrier. Such a device is not filled with the necessary information at the stage of creation, but receives all the necessary information in the process of functioning, especially at the beginning of its activity and in the process of education, just as the intellect of a person is being raised since childhood.

Openness of Natural Information Processing Systems and Short-Term and Long-Term Memory

In this paper, various types of memory are considered.Development of the nervous system and the brain of organisms ensures expansion of species with more developed brain. Different characteristics of the brain zones have created different conditions for forming traces of events. The various traces of trace events formed in different parts of the brain provided a wider range of functions and brain responses, which contributed to the survival and expansion of species. The work describes the appearance of memory varieties and the causes of these varieties.

Getting Past Your Past: Take Control of Your Life with Self-Help Techniques from EMDR TherapyNew York: Rodale Books, 2012. ISBN-10: 1609619951. ISBN-13: 978-1609619954 Hardcover: 352 Pages $26.99.

Getting Past Your Past: Take Control of Your Life with Self-Help Techniques from EMDR TherapyNew York: Rodale Books, 2012. ISBN-10: 1609619951. ISBN-13: 978-1609619954 Hardcover: 352 Pages $26.99.

Improvement of organizational performance and instructional design: An analogy based on general principles of natural information processing systems

The process of improving organizational performance through designing systemic interventions has remarkable similarities to designing instruction for improving learners' performance. Both processes deal with subjects (learners and organizations correspondingly) with certain capabilities that are exposed to novel information designed for producing a desired change. The article attempts to trace these similarities deeper to the underlying general principles that govern functioning of natural information processing systems that both human cognitive architecture (in case of instruction) and organizational systems may represent, and explores such connections and draws corresponding implications for research in human performance technology.

Architectural constraints on learning and memory function

Recent experimental and computational studies have identified relationships between architecture and functional performance in information processing systems ranging from natural neuronal ensembles [1,2] to artificial neural networks [3,4]. While these systems can vary greatly in their size and complexity, they share certain structural features, such as parallel and layered motifs [5]. Quantifying how these features influence functionality is a first step toward understanding the behavior of both natural and artificial information processing systems. Of particular interest is the impact of structural architecture on the ability of the system to balance stability with flexibility, for example in memory versus learning. In this study, we use neural networks as model information processing systems to examine tradeoffs in learning and memory processes arising from variations in structural organization. We compare the performance of parallel and layered structures during sequential function approximation, a task that requires networks to produce, retain, and dynamically adapt representations of external information. We measure network performance over a range of learning conditions by statistically analyzing the error in these representations while varying the initial network state, the structure of the external information, and the time allowed for learning. By characterizing local error landscape curvature, we can directly relate the functional performance of the system to its underlying architecture. Across a range of both parallel and layered system architectures, we find that variations in error landscape curvature give rise to tradeoffs between the ability of these networks to learn new versus retain old information, maximize success versus minimize failure, and produce specific versus generalizable representations of information. In particular, parallel networks generate smooth error landscapes with deep, narrow minima. Therefore, given sufficient time and through the adjustment of a large number of connection weights, parallel networks can find highly specific representations of the external information. Although accurate, however, these representations are difficult to generalize. In contrast, layered networks generate rough error landscapes with a variety of local minima, allowing them to quickly find coarse representations by adjusting a fewer number of weights. Although less accurate, these representations are more easily adaptable. We have conducted a detailed analysis of network performance over a range of parallel and layered architectures, thereby isolating learning and memory tradeoffs that arise from underlying structural complexity. A thorough understanding of small network systems is crucial for predicting the behavior of larger systems in which statistical studies of performance would not be possible. In particular, these results may provide insight into the behavior of composite systems, such as cortical layers composed of structurally distinct columns [6] or modular divide-and-conquer networks [7], which share features of both parallel and layered architectures. Additionally, the existence of tradeoffs inherent to a range of network structures may help explain the variability of architectural motifs observed in large-scale biological [5] and technical [3] systems. Identifying the structural mechanisms that impact performance has implications for understanding a wide variety of both natural and artificial learning systems.

Informing: A Cognitive Load Perspective

Introduction Informing science investigates how to provide clientele with information in a form, format, and schedule that maximizes its effectiveness (Cohen, 1999; 2009). According to its general, cross-disciplinary definition, information is a feature of objects of different nature that describes their structural aspects represented by patterns attributed to their organization (e.g., Stonier, 1997). Consequently, informing could be regarded as spreading structural patterns (patterns in form) among objects of different nature (Gackowski, 2009). In case of complex information systems such as living organisms (including humans), the aim of informing has been defined as expanding their control over environment (Gackowski, 2009) or, in evolutionary terms, enhancing their chances of survival. When informing is considered at the level of human information processing, the structure and characteristics of human cognitive architecture could have significant implications for the informing process. Cognitive aspects of human information processing may critically affect both the informing (informer) and perceiving (client) ends of the informing process. The intersection of cognitive and informing sciences has been related to the area of cognitive informatics (Cohen, 2009). Cognitive load theory as a branch of instructional psychology (see Sweller, 2003; Sweller, Ayres, & Kalyuga, 2011, for recent overviews) has applied cognitive science to enhance effectiveness and efficiency of instructional design. Since the major aim of instructional design is providing instructional formats and procedures that maximize learning, this could be regarded as a specific area of application of general principles of informing science with learners as clientele. Therefore, instructional and informing sciences are inherently connected. Theoretical frameworks and recommendations of informing science should be applied to instructional design, and established principles of cognitive load theory could also be potentially generalized to advance certain aspects of informing science. This paper explores a possibility of extending the established conceptual framework and principles of cognitive load theory to broader and more general situations than those in teaching and learning with the aim of enhancing the effectiveness of informing. The paper begins with an overview of major assumptions and principles of cognitive load theory based on a recently proposed evolutionary perspective, describes a corresponding model of human cognitive architecture, followed first by general implications of this architecture to informing science, and then by specific recommendations for improving processes of informing. Human Cognitive Architecture Cognitive load theory describes educational implications of human cognitive architecture (Sweller, 2003, 2004). In its basic assumptions, the theory uses the information processing aspects of biological evolution by natural selection as analogical to basic characteristics of human cognition (Sweller & Sweller, 2006). It considers both biological evolution and human cognition as examples of a broader category of natural information processing systems. It is assumed that such systems function based on the following five fundamental principles (Sweller, 2003; for an overview, see Sweller et al., 2011): * The information store principle: natural information processing systems include large stores of information that govern their activities. In human cognitive architecture, long-term memory provides this function. * The borrowing and reorganizing principle: information in the store is mostly borrowed from other information stores; however, it is reorganized in the process rather than copied exactly. For example, humans imitate other people, listen, and read in order to build long-term memory. * The randomness as genesis principle: all truly novel (not borrowed) information is acquired by a random generate-and-test process. …

Natural Information Processing Systems

Natural information processing systems such as biological evolution and human cognition organize information used to govern the activities of natural entities. When dealing with biologically secondary information, these systems can be specified by five common principles that we propose underlie natural information processing systems. The principles equate: (1) human long-term memory with a genome; (2) learning from other humans with biological reproduction; (3) problem solving through random generate and test with random mutation; (4) working memory when processing novel information with the epigenetic system managing environmental information; (5) long-term working memory with the epigenetic system managing genomic information. These five principles provide an integrated perspective for the nature of human learning and thought. They also have implications for the presentation of information.

No accident: genetic codes freeze in error-correcting patterns of the standard genetic code.

The standard genetic code poses a challenge in understanding the evolution of information processing at a fundamental level of biological organization. Genetic codes are generally coadapted with, or 'frozen' by, the protein-coding genes that they translate, and so cannot easily change by natural selection. Yet the standard code has a significantly non-random pattern that corrects common errors in the transmission of information in protein-coding genes. Because of the freezing effect and for other reasons, this pattern has been proposed not to be due to selection but rather to be incidental to other evolutionary forces or even entirely accidental. We present results from a deterministic population genetic model of code-message coevolution. We explicitly represent the freezing effect of genes on genetic codes and the perturbative effect of changes in genetic codes on genes. We incorporate characteristic patterns of mutation and translational error, namely, transition bias and positional asymmetry, respectively. Repeated selection over small successive changes produces genetic codes that are substantially, but not optimally, error correcting. In particular, our model reproduces the error-correcting patterns of the standard genetic code. Aspects of our model and results may be applicable to the general problem of adaptation to error in other natural information-processing systems.

DYNAMICS FOR A FUNDAMENTAL PROBLEM OF BIOLOGICAL INFORMATION PROCESSING

This paper presents results on the dynamics of asynchronous irregular cellular automata (as representations of natural information-processing systems). It is an approach to explaining global dynamics from local dynamics without the use of unrealistic intermediate structure (i.e., without synchronization or regular communication). The unrealistic intermediate structure is replaced by the the realistic assumption that local behavior is entropy reducing (an idea of E. Schrödinger). It has been shown that, for systems composed of cells programmed as cyclic finite-state automata, the observed global oscillation can be explained in terms of the structure of attractors in the global state space. The degree of local connectivity (i.e., of communication between cells) is shown here to determine the size of global attractors, and in turn the sharpness of global behavior. However, the primary result here is the extension of these results to systems whose cells are programmed as arbitrary strongly connected automata. Finally, these phenomena are demonstrated by the simulations.

Integrated precursor architecture as a framework for molecular computer design

Molecular computers are natural and artificial information processing systems in which individual macromolecules play a critical role. Multiple architectural possibilities can be contemplated, including conventional serial systems, parallel architectures, tactilizing processors, neural and neuromolecular networks, optomolecular electronic memory media, and hybrid silicon-carbon interfaces. The computational capabilities of these systems are governed by a tradeoff principle that links programmability, computational efficiency and evolutionary plasticity in a complementary fashion. Embedding this principle in the formalism of adaptability theory yields a framework for cost-benefit analysis of molecular computers and computational systems generally. The framework utilizes a stagewise approach to molecular computer development, beginning with a simulated integrated precursor architecture and proceeding to a bonafide integrated realization. The analysis highlights powerful computational synergies that emerge from the domain complementarities of programmable and nonprogrammable modes of computing and from the fabrication complementarities of conventional and carbon-based technologies.