A cognitive approach to autonomous incremental learning

Abstract

Contemporary work in learning algorithms has eclipsed the natural identity of lifelong learning. It is relevant to pursue an investigation of studies in cognitive and neurosciences in search of plausible means of artificial actualisation of this natural identity. The primary research interest of this thesis lies therein; the development of a biologically inspired self-learning algorithm with a knowledge acquiring disposition to address key concerns affecting continuity of learning, namely, catastrophic interference, stability-plasticity dilemma and lack of knowledge representation. The thesis documents the design, development and implementation of the Incremental Knowledge Acquiring Self-Learning (IKASL) algorithm followed by a convincing record of experiments and outcomes in incremental learning problems. Abstractions of the thought process from cognitive psychology motivated the morphological development of the IKASL algorithm while readings of neuroscientific discoveries instigated the development of its functionality. The primary application areas of the IKASL algorithm are incremental learning, sequential learning and knowledge accumulation problems in data mining and knowledge discovery. The IKASL process initiates with a dynamic topological organisation of input data. Subsequently, primary and secondary learning outcomes are extracted from the topological representations of the dynamic feature map. Identified as aggregate nodes of a generalisation layer, they resemble pyramidal neurons in the neocortex in both form and function as they contribute to sustain and continue the incremental learning process. Generalised outcomes from each phase of learning are maintained in an evolving columnar manner, an arrangement motivated by the columnar organisation of neurons in the neocortex. With the continuation of learning, the initial columns evolve into sub-columns. Thus the potential to super-impose new properties atop this columnar structure and the facility to examine these both medially and laterally, can facilitate the development of an evolving repository of knowledge with awareness of the problem space. The significance of associating continuously received sensory inputs with already acquired knowledge and the build up of knowledge from specific to general has been widely emphasized in cognitive psychology research. The multiplicity of continuous learning problems, specifically in streaming environments, influenced the design and development of an extension to the IKASL algorithm; the IKASL-stream algorithm. In addition to IKASL features, the IKASL-stream algorithm addresses key constraints in an online streaming scenario, data transience, randomness and processing limitations. Overall, the IKASL algorithm can be identified as a perpetual self-learning algorithm with a dynamic structure for acquisition and preservation of learned knowledge in a computationally efficient columnar architecture. It regulates generalised representations of past learning as the basis for subsequent learning and utilises accumulated knowledge to determine relevance and correlations to new inputs and thus facilitates the development of awareness of the problem space.