Self-Organization of Spatio-Temporal Hierarchy via Learning of Dynamic Visual Image Patterns on Action Sequences.

Jun Tani,Minju Jung,Jungsik Hwang,Xuchu Weng

doi:10.1371/journal.pone.0131214

Jun Tani, Minju Jung + Show 2 more

Open Access

https://doi.org/10.1371/journal.pone.0131214

Copy DOI

Journal: PloS one	Publication Date: Jul 6, 2015
Citations: 55	License type: CC BY 4.0

Affiliation: Korea Advanced Institute of Science and Technology

Abstract

It is well known that the visual cortex efficiently processes high-dimensional spatial information by using a hierarchical structure. Recently, computational models that were inspired by the spatial hierarchy of the visual cortex have shown remarkable performance in image recognition. Up to now, however, most biological and computational modeling studies have mainly focused on the spatial domain and do not discuss temporal domain processing of the visual cortex. Several studies on the visual cortex and other brain areas associated with motor control support that the brain also uses its hierarchical structure as a processing mechanism for temporal information. Based on the success of previous computational models using spatial hierarchy and temporal hierarchy observed in the brain, the current report introduces a novel neural network model for the recognition of dynamic visual image patterns based solely on the learning of exemplars. This model is characterized by the application of both spatial and temporal constraints on local neural activities, resulting in the self-organization of a spatio-temporal hierarchy necessary for the recognition of complex dynamic visual image patterns. The evaluation with the Weizmann dataset in recognition of a set of prototypical human movement patterns showed that the proposed model is significantly robust in recognizing dynamically occluded visual patterns compared to other baseline models. Furthermore, an evaluation test for the recognition of concatenated sequences of those prototypical movement patterns indicated that the model is endowed with a remarkable capability for the contextual recognition of long-range dynamic visual image patterns.

Highlights

Using a hierarchical structure has two main advantages in terms of computational efficiency and generalization mentioned by Kruger et al [1]
The core hypothesis driving the model is that the spatio-temporal hierarchy required for contextual recognition of dynamic visual image patterns can be self-organized by imposing multiple scales of spatial and temporal constraints on the neural activity during the supervised training of a set of exemplars
The capabilities of spatial and temporal processing in the convolutional neural network (CNN) and leaky integrator models have been extensively explored, the current study represents the first time that these two principles have been combined into one single model, the multiple spatio-temporal scales neural network (MSTNN)

Summary

Introduction

Using a hierarchical structure has two main advantages in terms of computational efficiency and generalization mentioned by Kruger et al [1]. By sharing the functionalities of lower levels, a hierarchical structure drastically reduces the computational cost compared to a flat structure which processes all tasks independently. The shared functionalities of the lower levels can be generalized because these are participating in many different tasks. The brain utilizes a hierarchy structure, especially in the visual cortex [1, 2]. PLOS ONE | DOI:10.1371/journal.pone.0131214 July 6, 2015

Methods

Results

Conclusion