Reservoir computing ensembles for multi-object behavior recognition

Abstract

Most available methods in computer vision can only detect one behavior at a time in a video sequence. Multi-object behavior recognition is still a very challenge problem. In this paper, we propose a novel model based on reservoir computing ensembles for multi-pattern recognition. In this new model, multiple interactive sub-reservoirs are connected to construct the reservoir model. The sub-reservoirs are competing with each other through inhibitory connections, and the internal states of all the sub-reservoirs are combined to form the output action potentials. Neurobiological studies have shown that cortical neural networks have a distinctive modular and laminar structure, which can provide powerful computational function. Therefore, cortical neural networks are employed to construct each sub-reservoir, whose parameters can be dynamically tuned by a gene regulatory network (GRN). Extensive experimental results on the MSR action dataset II have demonstrated the feasibility and efficiency of the proposed reservoir computing ensembles model on multi-object behavior recognition in video sequences.