A hierarchical memory network-based approach to uncertain streaming data

Abstract

Various interferences can cause uncertainties such as missing data, outliers, noise, and redundancies that are persistent either in stationary or streaming data. In such case, preprocessing methods are widely accepted, of which PCA (Principal Component Analysis), filter, and Bayesian are most popular. As batch learning schemes, they work well particularly in the stationary environment through making correction and revision for bad samples. However, for the streaming data an in-time response for the unusual is needed, where an online learning scheme is even more suitable. As we have known, human brain is a highly complex and intelligent information processing organ, in which memory serves as a crucial role. The more repeated information will be remembered more deeply, and the less repeated will be forgot soon. Considering the mechanisms of memory such as remembering, forgetting, and recalling, a human memory-inspired approach is proposed in this paper. Differ from any mathematic methods, it is an instance-replaced approach by retrieving superior historical instances in memory library to replace the abnormal ones. Through modeling these mechanisms and adding different priorities for each instance on storing and retrieving, a hierarchical memory network (HMN) is constructed, which contains three levels, namely, perpetual, long-term, and short-term. In HMN, data instances migrate dynamically to change their hierarchies to adapt changing circumstances. The perpetual level stores data with the deepest memory and never forgets them again. The long-term level interacts with other two levels through migrating instances with deeper memory into perpetual memory and forgetting shallower ones back to the short-term level. Likewise, data in short-term level will be migrated into long-term memory once up to a predefined threshold, meanwhile some instances under the lowest index will be forever discarded. Benchmark and real-world industrial datasets are utilized to build and test HMN, and simulation results verify its effectiveness.