Multiple Observations HMM Learning by Aggregating Ensemble Models

Abstract

Hidden Markov Model (HMM) is a widespread statistical model used in cases where the system involves not fully observable data sequences such as temporal pattern recognition and signal processing. The most difficult problem in dealing with HMMs is the training procedure, or parameter learning, for which several approaches has been proposed. Nevertheless, these methods suffer from trapping in local maxima and still no tractable algorithm is present to overcome this problem. On the other hand, good performances of ensemble methods, where multiple models are employed to obtain the target model, lead to considering ensemble learning in the HMM training problem. Until now, just a few ensemble methods have been proposed for HMMs which lack strong theoretical background, or do not involve all the basic models to construct the final model. Hence in this paper a new ensemble learning method for HMMs is proposed which takes advantage of information theory measures, specifically Rényi entropy, and addresses the mentioned problems of previous methods. In agreement with this claim, the results show superiority of the proposed method over other compared methods for both synthetic and real-world datasets. Besides, the proposed ensemble method succeeded to meet the performance of other methods with much lower required training samples.