Embedding Support Vector Machines into Localised Mixtures of Experts

Abstract

Aiming at distilling complex problems into simple subtasks, a mixture of expert (ME) may be implemented so that a gating network is in charge of learning how to softly divide the input space into overlapping regions to be each assigned to one or more expert networks. In this work, we focus mostly on the development and initial assessment of a new approach, named Localised Mixture of Support Vector Machine Experts (L-MSVME), that defines an SVM-based extension of localised ME models. Mixture models will then be used both to express prior belief in the existence of more than one process in the data generation model, and to provide an efficient method for increasing modelling power. An algorithm based on the maximum likelihood criteria (EM) was considered for training, and we demonstrate that it is possible to train each ME expert through an SVM perspective. This process also allows the decoupled training of gating and expert networks. Simulation results were then considered for performance assessment.