Distributed Reduced Kernel Extreme Learning Machine

Abstract

Extreme learning machines (ELM) are popular in the field of pattern recognition and machine learning. The kernel extension of ELM (KELM) presents a better performance than traditional ELM. Although the KELM is able to solve complex nonlinear problems, it is time-consuming and memorydemanding when dealing with a large-size kernel matrix. Introducing reduced kernel technique can dramatically cut down the computational load and memory usage. However, as the amount of training data grows exponentially, it is not effective for a single worker to store the kernel matrix, making it infeasible for data mining in a centralized manner. In this paper, a distributed reduced kernel method for training ELM over decentralized data (DRKELM) is proposed. In the DRKELM, we randomly assign data to different nodes. The communication between nodes is fixed and does not depend on the size of the training data on each node, but on the network topology. Different from the existing reduced kernel ELM, the DRKELM is a fully distributed training algorithm based on the method of alternating direction method of multiplier (ADMM). Experiment with the large scale data set finds that the distributed method can achieve almost the same results as the centralized algorithm and even takes less time to a large extent. It greatly reduces the computation time consumption.