Cholesky factorization based online regularized and kernelized extreme learning machines with forgetting mechanism

Abstract

In this paper, we propose two alternative schemes of fast online sequential extreme learning machine (ELM) for training the single hidden-layer feedforward neural networks (SLFN), termed as Cholesky factorization based online regularized ELM with forgetting mechanism (CF-FORELM) and Cholesky factorization based online kernelized ELM with forgetting mechanism (CF-FOKELM). First, the solutions of regularized ELM (RELM) and kernelized ELM (KELM) using the matrix Cholesky factorization are introduced; then the recursive method for calculating Cholesky factor of involved matrix in RELM and KELM is designed when RELM and KELM are applied to train SLFN online; consequently, the CF-FORELM and CF-FOKELM are obtained. The numerical simulation results show CF-FORELM demands less computational burden than Dynamic Regression ELM (DR-ELM), and CF-FOKELM also owns higher computational efficiency than both FOKELM and online sequential ELM with kernels (OS-ELMK), and CF-FORELM is less sensitive to model parameters than CF-FOKELM.