A joint optimization framework to semi-supervised RVFL and ELM networks for efficient data classification

Abstract

Due to the inefficiency of gradient-based iterative ways in network training, randomization-based neural networks usually offer non-iterative closed form solutions. The random vector functional link (RVFL) and extreme learning machine (ELM) are two popular randomized networks which provide us unified frameworks for both regression and multi-class classification. Currently, existing studies on RVFL and ELM focused mainly on supervised tasks even though we usually have only a small number of labeled samples but a large number of unlabeled samples. Therefore, it is necessary to make both models appropriately utilize both labeled and unlabeled samples; that is, we should develop their semi-supervised extensions. In this paper, we propose a joint optimization framework to semi-supervised RVFL and ELM networks. In the formulated JOSRVFL (jointly optimized semi-supervised RVFL) and JOSELM, the output weight matrix and the label indicator matrix of the unlabeled samples can be jointly optimized in an iterative manner. We provide a novel approach to optimize the JOSRVFL and JOSELM objective functions. Extensive experiments on benchmark data sets and Electroencephalography-based emotion recognition tasks showed the excellent performance of the proposed JOSRVFL and JOSELM models. Moreover, because the direct input–output connections help to regularize the randomization, JOSRVFL could obtain superior performance to JOSELM in most cases.