Transfer Extreme Learning Machine with Output Weight Alignment.

Shaofei Zang,Yongyi Yan,Yuhu Cheng,Xuesong Wang

doi:10.1155/2021/6627765

Abstract

Extreme Learning Machine (ELM) as a fast and efficient neural network model in pattern recognition and machine learning will decline when the labeled training sample is insufficient. Transfer learning helps the target task to learn a reliable model by using plentiful labeled samples from the different but relevant domain. In this paper, we propose a supervised Extreme Learning Machine with knowledge transferability, called Transfer Extreme Learning Machine with Output Weight Alignment (TELM-OWA). Firstly, it reduces the distribution difference between domains by aligning the output weight matrix of the ELM trained by the labeled samples from the source and target domains. Secondly, the approximation between the interdomain ELM output weight matrix is added to the objective function to further realize the cross-domain transfer of knowledge. Thirdly, we consider the objective function as the least square problem and transform it into a standard ELM model to be efficiently solved. Finally, the effectiveness of the proposed algorithm is verified by classification experiments on 16 sets of image datasets and 6 sets of text datasets, and the result demonstrates the competitive performance of our method with respect to other ELM models and transfer learning approach.

Highlights

Neural networks for solving classification problems have been widely researched in recent years [1, 2], which has powerful nonlinear fitting and approximation capabilities
Ough DAELM-S uses ‖HSβS − YS‖2 to transfer the knowledge from the source domain and increases the fitness of βS to source data, this decreases the fitness to the target domain comparing with TELM-OWA in which βa is more approximated to βT than βS by applying a subspace alignment mechanism
(ii) Transfer Component Analysis (TCA) [52] + SVM: classifier is built by combining TCA with SVM for the classification task of transfer learning

Summary

Introduction

Neural networks for solving classification problems have been widely researched in recent years [1, 2], which has powerful nonlinear fitting and approximation capabilities. Extreme Learning Machine (ELM), as a Single-Layer Feedforward Network (SLFN), has been proven to be an effective and efficient algorithm for pattern classification and regression [3, 4] It randomly generates the input weight and bias of the hidden layer without tuning and only updates the weight between the hidden layer and the output layer. With the regular least squares (or ridge regression) as prediction error, the output weight will be efficiently obtained in a closed form by Moore–Penrose generalized inverse [3] As a result, it has the advantages of strong generalization ability and fast training speed, and it has been widely used in various applications, such as face recognition [5], brain-computer interfaces [6,7,8,9], hyperspectral image classification [10], and malware hunting [11]. Raghuwanshi and Shukla [13] presented a novel SMOTE based Class-Specific Extreme Learning Machine (SMOTE-CSELM), a variant of ClassSpecific Extreme Learning Machine (CS-ELM), which exploits the benefit of both the minority oversampling and the class-specific regularization and has more comparable computational complexity than the Weighted Extreme

Results

Discussion

Conclusion