The generalized extreme learning machines: Tuning hyperparameters and limiting approach for the Moore–Penrose generalized inverse

Abstract

In this paper, we propose the generalized extreme learning machine (GELM). GELM is an ELM that incorporates the analyzed hyperparameters of ELM, such as sizes and ranks of weight matrices, and a limiting approach for the Moore–Penrose generalized inverse (M–P GI) into the learning process. ELM overcomes shortcomings of traditional deep learning, such as time-consuming due to iterative executions, as it learns quickly by removing the adjustment time of hyperparameters. There are desirable numbers of hidden nodes in ELM for single hidden layer feedforward neural networks, minimizing prediction error. However, it is difficult to use the desired number because it is related to the number of data used and datasets tend to be large. We consider ELM for multiple hidden layer feedforward neural networks. We analyze matrices derived in the network and figure out the characteristics of weight matrices and biases considering accurate prediction and learning speed, based on mathematical theories and a limiting approach for the M–P GI. The final output matrix of GELM is formulated explicitly. Experiments are conducted to verify the analysis using network traffic data, including DDoS attacks. The performances of GLEM, such as accuracies and learning speed, are compared for the networks with single and multiple hidden layers. Numerical results show the advantages of GELM in the performance measures, and the use of multiple hidden layers in GELM does not significantly affect performance. The theory-based prediction performances obtained from GELM will be the criterion for the margin of deep learning performance.