Abstract
The learning speed of online sequential extreme learning machine (OS-ELM) algorithms is much higher than that of convolutional neural networks (CNNs) or recurrent neural network (RNNs) on regression and simple classification datasets. However, the general feature extraction of OS-ELM makes it difficult to conveniently and effectively perform classification on some large and complex datasets, e.g., CIFAR. In this paper, we propose a flexible OS-ELM-mixed neural network, termed as fnnmOS-ELM. In this mixed structure, the OS-ELM can replace a part of fully connected layers in CNNs or RNNs. Our framework not only exploits the strong feature representation of CNNs or RNNs, but also performs at a fast speed in terms of classification. Additionally, it avoids the problem of long training time and large parameter size of CNNs or RNNs to some extent. Further, we propose a method for optimizing network performance by splicing OS-ELM after CNN or RNN structures. Iris, IMDb, CIFAR-10, and CIFAR-100 datasets are employed to verify the performance of the fnnmOS-ELM. The relationship between hyper-parameters and the performance of the fnnmOS-ELM is explored, which sheds light on the optimization of network performance. Finally, the experimental results demonstrate that the fnnmOS-ELM has a stronger feature representation and higher classification performance than contemporary methods.
Highlights
Classification tasks on various datasets have become a hot topic over the past decades
We extend the application of online sequential extreme learning machine (OS-extreme learning machine (ELM)) to more datasets, and our studies show that fnnmOS-ELM can optimize the network performance of convolutional neural networks (CNNs) or Recurrent neural networks (RNNs) without changing the original network structure
On the CIFAR-10 dataset, we examine the influence of hyper-parameters on the training effect in detail, and the performance of the fnnmOS-ELM model on the classification problems mentioned above did not achieve the best performance
Summary
Classification tasks on various datasets have become a hot topic over the past decades. Convolutional neural network (CNNs) [1] and many other network models are derived for feature extraction, which can directly take image data as input with their unique fine-grained feature extraction method without manual image preprocessing and other additional complex operations [2]. Recurrent neural networks (RNNs) [3] can remember the previous information and have more advantages over other network models in continuous, context-related, and feature extraction-related tasks, such as speech recognition. Similar to CNNs and RNNs, other types of neural networks have their own advantages in feature extraction, and great achievements have been made in recent studies [4,5,6,7]. Full connection layers play a major role in CNNs- or RNNs-based classifiers that use the back-propagation (BP) [8] algorithm to train networks.
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
