Abstract
SummaryExtreme learning machine (ELM) is a prominent example of neural network with its fast training speed, and good prediction performance. An online version of ELM called online sequential extreme learning machine (OS‐ELM) has also been proposed for the sequential training. Combined with the need for regularization to prevent over‐fitting in addition to the large number of neurons required in the hidden layer, OS‐ELM demands huge amount of computation power for the large‐scale data. In this article, a mixed norm () regularized online machine learning algorithm (MRO‐ELM) that is based on alternating direction method of multipliers (ADMM) is proposed. A linear combination of the mixed norm and the Frobenius norm regularization is applied using the ADMM framework and update formulas are derived. Graphics processing unit (GPU) accelerated version of MRO‐ELM (GPU‐MRO‐ELM) is also proposed to reduce the training time by processing appropriate parts in parallel using the implemented custom kernels. In addition, a novel automatic hyper‐parameter tuning method is incorporated to GPU‐MRO‐ELM using progressive validation with GPU acceleration. The experimental results show that the MRO‐ELM algorithm and its GPU version outperform OS‐ELM in terms of training speed, and testing accuracy. Also, compared to the cross validation, the proposed automatic hyper‐parameter tuning demonstrates dramatical reduction in the tuning time.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
More From: Concurrency and Computation: Practice and Experience
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.