Year-end Sale % OFF 
Abstract
Artificial neural network (ANN) has successfully provided solutions to many practical problems. One of the difficulties in training ANNs is finding the ideal solution to the network weights quickly. This paper designs an implementation of the hybrid particle swarm optimization (PSO) and quasi-Newton (QN) algorithm on CPU-GPU platform using OpenCL to accelerate ANN training. The PSO-QN implementation combines the strength of the PSO algorithm in global search and the advantage of the QN algorithm in fast convergence rate. A configurable parallel line search implementation and an efficient parallel reinitialization implementation are proposed to improve the performance and reduce data transmission. Experiments show the PSO-QN hybrid parallel implementation on CPU-GPU platform can achieve up to $362\times$ and $8.9\times$ acceleration compared with the C++ implementations of PSO and BFGS-QN on CPU, respectively. Compared with the PSO and BFGS-QN parallel implementations, the training loss of the PSO-QN hybrid implementation at given training time is reduced by 15.16% and 3.97%, and the testing error is reduced by 13.66% and 6.86%, respectively.
Highlights
Research on artificial neural network (ANN) has made great progress, and provides solutions for many practical problems in early detection [1], microwave circuit design [2], energy expenditure estimation [3], etc.ANN still faces difficulties in practical applications, one of which is training
To deal with the local optimum problem, global optimization algorithms like particle swarm optimization (PSO) algorithm [6] are used for ANN training
Compared with the PSO and BFGS-QN parallel implementations, the training loss of the PSO-QN hybrid implementation at given training time is reduced by 15.16% and 3.97%, and the testing error is reduced by 13.66% and 6.86%, respectively
Summary
Research on artificial neural network (ANN) has made great progress, and provides solutions for many practical problems in early detection [1], microwave circuit design [2], energy expenditure estimation [3], etc.ANN still faces difficulties in practical applications, one of which is training. Research on artificial neural network (ANN) has made great progress, and provides solutions for many practical problems in early detection [1], microwave circuit design [2], energy expenditure estimation [3], etc. Each iteration of the algorithm is carried out along the direction in which the loss function drops the fastest. This algorithm is easy to fall into local optimum and has slow convergence rate [5]. To deal with the local optimum problem, global optimization algorithms like particle swarm optimization (PSO) algorithm [6] are used for ANN training. As a population based stochastic optimization technique, the PSO algorithm has global search ability which increases the probability of finding the global optimal solution. The PSO algorithm requires a large number of particles to participate in the computation, The associate editor coordinating the review of this manuscript and approving it for publication was Aysegul Ucar
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
