The Implementation of Gradient Descent Based Methods Using Parallel Computing in R for Regression Tasks

Abstract

In the previous research, it has been developed the R package gradDescent, which contains the gradient descent algorithm and more than 10 methods of its variations. However, all algorithms in the package can only run in the standalone/single core mode. On the other hand, the amount of data processed in the future tends to increase. Therefore, this research is aimed to develop the previous package by extending 4 algorithms (i.e., mini batch gradient descent (MBGD), stochastic gradient descent (SGD), stochastic average gradient descent (SAGD), and accelerated gradient descent (AGD)) with the R high-performance computing packages: foreach, doSNOW, and pbdMPI, in order to have the ability to process data in multicore / parallel computing. By utilizing the parallel-computing tools, the computation cost can be reduced. After designing and implementing the proposed model, some simulations were conducted by using the California Housing dataset. Additionally, some comparisons with other software libraries were presented. The results show that the implementation of the proposed model provided competitive root mean squared errors and smallest computation cost.