A study of SLIDE Algorithm: Revolutionary AI Algorithm that Speeds Up Deep Learning on CPUs

Abstract

Deep learning is the emerging technology, having endless applications and heavy investment from industries. Deep learning constructs a model from many simple computational layers stacked up, resulting in a deep network. Deep networks empower many applications in different domains.Modern advancement in deep learning have made the use of large volume of data, and problem sizes keep increasing, data sizes grow, models use more features. However, specialized hardware’s are costly and challenging to generalize majority of tasks. It is becoming more challenging for training giant Neural Networks with millions of parameters and managing high speed computational power to obtain state-of-the-art accuracy.In the era of Deep Learning, it is widely accepted that for training models, Graphics Processing Units (GPUs) are chosen over Central Processing Units (CPUs) because of powerful speed of processing. But it deals with increase in computational cost and time.In this paper cost effective alternative to train Neural network on CPU has been discussed. SLIDE (Sub-Linear Deep Learning Engine) developed by Rice University Computer Scientists provides significant performance boosts and cost savings with the help of data and model parallelism and workload optimization. SLIDE is adequate to perform its machine learning algorithms using general purpose CPUs, thus eliminating the need of any specialized graphics hardware.