Acceleration of Neural Network training algorithms via FPGA devices

Abstract

FPGA devices have great benefits over traditional computational hardware like CPUs and GPUs that can be exploited in order to have greater performance on specialized tasks. This article aims to showcase some of these benefits and compare a small use-case with real world metrics. Previous researches have shown that ASIC devices can perform better on specialized tasks due to low latency but an FPGA device might be the key in developing a reconfigurable hardware that suits most Neural Network training algorithms for development purposes, without compromising performance. The test environment results show clearly the benefits and pitfalls of this approach. Using custom, modular processing cores can help developers in creating their own processing units that enhance the developer experience and have the benefit of faster processing times. In order to prove these claims we implement a few of the proposed cores, test them against conventional CPUs and GPUs and showcase the potential flexibility during development. The results highlight that FPGA devices can be faster than conventional processing units and developers can increase their productivity via the performance gains, but for the best results, the hardware choices need to be made according to project and budget constraints.