Study of neural net training methods in parallel and distributed architectures

Abstract

Artificial Neural Nets are among the most commonly used methods in high-energy applications for data pre-processing. The training phase of the ANN is critical in obtaining a net that can generalize the available data for use in new situations. However, from the computational viewpoint this phase is very costly and resource intensive. Therefore, the aim of this work is to parallelize and evaluate the performance and scalability of the kernel of a training algorithm of a multilayer perceptron artificial neural net used for analyzing data from the Large Electron Positron Collider at CERN. The training methods selected were linear-BFGS and hybrid linear-BFGS. Different approaches for the parallel implementation will be presented and evaluated in this paper. In order to perform a complete performance and scalability evaluation of the proposed approach, three different parallel architectures will be used: A shared memory multiprocessor, a cluster and a grid environment.