Chapter 5 - Training neural network

Abstract

Training a neural network is a process of finding a suitable set of weights that results in high performance on training and validation data. This chapter provides some of the most significant techniques that used in training network models, including backpropagation, weight initialization, and batch normalization. Then, we examine the effectiveness of each weight initialization technique and verify batch normalization by applying it to train the neural network model on the CIFAR-10 dataset. Finally, we summarize, compare, and visualize the performance of all the trained neural networks in Chapter 4 and this chapter on TensorBoard to see the impact of each technique in the convergence of the model during the training process.