Chapter 4 - Fully connected neural networks

Abstract

This chapter will introduce a type of neural network with the most basic structure, i.e., fully connected neural network or feed-forward neural network (FNN). At the early stage, when people say artificial neural network, they are normally referring to FNN. FNN with fully connections between neurons in different layers of the neural network can solve various tasks, like regression and classification tasks. It is worth to mention that both linear regression and logistic regression are building blocks of FNN layers, and these layers are the basic of deep neural networks. Thus, in this chapter we will introduce the basic structure of FNN starting, from linear regression to linear neural networks. Key components of deep neural networks, including layers, activation functions, and loss functions, are introduced. The back-propagation method as the core of neural network training is also elaborated to shed light on the power of deep neural networks. Furthermore, this chapter demonstrates transportation applications that can be easily solved by FNN, including traffic-state prediction and traffic-sign classification.