Chapter 6 - Recurrent neural networks

Abstract

This chapter introduces recurrent neural networks (RNN), which are suitable for dealing with sequential data. RNN and its variants have been widely used in the sequential data modeling, such as natural language processing (NLP), and other advanced neural network architectures. We begin by introducing the basic structure of RNN and two of its variants, i.e. the well-known long short-term memory network (LSTM) and the gated recurrent unit network (GRU). One important architecture for capturing the forward and backward temporal dependencies in time series, i.e., the bidirectional structure, is introduced that uses the bidirectional LSTM as an example. To learn a representation of time series data and better characterize the dependencies between input and output sequences, the widely used sequence-to-sequence structure, also called encoder–decoder, and a critical attention mechanism are also detailed in this chapter. After the methodology introduction, RNN-related transportation tasks are presented to give the reader a sense of how to apply RNNs and related models to the real transportation applications.