Reframing Difficult Deep Learning Problems

Abstract

To speak very roughly, the deep learning process can be represented with three central sequentially connected components: the creation of the architecture, the definition of training procedures and methods, and the training of the model itself. This is the general deep learning modeling workflow. Each of the chapters this book has covered up to this point can be categorized approximately into at least one of these categories. Chapter 2, on transfer learning and pretraining, discussed methods of training models to transfer and develop knowledge from sources other than the original dataset. Chapter 3, on autoencoders, discussed various usages of the versatile autoencoder concept and architecture. Chapter 4, on model compression, discussed various modifications to the neural network architecture, in addition to alterations in the training procedure. Chapter 5, on meta-optimization, discussed the automation of parameters in neural network architecture and training procedures. Chapter 6, on successful neural network design, discussed successful design patterns and techniques in neural network architectures and model implementation.