AI education matters: Model AI assignment

Abstract

Many of the most significant breakthroughs in artificial intelligence over the past decade have been based on progress in deep neural networks. That progress has been facilitated by deep-learning libraries like Theano (Al-Rfou et al., 2016), TensorFlow (Abadi et al., 2015) and PyTorch (Paszke et al., 2019) that allow rapid prototyping and efficient execution. The key algorithm at the heart of all of these libraries is reverse-mode automatic differentiation. This column introduces the Model AI Assignment ScalarFlow: Implementing Reverse Mode Automatic Differentiation. This assignment gives students the opportunity to gain a deeper understanding of modern deeplearning frameworks by building their own automatic differentiation engine and using it to experiment with some important concepts in deep learning. In this column we will review some basic background on training neural networks, provide a brief overview of the reverse-mode automatic differentiation algorithm, describe the model assignment and provide some pointers to additional resources.