What Is Missing from Contemporary AI? The World

Abstract

In the past three years, we have witnessed the emergence of a new class of artificial intelligence systems–—so-called foundation models, which are characterised by very large machine learning models (with tens or hundreds of billions of parameters) trained using extremely large and broad data sets. Foundation models, it is argued, have competence in a broad range of tasks, which can be specialised for specific applications. Large language models, of which GPT-3 is perhaps the best known, are the most prominent example of current foundation models. While foundation models have demonstrated impressive capabilities in certain tasks—natural language generation being the most obvious example—I argue that because they are inherently disembodied, and they are limited with respect to what they have learned and what they can do. Foundation models are likely to be very useful in many applications: but they are not the end of the road in artificial intelligence.