Limitations of and Lessons from the Learning of Large Language Models

Abstract

It is argued that the Curry-Howard correspondence for classical logic implies limitations for logical reasoning that can be learned and performed by large language models. The correspondence establishes an isomorphism between proofs in logic and programs in functional typed lambda calculus. While intuitionistic logic maps to a version of lambda calculus that can be carried out in a local way, i.e., considering local parts of the code in isolation, the version of lambda calculus that corresponds to classical logic requires non-local relations – and this non-locality cannot be learned by large language models due to their restriction to investigate a relative short sequence of tokens. A possible way to go beyond this limitation is sketched as well. Implications for other areas are investigated as well.