Abstract

In this paper, we attempt to link the inner workings of a neural language model to linguistic theory, focusing on a complex phenomenon well discussed in formal linguistics: (negative) polarity items. We briefly discuss the leading hypotheses about the licensing contexts that allow negative polarity items and evaluate to what extent a neural language model has the ability to correctly process a subset of such constructions. We show that the model finds a relation between the licensing context and the negative polarity item and appears to be aware of the scope of this context, which we extract from a parse tree of the sentence. With this research, we hope to pave the way for other studies linking formal linguistics to deep learning.

Highlights

  • In the past decade, we have seen a surge in the development of neural language models (LMs)

  • We conclude that the LM is able to detect a signal that indicates a strong relationship between an negative polarity items (NPIs) and its licensing context

  • We assessed the ability of a neural LM to handle NPI constructions, based on the probabilities returned by the LM

Read more

Summary

Introduction

We have seen a surge in the development of neural language models (LMs). As they are more capable of detecting long distance dependencies than traditional n-gram models, they serve as a stronger model for natural language. It is unclear what kind of properties of language these models encode. This does hinder further progress in the development of new models, and prevents us from using models as explanatory models and relating them to formal linguistic knowledge of natural language, an aspect we are interested in in the current paper. We follow up on this research by studying a phenomenon that has received much attention by linguists and for which the model requires – besides knowledge of syntactic structure – a semantic understanding of the sentence: negative polarity items (NPIs)

Methods
Results
Conclusion
Full Text
Published version (Free)

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call