Fusion is great, and interpretable fusion could be exciting for theory generation: Response to Pater

Abstract

From my perspective, Pater’s (2019) target article does a great service both to researchers who work in generative linguistics and to researchers who utilize neural networks—and especially to researchers who might find themselves wanting to do both by harnessing the insights of each tradition. The fusion of theories of linguistic representation and probabilistic learning techniques has certainly led to many interesting and valuable insights about the nature of both linguistic representation and the language acquisition process. However, I feel that the most exciting aspect of Pater’s article is the increasing interpretability of neural network models, especially when combined with insights from the theoretical framework of generative linguistics. This allows for the possibility that neural networks could be used to actually generate new theories of representation. I describe how I think this theory-generation process might work with interpretable neural networks.