Mapping Distributional Semantics to Property Norms with Deep Neural Networks

Dandan Li,Douglas Summers-Stay

doi:10.3390/bdcc3020030

Dandan Li, Douglas Summers-Stay

Open Access

https://doi.org/10.3390/bdcc3020030

Copy DOI

Abstract

Word embeddings have been very successful in many natural language processing tasks, but they characterize the meaning of a word/concept by uninterpretable “context signatures”. Such a representation can render results obtained using embeddings difficult to interpret. Neighboring word vectors may have similar meanings, but in what way are they similar? That similarity may represent a synonymy, metonymy, or even antonymy relation. In the cognitive psychology literature, in contrast, concepts are frequently represented by their relations with properties. These properties are produced by test subjects when asked to describe important features of concepts. As such, they form a natural, intuitive feature space. In this work, we present a neural-network-based method for mapping a distributional semantic space onto a human-built property space automatically. We evaluate our method on word embeddings learned with different types of contexts, and report state-of-the-art performances on the widely used McRae semantic feature production norms.

Highlights

Semantic word representation plays important roles in a broad range of natural language processing (NLP) tasks, including translation, query expansion, information extraction, and question answering
There are two main branches of previous work: (1) distributional semantic models learned with different types of contexts from large text corpora [1,2,3]; (2) property-based representation in terms of constituent properties generated by participants in property norming studies [4,5], extracted from manually-curated knowledge bases, such as FreeBase and Wikidata [6], or learned from a text [7]
We explore how to build a nonlinear mapping from a distributional semantic space to a human-built property space based on multilayer perceptrons (MLPs)

Summary

Introduction

Semantic word representation plays important roles in a broad range of natural language processing (NLP) tasks, including translation, query expansion, information extraction, and question answering. Distributional semantic models characterize the meaning of a word through the contexts in which it appears These models rely on the distributional hypothesis—that words occurring in similar contexts tend to have similar meanings [8,9]. Distributional models can tell us that airplane is similar to aircraft and pilot with different similarity scores, but it is difficult to differentiate how airplane is related to aircraft from how it is related to pilot based on these models. This is one of the main drawbacks of distributional models [10]

Objectives

Methods

Conclusion