Mapping Lexical Dialect Variation in British English Using Twitter.

Jack Grieve,Chris Montgomery,Andrea Nini,Akira Murakami,Diansheng Guo

doi:10.3389/frai.2019.00011

Abstract

There is a growing trend in regional dialectology to analyse large corpora of social media data, but it is unclear if the results of these studies can be generalized to language as a whole. To assess the generalizability of Twitter dialect maps, this paper presents the first systematic comparison of regional lexical variation in Twitter corpora and traditional survey data. We compare the regional patterns found in 139 lexical dialect maps based on a 1.8 billion word corpus of geolocated UK Twitter data and the BBC Voices dialect survey. A spatial analysis of these 139 map pairs finds a broad alignment between these two data sources, offering evidence that both approaches to data collection allow for the same basic underlying regional patterns to be identified. We argue that these results license the use of Twitter corpora for general inquiries into regional lexical variation and change.

Highlights

Regional dialectology has traditionally been based on data elicited through surveys and interviews, but in recent years there has been growing interest in mapping linguistic variation through the analysis of very large corpora of natural language collected online
Computing Lee’s L using 2, 5, and 20 nearest neighbors produced similar results, with all analyses finding that 78–80% of the map pairs exhibit positive correlations, and with the Lee’s L values across all 139 pairs of maps exhibiting strong correlations (r > 0.89), indicating the choice of spatial weights matrix does not have a large effect on our results
Twitter corpora are increasingly being used as the basis for dialect maps, their generalizability had not been established

Summary

Introduction

Regional dialectology has traditionally been based on data elicited through surveys and interviews, but in recent years there has been growing interest in mapping linguistic variation through the analysis of very large corpora of natural language collected online Such corpus-based approaches to the study of language variation and change are becoming increasingly common across sociolinguistics (Nguyen et al, 2016), but have been adopted most enthusiastically in dialectology, where traditional forms of data collection are so onerous. There have only been a handful of surveys completed in the UK and the US in over a century of research These studies have been immensely informative and influential, shaping our understanding of the mechanisms of language variation and change and giving rise to the modern field of sociolinguistics, but they have not allowed regional dialect variation to be fully understood, especially above the levels of phonetics and phonology. As was recently lamented in the popular press (Sheidlower, 2018), this shift from dialectology as a social science to a data science has led to a less personal form of scholarship, but it has reinvigorated the field, democratizing dialectology by allowing anyone to analyse regional linguistic variation on a large scale

Objectives

Methods

Results

Conclusion