An information-theoretic view on language complexity and register variation: Compressing naturalistic corpus data

Abstract

AbstractThis article utilises an innovative, information-theoretic metric to assess complexity variation across written and spoken registers of British English. This is novel because previous research on language complexity mainly analysed complexity variation in typological data, single language case studies or geographical varieties of the same language. The measure boils down to Kolmogorov complexity which can be conveniently approximated with off-the-shelf compression programs. Essentially, text samples that can be compressed more efficiently count as linguistically simple. The dataset covers a wide range of traditional written and spoken registers (e.g. broadsheet newspapers, courtroom debate or face-to-face conversation), as sampled in theBritish National Corpus. It turns out that Kolmogorov-based register variation coincides with register formality such that informal registers are overall and morphologically less complex than more formal registers, but more complex in regard to syntax (defined here as rigid word order). Generally, the results show that written and spoken registers vary along a continuum, and significantly trade-off morphological against syntactic complexity (and vice versa). Finally, the findings support proposals to view language as a complex adaptive system and demonstrate how language adapts to the situational context of language production and functional-communicative needs of its users.