Big Data Analytics, Text Mining and Modern English Language

Abstract

The modern English Language took centuries to convert from old English. The word ‘hath’ of old English for example, has taken centuries to become ‘have’ in the modern English Language. If these changes had not been occurred there would not have been the possibility of modern words. A text written in fifteen century can be difficult to read and if we go back a couple of more centuries, it would be like reading a different language. In this paper, we have used the text mining techniques to analyze the old and modern English languages. We have introduced the Common-Words Counting algorithm that identifies common words of 15th century that diminishes gradually in the later centuries. We computed the speed of linguistic changes and identified the reasons behind them. For this purpose, 34000 text books were downloaded from Project Gutenberg of different authors, between 15th to 19th centuries. These books were categorized into five centuries in the range from 15th to 19th centuries. We selected most common words from the books of 15th century and calculated their frequencies in other centuries. We calculated the sum of Term Frequency-Inverse Document Frequency (TF-IDF) of these words and proved that frequencies of words were decreasing from 15th century to 19th century with some words even disappeared in other centuries, such as ‘doth’, ‘hath’, punt, guise and ‘selfe’. We calculated the speed of changing of words using the slope formula. We proved that the words were changing during each century with the speed of changing of words being the lowest during 16th – 17th centuries and the highest during 18th – 19th centuries which shows that the old words or their spellings were changed to the modern words during 18th – 19th centuries. The industrialization, modernization, and British Empire invasion were the key factors, which changed the old English language into modern English language.