Polish Language Modelling Based on Deep Learning Methods and Techniques

Abstract

The article presents an example of deep learning methods application for language modelling in Polish. Language modelling helps to predict a sequence of recognized words or characters, and it can be used for improving speech processing and speech recognition. However, currently the field of language modelling is shifting from statistical language modelling methods to neural networks and deep learning methods. There are still many difficult problems to solve in natural language modelling. Nevertheless, deep learning methods achieve the most modern results for some specific language modelling problems. In this paper are presented the most interesting natural language modelling tasks, such as word-based and character-based language modelling, in which deep learning methods achieve some progress. New research results presented in this paper, in reference to previous articles, are focused on how to develop a character-based language model using a recurrent neural network and deep machine learning techniques. The use of both language modelling methods at the same time allow to develop hybrid language models that are characterized by even better properties and can greatly improve speech recognition. The presented results relate to the modelling of the Polish language but the achieved research results and conclusions can also be applied to language modelling application for other languages.