Design and Implementation of an Automatic Summarizer Using Extractive and Abstractive Methods

Abstract

Communication via a natural language requires two fundamental skills: producing ‘text’ (written or spoken) and understanding it. Here, the two terms, natural language processing (NLP) and natural language generation (NLG) become important. NLU, also known as natural language understanding, is where the system understands to disambiguate the input sentences (in some human or natural language) to produce the machine representation language. NLG, on the other hand, is the technique of generating natural language from a machine representation system (a database/logical form). An example of a simple NLG system is the Pollen Forecast for Scotland system (Turner et al., in generating spatio-temporal descriptions in pollen forecasts 2006, [1]) that could essentially be a template. NLG system takes as input six numbers, which predicts the pollen levels in different parts of Scotland. From these numbers, a short textual summary of pollen levels is generated by the system as its output. In this paper, our aim is to test and/or improve the preexisting algorithms for our NLG system and if required, to develop our own algorithms for the said system. The paper starts, first defining the stages and components of the NLG task and their distinctive roles in accounting for the coherence and appropriateness of natural texts. It then sets out the principal methods that have been developed in the field for building working computational systems. Thereafter, the attempts to define a new method for the application being developed are shown. Finally, the problem faced in developing an NLG system and potential applications are discussed.