Information Retrieval and Sentimental Analysis with Databricks

Abstract

With the rapid development of cloud computing, the information increases rapidly. Cheap cloud storage and computing power accelerate the development of massive data, and make the large data information collection and knowledge retrieval become necessary. The percentage of unstructured big data is more than 50%, so it is stored in the form of a file for the most part. Big data is split into many blocks that are stored within the server with some corresponding metadata of storage on the master server. How to collect the big data and keywords and retrieve the information are discussed in this paper. The proposed methodology is the information retrieval and sentimental analysis mechanism to improve text information retrieval and opinion mining using databricks. Information retrieval and analysis become more popular research fields within the world. Big data is a collection of heterogeneous structured, unstructured and semi-unstructured data. The basic aim of this paper is to present a broad picture of massive Data and to point out how information are often retrieved using evolutionary computation techniques i.e. databricks that help in the information retrieval process in a better way compared to traditional retrieval techniques. This paper also covered the setup and configuration of databricks.