Opportunities in Big Data Management and Processing

Abstract

Every day we witness new forms of data in various formats. Some example include structured data from transactions we make, unstructured data as text communications of different kinds, varieties of multimedia files and video streams. To ensure efficient processing of this data, often called 'Big Data', the use of highly distributed and scalable systems and new data management architectures, e.g. distributed file systems and NoSQL database, has been widely considered. However, volume, variety and velocity of Big Data and data analytics demands indicate that these tools solve Big Data problems partially only. To make effective use of such data, novel concepts of management is required, in particular, being able to efficiently access data within a tolerable time not only from the users own domain, but also from other domains which the user is authorized to access. This paper elaborates on some issues related to the efficient management of Big Data, discusses current trends, identifies the challenges and suggests possible research directions.