Challenges and prospects in big data analytics: a comprehensive review of developments, hurdles, and future research directions

Abstract

Big data and business analytics are trends that are positively affecting the business world. This comprehensive review article explores the shifting paradigms and dynamic trends within Big Data Technology (BDT), predominantly for last 5 years, based on an extensive literature review and comparative analysis methodology. It elucidates the transformative influence of big data analytics (BDA) in various sectors, emphasizing the rapid ascendance of cloud computing, Artificial Intelligence (AI) integration, and development of sophisticated analytics tools. The review leverages a wealth of academic literature and market research to underscore the predicted expansion of the big data market. This projected growth indicates the widespread adoption of BDT across industries, with healthcare becoming a significant consumer, motivated by the demand for personalized medicine and improved patient care. The review then navigates emerging trends such as open data usage and ethical concerns surrounding big data, indicating the increasing necessity for stringent guidelines for data use and robust individual data control mechanisms. This is derived from a methodical analysis of recent scholarly articles and industry reports. The article also scrutinizes the evolving definition of "big data" through comparative study of the 3V model and the expanded 7V model in various literature sources, reflecting the evolving nature of data and the unique challenges introduced by modern big data analytics. The review also outlines the challenges for successful implementation of big data projects and highlights the current open research directions of big data analytics. The reviewed areas of big data suggest that good management and manipulation of the large data sets using the techniques and tools of big data can deliver actionable insights that create business values.