Abstract
Most scientists are accustomed to make predictions based on consolidated and accepted theories pertaining to the domain of prediction. However, nowadays big data analytics (BDA) is able to deliver predictions based on executing a sequence of data processing while seemingly abstaining from being theoretically informed about the subject matter. This paper discusses how to deal with the shift from theory-driven to process-driven prediction through analyzing the BDA steps and identifying the epistemological challenges and various needs of theoretically informing BDA throughout data acquisition, preprocessing, analysis, and interpretation. We suggest a theory-driven guidance for the BDA process including acquisition, pre-processing, analytics and interpretation. That is, we propose—in association with these BDA process steps—a lightweight theory-driven approach in order to safeguard the analytics process from epistemological pitfalls. This study may serve as a guideline for researchers and practitioners to consider while conducting future big data analytics.Graphical abstractEpistemological challenges of big data analytics.
Highlights
Scientists are accustomed to make predictions based on consolidated and accepted theories pertaining to the domain of prediction
Addressing the epistemological challenges Trying not to let big data analytics (BDA) fall into empiricism, we looked at each BDA step and the related critical question in performing this step in order to identify the main epistemological challenges
Outliers detection In the following figure, if we look at the top-right most point, it seems like an outlier, since we are looking with regards to how far from rest of the data points which are depicted in two axes
Summary
Scientists are accustomed to make predictions based on consolidated and accepted theories pertaining to the domain of prediction. Nowadays big data analytics (BDA) is able to deliver predictions based on executing a sequence of processing while seemingly abstaining from being theoretically informed about the subject matter. Seizing these new opportunities is tempting: some researchers have been trapped by the sheer amount of datasets made available by leading data-driven companies, which are either directed towards the companies’ own prosperity or representing rather small subsets (e.g. of users). Big data analytics research applies machine learning, data mining, statistics, and visualization techniques in order to collect, process, analyze, visualize, and interpret results [1]. BDA, as a process, is based on many disciplines that analyze data to elucidate hidden
Sign-in/Register to view highlights & summary 
Published Version (
Free)
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call