Understanding the End-Users and Technical Requirements for Real-Time Streaming Data Analytics and Visualisation

Abstract

Today’s digital world deals with tremendous amounts of data with high complexity and velocity. The data being generated continuously from a variety of sources at very high speeds is considered as streaming data. The number of technologies to ingest, explore, analyze, and visualize the streaming data is growing rapidly across many industries. However, the use of real-time technologies in the Architecture, Engineering & Construction (AEC) sector is still at its early stages of development and further research is needed to make the tools truly commercial. The AEC is a data-intensive, project-based industry that relies on real-time information to help projects stay on schedule and enable companies to cut costs. The technologies supporting real-time data capability in the AEC sector must satisfy several key functional and nonfunctional requirements. This paper aims to identify and prioritize both end-users and technical requirements for AEC businesses when adopting a real-time streaming data analytics and visualization technology in their production control rooms. To this aim, an expert questionnaire survey is conducted to investigate the requirements in four distinct areas, namely: (i) data collection and ingestion, (ii) stream processing, (iii) real-time analytics, and (iv) presentation and visualization. A five-level Likert scale is used in the questionnaire to allow respondents to indicate their perceptions on the importance of each of the identified requirements. The survey is distributed using Qualtrics online tool to over twenty organizations all over the world and the responses are analyzed using SPSS statistical software. The findings of the study reveal that ‘interactivity’ and ‘user assistance’ are the most important requirements from the end-users’ perspective, whereas ‘fast data processing’, ‘real-time reporting’, ‘scalable data processing’, and ‘dashboard connectivity’ are the most important requirements from the technology developers’ perspective. The findings of this research can help stakeholders and researchers get a better picture of the current challenges facing the AEC sector in terms of streaming data analytics and provide insight into the possible directions of future technological changes.