Application of hyper-convergent platform for big data in exploring regional innovation systems

Abstract

The authors developed a decentralised hyper-convergent analytical platform for the collection and processing of big data in order to explore the monitoring processes of distributed objects in the regions on the basis of multi-agent approach. The platform is intended for modular integration of tools for searching, collecting, processing and big data mining from cyber-physical and cyber-social objects. The results of the intellectual analysis are used to assess the integrated criteria for the effectiveness of innovation systems of distributed monitoring and forecasting the dynamics of the influence of various factors on technological and socio-economic processes. The work analyses convergent and hyper-convergent systems, substantiates the necessity of creating a multi-agent decentralised platform for big data collection and analytical processing. The article proposes the principles of streaming architecture for the data integration analytical processing to resolve the problems of searching, parallel processing, data mining and uploading of information into a cloud storage. The paper also considers the main components of the hyper-convergent analytical platform. A new concept of distributed extraction, transformation, loading, mining (ETLM) system is considered.