Big data analytics approach for network core and edge applications

Abstract

The amount of data in the aerospace industry is being generated and collected at unprecedented levels. This is primarily due to the increasingly connected nature of aerospace systems and devices, which could be classified under the generic term of the Internet of Things (IoT). The nature of data being generated and collected is also changing, primarily in the attributes of amount, format, and sources of data. Hence, new approaches, techniques, and corresponding designs need to be developed to store, analyze and derive insight from these data sets or also called “Big Data.” The traditional approach for Analytics requires the movement of the collected dataset to a core location on the network like a data center or cloud. However, many modern and emerging applications and aerospace use cases require Network Edge Analytics capability, where analytics and insights need to be performed at the edge of the network without moving the entire dataset to the core location. Thus, a hybrid model of Big Data Analytics with Network Core and Edge Analytics needs to be reviewed and considered. This paper will discuss the Analytics capabilities that can be leveraged both at the Network Edge and Network Core, and will also review all the above considerations from analytics techniques, technologies and use cases point of view. In addition, the paper will discuss various technologies and their implementation both at the core and edge, including software and infrastructure elements. Lastly, this paper will focus on analytics technologies and approaches for structured, unstructured and semi-structured data, including, review of real time and batch analytics techniques in a distributed data environment. Consequently, an approach is required to integrate these disjointed analytics and data sources techniques in a hybrid model of network core and edge analytics. Select techniques and technologies like Hadoop, NoSql, and real time analytics will be discussed as examples. Finally, the paper will review Analytics use cases, including Aerospace industry use cases, where such a hybrid model of core and edge analytics can be leveraged for practical implementations.