Cloud Native Data Platform for Network Telemetry and Analytics

Abstract

In this manuscript, we present a prototype of a modular data platform that is able to continuously ingest, process, retain, and analyse large amounts of network telemetry data in a scalable and straightforward manner. It follows a recently proposed Data Lakehouse architectural pattern, which is an evolution of two well-known approaches used in this area – data warehouses and data lakes. The platform is based on open standards and open-source components, and it follows cloud native principles in order to be able to run in modern computing environments such as public, private, and hybrid clouds. The primary focus of the prototype is network telemetry and analytics over traffic flows and infrastructure logs for the purposes of cyber-security digital forensics and incident response. During the demonstration part, we will further describe internal workings of the presented data platform and showcase its capabilities and possible applications on a public dataset.