Real-time Monitoring System for Container Networks in the Era of Microservices

Abstract

Large-scale web services are increasingly adopting the microservice architecture that mainly utilizes container technologies. Microservices are operated on complex configured infrastructures, such as containers, virtual machines, and physical machines. To ensure service quality of microservices, it is important to monitor not only the quality of services but also the quality of the infrastructures utilized by the services. Therefore, the metrics of the infrastructure related with the services should be traced. An extended Berkeley Packet Filter (eBPF) is a relatively new Linux's function, which is effectively used as a sensor of container-network metrics. There are two key challenges in realizing the service-linked monitoring system. One challenge is making the full-stack topology between microservices, containers, and machines visible to set the sensor related with the services. Another challenge is dynamic sensor management that can relocate the sensor quickly after the topology's change. In this paper, we propose a real-time monitoring system that creates a full-stack topology and relocates the sensor in conjunction with events from a container orchestrator. The system enables a dynamic deployment of the sensors related with the monitored services.