Abstract

Recently, microservice has been a popular architecture to construct cloud-native systems. This novel architecture brings agility and accelerates the software development process significantly. However, it is not easy to manage and operate microservice systems due to their scale and complexity. Many approaches are proposed to automatically operate microservice systems such as anomaly detection. Nevertheless, those methods cannot be sufficiently validated and compared due to a lack of real microservice systems, which leads to the slow process of intelligent operation. These challenges inspire us to build a system named “VWR”, a framework of Virtual War Room for operating microservice applications which allows users to simulate their microservice architectures with low overhead and inject multiple types of faults into the microservice system with chaos engineering. VWR can mimic user requests and record the end-to-end tracing data (i.e., service call chains) for each request in a way consistent with OpenTracing. With easily designed tests and the produced streaming tracing data, the users can validate the performance of their intelligent operation algorithms and improve the algorithms as needed. Besides, based on the streaming tracing data generated by VWR, we introduce a novel unsupervised anomaly detection algorithm based on Matrix Sketch and set it as a default intelligent operation algorithm in VWR. This algorithm can detect anomalies by analyzing high-dimensional performance data collected from a microservice system in a streaming manner. The experimental result in VWR shows that the matrix sketch based method can precisely detect anomalies in microservice systems and outperform some widely used anomaly detection methods such as isolation forest in some scenario. We believe more approaches on the intelligent operation of microservice systems can be constructed based on VWR.

Highlights

  • Nowadays, the microservice has become a widely adopted architecture for enterprises to deploy their large cloud-native applications

  • To address the aforementioned challenges and shortages of previous work, we propose Virtual War Room(VWR) based on Spigo,1 a framework to mimic the behavior of the microservice architecture and to inject faults readily with chaos engineering [13], which has been open sourced in the Github

  • We introduce an unsupervised anomaly detection algorithm based on Matrix Sketch to detect anomalies in microservice systems

Read more

Summary

INTRODUCTION

The microservice has become a widely adopted architecture for enterprises to deploy their large cloud-native applications. Extensive studies have been done to resolve performance problems in distributed systems such as anomaly detection [6]–[8], root cause analysis [2], [9]–[11] In these studies, the researchers need to deploy their own microservice benchmarks and inject some faults into these systems in order to get abnormal cases. VWR can be leveraged to validate other intelligent operation algorithms such as performance prediction, root cause analysis, and so on These algorithms are out of scope of this paper. We introduce an unsupervised anomaly detection algorithm based on Matrix Sketch to detect anomalies in microservice systems This algorithm works in a realtime mode and analyzes the high-dimension run-time data.

RELATED WORK
MATRIX SKETCH BASED ANOMALY DETECTION
EXPERIMENT EVALUATION
CONCLUSION AND FUTURE WORK
Full Text
Published version (Free)

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call