Performance evaluation and benchmarking of event-based systems

Abstract

Event-based systems (EBS) are increasingly used as underlying technology in many mission critical areas and large-scale environments, such as environmental monitoring and location-based services [3]. Moreover, novel event-based applications are typically highly distributed and data intensive with stringent requirements for performance and scalability. Since their reliability is crucial for the whole IT infrastructure, a certain Quality-of-Service (QoS) level has to be ensured. The motivation for our work was to support the development and maintenance of EBS that meet their QoS requirements. Given that EBS differ from traditional software in fundamental aspects such as their underlying communications paradigm, specific solutions and concepts are needed. System architects and deployers need tools and methodologies, which allow them to evaluate and forecast system performance and behavior in certain situations to identify potential performance problems and bottlenecks. Common approaches are benchmarking and performance modeling. However, no general performance modeling methodologies focusing on EBS had been published. Furthermore, there was a lack of test harnesses and benchmarks using representative workloads for EBS. Consequently, we focused on the development of a performance modeling methodology of EBS as well as on approaches to benchmark them. We summarize now our main contributions and proposed approaches.