Position Paper: Towards a Structured Approach to Developing Benchmarks for Virtual Network Functions

Abstract

Network Functions Virtualization (NFV) aims at replacing proprietary hardware appliances with software running on a standardized, general purpose computing platforms. Recently, the concept has gained traction in the industry and major deployments have been announced. However, benchmarking the performance characteristics of virtualized network functions (VNFs) is still an active research topic. VNFs are supplied as software that is operated on the NFV infrastructure (NFVI) of a telecom provider. The performance as well as the accuracy and precision of performance measurements of VNFs are influenced by the NFVI they are running on. Furthermore, due to the introduction of new VNFs and NFVIs, a sharp increase in the need to conduct benchmarks is expected. Despite various efforts in this area by the research community and the industry there is no common understanding on how to approach this issue. To foster a common understanding in this area a structured approach to developing benchmarks for VNFs is proposed in this paper. The approach focuses on the most significant concepts and defines clear boundaries between them. The need to increase the efficiency of developing VNF benchmarks is addressed by the introduction of a hierarchical model with the aim to enable reuse of VNF benchmark components and results.