Benchmarking open source NFV MANO systems: OSM and ONAP

Abstract

With the increasing trend in softwarization and virtualization of network functions and systems, NFV management and orchestration (MANO) solutions are being developed to meet the agile and flexible management requirements of virtualized network services in the 5G era and beyond. In this regard, ETSI ISG NFV has specified a standard MANO system that is used as a reference by vendors as well as open-source MANO projects. These MANO systems are inherently very complex and have a direct impact on the overall performance of NFV systems. However, unlike traditional networking functions and systems, there are no well-defined test methods and KPIs based on which the performance of the NFV-MANO system can be tested, validated and benchmarked. Given the absence of formal MANO specific evaluation techniques based on which the performance and features of a MANO system can be quantified, and compared against, we introduce in this paper a formal benchmarking methodology and KPIs for MANO systems. For illustration purposes, we analyze and compare the performance of the two most popular open-source NFV MANO projects, namely ONAP and OSM, using a complex open-source virtual customer premises equipment (vCPE) VNF. Our results show the current features support, performance to be expected and gaps to be covered in future releases.