Analysis on Implementation of Optical Network Management Solution with Hadoop Architecture

Abstract

Optical network management and solution system provides services for a complete end-to-end provisioning on the transport network. In Optical networks, various technologies like SDH, PTN, EoSDH and WDM are integrated with each other and also it implemented with their own required network elements. This complex hierarchy and the large value of network data are in each and every individual EMS (Element Management System). While these various EMS data are collected through the north bound interface in primary Network Management and Solution System, it should be large scale data with multiple varieties of data set. Traditional data processing tools like MySQL cannot reach the requirement up to this level. This paper approaches the network big data handling with Hadoop distributed file system to overcome that incompetence. Using Hadoop advantages like automatic cluster distribution of data handling, parallel processing and open source with integration of multiple platforms, this is used for the network discovery, network alarm extraction and integrating with network application platforms. This paper discuss the performances of the traditional MySQL DBMS and Hadoop based data processing frameworks for Optical network management systems. Various sizes of optical network data in terms of links processed and results are discussed. Hadoop with Spark engine provides good efficiency compared to others for the optical network management system while it increases in size is observed.