Knowledge-based analytics for massively distributed networks with noisy data

Abstract

This article develops and implements an improving search algorithm that effectively and efficiently identifies pathways of interest using knowledge-based analytics for massively distributed networks with noisy data. The method developed in this article fundamentally changes how critical information is extracted from large data-sets. Many methods have been developed in the past to identify structures in large graphs. Most of these methods are computationally inefficient for large graphs and their outcome depends on the graph metrics and statistical measures. There has been limited research on using optimisation techniques for data mining in large networks with noisy data. The algorithm developed in this article converges to the optimal solution by traversing the interior of a feasible region. Experiments show that it identifies a pathway of interest from a network of 160,000 components in 10 hours using parallel computing. Future work will include customisation and implementation of the method to other large networks in a variety of applications.