A compact encoding of security logs for high performance activity detection

Abstract

The rise of cyber crime observed in recent years calls for more efficient and effective Data Exploration and analysis tools. In this respect, the need to support advanced analytics on activity logs and real time data is driving data scientist’ interest in order to design and implement scalable cyber security solutions. However, when data science algorithms are leveraged for huge amount of data, their fully scalable deployment faces a number of technical challenges that grow with the complexity of the algorithms involved and the task to be tackled. Thus algorithms, that were originally designed for classical scenarios, must often be redesigned in order to be effectively used for cyber security purposes. In this paper, we explore these problems and then propose a solution which has proven to be very effective for compressing suspicious activities in smaller and larger cyber environment in order to make the intelligent analysis and simulation of threats more efficient.