HADEC: Hadoop-based live DDoS detection framework

Sufian Hameed,Usman Ali

doi:10.1186/s13635-018-0081-z

Abstract

Distributed denial of service (DDoS) flooding attacks are one of the main methods to destroy the availability of critical online services today. These DDoS attacks cannot be prevented ahead of time, and once in place, they overwhelm the victim with huge volume of traffic and render it incapable of performing normal communication or crashes it completely. Any delays in detecting the flooding attacks completely halts the network services. With the rapid increase of DDoS volume and frequency, the new generation of DDoS detection mechanisms are needed to deal with huge attack volume in reasonable and affordable response time.In this paper, we propose HADEC, a Hadoop-based live DDoS detection framework to tackle efficient analysis of flooding attacks by harnessing MapReduce and HDFS. We implemented a counter-based DDoS detection algorithm for four major flooding attacks (TCP-SYN, HTTP GET, UDP, and ICMP) in MapReduce, consisting of map and reduce functions. We deployed a testbed to evaluate the performance of HADEC framework for live DDoS detection on low-end commodity hardware. Based on the experiment, we showed that HADEC is capable of processing and detecting DDoS attacks in near to real time.

Highlights

Distributed denial of service (DDoS) flooding attacks are one of the biggest threats faced by the critical IT infrastructure, from the simplest enterprise network to complex corporate networks
Based on the results presented in this paper, we can conclude that Hadoop-based live DDoS detection framework (HADEC) is capable of analyzing high volume of DDoS flooding attacks in scalable manner
7 Conclusions In this paper, we present HADEC, a scalable Hadoopbased live DDoS detection framework that is capable of analyzing DDoS attacks in affordable time

Summary

Introduction

DDoS flooding attacks are one of the biggest threats faced by the critical IT infrastructure, from the simplest enterprise network to complex corporate networks. These numbers present the total time required for capturing, processing, transferring, and detection with different file sizes. The best case of 1 GB file (10 node cluster) takes 178 s (77%) in capturing/transferring phase and just 50 s in detection phase On the whole, it takes somewhere between 4.3 and 3.82 min to analyze 1 GB of log file that can resolve 10K attackers and generated from an aggregate attack volume of 15.83 GBs. The variation of threshold and attack volume, Fig. 13b, and Fig. 14b slightly effects the overall detection time with couple of seconds. We observed that when parallelism occurs in Hadoop due to bigger file size, the NameNode CPU usage become lower

Comparison with existing work and optimization recommendations

Findings

Conclusions