A Memory Optimized Aggregated Bit Vector Algorithm for large rule-sets

Abstract

Packet classification consists of searching field values of incoming packets in rule database to find matching rule(s). It is a well-researched domain and currently many algorithms provide faster lookup. However, most of the existing solutions do not support large rule sets up to 4M. With high number of fields for IPv6 address, it further degrades the search performance and increases memory usage. We take Aggregated Bit Vector (ABV) algorithm as base algorithm and optimize it for better memory and search performance. Our solution Memory Optimized Aggregated Bit Vector (MOABV) Algorithm addresses these issues - Preprocessing performance, Memory Optimization and Classification Speed for large rule sets with some specific characterizes like more wild card fields, bigger ranges, similar search requests etc. We also give a solution to handle 16-bit or bigger range field without expanding the range into multiple ranges. Our test results show that we achieve around 50-80% memory optimization using wildcard field optimization and 60 to 70% search optimization using search key caching with 20% (worst case) additional cache processing overhead.