Lookup Speed Research Articles

An Information Centric Network for Distributed Registery withContent Unique Identifier (DID) and Verifiable Credentials

Decentralised Identifiers (DIDs), a new self-manageable method of authentication that is currently being standardised by the World Wide Web Consortium (W3C), are new standards for Content Unique Identifiers. Verifiable credentials (VCs), another ongoing standardisation (by the same W3C working group) that permits private and secure proofs of attribute ownership, are closely related to DIDs. Both of these methods rely on a central immutable decentralised registry (such as a blockchain or peer-to-peer network) where crucial meta-data is stored. Using the newly developing paradigm of Information Centric Networking (ICN), we design, construct, and evaluate a secure DID/VC registry service. The ''search by name'' feature of ICN, together with a secure protocol, are combined with the decentralised nature and our goal is to achieve technique for keeping in sync copies of an object at several locations. Our design has little network cost and provides quick lookup speeds because to ICN's built-in multicast and caching functionality

LPR-Trie: A fast IPv6 routing lookup algorithm with virtual nodes

The number of IPv6 routes in todays backbone routers has grown rapidly, which has put tremendous pressure on route lookup and storage. Based on the analysis of IPv6 address prefix length and distribution characteristics, this paper proposes an IPv6 route lookup architecture called LPR-Trie. The core idea of the algorithm is to utilize more spaces and accelerate routing lookup. Moreover, we put forward the concept of virtual nodes, and leverage the link between virtual nodes and ordinary nodes to accelerate routing lookup. We provide the longest prefix routing entry (LPR) calculation algorithm to achieve the longest prefix match. The experimental results show that the virtual node mechanism increases the search speed up to 244%, and the virtual nodes have better stability by setting an appropriate keep-alive time according to the characteristics of actual traffic. This paper shows that our design improves the routing lookup speed and have better memory utilization.

Smart Name Lookup for NDN Forwarding Plane via Neural Networks

Name lookup is a key technology for the forwarding plane of content router in Named Data Networking (NDN). To realize the efficient name lookup, what counts is deploying a highperformance index in content routers. So far, the proposed indexes have shown good performance, most of which are optimized for or evaluated with URLs collected from the current Internet, as the large-scale NDN names are not available yet. Unfortunately, the performance of these indexes is always impacted in terms of lookup speed, memory consumption and false positive probability, as the distributions of URLs retrieved in memory may differ from those of real NDN names independently generated by content-centric applications online. Focusing on this gap, a smart mapping model named Pyramid-NN via neural networks is proposed to build an index called LNI for NDN forwarding plane. Through learning the distributions of the names retrieved in the static memory, LNI can not only reduce the memory consumption and the probability of false positive, but also ensure the performance of real NDN name lookup. Experimental results show that LNI-based FIB can reduce the memory consumption to 58.258 MB for 2 million names. Moreover, as it can be deployed on SRAMs, the throughput is about 177 MSPS, which well meets the current network requirement for fast packet processing.

T-Cache: Efficient Policy-Based Forwarding Using Small TCAM

Ternary Content Addressable Memory (TCAM) is widely used by modern routers and switches to support policy-based forwarding due to its incomparable lookup speed and flexible matching patterns. However, the limited TCAM capacity does not scale with the ever-increasing rule table size due to the high hardware cost and high power consumption. At present, using TCAM just as a rule cache is an appealing solution, but one must resolve several tricky issues including the rule dependency and the associated TCAM updates. In this paper, we propose a new approach which can generate dependency-free rules to cache. By removing the rule dependency, the complex TCAM update problem also disappears. We provide the complete T-cache system design including slow path processing and cache replacement, and implement a T-cache prototype on Barefoot Tofino switches. We conduct comprehensive software simulations and hardware experiments based on real-world and synthesized rule tables and packet traces to show that T-cache is efficient and robust for network traffic in various scenarios.

Using XorOffsetTrie for high-performance IPv6 lookup in the backbone network

The development of IPv6 in scale and traffic requires the IPv6 lookup algorithm to perform fast longest prefix matching (LPM), especially in the backbone network. However, existing IPv6 lookup algorithms, e.g., SAIL, Poptrie, and Hi-BST have limited lookup speeds with too many memory accesses. To achieve the highest lookup speed with fewer memory accesses, we propose the XorOffsetTrie method, which is the combination of XorFilter and OffsetTrie. OffsetTrie reduces the memory accesses by combining the offset index and the next hop of rule into an integer. XorFilter further improves lookup speed by matching about 63% rules with one memory access. If a packet matches a rule in XorFilter, we can return the next hop of this rule immediately. Otherwise, we continue to lookup in OffsetTrie. With high efficient data structure for lookup process, the lookup speed of XorOffsetTrie achieves 1.6x, 4.2x, 8.5x that of SAIL, Poptrie, and Hi-BST.

ITOC: An Improved Trie-Based Algorithm for Online Packet Classification

With the development of SDN, packet classifiers nowadays need to be provided with low update latency besides fast lookup performance because switches need to respond to update control messages from controllers in time to guarantee real-time service in SDN implementations. Classification in this scenario is called online packet classification. In this paper, we put forward an improved trie-based algorithm for online packet classification (ITOC), in which we provide a trie selection strategy to avoid occasional high update latency in the update process of online trie-based algorithms. Experiments are conducted to validate the effectiveness of our optimization and compare the performance of ITOC with the offline methods, DPDK ACL. Experimental results demonstrate that ITOC has the same level of lookup speed with DPDK ACL and greatly decreased the update latency as well. The update latency of ITOC is only 6.85% of DPDK ACL library in the best case.

Bloom-Quotient Based Name Matching Technique in Content Centric Networks

Content Centric Networking (CCN) is a novel network paradigm in which the communication process focusses on content rather than the host. In the current network architecture, routers forward the incoming packets based on the routing information that already had. Whereas, forwarding decisions in CCN are taken depending on networks situation. The contents in CCN are retrieved depending on their names which are URL like names that are composing of a number of string parts separated by’/’. These names are stored inside a specific data structure, called Forwarding Information Base (FIB), inside CCN routers and they are used to forward any incoming packet. Therefore, the main two challenges that face the design of FIB table are: search speed and storage utilization. Consequently, in this paper, we propose a new name matching technique (named BF-QF FIB) to design and implement a FIB table in CCN routers in order to decrease the storage utilization and to increase the lookup speed. This technique utilizes two kinds of query data structures: Bloom filter (BF) and Quotient filter (QF) as its main data structures. The utilization of these two data structures will ensure low memory usage and high lookup speed. The results of our evaluation show that BF-QF FIB can guarantee high search rate and offer perfect scalability to large FIB tables.

An efficient pipeline processing scheme for programming Protocol-independent Packet Processors

OpenFlow is unable to provide customized flow tables, resulting in memory explosions and high switch retirement rates. This is the bottleneck for the development of SDN. Recently, P4 (Programming Protocol-independent Packet Processors) attracts much attentions from both academia and industry. It provides customized networking services by offering flow-level control. P4 can “produce” various forwarding tables according to packets. P4 increases the speed of custom ASICs. However, with the prevalence of P4, the multiple forwarding tables could explode when used in large scale networks. The explosion problem can slow down the lookup speed, which causes congestions and packet losses. In addition, the pipelined structure of forwarding tables brings additional processing delay. In this study, we will improve the lookup performance by optimizing the forwarding tables of P4. Intuitively, we will install the rules according to their popularity, i.e., the popular rules will appear earlier than others. Thus, the packets can hit the matched rule sooner. In this paper, we formalize the optimization problem, and prove that the problem is NP-hard. To solve the problem, we propose a heuristic algorithm called EPSP (Efficient Pipeline Processing Scheme for P4), which can largely reduce the lookup time while keeping the forwarding actions the same. Because running the optimization algorithm frequently brings additional processing burdens, wedesign an incremental update algorithm to alleviate this problem. To evaluate the proposed algorithms, we set up the simulation environments based on ns-3. The simulation results show that the algorithm greatly reduces both the lookup time and the number of memory accesses. The incremental algorithm largely reduces the processing burdens while the lookup time remains almost the same with the non-incremental algorithm. We also implemented a prototype using floodlight and mininet. The results show that our algorithm brings acceptable burder, and performs better than traditional algorithm.

FCTrees: A Front-Coded Family of Compressed Tree-Based FIB Structures for NDN Routers

Named data networking (NDN) is a nascent vision for the future Internet that replaces IP addresses with content names searchable at the network layer. One challenging task for NDN routers is to manage huge forwarding information bases (FIBs) that store next-hop routes to contents. In this article, we propose a family of compressed FIB data structures that significantly reduce the required storage space within the NDN routers. Our first compressed FIB data structure is FCTree. FCTree employs a localized front-coding compression, that eliminates repeated prefixes, to buckets containing partitions of routes. These buckets are then organized in self-balancing trees to speed up the longest prefix match (LPM) operations. We propose two enhancements to FCTree, a statistically compressed FCTree (StFCTree) and a dictionary compressed FCTree (DiFCTree). Both StFCTree and DiFCTree achieve higher compression ratios for NDN FIBs and can be used for FIB updates or exchanges between the forwarding and control planes. Finally, we provide the control plane with several knobs that can be employed to achieve different target trade-offs between the lookup speed and the FIB size in each of these structures. Theoretical analysis along with experimental results demonstrate the significant space savings and performance achieved by the proposed schemes.

Multibillion packet lookup for next generation networks

Fast Internet Protocol Version 4 (IPv4) lookup is one of the key challenges that have always been faced with growing internet speed. Next generation networks promise hundreds of gigabit communication bandwidths. To support such high data rates, core networking devices require very fast IPv4 lookup for incoming packets to sustain their functionality. Researchers from academia and industry have contributed widely towards this problem, presenting numerous techniques and algorithms to improve lookup times. In this paper, a bit vector-based IP lookup engine is presented that implements parallel units to achieve 4.3 Billion Packets Per Second (BPPS) lookup speeds for 5 fields. Implementations are done using dual port Distributed RAM (DRAM) on state-of-the-art Xilinx Virtex 7 series Field Programmable Gate Arrays (FPGA). Post place and route results for different configurations showed that the proposed design consumes much less memory, facilitating multiple engines on a single chip whilst maintaining a very low overall power profile.

Genetic Algorithm Based Prefix Partitioning for Memory-Efficient IPv4/6 Lookup

IP address lookup speed in core routers has a significant impact on service quality in communications in the Internet. The rapid growth of traffic on the Internet and the development of communication links working at speeds of several gigabits per second make IP lookup a challenge for IP routers. There are several IP lookup schemes based on counts of set bits of a bit-vector that encode the routing information in structures of a constant size that guarantee fast IP lookups. However, these schemes have exponential memory complexity and then prefix partitioning schemes are used to save memory consumption, performing the IP lookup in stages. In this work, we propose a prefix-partitioning scheme that optimizes the available memory usage in IP lookups in accordance with the prefix distribution in forwarding tables and an IP lookup scheme based in a precalculated count of set bits of a bit-vector. A complete IPv4/6 lookup has an instruction cost of two memory accesses in the best case and twice the number of stages in the worst case. The proposed prefix-partitioning scheme is based on heuristic search using genetic algorithms to find an optimal partitioning. Our proposal can be implemented in general purpose hardware for IPv4/6 lookups.

Towards Pending Interest Table Management Solutions in Named Data Networking

The transition of current Internet from host-centric communication to content-centric communication has been realized due to rapid growth in the number of users and their demands. Named Data Network (NDN) has emerged as Future Internet Architecture to meet the demands related to content oriented communication. Among other data structures such as Forwarding Information Base (FIB) and Content Store (CS), Pending Interest Table (PIT) holds record of the pending interests for which response has not yet received. Despite the benefits of PIT for fast packet forwarding (such as address less communication, loop detection, multi-path routing, packet aggregation, efficient bandwidth utilization), there are challenges faced by PIT. These challenges include name lookup speed, PIT memory consumption and PIT Entry Lifetime (PEL) that may render PIT as a bottleneck and hence will affect the system performance. In this paper, we thoroughly and systematically study the PIT management schemes. Our study covers the solutions for PIT management spanning over the last 7 years. As far as considering our learning, we are the first to study PIT based on both PIT memory size reduction to achieve fast look-up speed and different PEL calculation schemes. The study will help the research community in modeling better and optimized PIT management solutions. The paper also presents the research trends on the basis of different regions, publishing outlets, contributing patterns etc. in the field of PIT management in NDN.

A Composite Structure for Fast Name Prefix Lookup

Name-based forwarding plane is a critical but challenging component for Named Data Networking (NDN), where the hash table is an appealing candidate for data structure utilized in FIB on the benefit of its fast lookup speed. However, the hash table is flawed that it does not naturally support the longest-prefix-matching (LPM) algorithm for name-based forwarding. To support LPM in the hash table, besides the linear lookup, the random search (such as binary search) aims at increasing the lookup speed by reconstructing the FIB and optimizing the search path. We propose a composite data structure for random search based on the combination of hash table and trie; the latter is introduced preserve the logical associations among names, so as to recycle memory and prevent the so-called backtracking problem, thus enhancing the lookup efficiency. The experiment indicates the superiority of our scheme in lookup speed, the impact on memory consumption has also been evaluated.

Efficient security credential management for named data networking

As a promising future internet architecture, named data networking (NDN) shifts the focus from where the data is to what content it carries. In NDN, the communication is driven by the consumer requesting data with specified name, and is secured by the producer signing the data and optionally encrypting the content. Therefore, during NDN communications, consumers may need to fetch digital certificates to verify data's signature. And a chain of certificates may be involved in verifying some signatures. Maintaining those certificates efficiently, especially when the number of certificates to maintain is large, is challenging. This paper proposes a novel mechanism to store certificates efficiently, based on NDN certificate naming convention. According to our experimental results, the proposed approach gains a reduction of memory consumption around 80%, while providing faster lookup speed.

G-SD: Achieving Fast Reverse Lookup using Scalable Declustering Layout in Large-Scale File Systems

With the increasing popularity of cloud computing, current data centers contain petabytes of data in their datacenters. This requires thousands or tens of thousands of storage nodes at a single site. Node failure in these datacenters is normal instead of a rare situation. As a result, data reliability is a great concern. In order to achieve high reliability, data recovery or node reconstruction is a must. Although extensive research works have investigated how to sustain high performance and high reliability in case of node failure at large scale, a reverse lookup problem, namely finding the list of objects for the failed node is not well-addressed. As the first step of failure recovery, this process has a direct impact to the data recovery/node reconstruction. While existing solutions use metadata traversal or data distribution reversing methods for reverse lookup, which are either time consuming or expensive, the deterministic block placement schemes can achieve fast and efficient reverse lookup easily. However, they are designed for centralized, small-scale storage architectures such as RAID etc. Due to their lacking of scalability, they cannot be directly applied in large-scale storage systems. In this paper, we propose Group-Shifted Declustering (G-SD), a deterministic data layout for multi-way replication. G-SD addresses the scalability issue of our previous Shifted Declustering layout and supports fast and efficient reverse lookup. Our mathematical proofs demonstrate that G-SD is a scalable layout that maintains a high level of data availability. We implement a prototype of G-SD and its reverse lookup function on two open source file systems: Ceph and HDFS. Large scale experiments on the Marmot cluster demonstrate that the average speed of G-SD reverse lookup is more than $5\times$ faster than the reverse lookup speed of existing schemes.

A Multi-stage IPv6 Routing Lookup Algorithm Based on Hash Table and Multibit Trie

The address length of IPv6 is 128 and it has no classification addressing, which makes routing lookup processing burden of core router is heavier and the requirements are higher. As many existing algorithms cannot adapt to new requirements after expanding to IPv6, it is necessary to propose a new routing lookup algorithm based on IPv6. After thorough analysis of the IPv6 address prefix distribution regularity, this paper proposes a new scheme based on simple data structure: Hash Table and Multibit Trie (retrieval). This algorithm first classifies the prefix value and then starts its first lookup by using the first 48bits prefix of the IPv6 Address. Simulation results show that in most cases, the algorithm needs only one memory access to find the next hop routing information. This algorithm has high lookup speed and can be easily implemented with hardware.

Efficient name matching based on a fast two-dimensional filter in named data networking

ABSTRACTNamed data networking (NDN) is a recently proposed networking paradigm that aims to overcome the limitations of the Internet Protocol, while preserving its strengths by using named data rather than named hosts for communication. In NDN, the content is determined by its name, and each NDN packet has a name that determines the content it is bringing or forwarding. In this new paradigm, content names have a hierarchical structure and variable length and, therefore, these features will introduce new challenges to satisfy reasonable speed at large scale. In this paper we propose, design and implement a new approach called fast two-dimensional filter with hash table (FTDF-HT). In the proposed approach, a hash table storing the name prefixes, in a hierarchical manner, is only accessed when the proposed filter (FTDF) states that the name under querying exists. Thus, reducing the unnecessary access to the hash table. Moreover, the access to the hash table will be done with same hash function that is used for FTDF and this will minimize the search time. Our evaluation results show that FTDF-HT can achieve high lookup speed and exhibit good scalability to large-scale prefixes table.

BFAST: High-Speed and Memory-Efficient Approach for NDN Forwarding Engine

Named data networking (NDN) is a future Internet architecture that directly emphasizes accessible content by assigning each piece of content a unique name. Data transmission in NDN is realized via name-based routing and forwarding. Name-based forwarding information base (FIB) usually has much more and longer prefixes than IP-based ones, and therefore, name-based forwarding brings more challenges on the NDN router in terms of high forwarding throughput, low memory consumption, and fast FIB update. In this paper, we present an index data structure called BFAST for the name-based FIB. BFAST is designed based on a basic hash table, it employs a counting Bloom filter to balance the load among hash table slots, so that the number of items in each non-empty slot is close to 1, leading to low searching time in each slot. Meanwhile, the first-rank-indexed scheme is proposed to effectively reduce the massive memory consumption required by the pointers in all the hash table slots. Evaluation results show that, for the longest prefix match FIB lookup, BFAST achieves a speed of 2.14 MS/S using one thread, and meanwhile, the memory consumption is reasonably low. By leveraging the parallelism of today’s multi-core CPU, BFAST arrives at an FIB lookup speed of 33.64 MS/S using 24 threads, and the latency is around $0.71~\mu s$ .

Efficient Searching with a TCAM-based Parallel Architecture

Ternary Content-Addressable Memory (TCAM) is a popular hardware device for fast IP address lookup. High link transmission speed of Internet backbone demands more powerful IP address lookup engine. Restricted by the memory access speed, the lookup engine for next-generation routers demands exploiting parallelism among multiple TCAM chips. How to design an efficient engine with high IP lookup speed and less update time while keeping low power consumption is a great challenge in building the next-generation routers. At present, no parallel schemes can make full use of TCAM chips’ capability. In this paper, we propose a fast lookup, efficient update and power-saving scheme which can basically make full use of TCAM chips’ capability. With N parallel TCAM chips, our proposed scheme can achieve a worst-case speedup factor of (N-1)*90% in cache-update state, and a worst-case speedup factor of N*90% in normal work state. Compared with previous works, our scheme can apparently improve IP lookup performance and update efficiency while keeps low power consumption.

Steerable Name Lookup based on Classified Prefixes and Scalable One Memory Access Bloom Filter for Named Data Networking

Name Data Networking (NDN) is an entirely new network architecture, in which data packet forwarding rely on content names instead of fixed-length IP addresses. Under the new naming system, content names are hierarchical structure and have variable length, these pivotal features bring new challenges to meet line speed at large scale. In this paper, we propose Steerable Name Lookup based on Classified Prefixes and Scalable One Memory Access Bloom Filter for Named Data Networking (SNLBF) that prefixes are classified by the number of components and the length of prefix to reduce the size of collection and reduce the expansion rate of the Bloom filter by relieving the rate of collection growth. Then prefixes with different length and component numbers are mapped into different Bloom filter. In order to reduce unnecessary probes, we design the number of component steerable structure (NCS). One memory access Bloom filter and scalable Bloom filter are combined to solve scalability of name lookup engine and speed up name lookup. Because H3 in scalable Bloom filter is no longer applicable, we design new hash modular arithmetic. Our evaluation results show that SNLBF can achieve high lookup speed and exhibit good scalability to large-scale prefixes table.