Hash Collisions Research Articles

Towards load-balanced service chaining by Hash-based Traffic Steering on Softswitches

In this paper we study Hash-based Traffic Steering on Softswitches (HATS) which is a load balancing scheme for chaining virtualized network functions (VNFs), with the aim of mitigating the control and data plane overheads of existing methods. Our method uses a flow-hashing technique applied to softswitches to carry out server and network load balancing without triggering the controller. By exploiting the advantages of HATS, we then derive two algorithms, HATS with Flowcell-based Multipathing (HATS-Flowcell) and Dynamic Weight Adjustment for HATS (D-HATS), to address hash collision problems which downgrade system performance. The first algorithm divides an elephant flow into various equal-size flowcells, which are distributed over network paths as individual flows. The second algorithm periodically updates the hashing weights of VNFs and network paths according to their current load status. Our implementation demonstrates that HATS can be readily deployed on commodity network hardware. Furthermore, our experimental results show that D-HATS has roughly the same load balancing performance with Least Load First (LLF), a controller-based service chaining algorithm; while significantly reducing the number of flow entries and service chaining time by 54% and 93%.

Joint Virtual Switch Deployment and Routing for Load Balancing in SDNs

To better serve a diversity of flows, load balancing is crucial to ensure operational efficiency. However, previous works for load balancing have several disadvantages: 1) limited applicability with sub-flow scheduling (e.g., LetFlow); 2) hash collision (e.g., ECMP); or 3) transient network congestion due to reactive scheduling for traffic dynamics (e.g., Hedera and DevoFlow). An important reason for the above disadvantages is that it is difficult to provide fully fine-grained flow control for load balancing in an SDN as the flow table size of each SDN switch is usually limited. Inspired by the fact that a virtual switch (vswitch) has more powerful processing capacity and more flow entries compared with a physical switch, the previous work (e.g., Presto) deploys one vswitch for each ingress switch, and achieves the load balancing through efficient flow routing. However, this mechanism may lead to high cost and not well deal with topology asymmetry. Thus, this paper proposes to achieve the load balancing by incrementally deploying a certain number of vswitches in an SDN. We formulate the joint optimization of vswitch deployment and routing (JVR) problem as an integer linear program, and prove its NP-hardness. A rounding-based algorithm with bounded approximation factors is proposed to solve the JVR problem. We implement the proposed algorithm on an SDN testbed for experimental studies and use simulations for large-scale investigation. The experimental results and simulation results show high efficiency of our algorithm. For example, our proposed algorithm can reduce the link load ratio by about 41.5% compared with ECMP by deploying a small number of virtual switches.

Frequent pattern sub-space clustering optimisation algorithm for data mining from large database

Data mining environment gives a quick response to the user by fast and correctly pick-out the item from the large database is a very challenging task. Previously, multiple algorithms were proposed to identify the frequent item since they are scanning database at multiple times. To overcome those problems, we proposed Rehashing based Apriori Technique in which hashing technology is used to store the data in horizontal and vertical formats. Rehash Based Apriori uses hashing function to reduce the size of candidate item set and scanning of database, eliminate non-frequent items and avoid hash collision. After finding frequent item sets, perform level wise subspace clustering. We instigate generalised self organised tree based (GSTB) mechanism to adaptively selecting root to construct the path from the cluster head to neighbours when constructing the tree. Our experimental results show that our proposed mechanisms reduce the computational time of overall process.

Direct visualization of cryptographic keys for enhanced security

PGP public keys are relatively small binary data. Their hashes are used and also visualized for comparison and validation purposes. We pursue a direct, but previously unused approach. We produce colorful images of public keys and other binary data by generating drawing primitives from binary input. Optionally, we also include the hashes in the visualization. The visualization of raw data together with its hash provides a further security benefit. With it we can visually detect hash collisions. The primary focus of this paper is a direct visualization of public keys. We tune the transparency heuristics for better results. Our method visually detects key spoofing on real SHA1 collision data.

Flow Optimization in Data Centers With Clos Networks in Support of Cloud Applications

The proliferation of cloud computing has forced data centers to support a large number of dynamic resources such as virtual machines and storage volumes from any physical location within the data center. This has led to highly volatile and unpredictable traffic patterns. The emergence of cloud applications that exchange large volumes of information have created large persistent flows that need to coexist with other types of traffic. As a result, a number of data center topologies have been proposed to support cloud applications. One of the topologies that has gained prominence is the Clos network. On one hand, Clos networks provide many nice properties, such as equidistant hops, bandwidth, and latency. On the other hand the routing of diverse traffic flows still remains an issue. Current routing methods such as equal-cost multi-path (ECMP) routing that rely on flow hashing over equal cost paths do not take into consideration traffic volume nor network state. In this paper, we investigate routing in Clos topologies. We formulate optimal flow routing in Clos networks as a binary multicommodity flow problem and present optimized flow re-routing, a modified version of the binary multicommodity flow problem that uses a heuristics approach to minimize hash collisions. We also propose FlowFit, a practical optimization method that uses network state to optimally re-assign flows to links. Our experimental and simulation results show a significant improvement in bisection bandwidth and flow completion time over ECMP.

Graph regularized nonnegative sparse coding using incoherent dictionary for approximate nearest neighbor search

In this paper, we consider the problem of approximate nearest neighbor (ANN) retrieval with the method of sparse coding, which is a promising tool of discovering compact representation of high-dimensional data. A new study, exploiting the indices of the active set of sparse coded data as retrieval code, exhibits an appealing ANN route. Here our work aims to enhance the method via considering its shortages of the local structure of the data. Our primary innovation is two-fold: We introduce the graph Laplacian regularization to preserve the local structure of the original data into reduced space, which is indeed beneficial to ANN. And we impose non-negativity constraints such that the retrieval code can more effectively reflect the neighborhood relation, thereby cutting down on unnecessary hash collision. To this end, we learn an incoherent dictionary with both rules via a novel formulation of sparse coding. The resulting optimization problem can be provided with an available solution by the widely used iterative scheme, where we resort to the feature-sign search algorithm in the sparse coding step and exploit the method that uses a Lagrange dual for dictionary learning step. Experimental results on publicly available image data sets manifest that the rules are positive to promote the classification and ANN accuracies. Compared with several state-of-the-art ANN techniques, our methods can achieve an interesting improvement in retrieval accuracy.

A Collision-Mitigation Cuckoo Hashing Scheme for Large-Scale Storage Systems

With the rapid growth of the amount of information, cloud computing servers need to process and analyze large amounts of high-dimensional and unstructured data timely and accurately. This usually requires many query operations. Due to simplicity and ease of use, cuckoo hashing schemes have been widely used in real-world cloud-related applications. However, due to the potential hash collisions, the cuckoo hashing suffers from endless loops and high insertion latency, even high risks of re-construction of entire hash table. In order to address these problems, we propose a cost-efficient cuckoo hashing scheme, called MinCounter. The idea behind MinCounter is to alleviate the occurrence of endless loops in the data insertion by selecting unbusy kicking-out routes. MinCounter selects the “cold” (infrequently accessed), rather than random, buckets to handle hash collisions. We further improve the concurrency of the MinCounter scheme to pursue higher performance and adapt to concurrent applications. MinCounter has the salient features of offering efficient insertion and query services and delivering high performance of cloud servers, as well as enhancing the experiences for cloud users. We have implemented MinCounter in a large-scale cloud testbed and examined the performance by using three real-world traces. Extensive experimental results demonstrate the efficacy and efficiency of MinCounter.

Secure intelligent traffic light control using fog computing

As the number of vehicles grows, traffic efficiency is becoming a worldwide problem. Intelligent transportation system aims to improve the traffic efficiency, where intelligent traffic light control is an important component. Existing intelligent traffic light control systems face some challenges, e.g., avoiding heavy roadside sensors, resisting malicious vehicles and avoiding single-point failure. To cope with those challenges, we propose two secure intelligent traffic light control schemes using fog computing whose security are based on the hardness of the computational Diffie–Hellman puzzle and the hash collision puzzle respectively. The two schemes assume the traffic lights are fog devices. The first scheme is a simple extension of a recent scheme for defending denial-of-service attacks. We show this simple extension is not efficient when the vehicle density is high. The second scheme is much more efficient and is fog device friendly. Even the vehicle density is high, the traffic light may verify the validity of the vehicles efficiently.

FREQUENT PATTERN SUB-SPACE CLUSTERING OPTIMIZATION (FPSSCO) ALGORITHM FOR DATAMINING FROM LARGE DATA BASE

Data mining environment gives a quick response to the user by fast and correctly pick-out the item from the large database is a very challenging task. Previously, multiple algorithms were proposed to identify the frequent item since they are scanning database at multiple times. To overcome those problems, we proposed Rehashing based Apriori Technique in which hashing technology is used to store the data in horizontal and vertical formats. Rehash Based Apriori uses hashing function to reduce the size of candidate item set and scanning of database, eliminate non-frequent items and avoid hash collision. After finding frequent item sets, perform level wise subspace clustering. We instigate generalised self organised tree based (GSTB) mechanism to adaptively selecting root to construct the path from the cluster head to neighbours when constructing the tree. Our experimental results show that our proposed mechanisms reduce the computational time of overall process.

The Impact of MD5 File Hash Collisions On Digital Forensic Imaging

The Impact of MD5 File Hash Collisions On Digital Forensic Imaging

Delegation of signing rights for emerging 5G networks

Summary5G mobile networks are promising to offer mobile users unrivaled experiences with infinite networking capability at any period and from anywhere. However, it appears unnecessary and impractical for the customers and servers to be connected permanently in wireless networks because battery life is a bottleneck in such kind of networks. Therefore, the authorized entity needs to delegate its right to others in order to ensure the services available. This paper aims to address the issue of delegating authentication in 5G networks by proposing a new proxy signature scheme with efficient proxy signing operations. Concretely, we present a new and efficient proxy signature algorithm whose unforgeability can be reduced to the well‐known discrete logarithm assumption. In the proposal, the original signer delegates his signing right by signing an exposure‐free chameleon hash value on the warrant, and the proxy signer can generate a proxy signature on a message only by calculating a chameleon hash collision between the warrant and the message. This approach is computationally cost‐effective for the proxy signer. Further analysis of the new scheme demonstrates that it offers the desirable properties of delegation of signing sights. Copyright © 2015 John Wiley & Sons, Ltd.

Presto

Datacenter networks deal with a variety of workloads, ranging from latency-sensitive small flows to bandwidth-hungry large flows. Load balancing schemes based on flow hashing, e.g., ECMP, cause congestion when hash collisions occur and can perform poorly in asymmetric topologies. Recent proposals to load balance the network require centralized traffic engineering, multipath-aware transport, or expensive specialized hardware. We propose a mechanism that avoids these limitations by (i) pushing load-balancing functionality into the soft network edge (e.g., virtual switches) such that no changes are required in the transport layer, customer VMs, or networking hardware, and (ii) load balancing on fine-grained, near-uniform units of data (flowcells) that fit within end-host segment offload optimizations used to support fast networking speeds. We design and implement such a soft-edge load balancing scheme, called Presto, and evaluate it on a 10 Gbps physical testbed. We demonstrate the computational impact of packet reordering on receivers and propose a mechanism to handle reordering in the TCP receive offload functionality. Presto's performance closely tracks that of a single, non-blocking switch over many workloads and is adaptive to failures and topology asymmetry.

A Hash Table for Line-Rate Data Processing

FPGA-based data processing is becoming increasingly relevant in data centers, as the transformation of existing applications into dataflow architectures can bring significant throughput and power benefits. Furthermore, a tighter integration of computing and network is appealing, as it overcomes traditional bottlenecks between CPUs and network interfaces, and dramatically reduces latency. In this article, we present the design of a novel hash table, a fundamental building block used in many applications, to enable data processing on FPGAs close to the network. We present a fully pipelined design capable of sustaining consistent 10Gbps line-rate processing by deploying a concurrent mechanism to handle hash collisions. We address additional design challenges such as support for a broad range of key sizes without stalling the pipeline through careful matching of lookup time with packet reception time. Finally, the design is based on a scalable architecture that can be easily parameterized to work with different memory types operating at different access speeds and latencies. We have tested the proposed hash table in an FPGA-based memcached appliance implementing a main-memory key-value store in hardware. The hash table is used to index 2 million entries in 24GB of external DDR3 DRAM while sustaining 13 million requests per second, the maximum packet rate that can be achieved with UDP packets on a 10Gbps link for this application.

An Efficient Strategy for Collision Resolution in Hash Tables

paper presents NFO, a new and innovative technique for collision resolution based on single dimensional arrays. Hash collisions are practically unavoidable when hashing a random subset of a large set of possible keys and should be seen as an event that can disrupt the normal operations or flow of hash functions computing an index into an array of buckets or slots. Hash tables provide efficient table implementations but then its performance is greatly affected if there are high loads of collisions. This new approach intends to manage these collisions effectively and properly although there are some algorithms for handling collisions currently. NFO incorporates certain features to resolve some problems of existing techniques. The performance of our approach is quantified via analytical modeling and software simulations. Efficient implementations that are easily realizable and productive in modern technologies are discussed. The performance benefits are significant and require machines with moderate memory and speed specifications. Depending on observations of the results of implementation of the proposed approach or technique on a set of real data of several types, all results are registered and analyzed.

An Improved Method of Hash Table Based on Transform and Conquer

This paper presented a improved method of hash table, the algorithm based on transform and conquer, established a number of small-scale array, then imitated binary search processes and reduced the hash collision. Simulation results shows that , not only the continuous memory requirement is reduced greatly, but also that when the hash collisions occur, the searching efficiency is improved by an order of magnitude from O(n) to O(logn), the space utilization rate is also improved greatly.

Biometric-PKI Authentication System Using Fingerprint Minutiae

A digital certificate under Public Key Infrastructure has a defect of Man-in-the-Middle Attack that performs hash collision attacks. In this paper, we propose a robust biometric-PKI authentication system against Man-in-the-Middle Attack. The biometric-PKI authentication system consists of current PKI authentication and biometric authentication, which employs biometric data and a public key from a digital certificate. In the proposed biometric-PKI authentication system, an au- thentication process performs that it extracts consistent features of fingerprint images, encrypts consistent features, and matches features with prepared templates. The simulation results of the proposed authentication system prove that our system achieves low false acceptance rate and high accuracy rate.

An Online Framework for Flow Round Trip Time Measurement

With the advent of high speed links, online flow measurement for, e.g., flow round trip time (RTT), becomes difficult due to the enormous amount of requirements on computational resources. We address this problem by proposing the double-deletion bloom filter (DDBF) scheme, which alleviates potential hash collisions of a standard bloom filter by explicitly deleting used records and implicitly deleting out-of-date records. As a result, it accurately estimates the current TCP flow RTT distribution with limited memory space, even with the appearance of multipath routing and Syn flooding attacks. Theoretical analysis indicates that the DDBF scheme exhibits desirable accuracy with a constant and small amount of memory. We further validate our scheme using real traces and demonstrate significant memory-saving without degrading accuracy.

An efficient ECC-based mechanism for securing network coding-based P2P content distribution

Network coding has been demonstrated to be able to improve the performance of P2P content distribution. However, it is vulnerable to pollution attacks where malicious peers can flood the network with corrupted blocks easily, leading to substantial performance degradation. Moreover, existing corruption detection schemes for network coding are not well suited to P2P systems. Effective scheme to detect the corruption and identify the attacker is required to thwart such attacks. In this paper, we propose an efficient ECC-based mechanism for securing network coding-based P2P content distribution, namely ESNC, which includes an efficient network coding signature scheme and an identity-based malicious peer identification scheme. The two schemes cooperate to thwart pollution attacks on network coding effectively in P2P networks, not only detecting corrupted blocks on-the-fly efficiently, but also precisely identifying all the malicious peers quickly. ESNC is mainly based on elliptic curve cryptography (ECC) and can provide high level of security. It incurs significantly less computation and communication overheads than other comparable state-of-the-art schemes for P2P systems. ESNC can work with arbitrary topologies, as it is the case in P2P networks. Security analysis demonstrates that ESNC can resist hash collision attacks, signature forgery attacks, and collusion attacks with arbitrary number of colluding malicious peers. Simulation results show that ESNC effectively limits the corruption spread and identifies all the malicious peers in a short time under different practical settings.

Line speed accurate superspreader identification using dynamic error compensation

A superspreader is a source that connects with a large number of distinct destinations during a measuring interval. Identifying superspreaders provides very important information for many other applications, such as port scanner and Distributed Denial of Service (DDoS) attack detection, worm propagation measurement, and hot spots localization in peer-to-peer (p2p) and Content Delivery Networks (CDN). There are several challenges remaining in the area of superspreader identification, especially for high speed networks, exhibited by the trade-offs among three critical requirements, which are (1) fast data processing speed, (2) low memory space, and (3) high accuracy. In this paper, we propose a new Superspreader Identification System, which uses a bitmap to identify new network flows and a counting bloom filter to record flow numbers for each source. Several error compensation mechanisms are designed to dynamically compensate the system (e.g., hash collisions) while its states are changing, and thus substantially improve its accuracy in superspreaders identification. Multiple light-weight data operation techniques are also introduced to expedite data processing and reduce required memory. The Superspreader Identification System requires only 256KB SRAM to effectively identify superspreaders from high-speed data streams (e.g., OC-192) in line speed with more than 95% identification accuracy. Theoretical performance analysis is conducted to evaluate false negative ratio, false positive ratio, and compensation efficacy of the SIS. Comprehensive experiments using real Internet traces from a tier-1 Internet Service Provider (ISP) network demonstrate that the Superspreader Identification System is superior to other related approaches in terms of its achieved trade-offs among required memory space, identification accuracy, and data processing efficiency and capacity.

Improving Key Management Scheme Supporting Node Mobility in Heterogeneous Sensor Networks using Collision-Keys

Wireless Sensor Networks (WSNs) consist of a large number of sensor nodes that are batteries powered, equipped with limited memory and computational capabilities. These constrained devices face many security threats and thus there is a need of some cryptographic mechanism for secure communication. Key distribution is of critical importance to provide security in WSNs. Till now a number of key distribution schemes are proposed in the literature but there are very few schemes considering mobility of sensor nodes. In this paper we have proposed a modification to the key management scheme supporting node mobility in heterogeneous sensor network. Our modification uses Hash collision keys to improve the network resilience and connectivity between the nodes. We have evaluated our scheme analytically and obtained results show that our proposed solution assures better network connectivity and resilience while increasing an insignificant computational overhead.