An approach for improving searching algorithm in big data using hybrid technique

Abstract

Search algorithms play an essential role in wide variety of applications in computer science. Different search algorithms can be applied to different data structures. Each data structure has its own advantages and limitations which in turn affect the search algorithm complexity and performance. In the context of Big Data such limitations can be more obvious and noticeable in reducing the performance offered by the data structure and search algorithms applied to it because of big data characteristics such as its huge volume. Also, string data sets can be more challenging. They can cause more overhead in preprocessing. Some string datasets such as personal names may have some skew that has negative effect on hash indexes by increasing the ratio of collisions produced. The work in this paper offers a technique for improving the famous Hash table Data Structure by reducing the collision ratio in hashing in the average case, thus reducing the total number of comparisons made by search algorithm. The advantage of reducing the collision ratio was achieved by building a hybrid approach making use of three common data structures, namely Hash Table, B-Tree and Linked List. The results obtained showed noticeable improvement for time complexity of data searching algorithm over the traditional data structures that were used for comparisons.