Efficient Optimized Strategy of Big Data Retrieval

Abstract

With the development of new information technologies, the accumulation of data volume has been exploding, and big data retrieval has played an increasingly important role in big data technology. The challenge of data retrieval are the improvement of retrieval accuracy and retrieval speed. Aiming at the demand of big data platform for efficient data retrieval, an efficient optimized strategy is proposed. We found when the primary key query is used, the query response can be quick. However, when using a non-primary key query, the cache table needs to be comprehensively scanned and the longer response delay may be induced. This paper proposes a secondary index based on Solr to increase the accuracy of information retrieval and the quality of user experience. Then a cache-heat evaluation algorithm categorizes data according to access frequency to reduce query latency. Moreover, an index optimization method based on memory cache updates the cache to save space and enhance utilization. The experiments and simulation demonstrate that the proposed strategy can effectively improves the big data retrieval.