SSD In-Storage Computing for Search Engines

Abstract

SSD-based in-storage computing (called ”Smart SSDs”) allows application-specific codes to execute inside SSDs to exploit the high internal bandwidth and energy-efficient processors. As a result, Smart SSDs have been successfully deployed in many industry settings, e.g., Samsung, IBM, Teradata, and Oracle. Moreover, researchers have also demonstrated their potential opportunities in database systems, data mining, and big data processing. However, it remains unknown whether search engine systems can benefit from Smart SSDs. This work takes a first step to answer this question. The major research issue is what search engine query processing operations can be cost-effectively offloaded to SSDs. For this, we carefully identified the five most commonly used search engine operations that could potentially benefit from Smart SSDs: intersection, ranked intersection, ranked union, difference, and ranked difference. With close collaboration with Samsung, we offloaded the above five operations of Apache Lucene (a widely used open-source search engine) to Samsungs Smart SSD. Finally, we conducted extensive experiments to evaluate the system performance and tradeoffs by using both synthetic datasets and real datasets. The experimental results show that Smart SSDs significantly reduce the query latency by a factor of 2-3 and energy consumption by 6-10 for most of the aforementioned operations.