Towards a TRansparent I/O Solution

Abstract

The benefits of data distribution to multiple storage platforms with different characteristics have been widely acknowledged. Such systems are more tolerant to outages and bottlenecks and allow for more flexible policies regarding cost reduction, security and workload diversity. To leverage platforms simultaneously additional orchestration steps are needed. Existing approaches either implement such steps in the application's source code, resulting to minimum reusability across applications, or handle them at the infrastructure level. The latter usually involves over-engineering to handle different application behaviors and binds the system to a specific infrastructure. In this paper we present a middle-ware that decouples the I/O path from the application's source code and performs in-transit processing before data lands on the storage platforms. Abstracting the I/O process as a graph of reusable components allows the developers to easily implement complex storage solutions without the burden of writing custom code. Similarly, the administrators can create their own graph that reflects the infrastructure setup and append it to the preceding graph, so that various policies and infrastructure-related changes can be performed transparently to the application. Users can also extend the graph chain to enhance the application's functionality by using plug-ins. Our approach eliminates the need for custom I/O management code and allows for the applications to evolve independently of the storage back-end. To evaluate our system we employed a secure web service scenario that was seamlessly adapted to the changes in its storage back-end.