IS-HBase: An In-Storage Computing Optimized HBase with I/O Offloading and Self-Adaptive Caching in Compute-Storage Disaggregated Infrastructure

A management of large-scale data becomes more important, along with the spread of cloud service and the speed-up of networks. Since data management on a single machine can cause performance and scalability problems, data management across multiple machines has been proposed. Distributed Key Value Store(KVS) is a data store which manages data across multiple machines. Since distributed KVSs manage data which consists of simple key-value pair, they can achieve scalability easily. Distributed KVSs are widely used in many services managing large-scale data, such as Facebook and Twitter. Distributed KVSs provide interfaces to access key-value pair by simply specifying the key. In this paper, we refer to a query which only obtains a value from a key as a single query. Some distributed KVSs support a range query which obtains multiple values from a key range. However, under mixed query workloads that consist of single and range queries, single queries(which can be executed faster) are forced to wait until preceding range queries are finished. And this results in the increase of average response time. We propose an approach to reduce the average response time by query scheduling. We implemented our method on Cassandra, and evaluation results showed a reduction of the average response time.

Traditional Grid data management systems have been developed on the supposition that the network bandwidth is unconstrained with dedicated broadband network lines and the storage space is enough for storing replicas. This paper proposes a storage service for distributed computing systems without the supposition, by adopting the concept of pervasive storage in the distributed computing. The Sharable Storage space is recognized as the application server's local storage. Whenever the user commits, data transfer is automatically performed using iSCSI protocol. The storage service is destined for numerous data transfer of relatively small files, and simulated under the condition of a restrictive network bandwidth. File preloading function could make the applications start early without dedicated broadband network lines. The computing sites could benefit from the proposed storage service than waiting until the whole file is transferred for replication and keeping the replica stored without the users' explicit note for reuse.

Erasure codes are widely advocated as a viable means to ensure the dependability of key-value storage systems for big data applications (e.g., MapReduce). They separate user data to several data splits, encode data splits to generate parity splits, and store these splits in storage nodes. Reducing the disk Input and Output (I/O) latency is a well-known challenge to enhance the performance of erasure coding based storage systems. In this paper, we consider the problem of reducing the latency of read operations by caching splits in the memory of storage nodes. We find the key to solve this problem is that storage nodes need to cache enough splits in the memory, so that the application server can reconstruct the objects without reading data from disks. We design an efficient memory caching scheme, namely ECCS. The theoretical analysis verifies that ECCS can effectively reduce the latency of read operations. Accordingly, we implement a prototype storage systems to deploy our proposal. The extensive experiments are conducted on the prototype with the real-world storage cluster and traces. The experimental results show that our proposal can reduce the time of read operations by up to 32% and improve the throughput of read operations by up to 48% compared with current caching approaches.

iSCSI is proposed as a possible solution to building future storage systems. However, using iSCSI raises numerous questions about its implications on system performance. This lack of understanding of system I/O behavior in modern and future systems inhibits providing solutions at the architectural and system levels. Our main goals in this work are to understand the behavior of the application server (iSCSI initiator), to evaluate the overhead introduced by iSCSI compared to systems with directly-attached storage, and to provide insight about how future storage systems may be improved. We examine these questions in the context of commodity iSCSI systems that can benefit most from using iSCSI. We use commodity PCs with several disks as storage nodes and a Gigabit Ethernet network as the storage network. On the application server side we use a broad range of benchmarks and applications to evaluate the impact of iSCSI on application and server performance. We instrument the Linux kernel to provide detailed information about I/O activity and the various overheads of kernel I/O layers. Our analysis reveals how iSCSI affects application performance and shows that building next generation, network-based I/O architectures, requires optimizing I/O latency, reducing network and buffer cache related processing in the host CPU, and increasing the sheer network bandwidth to account for consolidation of different types of traffic.

The paper presents a new component for conventional video monitoring and security systems in air transportation, aiming at improving the overall system performance, eliminate nuisance alarms, optimize network bandwidth and systems storage occupation. Innovative analysis technologies were employed for the development of the new system. The proposed solution relies on a server application referred to as "Third Party Core" installed on a hardware platform residing in the same network with the video monitoring systems and other security systems. This server application uses data captured by video cameras, other security systems (e.g. access control, intrusion detection, radar surveillance) field equipment, data gathered and used in other server application (e.g. database servers, storage servers, other system servers) with to purpose of integrating different functionalities, unify the operator - machines interfaces, predict events, assist operators in video surveillance and improve the security of perimeter. The new server application can easily run on regular low cost hardware platforms (e.g. server or station available on the IT market) and was developed using SQL, C++ and languages connectors.

An explosive increment of data and a variety of data analysis make it indispensable to lower power and cooling costs of cloud datacenters. To address this issue, we investigate the thermal impact of I/O access patterns on data storage systems. Firstly, we conduct some preliminary experiments to study the thermal behavior of a data storage node. The experimental results show that disks have ignorable thermal impacts as processors to outlet temperatures of storage nodes. We raise an approach to model the outlet temperature of a storage node. The thermal models generated by our approach gains a precision error less than 6%. Next, we investigate the thermal impact of data placement strategies on storage systems. We compare the cooling cost of storage systems governed by different data placement schemes. Our study shows that evenly distributing the data leads to highest outlet temperature for the sake of shortest execution time and energy efficiency. According to the energy consumption of various data placement schemes, we propose a thermal-ware energy-efficient data placement strategy. We further show that this work can be extended to analyze the cooling cost of data centers with massive storage capacity. Big data, which is composed of a collection of huge and complex data sets, has been positioned as must have commodity and resource in industry, government, and academia. Processing big data requires a large-scale storage system, which increases both power and cooling costs. In this study, we investigate the thermal behavior of real storage systems and their I/O access patterns, which offer a guideline of building energy-efficient cloud storage systems. The cooling consumption of data centers can be considerably reduced by using an efficient thermal management for storage systems. However, disk is not considered in traditional thermal models for data centers. In this paper, we investigate the thermal impact of hard disks and propose a thermal modeling approach for storage systems. In addition, we estimate the outlet temperature of a storage server by applying the proposed

Discovering with QoS the geo-located Web services is a great challenge in the client mobility context. In this paper, we propose a mechanism that collects for a particular geo-located Web service in a specific domain controlled by a geo-located discovery server GLWSM (geo-located Web services manager) the network bandwidth and the utilization factor of a supplier application server (SAS). Data collected are used in the application servers' selection and migration criterion. For a specific geo-located Web service, a GLWSM sends periodically a collect traffic request to all SAS that offer the concerned service in his domain. Then, each implicated SAS executes two processes: the network bandwidth quotation and the SAS utilization factor collection. In the network bandwidth quotation, a SAS interact with the mobile anchor point (MAP) of a GLWSM domain, to collect the network bandwidth. Meanwhile, the SAS utilization factor consists of collecting the SAS processor rate. To justify the benefits of our concept, we built a prototype that uses the NS-2 simulator and the RSVP (resource reservation protocol) protocol and we analyzed the consistency of the GLWSA system. Results prove that the proposed concept has a better responsiveness compare to a similar discovery servers with QoS.

Enterprises usually store data in internal storage and install firewalls to protect against intruders to access the data. They also standardize data access procedures to prevent insiders to disclose the information without permission. In cloud computing, the data will be stored in storage provided by service providers. Service providers must have a viable way to protect their clients' data, especially to prevent the data from disclosure by unauthorized insiders. Storing the data in encrypted form is a common method of information privacy protection. If a cloud system is responsible for both tasks on storage and encryption/decryption of data, the system administrators may simultaneously obtain encrypted data and decryption keys. This allows them to access information without authorization and thus poses a risk to information privacy. This study proposes a business model for cloud computing based on the concept of separating the encryption and decryption service from the storage service. Furthermore, the party responsible for the data storage system must not store data in plaintext, and the party responsible for data encryption and decryption must delete all data upon the computation on encryption or decryption is complete. A CRM (Customer Relationship Management) service is described in this paper as an example to illustrate the proposed business model. The exemplary service utilizes three cloud systems, including an encryption and decryption system, a storage system, and a CRM application system. One service provider operates the encryption and decryption system while other providers operate the storage and application systems, according to the core concept of the proposed business model. This paper further includes suggestions for a multi-party Service- Level Agreement (SLA) suitable for use in the proposed business model.

We have proposed a distributed storage system which autonomously migrates a data block into a suitable storage node. In our storage system, heavily accessing data blocks will be migrated autonomously to a higher performance storage node (i.e. an upper storage tier), and rarely accessed data blocks will be migrated to a lower performance storage node (i.e. a lower storage tier). Our investigation shows that it may need several migrations in order to locate a data block onto the most suitable storage node when the destination storage tier has a lot of storage nodes. In this paper, we propose a method to enhance data migration efficiency to improve I/O performance of the whole system. Our simulation results show that the proposed method can greatly reduce I/O execution time than the previous one. The execution time was reduced about one-fifth at the maximum in a system with several hundred scale nodes.

To reduce the probability of data unavailability, it is extremely important to quickly recover failed data in a (k+r, k) erasure-coded storage cluster. In practice, storage nodes in a large-scale storage system have various network bandwidths and I/O capabilities, therefore, the heterogeneity of storage systems increases along with the growing scale. Both traditional recovery scheme and Fastest recovery scheme simply retrieve k surviving blocks from k surviving nodes, thereby resulting in low recovery performance in a heterogeneous storage cluster. In this paper, we propose a Load-aware Recovery Scheme (Lars) for heterogeneous RS-coded storage clusters. Lars not only takes into account both the heterogeneity and load of nodes, but also enables all surviving nodes to service reconstruction reads. The amount of surviving blocks retrieved by a surviving node depends on its load weight which is determined by both network bandwidth and I/O capacity. More blocks are fetched from faster nodes, and vice versa. The three recovery schemes are implemented on a 9-node heterogeneous RS-coded storage cluster, where a set of comparative experiments are conducted. The experimental results show that our Lars scheme outperforms the other two schemes by a factor of up to 1.58.

The paper presents a novel network RAID storage system based on the out-of-band virtualization architecture and the backend centralized redundant management. The application servers can inquire the mapping information of virtual disk from the out-of-band virtualization server and directly access the storage nodes. The read request can fetch the data from special storage node, while the write request is not only stored into the storage node, and also mirrored into the redundant server by the storage node. The redundant server can cache updated data in local disks with log-structured mode, and calculate the parity of RAID5 in the background process when the system is idle. It relieves the bottle problem of I/O performance and low reliability danger of single controller in the front-end centralized management system. And the layout of RAID1/RAID5 on the data block has acquired the trade-off among performance, reliability and cost.

The spread of cloud computing has increased the necessity of accumulating large amounts of data and performing high-speed data processing. Because strict consistency is not necessarily required for such large amount of data that cloud computing stores, a distributed Key Value Stores (KVS) is considered suitable for their data storage, based on an eventual consistency paradigm. In order to provide services such as SNS, mining and statistical processing of these data is indispensable. However because general distributed KVS systems are not designed for processing, these data must be transferred to distributed file systems such as HDFS, which enables data processing. The transfer cost issue has occurred in this case. To find a solution for this issue, we propose a method that performs high-speed data processing directly on a distributed KVS. In this paper, we extend the Apache Cassandra database, a distributed KVS that handles large amounts of data, to enable data affinity-based parallel processing. The parallel data processing mechanism runs the local processing on the stored values at each data node that stores the values, and it then returns only the results of the processing as an answer to a request. From the evaluation experiments, the proposed method is shown to be faster than the typically used Cassandra approach. In addition, even if the writing process is performed in the background while processing the data, the processing efficiency is appropriate for specific loads. The experimental results show that the data processing can be performed during the process of writing at approximately 10 Mbyte/sec if there are eight data nodes in the experiment environment.

The upgrade of data processing and storage system for EAST NBI

Traditional data scheme concepts for biologging data previously relied on traditional relational databases and fixed normalized tables. In practice, this means that a repository contains separate, fixed table structures for each type of sensor data. Prominent examples are the current Wireless Remote Animal Monitoring (WRAM) data schema or the now discontinued ZoaTrack approach. While the traditional approach worked fine as long as few sensors with fixed data types were used, rapid technological development continuously introduces new sensor types and more advanced sensor platforms. This means more data providers, new data types, and rapidly increasing amounts of data. Storage solutions using relational data models generate constant requirements for additional tables, changes to existing table structures, and as a consequence, changes to the overall data scheme in the repository. Further, it becomes very difficult to adapt to emerging international standards, as any change in a particular data field in a single table may have wide ranging consequences to the overall database structure. A concept better suited to deal with the growing amount of sensors and sensor types is the Key-Value Pair (KVP) concept: A KVP is a data type that includes two pieces of data that have a group of key identifiers and a set of associated values. The KVP concept has been used for a long time in data exchange/transport (e.g., JavaScript Object Notation (JSON), ‡ ‡ § §

From social networking websites to bank transactions, we interact with data-intensive applications every day. Such applications are typically hosted on an application server that interacts with a database server to manipulate persistent data. To make such applications efficient, developers face the daunting task of mastering the intricacies of both the programming system and the database management system. For instance, while many application features can be implemented in either the application or pushed into the database, it is difficult for a programmer to decide where to place a given computation as the decision is typically workload-driven. Unfortunately, making the wrong choice often results in drastic performance hit.In this talk, I will show how examining both the programming system and the database management system at the same time allows us to significantly improve the performance of data-intensive applications. To illustrate such cross-system optimization opportunities, I have designed, built, and evaluated three systems: Query By Synthesis, a tool that converts functionality written as imperative code into relational queries; Sloth, a system that combines queries embedded in applications into batches; and Pyxis, a system that seamlessly moves computation between application and database servers. Using real-world examples, I will show that these systems allow orders of magnitude performance improvement and graceful adaptation to changing server environments while preserving the high-level programming interface to the developer. I will furthermore highlight research opportunities and challenges in applying similar techniques to other system problems.