Data-intensive Cloud Computing Research Articles

Improvement of job completion time in data-intensive cloud computing applications

Task stragglers in MapReduce jobs dramatically impede job execution of data-intensive computing in cloud data centers. This impedance is due to the uneven distribution of input data, heterogeneous data nodes, resource contention situations, and network configurations. Data skew of intermediate data in MapReduce job causes delay failures due to the violation of job completion time. Data-intensive computing frameworks, such as MapReduce or Hadoop YARN, employ HashPartitioner. This partitioner may cause intermediate data skew, which results in straggler reducers. In this paper, we strive to make Hadoop YARN more efficient in cloud environments. We present, a new partitioning scheme, called balanced data clusters partitioner (BDCP), to handle straggler Reduce tasks based on sampling of input data and feedback information about the current processing task. Our extensive experimental results show that BDCP can outperform the default Hadoop HashPartitioner and Range partitioner. BDCP can assist in straggler mitigation during reduce phase and minimize the job completion time in MapReduce jobs within data-intensive cloud computing.

Using arced axes in parallel coordinates geometry for high dimensional BigData visual analytics in cloud computing

With the rapid growth of data in size and complexity, that are available on shared cloud computing platform, the threat of malicious activities and computer crimes has increased accordingly. Thus, investigating efficient data visualization techniques for visual analytics of such big data and visual intrusion detection over data intensive cloud computing is urgently required. In this paper, we first propose a new parallel coordinates visualization method that uses arced-axis for high-dimensional data representation. This new geometrical scheme can be efficiently used to identify the main features of network attacks by displaying recognizable visual patterns. In addition, with the aim of visualizing the clear and detailed structure of the dataset according to the contribution of each attribute, we propose a meaningful layout for the new method based on singular value decomposition algorithm, which possesses statistical property and can overcome the curse of dimensionality. Finally, we design a prototype system for network scan detection, which is based on our visualization approach. The experiments have shown that our approach is effective in visualizing multivariate datasets and detecting attacks from a variety of networking patterns, such as the features of DDoS attacks.

Data-Intensive Cloud Computing: Requirements, Expectations, Challenges, and Solutions

Data-intensive systems encompass terabytes to petabytes of data. Such systems require massive storage and intensive computational power in order to execute complex queries and generate timely results. Further, the rate at which this data is being generated induces extensive challenges of data storage, linking, and processing. A data-intensive cloud provides an abstraction of high availability, usability, and efficiency to users. However, underlying this abstraction, there are stringent requirements and challenges to facilitate scalable and resourceful services through effective physical infrastructure, smart networking solutions, intelligent software tools, and useful software approaches. This paper analyzes the extensive requirements which exist in data-intensive clouds, describes various challenges related to the paradigm, and assess numerous solutions in meeting these requirements and challenges. It provides a detailed study of the solutions and analyzes their capabilities in meeting emerging needs of widespread applications.

Environmental Modeling for Automated Cloud Application Testing

Platforms such as Windows Azure let applications conduct data-intensive cloud computing. Unit testing can help ensure high-quality development of such applications, but the results depend on test inputs and the cloud environment's state. Manually providing various test inputs and cloud states is laborious and time-consuming. However, automated test generation must simulate various cloud states to achieve effective testing. To address this challenge, a proposed approach models the cloud environment and applies dynamic symbolic execution to generate test inputs and cloud states. Applying this approach to open-source Azure cloud applications shows that it can achieve high structural coverage.