Resource Broker Research Articles

Semantic-enabled CARE Resource Broker (SeCRB) for managing grid and cloud environment

Grid computing is mainly helpful for executing high-performance computing applications. However, conventional grid resources sometimes fail to offer a dynamic application execution environment and this increases the rate at which the job requests of users are rejected. Integrating emerging virtualization technologies in grid and cloud computing facilitates the provision of dynamic virtual resources in the required execution environment. Resource brokers play a significant role in managing grid and cloud resources as well as identifying potential resources that satisfy users' application requests. This research paper proposes a semantic-enabled CARE Resource Broker (SeCRB) that provides a common framework to describe grid and cloud resources, and to discover them in an intelligent manner by considering software, hardware and quality of service (QoS) requirements. The proposed semantic resource discovery mechanism classifies the resources into three categories viz., exact, high-similarity subsume and high-similarity plug-in regions. To achieve the necessary user QoS requirements, we have included a service level agreement (SLA) negotiation mechanism that pairs users' QoS requirements with matching resources to guarantee the execution of applications, and to achieve the desired QoS of users. Finally, we have implemented the QoS-based resource scheduling mechanism that selects the resources from the SLA negotiation accepted list in an optimal manner. The proposed work is simulated and evaluated by submitting real-world bio-informatics and image processing application for various test cases. The result of the experiment shows that for jobs submitted to the resource broker, job rejection rate is reduced while job success and scheduling rates are increased, thus making the resource management system more efficient.

An effective monitoring framework and user interface design

SummaryA distributed environment requires a monitoring system to oversee the operation of various distributed nodes. A monitoring service is crucial because it ensures a high‐quality computing environment and a reliable service. The interface and framework determine the effectiveness of a monitoring system. This paper uses the concept of user‐adaptive visualization to design its interface and proposes a flexible modular framework. Designers can use the proposed modular framework to flexibly extend existing modules, design visual interfaces to satisfy user requirements, and improve system failover schemes. The implementation of such a monitoring system for monitoring data preservation nodes is also provided. The system including fault‐tolerance and notification functions supports full monitoring services for Storage Resource Broker (SRB) or integrated Rule‐Oriented Data System (iRODS) based systems. The experimental results show that the proposed framework is suitable for data preservation services and is robust and responsive when faced with system failures. Copyright © 2014 John Wiley & Sons, Ltd.

A Novel Resource Pricing Mechanism based on Multi-Player Gaming Model in Cloud Environments

Recently, cloud computing has become a promising network platform for non-trivial applications. The key feature of cloud computing is on-demand resource provision by utility paradigm. Therefore, resource pricing mechanism plays an important role for realizing on-demand resource provision. Unfortunately, existing resource pricing mechanism will result in many negative effects on system performance, such as low-efficient negotiation, extra communication costs and etc. In this paper, we present a novel resource pricing mechanism, which is based on multi- player gaming model and capable of realizing batch resource allocation in an efficient manner. In this mechanism, we introduce a set of virtual resource brokers which is responsible for figuring most rationale resource prices in elastic cloud environment. To investigate the effectiveness and performance of our pricing mechanism, extensive experiments are conducted, and we compare its performance with various approaches, include market- based approaches and auction mechanisms. The results show that our pricing mechanism is more effective to realizing batch resource allocation, especially when the system workload is intensive. In addition, our mechanism can significantly reduce the negotiation costs comparing with exiting approaches. Index Terms—cloud computing, pricing mechanism, virtual resources, computing economy, cooperative game

On Construction of a Multi-Grid Resource Selection Strategy on Grids

Grid computing is now in widespread use, integrating geographical computing resources across multiple virtual organizations to achieve high performance computing. A single grid does not often provide a vast resource because virtual organizations have inadequate computing resource restrictions for management on an organizational scale. This paper presents a new grid architecture named Multi-Grid, which integrates multiple computational grids from different virtual organizations. This study builds a resource broker on multiple grid environments, integrating a number of single grids from different virtual organizations without the limit of organizations. The purpose of the multiple-grid resource is to avoid wasting resources. In addition, this study proposes a Multi-Grid Resource Selection Strategy (MRGSS) for the resource broker to better allocate resources before submitting jobs, to avoid network congestion that consequently causes a decrease in performance.

SLA-based negotiation and brokering of cloud resources

Cloud computing represents an opportunity for IT users to reduce costs and increase efficiency providing an alternative way of using IT services. Elastic provisioning plays an important role by giving the possibility to get the best resources configuration that satisfies the application requirements. Even if there are many cloud providers, with a rich and various offer of technological solutions, above all at IAAS, however there is neither support of SLA negotiation, nor of SLA management yet. One of the most important issue in such a context is the lack of negotiation interfaces and mechanisms by current providers for dynamic provisioning, which instead only make available a configuration form to submit the request for a specific resource among the available ones. In this paper, we propose the design of a tool for SLA-based dynamic provisioning of cloud resources at IAAS, that offers to the user negotiation and brokering facilities by integrating multiple models. We present a prototype implementation of our architecture using the mOSAIC framework.

Cloud Resource Recycling: An Addition of Species to the Zoo of Virtualised, Overlaid, Federated, Multiplexed and Nested Clouds

The foundation of cloud computing is to leverage virtualised physical computing resources by offering them on demand as elastic infrastructure services. Processing, storage and networking resources can thus be perceived and traded comparable to existing utility services. A restriction of conventional infrastructure services is the missing flexibility for scenarios in which coarse-grained provisioning from a single provider is not sufficient anymore. Cloud federation and multiplexing are recent techniques to overcome the provider binding and lock-in limitation. In this article, we give a brief overview about this research direction and then present a new variety, nested clouds, which helps overcoming the coarse-grained virtualisation, billing and trading limitations. In particular, we contribute a design for the Nested Cloud virtual machine and the corresponding spot market Highly-Virtualising Cloud Resource Broker and report on our experience with these systems. We argue that nested clouds help consumers in decreasing their costs and increasing their flexibility for infrastructure services by recycling and repurposing unused capacities.

CLOUDRB: A framework for scheduling and managing High-Performance Computing (HPC) applications in science cloud

In recent years, the Cloud environment has played a major role in running High-Performance Computing (HPC) applications, which are computationally intensive and data intensive in nature. The High-Performance Computing Cloud (HPCC) or Science Cloud (SC) provides the resources to these types of applications in an on demand and scalable manner. Scheduling of jobs or applications in a Cloud environment is NP-Complete and complex in nature due to the dynamicity of resources and on demand user application requirements. The main motivation behind this research study is to design and develop a CLOUD Resource Broker (CLOUDRB) for efficiently managing cloud resources and completing jobs for scientific applications within a user-specified deadline. It is implemented and integrated with a Deadline-based Job Scheduling and Particle Swarm Optimization (PSO)-based Resource Allocation mechanism. Our proposed approach intends to achieve the objectives of minimizing both execution time and cost based on the defined fitness function. It is simulated by modeling the HPC jobs and Cloud resources using the Matlab programming environment. The simulation results prove the effectiveness of the proposed research work by minimizing the completion time, cost and job rejection ratio and maximizing the number of jobs completing their applications within a deadline and meeting the user’s satisfaction. The proposed work has been tested in our Eucalyptus-based cloud environments by submitting real-world HPC applications and observed the improvements in performance.

Virtual Optical Network Resource Allocation Using PCE Global Concurrent Optimization for Dynamic Deployment of Virtual GMPLS-Controlled WSON

We present dynamic virtual optical network resource allocation (VON RA) algorithms for on-demand deployment of virtual wavelength switched optical networks (WSONs), which is handled by a resource broker with a virtual network topology manager (VNTM). The resource broker with VNTM dynamically configures and deploys independent instances of virtual WSONs over the same physical optical infrastructure and their own independent virtual generalized multiprotocol label switching control plane. The VON RA is computed by a path computation element using global concurrent optimization, which assigns the resources for the virtual optical links (e.g., wavelengths) and nodes. The experimental assessment and performance evaluation have been carried out in the ADRENALINE testbed. We discuss the obtained results on a National Science Foundation Network topology in terms of the VON request blocking rate and VON setup delay, which are used to compare the different proposed algorithms.

Multi-Attribute Auctioning Resource in Grids: Model and Protocols

Auction models and protocols are found efficient in managing resources allocation, which are a key technology in grid computing system. In this paper, a new multi-attribute multi-round reverse auction is proposed, and related reverse auction based protocols are designed. The resource user’s satisfaction degree is introduced into the traditional grid resource allocation problem to help the grid resource broker make multi-attribute decisions with incomplete information. Numerical simulating experiments show that our model and protocols can satisfy the resource user’s quality demand on multiple attributes, and achieve high efficiency in user utility. The results also illustrate that the on-line multi-attribute algorithm in ONMRA protocol has better performance in an on-line setting for grid allocation. DOI: http://dx.doi.org/10.11591/telkomnika.v11i10.3347

Reliable resources brokering scheme in wireless grids based on non-cooperative bargaining game

Wireless computational grids are evolving due to rapid growth in wireless devices to provide any time any where computing and communication services. Wireless devices are considered as important resources for job execution in a wireless grid. However, they are unreliable due to unstable wireless connections, frequent mobility, and device resource constraints such as limited processing capacity, limited battery life, low communication bandwidth, etc. Consumers and providers in a wireless grid have potential conflicting interests: consumers prefer reliable resources at minimum cost for job execution whereas providers prefer efficient utilization of resources with a maximum profit. Hence it is necessary to design a resource brokering mechanism that assures job(s) execution and satisfies needs of consumers and providers. This paper proposes a software agent based resource brokering scheme that employs a resource brokering agency consisting of a set of software agents which discover and recommend reliable devices, negotiate resource cost using non-cooperative bargaining game, and schedule jobs. Device reliability is modeled by using parameters like processing rate, memory, bandwidth and battery power. Discount function is employed in bargaining game based on job deadline, previous offers of an opponent, and dynamics in the grid market. Adaptive negotiation process is applied by exercising different discount strategies (aggressive or conservative or linear) based on the negotiation status of a player. The scheme is simulated to evaluate performance parameters like resource utilization rate, expected profit, job completion time and job execution rate. We observed that the proposed scheme performs better than the existing resource brokering scheme.

Multiple equivalent simultaneous offers strategy in an agent-based grid resource brokering system - initial considerations

The Agent in Grid project (AiG) attempts to integrate the concept of the Grid and an agent-based system to facilitate efficient resource management in the Grid. In this paper, we present preliminary considerations concerning multiple equivalent simultaneous offers strategy that can be used in the Service Level Agreement (SLA) negotiations. These negotiations are the key part of the main use case scenarios within the AiG project. In this context, first, we describe the AiG system. Second, we introduce the simultaneous offer strategy, as a mechanism known from economy, and suggest an approach for using it in the AiG negotiations.

Enabling Interoperability among Grid Meta-Schedulers

The goal of Grid computing is to integrate the usage of computer resources from cooperating partners in the form of Virtual Organizations (VO). One of its key functions is to match jobs to execution resources efficiently. For interoperability between VOs, this matching operation occurs in resource brokering middleware, commonly referred to as the meta-scheduler or meta-broker. In this paper, we present an approach to a meta-scheduler architecture, combining hierarchical and peer-to-peer models for flexibility and extensibility. Interoperability is further promoted through the introduction of a set of protocols, allowing meta-schedulers to maintain sessions and exchange job and resource state using Web Services. Our architecture also incorporates a resource model that enables an efficient resource matching across multiple Virtual Organizations, especially where the compute resources and state are dynamic. Experiments demonstrate these new functional features across three distributed organizations (BSC, FIU, and IBM), that internally use different job scheduling technologies, computing infrastructure and security mechanisms. Performance evaluations through actual system measurements and simulations provide the insights on the architecture's effectiveness and scalability.

Genetic Algorithm based Resource Broker for Computational Grid

In this paper, we present a resource broker architecture for a computational Grid which uses Genetic Algorithm (GA) for brokering. Resource brokering implies selection of appropriate resource providers for jobs submitted to the Grid. Resource brokering is normally done with the objective of optimizing some performance parameter such as minimizing the total cost of running the jobs or maximizing the utilization of Grid resources. It is a challenging task since the search space for the problem consists of all possible allocations of submitted jobs to available resource providers in a Grid and may be very large. GAs are found to be efficient for such optimization problems. Moreover, the configuration and workload of a Grid is dynamic in nature. Our GA based resource broker tries to address these issues so that jobs are scheduled efficiently.

Heuristic Scheduling Algorithms for Allocation of Virtualized Network and Computing Resources

Cloud computing technology facilitates computing-intensive applications by providing virtualized resources which can be dynamically provisioned. However, user’s requests are varied according to different applications’ computation ability needs. These applications can be presented as meta-job of user’s demand. The total processing time of these jobs may need data transmission time over the Internet as well as the completed time of jobs to execute on the virtual machine must be taken into account. In this paper, we presented V-heuristics scheduling algorithm for allocation of virtualized network and computing resources under user’s constraint which applied into a service-oriented resource broker for jobs scheduling. This scheduling algorithm takes into account both data transmission time and computation time that related to virtualized network and virtual machine. The simulation results are compared with three different types of heuristic algorithms under conventional network or virtual network conditions such as MCT, Min-Min and Max-Min. e evaluate these algorithms within a simulated cloud environment via an abilenenetwork topology which is real physical core network topology. These experimental results show that V-heuristic scheduling algorithm achieved significant performance gain for a variety of applications in terms of load balance, Makespan, average resource utilization and total processing time.

Dynamic virtual GMPLS-controlled WSON using a Resource Broker with a VNT Manager on the ADRENALINE testbed

We present a Resource Broker with a Virtual Network Topology Manager (VNTM) which dynamically deploys virtual GMPLS-controlled WSON networks. Virtual Optical links are constructed by grouping established optical connections which are managed by the VNTM. We evaluate the performance of the Resource Broker in the ADRENALINE testbed.

An SLA-based Broker for Cloud Infrastructures

The breakthrough of Cloud comes from its service oriented perspective where everything, including the infrastructure, is provided "as a service". This model is really attractive and convenient for both providers and consumers, as a consequence the Cloud paradigm is quickly growing and widely spreading, also in non commercial contexts. In such a scenario, we propose to incorporate some elements of volunteer computing into the Cloud paradigm through the Cloud@Home solution, involving into the mix nodes and devices provided by potentially any owners or administrators, disclosing high computational resources to contributors and also allowing to maximize their utilization. This paper presents and discusses the first step towards Cloud@Home: providing quality of service and service level agreement facilities on top of unreliable, intermittent Cloud providers. Some of the main issues and challenges of Cloud@Home, such as the monitoring, management and brokering of resources according to service level requirements are addressed through the design of a framework core architecture. All the tasks committed to the architecture's modules and components, as well as the most relevant component interactions, are identified and discussed from both the structural and the behavioural viewpoints. Some encouraging experiments on an early implementation prototype deployed in a real testing environment are also documented in the paper.

Enhancing performance of failure-prone clusters by adaptive provisioning of cloud resources

In this paper, we investigate Cloud computing resource provisioning to extend the computing capacity of local clusters in the presence of failures. We consider three steps in the resource provisioning including resource brokering, dispatch sequences, and scheduling. The proposed brokering strategy is based on the stochastic analysis of routing in distributed parallel queues and takes into account the response time of the Cloud provider and the local cluster while considering computing cost of both sides. Moreover, we propose dispatching with probabilistic and deterministic sequences to redirect requests to the resource providers. We also incorporate checkpointing in some well-known scheduling algorithms to provide a fault-tolerant environment. We propose two cost-aware and failure-aware provisioning policies that can be utilized by an organization that operates a cluster managed by virtual machine technology, and seeks to use resources from a public Cloud provider. Simulation results demonstrate that the proposed policies improve the response time of users' requests by a factor of 4.10 under a moderate load with a limited cost on a public Cloud.

Брокер ресурсов для поддержки проблемно-ориентированных грид-сред

This paper describes scheduling methods and algorithms of resources, and also the development on their base of the broker resource for search optimum resources in problem-oriented grid-environment. The developed scheduling algorithm considers additional knowledge about subject domain specifics of tasks and the representation about a workflow. The algorithm is based on a dominant sequence clustering algorithm (DSC). Unlike the original algorithm is that, for mapping tasks on the computing resources used by task graph coloring and the merging of tasks in a cluster is based on the availability of free slots on computing resources. Proposed a diphasic reservation method of resources and accounting problem of the problem parameters to estimate the time of its execution. Use cases of the broker resource are also given, process of resource allocation and architecture of the resource broker CAEBeans Broker are described.

Replication based fault tolerant job scheduling strategy for economy driven grid

In this paper, the problem of fault tolerance in grid computing is addressed and a novel adaptive task replication based fault tolerant job scheduling strategy for economy driven grid is proposed. The proposed strategy maintains fault history of the resources termed as resource fault index. Fault index entry for the resource is updated based on successful completion or failure of an assigned task by the grid resource. Grid Resource Broker then replicates the task (submitting the same task to different backup resources) with different intensity, based on vulnerability of resource towards faults suggested by resource fault index. Consequently, in case of possible fault at a resource the results of replicated task(s) on other backup resource(s) can be used. Hence, user job(s) can be completed within specified deadline and assigned budget, even on the event of faults at the grid resource(s). Through extensive simulations, performance of the proposed strategy is evaluated and compared with the Time Optimization and Checkpointing based Strategy in an economy driven grid environment. The experimental results demonstrate that in the presence of faults, proposed fault tolerant strategy improves the number of tasks completed with varied deadline and fixed budget as well as number of tasks completed with varied budget and fixed deadline. Additionally, the proposed strategy used a smaller percentage of deadline time as compare to both Time Optimization and Checkpointing based Strategy. Although the proposed strategy has a percentage of budget spent greater than that of Time Optimization Strategy and Checkpointing based Strategy, it is accepted as a proposed strategy in time optimization where the main objective is to maximize tasks completed within a given deadline. It can be concluded from the experiments that the proposed strategy shows improvement in satisfying the user QoS requirements. It can effectively schedule tasks and tolerate faults gracefully even in the presence of failures, but the costs are slightly higher in terms of budget consumption. Hence, the proposed fault tolerant strategy helps in sustaining user's faith in the grid, by enabling the grid to deliver reliable and consistent performance in the presence of faults.

Modeling and Simulation of Grid Resource Brokering Algorithms

Computing is concerned with applying the heteregenous resources of many computers to solve a single problem and involves managing the diverse resources towards a common objective. Successful utilization of grid infrastructure to solve resource intensive and computing problems requires performance modeling and evaluation to meet the QoS requirements of end users. Resource management and scheduling is the most important component of grid systems. A Grid scheduler must make resource selection decisions in an environment where it has no control over the local resources, the resources are distributed, and information about the systems is often limited or dated. Grid Resource broker or meta-scheduler uses local schedulers of the different grid middleware and local schedulers of clusters to allocate jobs to distributed resources. To address different issues in grid scheduling, different scheduling approaches and algorithms have been proposed in the literature. However, evaluation and comparative analysis of these algorithms and research experiments are often difficult to perform due to problems like large number of heterogeneous resources, dynamic nature of grid, not able to create different types of realistic workloads and jobs with different parameters and lack of certain functionalities in available resource management systems like advance reservation (AR) and grid usage accounting. This paper focuses on modeling and simulation of grid towards achieving various grid performance metrics and QoS requirements. This paper also presents the experimental results on grid simulation and performance evaluation different grid resource brokering approaches using synthetic workloads. Different simulation experiments are used to compare different aspects of scheduling using different types of grid job, input resources and workloads. The simulation results indicate the effect of resource brokering approach used on efficient execution of grid jobs, success ratio of jobs with QoS requirement, resource utilization and load balancing of grid system.