Grid Resource Discovery Research Articles

Discovery of resources using MADM approaches for parallel and distributed computing

Grid, a form of parallel and distributed computing, allows the sharing of data and computational resources among its users from various geographical locations. The grid resources are diverse in terms of their underlying attributes. The majority of the state-of-the-art resource discovery techniques rely on the static resource attributes during resource selection. However, the matching resources based on the static resource attributes may not be the most appropriate resources for the execution of user applications because they may have heavy job loads, less storage space or less working memory (RAM). Hence, there is a need to consider the current state of the resources in order to find the most suitable resources. In this paper, we have proposed a two-phased multi-attribute decision making (MADM) approach for discovery of grid resources by using P2P formalism. The proposed approach considers multiple resource attributes for decision making of resource selection and provides the best suitable resource(s) to grid users. The first phase describes a mechanism to discover all matching resources and applies SAW method to shortlist the top ranked resources, which are communicated to the requesting super-peer. The second phase of our proposed methodology applies integrated MADM approach (AHP enriched PROMETHEE-II) on the list of selected resources received from different super-peers. The pairwise comparison of the resources with respect to their attributes is made and the rank of each resource is determined. The top ranked resource is then communicated to the grid user by the grid scheduler. Our proposed methodology enables the grid scheduler to allocate the most suitable resource to the user application and also reduces the search complexity by filtering out the less suitable resources during resource discovery.

A Weight-based Query Forwarding Technique for Super-peer-based Grid Resource Discovery

A Weight-based Query Forwarding Technique for Super-peer-based Grid Resource Discovery

A Weight-based Query Forwarding Technique for Super-peer-based Grid Resource Discovery

Grid computing environments include heterogeneous resources shared by a large number of computers to handle data and process intensive applications. The required resources must be accessible for the grid applications on demand, which makes resource discovery a critical service. In recent years, different techniques are provided to index and discover grid resources. Response time and message load during the search process highly affect the efficiency of resource discovery. This paper proposes a technique to forward the queries based on the resource types accessible through each neighbor in super-peer-based grid resource discovery approaches. The proposed technique is simulated in GridSim and the experimental results indicated that it is able to reduce the response time and message load during the search process especially when the grid environment contains a large number of nodes.

Semantic Grid resource discovery based on SKOS ontology

Semantic Grid resource discovery based on SKOS ontology

Semantic Grid resource discovery based on SKOS ontology

In large-scale distributed environments like dynamic Grids, an efficient and scalable resource discovery mechanism is required. Hybrid techniques based on node clustering are considered among solutions that guarantee the efficiency and the scalability at the same time. However, lack of a good representation of nodes in these environments often results in an irrelevant grouping of nodes. Combining hybrid techniques with Semantic Web technologies would bring more benefits to resource discovery. This paper introduces a semantic clustering of nodes based on their domains of interest to form groups called federations. It presents a new process for constructing federations and a three-layered overlay network. The proposed process is based on a SKOS lightweight ontology that describes domains of applications in the Grid. We conduct extensive simulations to evaluate the performance of the proposed approach; the experimental results demonstrate its efficiency and its ability to scale up with the system size.

Using Network Traffic to Infer Hardware State

In this article, we illustrate that the boundary of a general-purpose node can be extended into the network by extracting information from network traffic generated by that general-purpose node to infer the state of its hardware components. This information is represented in a delay signature latent within the network traffic. In contrast, the traditional approach to determine the internal state of a node’s resources meant that a software application with internal processes had to be resident on the node. The aforementioned delay signature is the keystone that provides a correlation between network traffic and the internal state of the source node. We characterize this delay signature by (1) identifying the different types of assembly language instructions that source this delay and (2) describing how architectural techniques, such as instruction pipelining and caching, give rise to this delay signature. In theory, highly utilized nodes (due to multiple threads) will contain excessive context switching and contention for shared resources. One important shared resource is main memory, and excessive use of this resource by applications and internal processes eventually leads to a decrease in cache efficiency that eventually stalls the instruction pipeline. Our results support this theory; specifically, we have observed that excessive context switching in active applications increases the effective memory access time and wastes precious CPU cycles, thus adding additional delay to the execution of load, store, and other instructions. Because the operating system (OS) kernel accesses memory to send network packets, the delay signature is induced into network traffic in situations where user-level utilization is high. We demonstrate this theory in two case studies: (1) resource discovery in cluster grids and (2) network-based detection of bitcoin mining on compromised nodes.

A Comparative Analysis of Adaptive Solutions for Grid Environments

Grid computing environments are distributed systems composed by heterogeneous and geographically distributed resources. This type of systems mainly emerged to satisfy the increasing computing power demand within the scientific community. Despite the advantages of such paradigm, there are still several challenges related to the discovery, monitoring and selection of grid resources. Moreover, the dynamic nature and changing characteristics of such environments worsen the applications performance. Thus, improving their efficiency is a fundamental issue. The present contribution analyses two self-adaptive solutions focused on enhancing the grid resource selection process by using resources in an efficient way. On the one hand, the Efficient Resources Selection model which is defined from the user's point of view (it avoids controlling or modifying the infrastructure) and it is based on the Scatter Search method for achieving a suitable selection of resources. On the other hand, Montera2, a framework designed for addressing an efficient execution of distributed applications on the grid; it defines and employs a dynamic scheduling algorithm to determine the size and number of tasks to be executed. Both approaches have been tested on a real European infrastructure belonging to the well-known European Grid Infrastructure (EGI) project. The study also compares both solutions with the standard scheduling technique that governs this infrastructure, the gLiteWMS scheduler, showing a much better performance by reducing the final makespan by a factor of 20 if compared to the gLiteWMS scheduler. An analysis of task and time overheads for both approaches is also included. Furthermore, comparisons with many other solutions proposed in the literature are presented, showing the advantages of our approaches.

THE EFFICIENCY CHALLENGES OF RESOURCE DISCOVERY IN GRID ENVIRONMENTS

Resource discovery is one of the most important services that significantly affects the efficiency of grid computing systems. The inherent dynamic and large-scale characteristics of grid environments make their resource discovery a challenging task. In recent years, different approaches have been proposed for resource discovery, attempting to tackle the challenges of grid environments and improve the efficiency. Being aware of these challenges and approaches is worthwhile in order to choose an appropriate approach according to the application in different organizations. This study reviews the most important factors that should be considered and challenges to be tackled in order to develop an efficient grid resource discovery system.

Distributed optimization Grid resource discovery

Grid computing is a framework for large-scale resource sharing and indexing that evolves with the goal of resource provisioning. In this paper, we develop a distributed learning automata (DLA) based on multi-swarm discrete particle swarm optimization (PSO) approach for Grid resource discovery, called distributed optimization grid (DOG) resource discovery algorithm. This algorithm makes use of swarms of particles for different computational resource metrics while a group of DLA is the control unit of each swarm of particles. The algorithm takes advantage of the PSO solution diversity to optimize the quality of delivered resource. Moreover, the recommended algorithm uses DLA as a fully distributed model for imitating the Grid infrastructure topology. Our experimental results show that DOG is fast as well as efficient and accurate.

THRD: Threshold-based hierarchical resource discovery for Grid environments

The resource discovery is one of the most important services in Grid systems because providing the resources is critical to handle the applications. But some inherent characteristics of Grid environments such as large-scale and dynamicity make the resource discovery a challenging task. The hierarchical structure is widely used in Grid resource discovery but it suffers from high message load especially in upper level indexing nodes. This paper proposes a technique to reduce the message load of upper level indexing nodes in the hierarchical resource discovery. It applies a threshold value for in-process messages of indexing nodes to limit the queries passed to upper levels. The queries are checked in the sibling nodes of current level when they are not allowed to be passed to the parent node.The proposed technique is simulated in GridSim and experimented by different configurations of threshold values and number of Grid nodes. The experimental results showed that the proposed technique is able to reduce the message load of indexing nodes as 21.2 % however it also affects negatively on the response time of issued resource queries as 19.3 %. Therefore, the proposed technique is an appropriate solution for the applications with priority of lower message loads relative to the response time.

Grid Electricity Market Bidding Model Based on Agent

Grid technology can provide electricity market bidding to achieve the ability to provide large-scale computing. Agent technology has the autonomy, interoperability, and self-learning, can effectively solve the grid resource discovery and management, computing task allocation and scheduling problems, shared grid environment of heterogeneous computing resources. Introduction grid technology related content, Agent technology and proposes a use of Agent technology in the electricity market bidding model grid environment. Bidding able to provide the technology platform the parties a fair trade.

A New Method for Resource Discovery in a Grid Resource Service Chain

A new decentralized method is proposed to resource discovery in Grid computing systems. This agent-based method supports semantic description and is flexible for resource identification and allocation. It allows each of the individual resource agents to interact only with its neighbor agents semantically and form a resource chain for a specific task dynamically. We discuss about the success of probability of our method under different task loading rates and compare it with some other known Algorithms. The experiments show the algorithm could flexibly and dynamically discover resources.

Data resource discovery model based on hybrid architecture in data grid environment

SummaryToday, the management of massive data collections draws much attention as data grids have been developed to deal with large computational problems and provide the opportunity for sharing geographically distributed resources for large‒scale data‒intensive applications. Therefore, finding an effective approach to discover data resources in order to promote better interactions between application communities or virtual organizations becomes a critical challenge. Traditional grid resource discovery models are mostly based on central and hierarchical architecture that can lead to bottlenecking with the expansion of the grid scale. Although the Peer‒to‒Peer (P2P) technique is integrated into the grid in order to improve the performance in recent years, each P2P structure still has drawbacks that require several compensatory strategies. In this paper, based on the unstructured super‒node‒based architecture from the P2P system, we design a structured logic resource tree in each domain in order to effectively alleviate the load on the super‒node, and we propose a query recording learning algorithm based on this hybrid architecture to reduce traffic in the network and greatly shorten the response time. The model and algorithm are validated by simulations and compared with the traditional super‒peer model and the flooding‒based approach. Copyright © 2014 John Wiley & Sons, Ltd.

Grid Resource Discovery Algorithm Based on Distance

Resource discovery in a Grid environment is a critical problem and also a fundamental task which provides searching and locating necessary resources for given processes. Based on the domain and resource routing nodes, a grid resource discovery model of multilayer overlay network is given in this paper. And on this basis, using a linear combination of the block distance and chessboard distance instead of Euclidean distance, grid resource discovery algorithm based on distance is proposed. Experiment shows that the algorithm has low cost, fast response and can obtain better success rate of lookup, as well as the effectiveness of the resource discovery algorithm.

Grid Resource Discovery Algorithm of the Multi-Layer Overlay Network Model Based on Distance

The resource discovery mechanism is a hot topic issue in grid environments and It has great impact on the efficiency of the resource sharing and cooperative computing. Based on the domain and resource routing nodes, a grid resource discovery model of multilayer overlay network was given in this paper. And on this basis, using a linear combination of the block distance and chessboard distance instead of Euclidean distance, grid resource discovery algorithm based on distance was proposed. Experiment shows that the algorithm has low cost, fast response and can obtain better success rate of lookup, as well as the effectiveness of the resource discovery algorithm.

New Approaches with Chord in Efficient P2p Grid Resource Discovery

New Approaches with Chord in Efficient P2p Grid Resource Discovery

Dual Heap Overlay for Range Queries for Handling Hardware Resources

Grid resource discovery involves resolving various types of queries including exact match and range queries. Our recent effort resulted in Dual Heap Overlay Resource Discovery Protocol for a Mobile Grid (DEEPG) to resolve exact match queries. DEEPG attempted to reduce the search set for query resolution by rejecting nodes which do not satisfy the sum of the attributes specified in the query. In this paper, we propose DEEPG-R, a Dual Heap Overlay Resource Discovery Protocol for Range Queries in Grid. Distributed Hash Table (DHT) based resource discovery mechanism is common in the field and fails to handle few challenges. Hence, DEEPG-R employs the non-DHT based overlays of DEEPG and is tuned to handle range queries. The focus of the extended protocol is on reducing the number of comparisons further by comparing the resource attributes with the lower and upper bounds specified in the range query. Simulation results prove that the proposed protocol is efficient with reduced search time complexities and low maintenance overheads.

P2P-based resource discovery in dynamic grids allowing multi-attribute and range queries

A key point for the efficient use of large grid systems is the discovery of resources, and this task becomes more complicated as the size of the system grows up. In this case, large amounts of information on the available resources must be stored and kept up-to-date along the system so that it can be queried by users to find resources meeting specific requirements (e.g. a given operating system or available memory). Thus, three tasks must be performed, (1) information on resources must be gathered and processed, (2) such processed information has to be disseminated over the system, and (3) upon users' requests, the system must be able to discover resources meeting some requirements using the processed information. This paper presents a new technique for the discovery of resources in grids which can be used in the case of multi-attribute (e.g. {OS=Linux &memory=4GB}) and range queries (e.g. {50GB

A MULTI-ATTRIBUTE RESOURCE DISCOVERY ALGORITHM FOR PEER-TO-PEER GRIDS

Centralized or hierarchical administration of the classical Grid resource discovery approaches is unable to efficiently manage the highly dynamic large‐scale Grid environments. In this study, a multi-attribute distributed learning automata-based resource discovery algorithm called MDLRD is proposed for large-scale peer-to-peer (P2P) Grids. Taking advantage of the learning automata theory, the proposed method routes the resource query through the path having the minimum expected hop count toward the Grid peers including the requested resources. Therefore, MDLRD significantly reduces the message overhead of the unstructured P2P resource discovery methods in which the resource queries are flooded within the network. Furthermore, MDLRD fully supports the multi-attribute range query that is impossible in structured P2P resource discovery approaches. A strong theorem is presented to show the convergence of the proposed distributed learning automata-based algorithm to the optimal solution. To investigate the performance of the proposed method, several simulation experiments are conducted. The obtained results confirm that MDLRD significantly outperforms the other methods in terms of the average hop count, average hit ratio, and control message overhead.

High Performance Hypercube for Better Resource Discovery and Allocation in Simulated Grid based Environment

High Performance Hypercube for Better Resource Discovery and Allocation in Simulated Grid based Environment