Grid Information Service Research Articles

Resource selection in computational grids based on learning automata

Grid computing, in simplest terms, is distributed computations that have reached higher evolution level. Grid scheduler is part of Grid. Scheduler generates an assignment of jobs to resources using the resource information provided by the Grid information service. Since the problems raised in the resource management system are NP-hard, classical methods, such as dynamic programming, are useful only for small-size problems. Being capable of producing efficient schedules at an acceptable time, even for large samples of the problem, heuristic algorithms are promising methods for solving the scheduling problem. The resource scheduling process in the Grid consists of three main phases: resource discovery, resource selection and job execution. In this paper, we propose an algorithm based on learning automata to resource selection in computational Grid. In this algorithm, decisions are made based on the list of resources that discovered at the resource discovery phase, and after being selected based on predicted time to execution or completion job, they would sent to the next phase namely the execution phase. The efficiency of the proposed algorithm is evaluated through conducting several simulation experiments under different Grid scenarios. The obtained results are compared with several existing methods in terms of the average turn-around time, average response time and throughput.

Exploiting resource profiling mechanism for large-scale scientific computing on grids

Large-scale scientific applications from various scientific domains (e.g., astronomy, physics, pharmaceuticals, chemistry, etc.) usually require substantial amounts of computing resources and storage space. International Grid computing resources can be a viable choice for supporting these challenging applications so that effectively locating suitable computing resources with minimal allocation overhead can be crucial. However, Grid resource availability is highly unstable and current Grid Information Service (GIS) cannot provide accurate state information. This can make it very difficult for users to schedule the jobs on the Grid system and to map tasks on appropriate available resources. In this paper, we present SCOUT system that can periodically profile Grid computing elements based on available number of CPU cores and average response time, and monitor the performance of each CE in the Virtual Organizations (VO). Micro-benchmark experimental results demonstrate that leveraging profiled data by SCOUT can improve the success rate of task executions and reduce the average response time.

Grid Resource Ontology: a Keystone of Semantic Grid Information Service

This work presents a set of ontologies to facilitate the adoption of semantic technologies in Grid middleware and to provide a solid foundation for development of semantic Grid information services. The scope of the proposed ontologies includes a description of Grid resources and functionalities, interrelationship of all Grid components, subsystems and services, definition of administrative and usage policies, etc. The work also presents a set of tools to populate the provided Grid resource ontologies with real-data instance assertions by interrogating a traditional Grid information service and discusses the benefits of such representation of Grid resources.

Configuration management and monitoring of the middleware at GridKa

GridKa is a computing centre located in Karlsruhe. It serves as Tier-1 centre for the four LHC experiments and also provides its computing and storage resources for other non-LHC HEP and astroparticle physics experiments as well as for several communities of the German Grid Initiative D-Grid. The middleware layer at GridKa comprises three main flavours: Globus, gLite and UNICORE. This layer provides the access to the several clusters, according to the requirements of the corresponding communities. The heterogeneous structure of middleware resources and services requires their effective administration for stable and sustainable operation of the whole computing centre. In the presentation the overview of the middleware system at GridKa is given with focus on the configuration management and monitoring. These are the crucial components of the administration task for the system with high-availability setup. The various configuration tools used at GridKa, their benefits and limitations as well as developed automation procedures of the configuration management will be discussed. The overview of the monitoring system which evaluates the information delivered by central and local grid information services and provides status and detailed diagnostics for the middleware services is presented.

Proposal of Grid Monitoring System with Fault Tolerance

A Grid monitoring system is differentiated from a general monitoring system in that it must be scalable across wide-area networks, include a large number of heterogeneous resources, and be integrated with the other Grid middleware in terms of naming and security issues. A Grid Monitoring is the act of collecting information concerning the characteristics and status of resources of interest. The Grid Monitoring Architecture (GMA) specification sets out the requirements and constraints of any implementation. It is based on simple Consumer/Producer architecture with an integrated system registry and distinguishes transmission of monitoring data and data discovery logically. There are many systems that implement GMA but all have some drawbacks such as, difficult installation, single point of failure, or loss of message control. So we design a simple model after we analyze the requirements of Grid monitoring and information service. We propose a grid monitoring system based on GMA. The proposed Grid monitoring system consists of producers, registry, consumers, and failover registry. The registry is used to match the consumer with one or more producers, so it is the main monitoring tool. The failover registry is used to recover any failure in the main registry. The structure of a proposed grid monitoring system depends on java Servlet and SQL query language. This makes the system more flexible and scalable. We try to solve some problems of the previous works in a Grid monitoring system such as, lack of data flow and single point of failure in R-GMA, and difficulty of installing in MDS4. Firstly, we solve the problem of single point of failure by adding failover registry to the system. It can recover any failure in Registry node. Secondly, we take into consideration the system components to be easy to install/maintain. The proposed system is combination of few systems and frequency of update is low. Thirdly, load balancing should be added to the system to overcome the message overloaded. We evaluate the performance of the system by measuring the response time, utilization, and throughput. The result with load balancing is better than that without load balancing in all evaluation results. Finally, we make a comparison between the proposed system and the other three monitoring systems. We also make a comparison between the four types of load balancing algorithms.

WiGriMMA: A Wireless Grid Monitoring Model Using Agents

The complexity, heterogeneity, device mobility and the unpredictable user behavior demands proper automation of monitoring activity in the wireless Grid to enable the user needs. Since the wireless devices can dynamically join/leave the Grid, and its state may be affected by various parameters (like the battery power, signal strength, the number of jobs submitted to it, device mobility, etc.) leading to overload state, it is essential to monitor the devices so that long term resource planning can be achieved. This paper proposes a Wireless Grid Monitoring Model using Agents (WiGriMMA) that monitor the device mobility and state, communicates the state to Grid information server (GIS), provides the resource availability information, controls the selfish users and the device state so that the device is not overloaded. The model is simulated to test its operation effectiveness considering the performance parameters such as resource availability, resource stability, device state, job execution rate, user behavior and agent overhead. The results show that the proposed WiGriMMA performs better than the existing Grid monitoring model (GridView) in terms of the resource availability, device states and the job execution rate.

Provide Virtual Machine Information for Grid Computing

Distributed virtual machines can help to build scalable, manageable, and efficient grid infrastructures. The work proposed in this paper focuses on employing virtual machines for grid computing. In order to efficiently run grid applications, virtual machine resource information should be provided. This paper first discusses the system architecture of virtual machine pools and the process of information retrieval from virtual machines. Based on the characterization of the system model, this paper presents the work on how to retrieve resource information from Xen/VMware virtual machines via VMware Common Information Model Software Development Kit and lightweight Java agents. The resource information is integrated into a grid information service. The work is implemented in a test bed with Xen/VMware virtual machines and the Globus Toolkit. With a performance evaluation and discussion on a real test bed, it is declared that the design and implementation of information services for virtual-machine-based grid systems are feasible, efficient, and scalable.

PIVOT: An adaptive information discovery framework for computational grids

In a traditional computational grid environment, the owners of resources usually provide information about their resources extracted by pre-configured information services or web services. However, such information is not sufficient for the scheduler in the high-performance distributed computing. To solve this problem, we propose a scalable grid information service framework, named PIVOT (adaPtive Information discoVery framewOrk for compuTational grid). By using deadline-constrained flooding collector dissemination and P2P-like information collection schemes, PIVOT provides an active mechanism to collect application-specific resource information. In particular, PIVOT provides a resource information service for application-specific schedulers. The best-effort performance on overhead traffic and communication latency during information discovery is guaranteed by two new distributed cooperative algorithms. The experimental results in the simulations and real computational grid platform demonstrate that PIVOT has a high level of adaptability for application-specific resource information discovery, and also improves the accuracy of resource allocation and the efficiency of executing parallel tasks in traditional information services.

On gLite WMS/LB monitoring and Management through WMSMonitor

The Workload Management System is the gLite service supporting the distributed production and analysis activities of various HEP experiments. It is responsible of dispatching computing jobs to remote computing facilities by matching job requirements and the resource status information collected from the Grid information services. Given the distributed and heterogeneous nature of the Grid, the monitoring of the job lifecycle and of the aggregate workflow patterns generated by multiple user communities, and the reliability of the service are of great importance. In this paper we deal with the problem of WMS monitoring and management. We present the architecture and implementation of the WMSMonitor, a tool for WMS monitoring and management, which has been designed to meet the needs of various WMS user categories: administrators, developers, advanced Grid users and performance testers. The tool was successfully deployed to monitor the progress of WMS job submission activities during HEP computing challenges. We also describe how, for each WMS in a cluster, WMSMonitor produces status indexes and a load metric that can be used for automated notification of critical events via Nagios, or for ranking of service instances deployed in load balancing mode.

Grid computing enhances standards-compatible geospatial catalogue service

A catalogue service facilitates sharing, discovery, retrieval, management of, and access to large volumes of distributed geospatial resources, for example data, services, applications, and their replicas on the Internet. Grid computing provides an infrastructure for effective use of computing, storage, and other resources available online. The Open Geospatial Consortium has proposed a catalogue service specification and a series of profiles for promoting the interoperability of geospatial resources. By referring to the profile of the catalogue service for Web, an innovative information model of a catalogue service is proposed to offer Grid-enabled registry, management, retrieval of and access to geospatial resources and their replicas. This information model extends the e-business registry information model by adopting several geospatial data and service metadata standards—the International Organization for Standardization (ISO)’s 19115/19119 standards and the US Federal Geographic Data Committee (FGDC) and US National Aeronautics and Space Administration (NASA) metadata standards for describing and indexing geospatial resources. In order to select the optimal geospatial resources and their replicas managed by the Grid, the Grid data management service and information service from the Globus Toolkits are closely integrated with the extended catalogue information model. Based on this new model, a catalogue service is implemented first as a Web service. Then, the catalogue service is further developed as a Grid service conforming to Grid service specifications. The catalogue service can be deployed in both the Web and Grid environments and accessed by standard Web services or authorized Grid services, respectively. The catalogue service has been implemented at the George Mason University/Center for Spatial Information Science and Systems (GMU/CSISS), managing more than 17 TB of geospatial data and geospatial Grid services. This service makes it easy to share and interoperate geospatial resources by using Grid technology and extends Grid technology into the geoscience communities.

A Federated Approach to Information Management in Grids

We propose a novel approach to managing information in grids. The proposed approach is an add-on information system that provides unification and federation of grid information services. The system interacts with local information services and assembles their metadata instances under one hybrid architecture to provide a common query/publish interface to different kinds of metadata. The system also supports interoperability of major grid information services by providing federated information management. We present the semantics and architectural design for this system. We introduce a prototype implementation and present its evaluation. As the results indicate, the proposed system achieves unification and federation of custom implementations of grid information services with negligible processing overheads.

Modeling Scalable Grid Information Services with Colored Petri Nets

Information services play a crucial role in grid computing environments in that the state information of a grid system can be used to facilitate the discovery of resources and services available to meet user requirements and help tune the performance of the grid. This article models PIndex, which is a grouped peer-to-peer network with Colored Petri Nets (CPNs) for scalable grid information services. Based on the CPN model, a simulator is implemented for PIndex simulation and performance evaluation. The correctness of the simulator is further verified by comparing the results computed from the CPN model with the results generated by the PIndex simulator.

Research and Implementation of Information Service Model on Grid with Multiple Autonomous Domains

Grid is an effective pattern for implementing heterogeneous resource sharing.Information service is a vital component of grid system,it carries out efficient management of system services and resources.ChinaGrid is a large-scale grid containing multiple autonomous domains.Existing information services can not meet the requirements of systems and applications with such kind of characteristics.This paper presents GISA2.0(Grid Information Service Architecture,GISA),focusing on autonomous domain management and resource information security.GISA2.0 implements a scalable grid information model and service-oriented hierarchical information management framework,which supports the aggregation of monitoring information from multiple monitoring systems.To achieve secure,fast,and user-related virtual global resource view,GISA2.0 uses a cross-domain information retrieval mechanism based on distributed XPath engine.

Implementation of cross grid information services for resource broker on computational multi‐grid environments

Grid systems integrate geographical computing resources across multiple administrative domains to achieve high performance computing. To exploit the full potential of grid computing, it is expected that grid participants are able to use distributed grid resources. In this paper, we present a new resource broker system with the Cross‐grid Information Service (CGIS), which integrates the information from multiple computational grids. Due to that gathering information across different grids has the imprecise problem; the accuracy of information probably affects the computational efficiency. Furthermore, this study proposes a gathering algorithm to balance the load of the queries and to ensure the accuracy of the resource information. We implement the resource broker with cross‐grid information services on multi‐grid computational environments. Experimental results show that the proposed approach could improve the system performance.

Scalable DHT- and ontology-based information service for large-scale grids

Current grid information services are centralized or hierarchical and prove inefficient as the scale of the grid rapidly increases. The introduction of the P2P DHT technique into grids brings an encouraging path. However, current applications of the P2P DHT technique to grids do not consider the Virtual Organization (VO) management mode of grid resources. Frequent joining and leaving of resources requires strong self-organization capacity of the system to maintain the rigid structure. Moreover, arranging a moderate identifier space for a DHT ring is knotty. A large identifier space will make some nodes overloaded, while a small identifier space will bring forth the same problem as the millennium bug. In addition, current grid services are described in XML-based standards, which only support syntactic keyword and taxonomy-based queries without reasoning ability at the semantic level, leading to poor precision and integrality. To address these issues, the paper proposes a scalable DHT- and ontology-based Information Service (DIS) for a grid system, which organizes resources into a DHT ring based on the VO mode. To save the identifier space, only stable VOs can join DIS via a new DHT node, whereas volatile VOs join DIS via being the sub-domain of other VOs. Furthermore, ontology-based information integration is adopted in DIS and a novel ontology for grid resources is designed, which supports semantic-based information queries. By integrating DHT- and semantic-based query techniques, DIS speeds up the information query and improves the query precision and integrality. DIS is evaluated over the ChinaGrid environment and experimental results show that DIS provides rapid resource query, high throughput and strong scalability.

The wandering token: Congestion avoidance of a shared resource

In a distributed system where scalability is an issue, the problem of enforcing mutual exclusion often arises in a soft form: the infrequent failure of the mutual exclusion predicate is tolerated, without compromising the consistent operation of the overall system. For instance this occurs when the operation subject to mutual exclusion requires massive use of a shared resource. We introduce a scalable soft mutual exclusion algorithm, based on token passing: one distinguished feature of our algorithm is that instead of introducing an overlay topology we adopt a random walk approach. The consistency of our proposal is evaluated by simulation, and we exemplify its use in the coordination of large data transfers in a backbone based network. This algorithm is studied in the frame of the CoreGRID Institute of Grid Information, Resource and Workflow Monitoring Services, in cooperation with the FORTH Institute, in Greece.

NodeWiz: Fault-tolerant grid information service

Large scale grid computing systems may provide multitudinous services, from different providers, whose quality of service will vary. Moreover, services are deployed and undeployed in the grid with no central coordination. Thus, to find out the most suitable service to fulfill their needs, or to find the most suitable set of resources on which to deploy their services, grid users must resort to a Grid Information Service (GIS). This service allows users to submit rich queries that are normally composed of multiple attributes and range operations. The ability to efficiently execute complex searches in a scalable and reliable way is a key challenge for current GIS designs. Scalability issues are normally dealt with by using peer-to-peer technologies. However, the more reliable peer-to-peer approaches do not cater for rich queries in a natural way. On the other hand, approaches that can easily support these rich queries are less robust in the presence of failures. In this paper we present the design of NodeWiz, a GIS that allows multi-attribute range queries to be performed efficiently in a distributed manner, while maintaining load balance and resilience to failures.

Active Grid Information Server for grid computing

We describe the Active Grid Information Server for performing expressive resource discovery searches and resource management in a grid environment. We represent server meta-data, such as their CPU power, storage capacity and bandwidth as points in a multi-dimensional space and then express queries as predicates over these points. The design of the system is driven by its application as a part of the information infrastructure for computational grids. Such grids provide an infrastructure for sharing computing resources; an information infrastructure is their inherent part which collects resource data and provides search functionality. Our approach complements current solutions such as MDS by adding Event Condition Action rules and an ability to efficiently handle dynamic attributes. An Event Condition Action rule-based system can support ad hoc, adaptive, flexible, and dynamic schedulers that are modifiable at runtime. We evaluate our Active Grid Information Server by evaluating Event Condition Action rules for termination, confluence, and conflict.

A grouped P2P network for scalable grid information services

Information services play a crucial role in grid environments in that the state information can be used to facilitate the discovery of resources and the services available to meet user requirements, and also to help tune the performance of a grid system. However, the large size and dynamic nature of the grid brings forth a number of challenges for information services. This paper presents PIndex, a grouped peer-to-peer network that can be used for scalable grid information services. PIndex builds on Globus MDS4, but introduces peer groups to dynamically split the large grid information search space into many small sections to enhance its scalability and resilience. PIndex is subsequently modeled with Colored Petri Nets for performance evaluation. The simulation results show that PIndex is scalable and resilient in dealing with a large number of peer nodes.

AN IMPORTANCE-AWARE ARCHITECTURE FOR LARGE-SCALE GRID INFORMATION SERVICES

This paper is concerned with the scalability of large-scale grid monitoring and information services, which are mainly used for the discovery of resources of interest. Large-scale grid monitoring systems have to balance between three competing performance metrics: query response time, imposed network overhead, and information freshness. Improving one of the three metrics will affect another; any solution will be based on a trade-off. The paper is motivated by the observation that existing grid monitoring systems can only be manually configured for a trade-off among the three metrics, which applies equally to all monitored resources; this implies that all resources in a grid are considered to be of equal importance. Assuming that in a large-scale grid setting this is unlikely to hold, the paper proposes an importance-based monitoring architecture for large-scale grid information services, based on an adaptation of the web crawling paradigm. The main idea is that, since not all resources are of equal importance, one can vary the trade-off based on the relative importance of the monitored resources. The proposed architecture is described and evaluated based on large-scale deployments of a prototype implementation on PlanetLab.