Grid Computing Environments Research Articles

Grid scheduling with NBU service times

In the highly unpredictable environment of grid computing, it often makes sense to replicate the same job on several of the available servers. We prove that never replicating is optimal for a heterogeneous multi-server system in a random environment when service times are New Better than Used.

PIVOT: An adaptive information discovery framework for computational grids

In a traditional computational grid environment, the owners of resources usually provide information about their resources extracted by pre-configured information services or web services. However, such information is not sufficient for the scheduler in the high-performance distributed computing. To solve this problem, we propose a scalable grid information service framework, named PIVOT (adaPtive Information discoVery framewOrk for compuTational grid). By using deadline-constrained flooding collector dissemination and P2P-like information collection schemes, PIVOT provides an active mechanism to collect application-specific resource information. In particular, PIVOT provides a resource information service for application-specific schedulers. The best-effort performance on overhead traffic and communication latency during information discovery is guaranteed by two new distributed cooperative algorithms. The experimental results in the simulations and real computational grid platform demonstrate that PIVOT has a high level of adaptability for application-specific resource information discovery, and also improves the accuracy of resource allocation and the efficiency of executing parallel tasks in traditional information services.

The Representation and Computation of QoS Preference with Its Applications in Grid Computing Environments

In Grid computing environment, quality of service (QoS) provisioning must be provided to the end users on the basis of their specific requirements. This paper proposes in a first step QoS attributes for Grid applications. In this matter, a mix of quantitative and qualitative parameters have to be considered. In the context, the analytical hierarchy process (AHP) technique [1] is a possible approach to formulate the QoS requirements of the users for Grid services. In order to apply QoS preference to actual application, we introduce a QoS function and a metric for user's satisfaction degrees. These both tools can be used as an evaluation criterion by the user. Subsequently, an algorithm of service selection considering the user's QoS preference is presented. Our empirical studies indicate that the application can reliably select the optimal service for users.

A general model for the generation and scheduling of parameter sweep experiments in computational grid environments

Parameter sweep experiments (PSE) involve several issues. In this work, we consider two of them: the generation of the parameters space and the scheduling of the associated tasks. Thus, we propose a general model to generate the parameters space of any PSE applying the Nested Summation Symbol operator. On the other hand, for the scheduling of these kinds of problems, we test an adaptive scheduling approach with fault tolerance. This approach has been implemented, using the DRMAA-C version for GridWay, to allocate tasks in a Grid environment. In the tests, the scheduler shows a good performance. Moreover, the CPU usage of the scheduler is quite low.

Heuristic computation of the rovibrational G matrix in optimized molecule-fixed axes. Gmat 2.1

Gmat 2.1 is a program able to compute the rovibrational G matrix in different molecule-fixed axes extending the capabilities of Gmat 1.0. The present version is able to select optimal molecule-fixed axes minimizing the pure rotational kinetic elements, the rovibrational kinetic elements or both simultaneously. To such an end, it uses a hybrid minimization approach. Thus, it combines a global search heuristic based in simulated annealing with a gradient-free local minimization. As the previous version, the program handles the structural results of potential energy hypersurface mappings computed in computer clusters or computational Grid environments. However, since now more general molecule-fixed axes can be defined, a procedure is implemented to ensure the same minimum of the cost function is used in all the molecular structures. In addition, an algorithm for the unambiguous definition of the molecule-fixed axes orientation is used.

Computing activities for the P̄ANDA experiment at FAIR

The P̄ANDA experiment at the future facility FAIR will provide valuable data for our present understanding of the strong interaction. In preparation for the experiments, large-scale simulations for design and feasibility studies are performed exploiting a new software framework, P̄ANDAROOT, which is based on FairROOT and the Virtual Monte Carlo interface, and which runs on a large-scale computing GRID environment exploiting the AliEn2 middleware. In this paper, an overview is given of the P̄ANDA experiment with the emphasis on the various developments which are pursuit to provide a user and developer friendly computing environment for the P̄ANDA collaboration.

Multi-agent framework for performance-based resource management in computational grid environment

Resource management is one of the focus areas in Grid computing research. One major objective of resource management in a computational Grid environment is to allocate jobs to make efficient use of the computational resources under different resource providers and, thereby, achieve high performance of the jobs. Therefore, performance analysis is also an important issue in Grid resource management. This paper aims at providing an integrated framework for performance-based resource management in computational Grid environment. The framework is supported by a multi-agent system (MAS) which has been developed using a firm software engineering approach based on Gaia methodology. The MAS provides adaptive execution facility either by rescheduling the jobs onto different resource providers or through application of local tuning techniques to jobs in case of any performance problem (may be due to some change in the resource availability or usage scenario). Thus it always guarantees to maintain the quality of service as desired by the client.

Formal Modelling of a Usable Identity Management Solution for Virtual Organisations

This paper attempts to accurately model security requirements for computational grid environments with particular focus on authentication. We introduce the Audited Credential Delegation (ACD) architecture as a solution to some of the virtual organisations identity management usability problems. The approach uses two complementary models: one is state based, described in Z notation, and the other is event-based, expressed in the Process Algebra of Hoare's Communicating Sequential Processes (CSP). The former will be used to capture the state of the WS and to model back-end operations on it whereas the latter will be used to model behavior, and in particular, front-end interactions and communications. The modelling helps to clearly and precisely understand functional and security requirements and provide a basis for verifying that the system meets its intended requirements.

Adaptive Execution of Jobs in Computational Grid Environment

In a computational grid, jobs must adapt to the dynamically changing heterogeneous environment with an objective of maintaining the quality of service. In order to enable adaptive execution of multiple jobs running concurrently in a computational grid, we propose an integrated performance-based resource management framework that is supported by a multi-agent system (MAS). The multi-agent system initially allocates the jobs onto different resource providers based on a resource selection algorithm. Later, during runtime, if performance of any job degrades or quality of service cannot be maintained for some reason (resource failure or overloading), the multi-agent system assists the job to adapt to the system. This paper focuses on a part of our framework in which adaptive execution facility is supported. Adaptive execution facility is availed by reallocation and local tuning of jobs. Mobile, as well as static agents are employed for this purpose. The paper provides a summary of the design and implementation and demonstrates the efficiency of the framework by conducting experiments on a local grid test bed.

A novel time computation model based on algorithm complexity for data intensive scientific workflow design and scheduling

AbstractScientific workflow offers a framework for cooperation between remote and shared resources on a grid computing environment (GCE) for scientific discovery. One major function of scientific workflow is to schedule a collection of computational subtasks in well‐defined orders for efficient outputs by estimating task duration at runtime. In this paper, we propose a novel time computation model based on algorithm complexity (termed as TCMAC model) for high‐level data intensive scientific workflow design. The proposed model schedules the subtasks based on their durations and the complexities of participant algorithms. Characterized by utilization of task duration computation function for time efficiency, the TCMAC model has three features for a full‐aspect scientific workflow including both dataflow and control‐flow: (1) provides flexible and reusable task duration functions in GCE; (2) facilitates better parallelism in iteration structures for providing more precise task durations; and (3) accommodates dynamic task durations for rescheduling in selective structures of control flow. We will also present theories and examples in scientific workflows to show the efficiency of the TCMAC model, especially for control‐flow. Copyright © 2009 John Wiley & Sons, Ltd.

Special issue on architectures and techniques for real-time processing of remotely sensed images

Advances in sensor technology are revolutionizing the way remotely sensed images are collected, managed and processed. The incorporation of latest generation sensors to airborne and satellite platforms is currently producing a nearly continual stream of high-dimensional image data [6], and this explosion in the amount of collected information has rapidly created new processing challenges [4]. In particular, many current and future applications of remote sensing in Earth science, space science, and soon in planetary exploration science, require real or near realtime processing capabilities. Relevant examples include monitoring of natural disasters, such as earthquakes and floods, military applications, tracking of man-induced hazards, such as wild land and forest fires, oil spills and other types of biological agents. Most of these applications require timely responses for swift decisions, which depend on realtime performance of algorithm analysis. Owing to the diversity and large dimensionality of the image data sets provided by observation instruments in Earth and planetary applications, there are no commonly accepted real-time image processing architectures and techniques in the context of remote sensing missions. In future years, remote sensing imaging spectrometers [2] such as hyperspectraland ultraspectral imagers [1] will continue increasing their spatial and spectral resolutions (imagers with thousands of narrow spectral bands are currently under development), thus producing a nearly continual stream of high-dimensional image data [5] and increasing the possibility of exploitation of the data with regards to multi-spectral imagers [3]. Such technological advances are not only expected in optical remote sensing instruments, but also in radar and other types of remote sensing systems. Such explosion in the amount, size and dimensionality of the information collected on a daily basis presents new challenges for real-time processing in certain remote sensing applications, which demand for quick response in practical exploitation. This special issue on ‘‘Architectures and techniques for real-time processing of remotely sensed images’’ is intended to present the state of the art and the most recent developments in the incorporation of (near) real-time image processing techniques and specialized architectures in the context of remote sensing applications. This special issue covers different techniques and architectures for (near) real-time image processing in remote sensing. The techniques addressed in the special issue deal with relevant problems in Earth observation, such as damage mapping in case of earthquakes, flood monitoring, target and anomaly detection for military applications, cloud cover assessment or precision agriculture. The set of implementation strategies discussed in the special issue is also extensive, including parallel and distributed platforms, grid computing and multi-core environments, and specialized architectures for real-time on-board processing such as field programmable gate arrays (FPGAs) or commodity graphics processing units (GPUs). Combined these topics are expected to provide an excellent snapshot of the state of the art in those areas, and to offer a thoughtful perspective of the potential and emerging challenges of applying real-time image processing architectures and algorithms to realistic remote sensing problems. The special issue comprises eight articles contributed by authors with extensive experience in the field of efficient A. J. Plaza (&) Department Technology of Computers and Communications, Escuela Politecnica, University of Extremadura, Avenida de la Universidad s/n, 10071 Caceres, Spain e-mail: aplaza@unex.es URL: http://www.umbc.edu/rssipl/people/aplaza

Orbweb—A Network Substrate for Peer-to-Peer Desktop Grid Computing Based on Open Standards

While Desktop Grid Computing has become a promising alternative to classical Grids for certain kinds of applications, it has not yet reached the same level of standardization. In this article, we present Orbweb, a network substrate for Peer-to-Peer Desktop Grid Computing based on the open industrial-strength eXtensible Messaging and Presence Protocol (XMPP). We discuss, how XMPP can be leveraged to tackle domain-specific challenges, including high scalability, support for volatility, NAT/Firewall traversal, and protocol efficiency. Where XMPP fails to meet these requirements, we contribute pertinent extensions. In particular, we boost the scalability of XMPP by taking load of the XMPP servers through dynamically negotiated direct Peer-to-Peer communication channels between XMPP peers. We pave the way for scalable group membership management by substituting the existing XMPP Multi-User Chat protocol for one that does not suffer from limitations imposed by an »everyone knows everyone« visibility model and allows for deploying custom virtual topologies that best match the interaction patterns of a given application. As efficient multicasting is an essential prerequisite for many distributed algorithms, we adapt the well-known Bimodal Multicast protocol to work in a highly volatile Peer-to-Peer Desktop Grid Computing environment. Finally, we show how to improve the protocol efficiency of XMPP by leveraging a standardized binary encoding of the XML Information Set for XMPP packet transmission. To substantiate the applicability of our approach and the effectiveness of our extensions, we describe how some important higher-level services used in Peer-to-Peer Desktop Grid Computing can be implemented on top of Orbweb and provide a detailed experimental analysis with up to 10K nodes.

A parallel Aitken-additive Schwarz waveform relaxation suitable for the grid

The objective of this paper is to describe a grid-efficient parallel implementation of the Aitken-Schwarz waveform relaxation method for the heat equation problem. This new parallel domain decomposition algorithm, introduced by Garbey [M. Garbey, A direct solver for the heat equation with domain decomposition in space and time, in: Springer Ulrich Langer et al. (Ed.), Domain Decomposition in Science and Engineering XVII, vol. 60, 2007, pp. 501-508], generalizes the Aitken-like acceleration method of the additive Schwarz algorithm for elliptic problems. Although the standard Schwarz waveform relaxation algorithm has a linear rate of convergence and low numerical efficiency, it can be easily optimized with respect to cache memory access and it scales well on a parallel system as the number of subdomains increases. The Aitken-like acceleration method transforms the Schwarz algorithm into a direct solver for the parabolic problem when one knows a priori the eigenvectors of the trace transfer operator. A standard example is the linear three dimensional heat equation problem discretized with a seven point scheme on a regular Cartesian grid. The core idea of the method is to postprocess the sequence of interfaces generated by the additive Schwarz wave relaxation solver. The parallel implementation of the domain decomposition algorithm presented here is capable of achieving robustness and scalability in heterogeneous distributed computing environments and it is also naturally fault tolerant. All these features make such a numerical solver ideal for computational grid environments. This paper presents experimental results with a few loosely coupled parallel systems, remotely connected through the internet, located in Europe, Russia and the USA.

Performance analysis of a cross-layer SSO mechanism for a roaming infrastructure

Over the last decade, several international initiatives have tried to provide different solutions to a common issue: resource sharing among several institutions. Some have been mainly designed for Web resources or computing resources, like Grid Computing environments, or even for network access for roaming users. A common aspect in most of those approaches is the management of identities, that is, the representation of the information related to specific individuals or other entities and its use for authentication and authorization purposes. However, since the different solutions are focused on different application scenarios (Web, Grid, and network) it has been really difficult to create a unified point of view (cross-layer) for identity management and, therefore, mechanisms like Single Sign On (SSO) across different layers are considered to be a main gap in current efforts. In this article, we present an architecture based on an existing solution for roaming in educational environments (eduroam). The architecture is able to provide what has been called a unified SSO mechanism, that is, once the users have been authenticated during the network access, they are enabled to obtain protected resources at higher layers (like Web resources) without further re-authentication. Additionally, we include a performance analysis to illustrate the feasibility of this architecture, which has been tested in a real production environment like eduroam.

Gmat. A software tool for the computation of the rovibrational G matrix

Gmat is a C++ program able to compute the rovibrational G matrix in molecules of arbitrary size. This allows the building of arbitrary rovibrational Hamiltonians. In particular, the program is designed to work with the structural results of potential energy hypersurface mappings computed in computer clusters or computational Grid environments. In the present version, 1.0, the program uses internal coordinates as vibrational coordinates, with the principal axes of inertia as body-fixed system. The main design implements a complete separation of the interface and functional parts of the program. The interface part permits the automatic reading of the molecular structures from the output files of different electronic structure codes. At present, Gamess and Gaussian output files are allowed. To such an end, use is made of the object orientation polymorphism characteristic. The functional part computes numerically the derivatives of the nuclear positions respect to the vibrational coordinates. Very accurate derivatives are obtained by using central differences embedded in a nine levels Richardson extrapolation procedure. Program summary Program title: Gmat Catalogue identifier: AECZ_v1_0 Program summary URL: http://cpc.cs.qub.ac.uk/summaries/AECZ_v1_0.html Program obtainable from: CPC Program Library, Queen's University, Belfast, N. Ireland Licensing provisions: Standard CPC licence, http://cpc.cs.qub.ac.uk/licence/licence.html No. of lines in distributed program, including test data, etc.: 17 023 No. of bytes in distributed program, including test data, etc.: 274 714 Distribution format: tar.gz Programming language: Standard C++ Computer: All running Linux/Windows Operating system: Linux, Windows Classification: 16.2 Nature of problem: Computation of the rovibrational G matrix in molecules of any size. This allows the building of arbitrary rovibrational Hamiltonians. It must be possible to obtain the input data from the output files of standard electronic structure codes. In addition, the program should handle the large number of files generated in massive explorations of molecular potential energy hypersurfaces. In these cases, Gmat will provide the G matrix as a function of the molecular structure. Solution method: To reach its objectives, Gmat has been organized in two components: an interface and a functional part. This organization allows for separating the input/output tasks, which are dependent on the human–machine interaction model selected, from the functional requirements, which are not. An object-oriented approach has been used in both parts. In the interface, polymorphism is used to allow the data acquisition from output files of different electronic structure codes. In the functional part, Gmat computes numerically the derivatives of the Cartesian coordinates respect to the vibrational coordinates needed to build the G matrix. Extremely accurate numerical derivatives are obtained in a double procedure. First, the truncation plus roundoff errors are minimized in the central differences expression. Second, the result is embedding in a nine levels Richardson extrapolation process. In the present version, the program allows the use of internal coordinates as vibrational coordinates, with the principal axes of inertia as body-fixed system. Running time: Sample test runs provided with the distribution take a few seconds to execute.

Research on Tasks Scheduling Algorithms for Dynamic and Uncertain Computing Grid Based on a+bi Connection Number of SPA

Task scheduling algorithms are key techniques in task management system of computing grid. Because of the uncertainty nature of a grid, traditional task scheduling algorithms do not work well in an open, heterogeneous and dynamic grid environment of real world. In this paper, Set Pair Analysis (SPA), a new soft computation method is used to process the synthetic uncertainty in the task scheduling of a computing grid. After introducing SPA and its application, the paper goes on to introduce the definition of connection number to express the uncertain Expected Time to Compute of tasks, analysis operation properties and linear order relation suitable for computing grid scheduling. Three online uncertain dynamic scheduling algorithms, OUD_OLB, OUD_MET, OUD_MCT, and three batch uncertain dynamic scheduling algorithms BUD_Min-min, BUD_Min-max, BUD_Surferage, are presented for the uncertain dynamic computing grid. Theoretical analysis and experimental results illustrate that these algorithms are capable of representing the dynamics and uncertainty in a computing grid environment. These algorithms are the generalization of traditional grid scheduling algorithms, and they possess high value in theory and application in a grid environment. Certainly it will be a new method to design tasks scheduling algorithm in uncertain computing grid environment.

Adaptive Processor Allocation for Moldable Jobs in Computational Grid

In a computational grid environment, a common practice is try to allocate an entire parallel job onto a single participating site. Sometimes a parallel job, upon its submission, cannot fit in any single site due to the occupation of some resources by running jobs. How the job scheduler handles such situations is an important issue which has the potential to further improve the utilization of grid resources as well as the performance of parallel jobs. This article develops adaptive processor allocation policies based on the moldable property of parallel jobs to deal with such situations in a heterogeneous computational grid environment. The proposed policies are evaluated through a series of simulations using real workload traces. The results indicate that the proposed adaptive processor allocation policies can further improve the system performance of a heterogeneous computational grid significantly.

Electricity Market Forecasting using Artificial Neural Network Models Optimized by Grid Computing

This paper reports a grid computing approach to parallel-process a neural network time-series model for forecasting electricity market prices. The grid computing of the neural network model not only processes several times faster than a single iterative process but also provides chances of improving forecasting accuracy. A grid-computing environment implemented in a university computing laboratory improves utilization rate of otherwise underused computing resources. Results of numerical tests using the real market data by more than twenty grid-connected PCs are presented.

Analysis of stochastic problem decomposition algorithms in computational grids

Stochastic programming usually represents uncertainty discretely by means of a scenario tree. This representation leads to an exponential growth of the size of stochastic mathematical problems when better accuracy is needed. Trying to solve the problem as a whole, considering all scenarios together, yields to huge memory requirements that surpass the capabilities of current computers. Thus, decomposition algorithms are employed to divide the problem into several smaller subproblems and to coordinate their solution in order to obtain the global optimum. This paper analyzes several decomposition strategies based on the classical Benders decomposition algorithm, and applies them in the emerging computational grid environments. Most decomposition algorithms are not able to take full advantage of all the computing power available in a grid system because of unavoidable dependencies inherent to the algorithms. However, a special decomposition method presented in this paper aims at reducing dependency among subproblems, to the point where all the subproblems can be sent simultaneously to the grid. All algorithms have been tested in a grid system, measuring execution times required to solve standard optimization problems and a real-size hydrothermal coordination problem. Numerical results are shown to confirm that this new method outperforms the classical ones when used in grid computing environments.

On improving resource utilization and system throughput of master slave job scheduling in heterogeneous systems

The rapid advances of network technologies shed light on many aspects of the practicability of large scale ubiquitous computing. Grid technology has been recognized as an efficient solution to coordinate large-scale shared resources and execute complex applications in heterogeneous network environments. The problem of resource management and task allocation has always been one of the main challenges. In this paper, we present an efficient task allocation strategy for distributing tasks onto computing nodes in the underlying heterogeneous networks. The contribution of the proposed technique is to minimize average turnaround time by dispatching tasks to processors with smallest communication ratio. System throughput could be also enhanced by dispersing processor idle time. The proposed technique can be applied to heterogeneous cluster systems as well as computational grid environments, in which the communication costs vary in different clusters. Experimental results show that the proposed scheme outperforms other previous algorithms in terms of throughput and turnaround time.