Federation Of Resources Research Articles

Cloud services for the Fermilab scientific stakeholders

As part of the Fermilab/KISTI cooperative research project, Fermilab has successfully run an experimental simulation workflow at scale on a federation of Amazon Web Services (AWS), FermiCloud, and local FermiGrid resources. We used the CernVM-FS (CVMFS) file system to deliver the application software. We established Squid caching servers in AWS as well, using the Shoal system to let each individual virtual machine find the closest squid server. We also developed an automatic virtual machine conversion system so that we could transition virtual machines made on FermiCloud to Amazon Web Services. We used this system to successfully run a cosmic ray simulation of the NOvA detector at Fermilab, making use of both AWS spot pricing and network bandwidth discounts to minimize the cost. On FermiCloud we also were able to run the workflow at the scale of 1000 virtual machines, using a private network routable inside of Fermilab. We present in detail the technological improvements that were used to make this work a reality.

Inter-governmental political relations in a federation and illegal mining of natural resources

The paper studies the impact of inter-governmental political relations on illegal resource extraction in a federal setup. In a stylized model, a corruptible sub-national politician accepts bribes from a private miner to allow under-reporting of mineral extraction and evasion of royalty payment. The probability that a mine is audited by a national audit agency increases with the extent of illegal mining. In addition, when the audit agency is politicized, the probability of audit declines with the political weight of the politician in the ruling national coalition government. We find that there is an endogenous threshold political weight beyond (below) which greater politicization of the audit agency leads to higher (lower) illegal mining. The threshold weight increases with the rate of fine on royalty evasion but decreases with the rate of royalty itself. The model also suggests that there is no unambiguous relationship between economic distortions in the form of illegal mining and the degree of rent extraction through bribes. As the audit agency becomes more politicized, the possibility of higher illegal mining but lower bribes or lower illegal mining but higher bribes cannot be ruled out.

Federated Big Data for Resource Aggregation and Load Balancing with DIRAC

BigDataDIRAC is a Big Data solution with a Distributed Infrastructure with Remote Agent Control (DIRAC) access point. Users have the opportunity to access multiple Big Data resources scattered in different geographical areas, such as access to grid resources. This approach opens the possibility of offering not only grid and cloud to the users, but also Big Data resources from the same DIRAC environment. In this work, we describe a system to allow access to a federation of Big Data resources, including load balancing, using DIRAC. Our results demonstrate the ability of BigDataDIRAC to manage jobs driven by dataset location stored in the Hadoop File System (HDFS) of the Hadoop distributed clusters. DIRAC is used to monitor the execution, collect the necessary statistical data, and upload the results from the remote HDFS to the SandBox Storage machine. Performance results demonstrate that BigDataDIRAC load balancing is able to aggregate resources in an efficient manner.

From Calgary to Canberra: Resource Taxation and Fiscal Federalism in Canada and Australia

Intergovernmental conflict over the allocation of resource revenue in federal systems is not uncommon. However, there is remarkably little scholarship on how such political tensions are managed in different federations and whether, over time, resource politics can reshape federal dynamics. This article assesses these questions through a strategic comparison of resource federalism in Canada and Australia since the 1970s. We argue that the political and institutional strategies employed within the Canadian federation, with its greater commitment to fiscal and political decentralization, has been more adept at managing short term political tensions associated with resource booms relative to the Australian model, with its creeping centrism and entrenched system of horizontal equalization. The study also finds that, over time, resource conflict has the potential to shape the practice of federalism in resource-rich jurisdictions by providing political incentives to decentralize nonrenewable resource revenue collection.

Expert Grid: New Type of Grid to Manage the Human Resources and Study the Effectiveness of Its Task Scheduler

Grid systems are the federation of resources from multiple locations to reach a common goal. In previous researches, the focused resources were only CPU, electronic devices, storage devices, network, and software applications, but till now, human resource (HR) has not been considered as Grid resources in details, despite its importance in many fields of science and society. HR is one of the important assets of organizations and plays a significant role in their success. Common and local methods to share and manage the human’s skills and knowledge in large geographically dispersed organizations are almost centralized, not reconfigurable and are based on the structure of organization; therefore, the optimal trade-off between the HR and job demands is a challenging problem. In addition, these methods did not consider some important parameters such as security issues, load balancing, organizations costs, unnecessary overtime, and employment issues. In order to overcome these defects, in this paper the new Grid architecture, named Expert Grid (EG), is proposed to find, employ, exploit, share, and manage the HR, its skills and knowledge effectively. EG provides infrastructure to increase the customer satisfactions by optimal distributions of requested jobs from customers. Experimental results show that the EG decreases the HR free time, customers delay time, and HR overtime in order to achieve high level of customers satisfaction and better performance of organizations.

Behavioral modeling and formal verification of a resource discovery approach in Grid computing

Grid computing is the federation of resources from multiple locations to facilitate resource sharing and problem solving over the Internet. The challenge of finding services or resources in Grid environments has recently been the subject of many papers and researches. These researches and papers evaluate their approaches only by simulation and experiments. Therefore, it is possible that some part of the state space of the problem is not analyzed and checked well. To overcome this defect, model checking as an automatic technique for the verification of the systems is a suitable solution. In this paper, an adopted type of resource discovery approach to address multi-attribute and range queries has been presented. Unlike the papers in this scope, this paper decouple resource discovery behavior model to data gathering, discovery and control behavior. Also it facilitates the mapping process between three behaviors by means of the formal verification approach based on Binary Decision Diagram (BDD). The formal approach extracts the expected properties of resource discovery approach from control behavior in the form of CTL and LTL temporal logic formulas, and verifies the properties in data gathering and discovery behaviors comprehensively. Moreover, analyzing and evaluating the logical problems such as soundness, completeness, and consistency of the considered resource discovery approach is provided. To implement the behavior models of resource discovery approach the ArgoUML tool and the NuSMV model checker are employed. The results show that the adopted resource discovery approach can discovers multi-attribute and range queries very fast and detects logical problems such as soundness, completeness, and consistency.

Scheduling System for Cloud Federation across Multi-Data Center

In the cloud computing federation, the method of cloud computing federation resource scheduling has been introduced to allocate the users requested tasks reasonably to the providers. In the present, single cloud providers method and system of resource scheduling dont apply to the cloud federation environment. Therefore, the solution to the problem of resource scheduling in cloud federation cross data center has become a key technology. The system, mainly about the resource scheduling algorithms across data center, presents the framework and major function of the cloud federation environment cross the datacenter. Furthermore, through Cloudsim, a Web server based system platform was built. Finally, the system proved that it can meet the complicated large-scale demand of users as well as increase the efficiency and profit of resource scheduling among the providers in cloud federation.

Optimization of the assets of the living labor of agricultural organizations

The program on the modernization of the national economy is impossible without an active innovative policy. The subject matter of the research in the article is the economy of agricultural organizations. Methodological aspect of research is the optimizing methods as the leading idea of the effective innovation policy. The continuous economic increase in the agricultural production is required for the development of investment and innovation-based component of the Russian Federation economy and competition in the conditions of WTO accession. Scientific and technical potential is a basis of investment and innovation-based economy and strategically renewable Russian Federation resource. New equipment and new technologies have the future. The intensification of all processes connected to innovation policy assumes development and implementation of the state investment and innovation policy both on federal and regional levels, applications of modern methods of a program goals management, development of strategy of innovative development of territories are necessary. The developed technique can successfully be applied in the agricultural organizations of the country, and after the corresponding completion — at any enterprise. The developed scientific and methodological knowledge allows to estimate objectively the level of technical capacity of an enterprise and, respectively, its maturity for investments without risk of non-return of means.

Heterogeneous resource federation with a centralized security model for information extraction

With the continuous growth of data generated in various scientific and commercial endeavors and the rising need for interdisciplinary studies and applications in e-Science easy exchange of information and computation resources capable of processing large amounts of data to allow ad-hoc co-operation becomes ever more important. Unfortunately different communities often use incompatible resource management systems. In this work we try to alleviate the difficulties occurring on bridging the gap between different research eco-systems by federating resources and thus unifying resource access.

SLA-BASED JOB SUBMISSION AND SCHEDULING WITH THE GLOBUS TOOLKIT 4

High performance computing is nowadays mostly performed in a best effort fashion. This is surprising as the closely related topic of grid computing, which deals with the federation of resources from multiple domains in order to support large jobs, and cloud computing, which promises seemingly infinite amounts of compute and storage, both offer quality of service (QoS), albeit in different ways. Long-term service level agreements (SLAs), which require the establishment of SLAs long in advance of their actual usage, seem a promising way for the offering of QoS guarantees in an HPC environment in a way that is not disruptive to the business models employed today. This work uses the long-term SLA approach as a basis for the provisioning of service levels for HPC resources and presents an SLA management framework to support this. Flexibility is provided by providing SLAs with different service levels, support for which is integrated into job submission and scheduling. The SLA management framework can, on a high level, be used in a generic fashion and an implementation is presented that is evaluated against a motivating scenario.

QoS-Ensured Network Resource Federation Driven by Bandwidth Brokers in Open Access Network Architecture

Various network services, such as virtual private network, cloud computing and Internet protocol television, are often provided across multiple network operators. The difficulty in managing the quality of service across multiple operator domains is the barrier to adoption especially to service level agreement-sensitive and mission critical cases. Federating network resources among operators is necessary to manage the quality of service across operators. To manage network resources of other operator domains, the network operator's federation mechanisms aiming at a future open access network model are essential. In this paper, the mechanisms of the signaling process as well as the capability of the bandwidth broker are proposed for open access networking, where multiple operators are connected via a common access network operator. Considering that both the next generation network and the non-next generation network architectures must coexist, we have analyzed federation mechanisms for establishing practical functional extensions to existing bandwidth broker implementations for the federation signaling. Based on the analysis, the designs of the federation signaling and the required bandwidth broker functional models are proposed. The proposed design is prototyped and the demonstration of the federation signaling shows that the federation mechanism can assure the bandwidth of a targeted live data stream on demand across the trunk and the access network operators even under a congested situation.

Autonomic live adaptation of virtual networked environments in a multidomain infrastructure

Abstract The trend toward cloud-based services is creating the need for large scale shared distributed infrastructures. Behind many clouds lay shared distributed infrastructures formed through the federation of many resources residing in multiple domains. Such shared infrastructures enable massive amounts of aggregated computation resources to be shared among large numbers of users. The core technologies enabling these distributed clouds are machine and network virtualization. Virtualization is the technology that enables the execution of arbitrary distributed applications on top of these increasingly popular shared physical infrastructures. In this paper, we go beyond supporting applications in the cloud and support autonomic adaptation of virtual computation environments as active, integrated entities. More specifically, driven by both dynamic availability of infrastructure resources and dynamic application resource demand, a virtual computation environment is able to automatically relocate itself across the infrastructure and scale its share of infrastructural resources. Such autonomic adaptation is transparent to both users of virtual environments and administrators of infrastructures, maintaining the look and feel of a stable, dedicated environment for the user. As our proof-of-concept, we present the design, implementation, and evaluation of a system called VIOLIN, which is composed of a virtual network of virtual machines capable of live migration across a multidomain physical infrastructure.

Infrastructure Federation Through Virtualized Delegation of Resources and Services

Infrastructure federation is becoming an increasingly important issue for modern Distributed Computing Infrastructures (DCIs): Dynamic elasticity of quasi-static Grid environments, incorporation of special-purpose resources into commoditized Cloud infrastructures, cross-community collaboration for increasingly diverging areas of modern e-Science, and Cloud Bursting pose major challenges on the technical level for many resource and middleware providers. Especially with respect to increasing costs of operating data centers, the intelligent yet automated and secure sharing of resources is a key factor for success. With the D-Grid Scheduler Interoperability (DGSI) project within the German D-Grid Initiative, we provide a strategic technology for the automatically negotiated, SLA-secured, dynamically provisioned federation of resources and services for Grid-and Cloud-type infrastructures. This goal is achieved by complementing current DCI schedulers with the ability to federate infrastructure for the temporary leasing of resources and rechanneling of workloads. In this work, we describe the overall architecture and SLA-secured negotiation protocols within DGSI and depict an advanced mechanism for resource delegation through means of dynamically provisioned, virtualized middleware. Through this methodology, we provide the technological foundation for intelligent capacity planning and workload management in a cross-infrastructure fashion.

OpenMeetings as a browser-based teleconferencing tool for EFDA laboratories

Remote participation is a key success factor for worldwide organizations and video-conference is an important aspect when laboratory collaborators are geographically dispersed. Several tools for video-conference are available worldwide. However, there is a need for a default integrated set of tools which provide all the needed features for remote participation between cooperating laboratories. Moreover, laboratories currently use different video-conference tools to communicate leading to a non-compatible method among them. OpenMeetings is open source software which provides video-conferences through web-browsing. It is operating system independent, requires no computer installation providing easy integration with already existing tools and customization of requirements. This paper intends to give an OpenMeetings overview as a video-conference application between worldwide laboratories and to demonstrate its integration with the European Fusion Development Agreement (EFDA) Federation authentication and authorization features. EFDA Federation is a multi-organization security infrastructure, whose purpose is to provide sharing and provision of a wide set of resources so that authenticated users may access them in a transparent and secure mode. This infrastructure is based on a distributed security technology called PAPI (Point of Access to Providers of Information). OpenMeetings can be a valuable EFDA Federation resource and a good approach as a method to provide a worldwide laboratory video-conferencing application, based on standard browsers.

Measuring TeraGrid: workload characterization for a high-performance computing federation

TeraGrid has deployed a significant monitoring and accounting infrastructure in order to understand its operational success. In this paper, we present an analysis of the jobs reported by TeraGrid for 2008. We consider the workload from several perspectives: traditional high-performance computing (HPC) workload characteristics; grid-oriented workload characteristics; and finally user- and group-oriented characteristics. We use metrics reported in prior studies of HPC and grid systems in order to understand whether such metrics provide useful information for managing and studying resource federations. This study highlights the importance of distinguishing between analyses of job patterns and work patterns; that small sets of users dominate the workload both in terms of job and work patterns; and that aggregate analyses across even loosely coupled federations, with incomplete information for individual systems, reflect patterns seen in more tightly coupled grids and in single HPC systems.

Resource federation in grid using automated intelligent agent negotiation

This paper provides a novel approach for enabling grid users to perform resource federation using intelligent agent negotiation. The features of agents including autonomous and distributed are designed to address the federation problems in grid, such as resource selection and policy reconciliation. Early work of resource federation is still highly depending on resource administrator (a person who manages the resources for corresponding institute). In this paper, automated agent negotiation in grid is presented to maximize the satisfaction level of federation outcome for involving parties. Using a new conflict resolution technique–creative negotiation, is introduced in solving communication problems during negotiation. A framework of automated negotiation with conflict resolution–select, match, negotiate and expand (SMNE) protocol, is proposed to guide resource administrators in performing resource federation with intelligent agent. According to the experimental result, the proposed SMNE protocol is able to guide resource administrator in conflict resolution when negotiated parties are unable to make further a concession in resource federation.

Pan-European testbed and experimental facility federation – architecture refinement and implementation

Large future internet research programs such as GENI, FIND, and FIRE are under way that will have a global impact on the current and future internet. Experimental facilities and testbeds are needed to support both revolutionary and evolutionary approaches for re-thinking and improving the internet architecture. The resource federation framework proposed by the authors makes available multi-domain testbeds providing heterogeneous crosslayer infrastructures for broad testing and experimentation purposes. Currently, it is unclear how new concepts and ideas developed by the future internet research community can be integrated into production systems while preserving infrastructure operator investments. Service oriented experimental facilities are required that integrate technologies and concepts across various domains and layers and actively incorporate different market players. In this article, we briefly introduce the network domain federation model as the basis and architectural mindset for the Pan-European Laboratory (Panlab) concept, introduce technical architecture refinements, and outline first implementation results.

Service-Oriented Models for Educational Resource Federations

Service-Oriented Models for Educational Resource Federations

Distributed Radiotherapy Simulation with the Webcom Workflow System

Accurate radiotherapy plans are a vital tool in combating cancer. The verification of such plans is a computationally intensive task, and providing clinical experts with access to sufficient resources to conduct plan verification simulations in a suitable and timely manner is a genuine challenge. In this paper we present a new approach to the problem, incorporating the Monte Carlo method for treatment verification. A fully integrated radiotherapy treatment verification workflow built on the BEAM simulation package has been developed within the scope of this work. The Monte Carlo approach is recognized as being superior to the standard clinical techniques available. To be useful in clinical practice, accurate results must be generated within a short time frame. Consequently, turnaround times must be predictable, and results must be of a consistently high standard. These requirements are the key challenges that drive this work. The development of this application is being conducted within the context of the Webcom project. Webcom is an interpreter for a graph-oriented model of computing, implemented as a distributed virtual machine. This platform has been used to construct a workflow tool suite and a novel methodology for dynamic resource federation. These components are applied to the execution of Monte Carlo radiotherapy simulation application on heterogeneous dynamically coordinated resources. The Webcom-based model of workflow management facilitates the execution of resource intensive workflows and provides a basis for the development of scalable services in the heterogeneous environments formed through the dynamic aggregation of mixed autonomous resources. We discuss the motivation behind the project and present the methodology, describe the software design of the current implementation, and demonstrates the utility of the system via experiments conducted in a real and deeply heterogeneous testbed environment.

Medical Grid: Using Grid Technology for Brain Studies

ABSTRACT The Medical Grid (MedGrid) Project was initiated in 2004 to study and demonstrate the use of grid technology in medical applications, in particular, in the management and analysis of functional magnetic resonance imaging (fMRI) datasets. For this, the MedGrid testbed was formed to provide a platform for the study and to demonstrate the feasibility of the idea. In this paper, we summarize the progress we’ve made in the last four years in implementing a grid-based fMRI data management and analysis framework within the MedGrid testbed. Keywords Grid computing, functional MRI, fMRI, BAXGrid, BAXSQL 1. INTRODUCTION Advances in digital medical imaging systems such as magnetic resonance imaging (MRI), computed tomography, among others, have led to an explosion in the number of medical digital images produced in hospitals worldwide. As this growing number of datasets necessitates more storage capacity and more computational resources, imaging sites are facing several new challenges in terms of the management and processing of these datasets. In the study of human brain functions using functional magnetic resonance imaging (fMRI), the demand for more storage and processing resources is also rising. With the availability of higher field strength MRI scanners, acquiring veryhigh-resolution functional MR images becomes possible. This raises the storage and computational requirement for these datasets. This is further amplified by the increasing complexity in the type of analysis performed on acquired datasets and the growing trend of fusing several imaging modalities to obtain more reliable results. Because of this, we explored the potential use of grid technology[1] to meet the storage and computational demands for brain studies. Advances in grid technology enabled the sharing of geographically distributed resources, such as supercomputers, PC clusters, ultra-high capacity storage devices, and scientific instruments. With grid computing, on-demand, collaborative, and data-intensive computing is also now feasible. With on-demand computing, imaging sites can access high-end computational resources from service providers when only needed, minimizing acquisition and maintenance cost. Collaborative computing allows the federation of several resources from different organizations, forming a virtual organization with computing and/or storage capacity much higher than individual members. Computing requirements for data-intensive applications can be shared among participating organizations to minimize computational load. To study and demonstrate the use of grid technology, the Medical Grid (MedGrid) Project