Local Resource Management Systems Research Articles

Socioecology and Biodiversity Conservation

The discipline of Socioecological Systems (SES) was conceptualized in 1998 with the intention of understanding the effects of human activities on natural ecosystems by analyzing resilience in local resource management systems [...]

Who Sets the Rules? Institutional Misfits and Bricolage in Hunting Management in Brazil

Hunting management in Brazil in not regulated by a unique legal framework, but by a combination of national environmental legislation and local rules and norms. Therefore, a multidimensional perspective combining structural and procedural analysis of the different institutional arrangements is necessary for better understanding of this local natural resource management system. We combined three analytical approaches in the study case of hunting management in the Piagacu-Purus Sustainable Development Reserve (RDS-PP) in the Brazilian Amazon. The institutional analysis of the rules revealed gaps and inconsistencies between different structures and decision levels. The meta-governance analysis indicated incongruities arising from different perceptions and values among stakeholders. Finally, the bricolage analysis revealed hybrid strategies used by local stakeholders and state agents. We conclude that complementarity of analytical frameworks is necessary in order to understand how multiple perceptions and power asymmetries in co-management arenas may influence the dynamics and porosity of such institutional arrangements.

Improved Cloud resource allocation: how INDIGO-DataCloud is overcoming the current limitations in Cloud schedulers

Performing efficient resource provisioning is a fundamental aspect for any resource provider. Local Resource Management Systems (LRMS) have been used in data centers for decades in order to obtain the best usage of the resources, providing their fair usage and partitioning for the users. In contrast, current cloud schedulers are normally based on the immediate allocation of resources on a first-come, first-served basis, meaning that a request will fail if there are no resources (e.g. OpenStack) or it will be trivially queued ordered by entry time (e.g. OpenNebula). Moreover, these scheduling strategies are based on a static partitioning of the resources, meaning that existing quotas cannot be exceeded, even if there are idle resources allocated to other projects. This is a consequence of the fact that cloud instances are not associated with a maximum execution time and leads to a situation where the resources are under-utilized. These facts have been identified by the INDIGO-DataCloud project as being too simplistic for accommodating scientific workloads in an efficient way, leading to an underutilization of the resources, a non desirable situation in scientific data centers. In this work, we will present the work done in the scheduling area during the first year of the INDIGO project and the foreseen evolutions.

The Legnaro-Padova distributed Tier-2: challenges and results

The Legnaro-Padova Tier-2 is a computing facility serving the ALICE and CMS LHC experiments. It also supports other High Energy Physics experiments and other virtual organizations of different disciplines, which can opportunistically harness idle resources if available.The unique characteristic of this Tier-2 is its topology: the computational resources are spread in two different sites, about 15 km apart: the INFN Legnaro National Laboratories and the INFN Padova unit, connected through a 10 Gbps network link (it will be soon updated to 20 Gbps). Nevertheless these resources are seamlessly integrated and are exposed as a single computing facility. Despite this intrinsic complexity, the Legnaro-Padova Tier-2 ranks among the best Grid sites for what concerns reliability and availability. The Tier-2 comprises about 190 worker nodes, providing about 26000 HS06 in total. Such computing nodes are managed by the LSF local resource management system, and are accessible using a Grid-based interface implemented through multiple CREAM CE front-ends.dCache, xrootd and Lustre are the storage systems in use at the Tier-2: about 1.5 PB of disk space is available to users in total, through multiple access protocols.A 10 Gbps network link, planned to be doubled in the next months, connects the Tier-2 to WAN. This link is used for the LHC Open Network Environment (LHCONE) and for other general purpose traffic.In this paper we discuss about the experiences at the Legnaro-Padova Tier-2: the problems that had to be addressed, the lessons learned, the implementation choices. We also present the tools used for the daily management operations. These include DOCET, a Java-based webtool designed, implemented and maintained at the Legnaro-Padova Tier-2, and deployed also in other sites, such as the LHC Italian T1. DOCET provides an uniform interface to manage all the information about the physical resources of a computing center. It is also used as documentation repository available to the Tier-2 operations team.Finally we discuss about the foreseen developments of the existing infrastructure. This includes in particular the evolution from a Grid-based resource towards a Cloud-based computing facility.

VM-based infrastructure for simulating different cluster and storage solutions used on ATLAS Tier-3 sites

The current ATLAS Tier-3 infrastructure consists of a variety of sites of different sizes and with a mix of local resource management systems (LRMS) and mass storage system (MSS) implementations. The Tier-3 monitoring suite, having been developed in order to satisfy the needs of Tier-3 site administrators and to aggregate Tier-3 monitoring information on the global VO level, needs to be validated for various combinations of LRMS and MSS solutions along with the corresponding Ganglia plugins. For this purpose the testbed infrastructure, which allows simulation of various computational cluster and storage solutions, had been set up at JINR (Dubna, Russia). This infrastructure provides the ability to run testbeds with various LRMS and MSS implementations, and with the capability to quickly redeploy particular testbeds or their components. Performance of specific components is not a critical issue for development and validation, whereas easy management and deployment are crucial. Therefore virtual machines were chosen for implementation of the validation infrastructure which, though initially developed for Tier-3 monitoring project, can be exploited for other purposes. Load generators for simulation of the computing activities at the farm were developed as a part of this task. The paper will cover concrete implementation, including deployment scenarios, hypervisor details and load simulators.

Enabling data analysis à la PROOF on the Italian ATLAS Tier-2s using PoD

We describe our experience using PROOF for data analysis on the Italian ATLAS-Tier2 in Frascati, Napoli and Roma1. To enable PROOF on the cluster we used PoD, Proof-on-Demand. PoD is a set of tools designed to interact with any Local Resource Management System (LRMS) to start the PROOF daemons. In this way any user can quickly setup its own PROOF cluster on the resources, with the LRMS taking care of scheduling, priorities and accounting. Usage of PoD has steadily increased in the last years, and the product has now reached a production level quality. PoD features an abstract interface to LRMSs and provides plugins for several LRMSs. In our tests we used both the gLite and PBS plug-ins, the latter being the native LRMS handling the resources under test. Data were accessed via XRootD with file discovery provided by the standard ATLAS tools. The Storage Element was Disk Pool Manager (DPM) which traditionally uses RFIO rfio data access protocol; we added XRootD on top of this system so PoD could access the data. We describe the configuration and setup details and the results of some benchmark tests we run on the facility.

New developments in the CREAM Computing Element

The EU-funded project EMI aims at providing a unified, standardized, easy to install software for distributed computing infrastructures. CREAM is one of the middleware products part of the EMI middleware distribution: it implements a Grid job management service which allows the submission, management and monitoring of computational jobs to local resource management systems. In this paper we discuss about some new features being implemented in the CREAM Computing Element. The implementation of the EMI Execution Service (EMI-ES) specification (an agreement in the EMI consortium on interfaces and protocols to be used in order to enable computational job submission and management required across technologies) is one of the new functions being implemented. New developments are also focusing in the High Availability (HA) area, to improve performance, scalability, availability and fault tolerance.

List-based Data Structures for Efficient Management of Advance Reservations

Complex eScience and other sophisticated applications in the field of HPC imply new demands that queuing based resource management systems cannot meet. To guarantee Quality of Service and co-allocation in the Grid, planning based resource management systems implementing advance reservation are needed. These systems face new challenges as a planning based management system has to keep track of the jobs and reservations in the future. Additionally, during the negotiation process of incoming reservations, a good overview of the remaining, not-yet reserved capacity is needed--not only for the current allocation, but also for the whole book-ahead time. Therefore, the resource management problem becomes a two dimensional problem for advance reservations in this field. In this paper different data structures are investigated and discussed in order to fit to planning based resource management. As a result the benefits of using lists of resource allocation or free blocks are exposed. This general idea widely used to manage continuous resources is extended to cover not only the resource dimension but also the time dimension. The list of blocks approach is evaluated in a Grid level and a local resource management system for a computing cluster. The extensive simulations showed a better runtime and higher reservation success rate compared with the currently favored approach of a slotted time and the more sophisticated approach based on AVL trees.

Interactive Analysis using PROOF in a GRID Infrastructure

Current high energy physics experiments aim to explore new territories where new physics is expected. In order to achieve that, a huge amount of data has to be collected and analyzed. The accomplishment of these scientific projects require computing resources beyond the capabilities of a single user or group, thus the data is treated under the grid infrastructure. Despite the reduction applied to the data, the sample used in the last step of the analysis is still large. At this phase, interactivity contributes to a faster optimization of the final cuts in order to improve the results. The Parallel ROOT Facility (PROOF) is intended to speed up even further this procedure providing the user analysis results within a shorter time by simultaneously using more cores. Taking profit of the computing resources and facilities available at Instituto de Física de Cantabria (IFCA), shared between two major projects LHC-CMS Tier-2 and GRID-CSIC, we have developed a setup that integrates PROOF with SGE as local resource management system and GPFS as file system, both common to the grid infrastructure. The setup was also integrated in a similar infrastructure for the LHC-CMS Tier-3 at Universidad de Oviedo that uses Torque (PBS) as local job manager and Hadoop as file system. In addition, to ease the transition from a sequential analysis code to PROOF, an analysis framework based on the TSelector class is provided. Integrating PROOF in a cluster provides users the potential usage of thousands of cores (1,680 in the IFCA case). Performance measurements have been done showing a speed improvement closely correlated with the number of cores used.

ARC and gLite interoperability in ATLAS sites

In the Worldwide Large Hadron Collider Computing Grid several sites deploy both gLite and ARC middlewares. In this manner they are simultaneously providing resources to more than one federation of sites. In this article the interoperability between the two middleware flavors at sites that provide distributed resources to the ATLAS experiment is described. Some motivations for dual deployments, the coexistence of a gLite computing element and an ARC front-end on the same local resource management system, consistent monitoring, accounting and ticket handling are discussed.

WNoDeS, a tool for integrated Grid and Cloud access and computing farm virtualization

INFN CNAF is the National Computing Center, located in Bologna, Italy, of the Italian National Institute for Nuclear Physics (INFN). INFN CNAF, also called the INFN Tier-1, provides computing and storage facilities to the International High-Energy Physics community and to several multi-disciplinary experiments. Currently, the INFN Tier-1 supports more than twenty different collaborations; in this context, optimization of the usage of computing resources is essential. This is one of the main drivers behind the development of a software called WNoDeS (Worker Nodes on Demand Service).WNoDeS, developed at INFN CNAF and deployed on the INFN Tier-1 production infrastructure, is a solution to virtualize computing resources and to make them available through local, Grid or Cloud interfaces. It is designed to be fully integrated with a Local Resource Management System; it is therefore inherently scalable and permits full integration with existing scheduling, policing, monitoring, accounting and security workflows. WNoDeS dynamically instantiates Virtual Machines (VMs) on-demand, i.e. only when the need arises; these VMs can be tailored and used for purposes like batch job execution, interactive analysis or service instantiation. WNoDeS supports interaction with user requests through traditional batch or Grid jobs and also via the Open Cloud Computing Interface standard, making it possible to allocate compute, storage and network resources on a pay-as-you-go basis. User authentication is supported via several authentication methods, while authorization policies are handled via gLite Argus. WNoDeS is an ambitious solution aimed at virtualizing cluster resources in medium or large scale computing centers, with up to several thousands of Virtual Machines up and running at any given time.In this paper, we descrive the WNoDeS architecture.

Integration of cloud, grid and local cluster resources with DIRAC

Grid computing was developed to provide users with uniform access to large-scale distributed resources. This has worked well, however there are significant resources available to the scientific community that do not follow this paradigm - those on cloud infrastructure providers, HPC supercomputers or local clusters. DIRAC (Distributed Infrastructure with Remote Agent Control) was originally designed to support direct submission to the Local Resource Management Systems (LRMS) of such clusters for LHCb, matured to support grid workflows and has recently been updated to support Amazon's Elastic Compute Cloud.This raises a number of new possibilities - by opening avenues to new resources, virtual organisations can change their resources with usage patterns and use these dedicated facilities for a given time.For example, user communities such as High Energy Physics experiments, have computing tasks with a wide variety of requirements in terms of CPU, data access or memory consumption, and their usage profile is never constant throughout the year. Having the possibility to transparently absorb peaks on the demand for these kinds of tasks using Cloud resources could allow a reduction in the overall cost of the system.This paper investigates interoperability by following a recent large-scale production excercise utilising resources from these three different paradigms, during the 2010 Belle Monte Carlo run. Through this, it discusses the challenges and opportunities of such a model.

Status and Developments of the CREAM Computing Element Service

The CREAM CE implements a Grid job management service available to end users and to other higher level Grid job submission services. It allows the submission, management and monitoring of computational jobs to local resource management systems. CREAM, which is part of the gLite Grid middleware, is available in the EGI production Grid where it is used by several user communities in different job submission scenarios. In this paper, after a quick description of the CREAM CE architecture and functionality, we report on the status of this Grid service, focusing on the results, feedback and issues that had to be addressed. We also discuss about its integration with other job submission services, in particular the gLite Workload Management System. The planned future activities, concerning the maintenance and evolution of the CREAM CE, are reported as well.

Using a Local Electronic Resource Management System to Manage E-journals: Can It Get Any Better Than This?

The Smithsonian Institution Libraries (SIL) use a locally developed and maintained database for managing electronic resources. Journal and database titles, vendors, interlibrary loan, usage, and SIL persistent URLs are collected and tracked for free and subscribed content in one central access tool. This session discussed the current workflows along with the pros and cons of such an arrangement, and some possible future directions for SIL.

Design and implementation of the gLite CREAM job management service

Job execution and management is one of the most important functionalities provided by every modern Grid systems. In this paper we describe how the problem of job management has been addressed in the gLite middleware by means of the CREAM and CEMonitor services. CREAM (Computing Resource Execution and Management) provides a job execution and management capability for Grids, while CEMonitor is a general purpose asynchronous event notification framework. Both components expose a Web Service interface allowing conforming clients to submit, manage and monitor computational jobs to a Local Resource Management System.

BaBar MC production on the Canadian grid using a web services approach

The present paper highlights the approach used to design and implement a web services based BaBar Monte Carlo (MC) production grid using Globus Toolkit version 4. The grid integrates the resources of two clusters at the University of Victoria, using the ClassAd mechanism provided by the Condor-G metascheduler. Each cluster uses the Portable Batch System (PBS) as its local resource management system (LRMS). Resource brokering is provided by the Condor matchmaking process, whereby the job and resource attributes are expressed as ClassAds. The important features of the grid are automatic registering of resource ClassAds to the central registry, ClassAds extraction from the registry to the metascheduler for matchmaking, and the incorporation of input/output file staging. Web-based monitoring is employed to track the status of grid resources and the jobs for an efficient operation of the grid. The performance of this new grid for BaBar jobs, and the existing Canadian computational grid (GridX1) based on Globus Toolkit version 2 is found to be consistent.

Job submission and management through web services: the experience with the CREAM service

Modern Grid middleware is built around components providing basic functionality, such as data storage, authentication, security, job management, resource monitoring and reservation. In this paper we describe the Computing Resource Execution and Management (CREAM) service. CREAM provides a Web service-based job execution and management capability for Grid systems; in particular, it is being used within the gLite middleware. CREAM exposes a Web service interface allowing conforming clients to submit and manage computational jobs to a Local Resource Management System. We developed a special component, called ICE (Interface to CREAM Environment) to integrate CREAM in gLite. ICE transfers job submissions and cancellations from the Workload Management System, allowing users to manage CREAM jobs from the gLite User Interface. This paper describes some recent studies aimed at assessing the performance and reliability of CREAM and ICE; those tests have been performed as part of the acceptance tests for integration of CREAM and ICE in gLite. We also discuss recent work towards enhancing CREAM with a BES and JSDL compliant interface.

Groundwater management in rice terraces: a case study of a lakeside community in Shiga Prefecture, Japan

This paper aims to clarify the process of norm formation to reach consensus on groundwater management among farmers in rice terraces in Japan. The modernisation and globalisation of water management have led to the disorganisation and reorganisation of local resource management systems and local knowledge. Field research was conducted in a lakeside community in Shiga Prefecture. Most of this area is covered with steep ranges of forests and rice paddies. The farmers in the area who had no water rights invented a device to share groundwater. The resulting new norms concerning groundwater management help to avoid conflicts among water users. The individual interests of farmers in disadvantaged areas have been adjusted to form unique communal norms, equalising individual differences, as well as to give birth to an alternative water management system based on consensus among users.

Local approaches to biodiversity conservation: lessons from Oaxaca, southern Mexico

Large areas of Oaxaca, southern Mexico, exhibit high biodiversity in the absence of official protected areas. This paper discusses some of the key mechanisms and practices employed by local communities to help conserve their forest resources. The findings suggest that learning from local resource management systems should become an important component of future conservation planning in Mexico. This will require conservationists and the wider public to consider local communities to be a necessary part of territorial and ecological processes and, in some instances, to give them a greater role in biodiversity conservation and stewardship of the country's forest commons. However, such a shift in thinking is unlikely to occur until more research is carried out to determine the specific impacts of these land-use systems on biodiversity and ecological integrity.

Presence, performance, and institutional resilience of sasi, a traditional management institution in Central Maluku, Indonesia

This article presents the results of a study performed in 1997–98 on the presence, performance and institutional resilience of sasi, a local traditional resource management system in Central Maluku, Indonesia. The presence and functionality of sasi in 63 villages was determined by way of structured key informant interviews that explored the existence of sasi ceremonies, presence of written rules and sanctions, level and consistency of activity over time, area of application of rules, and governing authorities. Performance was measured in 11 villages with strong sasi and 11 villages where it was weak. Performance was expressed in terms of biological and social sustainability, efficiency and equity. A number of factors were identified that contribute positively to the resilience of sasi as a local institution. The results of the study can be useful in the revitalization of traditional institutions or the establishment of new institutions in the context of decentralization and co-management in Indonesia and elsewhere.