Data Location Service Research Articles

HDS: A Fast Hybrid Data Location Service for Hierarchical Mobile Edge Computing

The hierarchical mobile edge computing satisfies the stringent latency requirements of data access and processing for emerging edge applications. The data location service is a basic function to provide data storage and retrieval to enable these applications. However, it still lacks research of a scalable and low-latency data location service in the environment. The existing solutions, such as DNS and DHT, fail to meet the requirement of those latency-sensitive applications. Therefore, in this article, we present a low-latency hybrid data-sharing framework, HDS. The HDS divides the data location service into two parts: intra-region and inter-region. More precisely, we design a data sharing protocol called Cuckoo Summary to achieve fast data localization in intra-region. Furthermore, for the inter-region data sharing, we develop a geographic routing based scheme to achieve efficient data localization with only one overlay hop. The advantages of HDS include short response latency, low implementation overhead, and few false positives. We implement the HDS framework based on a P4 prototype. The experimental results show that, compared to the state-of-the-art solutions, our design achieves 50.21% shorter lookup paths and 92.75% fewer false positives.

The Algorithm of the Snail: An Example to Grasp the Window of Opportunity to Boost Big Data

This work explores a new application which can effectively meet different localization accuracy requirements of most data location services studying the interactions between customers and suppliers. It helps to have the status or position of what is sought with respect to an address that summarizes thus a reference point which is the point of research. This proposal explains what snail algorithm is and how we can benefit from using it for the localization of information for business applications especially in the field of analytics. A business application using our algorithm has been developed by the Autour.com company (located in the department of Herault, Montpellier city) to illustrate its feasibility and availability. The results show that our algorithm can improve the localization accuracy.

A Lightweight Data Location Service for Nondeterministic Exascale Storage Systems

In this article, we present LWDLS, a lightweight data location service designed for Exascale storage systems (storage systems with order of 10 18 bytes) and geo-distributed storage systems (large storage systems with physically distributed locations). LWDLS provides a search-based data location solution, and enables free data placement, movement, and replication. In LWDLS, probe and prune protocols are introduced that reduce topology mismatch, and a heuristic flooding search algorithm (HFS) is presented that achieves higher search efficiency than pure flooding search while having comparable search speed and coverage to the pure flooding search. LWDLS is lightweight and scalable in terms of incorporating low overhead, high search efficiency, no global state, and avoiding periodic messages. LWDLS is fully distributed and can be used in nondeterministic storage systems and in deterministic storage systems to deal with cases where search is needed. Extensive simulations modeling large-scale High Performance Computing (HPC) storage environments provide representative performance outcomes. Performance is evaluated by metrics including search scope, search efficiency, and average neighbor distance. Results show that LWDLS is able to locate data efficiently with low cost of state maintenance in arbitrary network environments. Through these simulations, we demonstrate the effectiveness of protocols and search algorithm of LWDLS.

A multi-dimensional view on information retrieval of CMS data

The CMS Dataset Bookkeeping System (DBS) search page is a web-based application used by physicists and production managers to find data from the CMS experiment. The main challenge in the design of the system was to map the complex, distributed data model embodied in the DBS and the Data Location Service (DLS) to a simple, intuitive interface consistent with the mental model of physicists analyzing the data. We used focus groups and user interviews to establish the required features. The resulting interface addresses the physicist and production manager roles separately, offering both a guided search structured for the common physics use cases as well as a dynamic advanced query interface.

CMS workload management

From september 2007 the LHC accelerator will start its activity and CMS, one of the four experiments, will begin to take data. The CMS computing model is based on the the Grid paradigm where data is deployed and accessed on a number of geographically distributed computing centers. In addition to real data events, a large number of simulated ones will be produced in a similar, distributed manner. Both real and simulated data will be analyzed by physicist, at an expected rate of 100000 jobs per day submitted to the Grid infrastructure. In order to reach these goals, CMS is developing two tools for the workload management (plus a set of services): ProdAgent and CRAB. The ProdAgent deals with MonteCarlo production system: it creates and configures jobs, interacts with the Framework, merges outputs to a reasonable filesize and publishes the simulated data back into CMS data bookkeeping and data location services. CRAB (Cms Remote Analysis Builder) is the tool deployed ad hoc by CMS to access those remote data. CRAB allows a generic user, without specific knowledge of the Grid infrastructure, to access data and perform its analysis as simply as in a local environment. CRAB takes care to interact with all Data Management services, from data discovery and location to output file management. An overview of the current implementation of the components of the CMS workload management is presented in this work.

Evaluating Performance Tradeoffs in a One-to-Many Peer Content Distribution Architecture

Peer-to-Peer (P2P) content exchange has recently gained attention from both the research and industrial communities. The dynamic nature of peer networks and the resource constraints of peer hosts have introduced a number of unique technical challenges that must be addressed to make large-scale P2P content exchange applications more efficient. In this work, we expand our previous work on Pixie, an architecture that integrates one-to-many distribution of content and peer networks. Pixie uses a jukebox-style scheduling mechanism to provide a valuable data location service. Users can browse a listing of all content scheduled to be distributed across the network thus reducing search overhead. Moreover, Pixie’s use of one-to-many content distribution provides additional scalability. Our results indicate that, using Pixie, we can significantly reduce the resources required to locate and distribute content in peer networks. The properties Pixie embodies will become increasingly important as peer content exchange is extended to support more advanced, and possibly commercial applications.

Integrated access to distributed data and information services in astrophysics and the space sciences

Scientific advances do not necessarily follow strict research discipline boundaries. In the area of astronomy and space science, data from multiple missions and observatories operating in various parts of electromagnetic spectrum are necessary in order to answer fundamental scientific questions. However, a researcher attempting to locate, understand, and use data from a variety of sources now faces serious difficulties. Many datasets are available on the Internet but finding the ones of relevance, especially outside of one's immediate field of expertise, is difficult. The metadata used to annotate datasets in different fields can be unfamiliar and obscure, even though the same basic data attributes are being described. Once data has been located, it is often in different and incompatible formats. We are now developing a distributed information service for the space sciences – ISAIA – which covers many different subdisciplines with datasets of common interest and which will improve the researcher's abilities to locate and use data from a wide variety of on-line resources. This service builds upon experience in implementing a data location service for astronomy, Astrobrowse. The key to implementing such services is the concept of profiles. Profiles are a generalization of data dictionaries: they define the metadata labels and content for information resources and provide mappings of those labels onto site or service-specific terms and query protocols. In ISAIA profiles may be hierarchical – general at the highest levels, with sub-profiles for certain disciplines or types of instrumentation. In practical terms, a distributed system such as ISAIA will only succeed if existing data providers can make their systems and services available to it with a minimum amount of effort. ISAIA must be “light weight”, imposing no constraints on how an organization manages its data services internally. It must support participation at different levels, from simple registration of a service to full support for query and response interpretation. In addition, ISAIA must preserve the identity and visibility of the participating services. This is important for end-users (who must understand the provenance of the data they are using) and the providers (who need to understand their users' needs and the demands upon their resources).