Distributed Data Resources Research Articles

Spatio-temporal data fusion techniques for modeling digital twin City

ABSTRACT The digital twin city technique maps the massive city environmental and social data on a three-dimensional virtual model. It presents the operational status of physical world and supports intelligent city governance. However, the inefficient utilization of distributed data resources, and the lack of sharing and collaboration among multiple departments restrict the data formulation of digital twin city construction. This research proposes a new cross-domain spatio-temporal data fusion framework for supporting complex urban governance. It integrates the heterogeneous urban information generated and stored by different government departments using multiple-information techniques. A specified geographic base reflecting the real city status is established, using geographical entities with unified address as identifiers to encapsulate the urban elements information. We introduce a comprehensive urban spatio-temporal data center construction process, which has already supported multiple urban governance projects. The two distinct advantages in using this data fusion system are: 1) The proposed Bert+PtrNet+ESIM-based address mapping method associates the urban elements information to their corresponding geographic entities with 99.3% F1-Score on real-world dataset. 2) The Wuhan spatio-temporal data center operation illustrates the capability of our framework for complex urban governance, which significantly improves the efficiency of urban management and services. This integrated system engineering provides reference and inspiration for further spatio-temporal data management, which contributes to the future social governance in digital twin city platform.

TDLearning: Trusted Distributed Collaborative Learning Based on Blockchain Smart Contracts

Massive amounts of data drive the performance of deep learning models, but in practice, data resources are often highly dispersed and bound by data privacy and security concerns, making it difficult for multiple data sources to share their local data directly. Data resources are difficult to aggregate effectively, resulting in a lack of support for model training. How to collaborate between data sources in order to aggregate the value of data resources is therefore an important research question. However, existing distributed-collaborative-learning architectures still face serious challenges in collaborating between nodes that lack mutual trust, with security and trust issues seriously affecting the confidence and willingness of data sources to participate in collaboration. Blockchain technology provides trusted distributed storage and computing, and combining it with collaboration between data sources to build trusted distributed-collaborative-learning architectures is an extremely valuable research direction for application. We propose a trusted distributed-collaborative-learning mechanism based on blockchain smart contracts. Firstly, the mechanism uses blockchain smart contracts to define and encapsulate collaborative behaviours, relationships and norms between distributed collaborative nodes. Secondly, we propose a model-fusion method based on feature fusion, which replaces the direct sharing of local data resources with distributed-model collaborative training and organises distributed data resources for distributed collaboration to improve model performance. Finally, in order to verify the trustworthiness and usability of the proposed mechanism, on the one hand, we implement formal modelling and verification of the smart contract by using Coloured Petri Net and prove that the mechanism satisfies the expected trustworthiness properties by verifying the formal model of the smart contract associated with the mechanism. On the other hand, the model-fusion method based on feature fusion is evaluated in different datasets and collaboration scenarios, while a typical collaborative-learning case is implemented for a comprehensive analysis and validation of the mechanism. The experimental results show that the proposed mechanism can provide a trusted and fair collaboration infrastructure for distributed-collaboration nodes that lack mutual trust and organise decentralised data resources for collaborative model training to develop effective global models.

Boosting for Distributed Online Convex Optimization

Decentralized Online Learning (DOL) extends online learning to the domain of distributed networks. However, limitations of local data in decentralized settings lead to a decrease in the accuracy of decisions or models compared to centralized methods. Considering the increasing requirement to achieve a high-precision model or decision with distributed data resources in a network, applying ensemble methods is attempted to achieve a superior model or decision with only transferring gradients or models. A new boosting method, namely Boosting for Distributed Online Convex Optimization (BD-OCO), is designed to realize the application of boosting in distributed scenarios. BD-OCO achieves the regret upper bound <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathcal{O}\left(\frac{M+N}{MN}T\right)$</tex> , where <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$M$</tex> measures the size of the distributed network and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$N$</tex> is the number of Weak Learners (WLs) in each node. The core idea of BD-OCO is to apply the local model to train a strong global one. BD-OCO is evaluated on the basis of eight different real-world datasets. Numerical results show that BD-OCO achieves excellent performance in accuracy and convergence, and is robust to the size of the distributed network.

RETRACTED: Development of Island tourism resources based on distributed database and embedded system

This article describes the sustainable production system made islands. Distributed data resources used to produce goods for consumption and disposal. Production activities are supported by global non-critical resources, such as capital, technology and expertise in embedded systems, patents and data: analysis and business plan. Provide a case study to explain this arrangement. Understand the potential of the Island's manufacturing sector to add value to the debate on sustainable manufacturing. This study aims to develop an embedded system as a model recognized as a highly developed modern development model, and tourism destination as a system bound essentially regional representatives with a specific development framework. Find similarities with the indispensable support of the overall product at your destination for a distributed database server. By this analogy, find a long-term development model suitable for local commercial embedded systems tourist destinations. A reasonable solution is the concept of sustainable development; the basic principle is coherent. Tourism destination management principles function.

Gap analysis on open data interconnectivity for disaster risk research

ABSTRACTOpen data strategies are being adopted in disaster-related data particularly because of the need to provide information on global targets and indicators for implementation of the Sendai Framework for Disaster Risk Reduction 2015–2030. In all phases of disaster risk management including forecasting, emergency response and post-disaster reconstruction, the need for interconnected multidisciplinary open data for collaborative reporting as well as study and analysis are apparent, in order to determine disaster impact data in timely and reportable manner. The extraordinary progress in computing and information technology in the past decade, such as broad local and wide-area network connectivity (e.g. Internet), high-performance computing, service and cloud computing, big data methods and mobile devices, provides the technical foundation for connecting open data to support disaster risk research. A new generation of disaster data infrastructure based on interconnected open data is evolving rapidly. There are two levels in the conceptual model of Linked Open Data for Global Disaster Risk Research (LODGD) Working Group of the Committee on Data for Science and Technology (CODATA), which is the Committee on Data of the International Council for Science (ICSU): data characterization and data connection. In data characterization, the knowledge about disaster taxonomy and data dependency on disaster events requires specific scientific study as it aims to understand and present the correlation between specific disaster events and scientific data through the integration of literature analysis and semantic knowledge discovery. Data connection concepts deal with technical methods to connect distributed data resources identified by data characterization of disaster type. In the science community, interconnected open data for disaster risk impact assessment are beginning to influence how disaster data are shared, and this will need to extend data coverage and provide better ways of utilizing data across domains where innovation and integration are now necessarily needed.

A Semantic Framework to Improve Interoperability of Malaria Surveillance Systems

ObjectiveMalaria is one of the top causes of death in Africa and some other regions in the world. Data driven surveillance activities are essential for enabling the timely interventions to alleviate the impact of the disease and eventually eliminate malaria. Improving the interoperability of data sources through the use of shared semantics is a key consideration when designing surveillance systems, which must be robust in the face of dynamic changes to one or more components of a distributed infrastructure. Here we introduce a semantic framework to improve interoperability of malaria surveillance systems (SIEMA).IntroductionIn 2015, there were 212 million new cases of malaria, and about 429,000 malaria death, worldwide. African countries accounted for almost 90% of global cases of malaria and 92% of malaria deaths. Currently, malaria data are scattered across different countries, laboratories, and organizations in different heterogeneous data formats and repositories. The diversity of access methodologies makes it difficult to retrieve relevant data in a timely manner. Moreover, lack of rich metadata limits the reusability of data and its integration. The current process of discovering, accessing and reusing the data is inefficient and error-prone profoundly hindering surveillance efforts.As our knowledge about malaria and appropriate preventive measures becomes more comprehensive malaria data management systems, data collection standards, and data stewardship are certain to change regularly. Collectively these changes will make it more difficult to perform accurate data analytics or achieve reliable estimates of important metrics, such as infection rates. Consequently, there is a critical need to rapidly re-assess the integrity of data and knowledge infrastructures that experts depend on to support their surveillance tasks.MethodsIn order to address the challenge of heterogeneity of malaria data sources we recruit domain specific ontologies in the field (e.g. IDOMAL (1)) that define a shared lexicon of concepts and relations. These ontologies are expressed in the standard Web Ontology Language (OWL).To over come challenges in accessing distributed data resources we have adopted the Semantic Automatic Discovery &amp; Integration framework (SADI) (2) to ensure interoperability. SADI provides a way to describe services that provide access to data, detailing inputs and outputs of services and a functional description. Existing ontology terms are used when building SADI Service descriptions. The services can be discovered by querying a registry and combined into complex workflows. Users can issue SPARQL syntax to a query engine which can plan complex workflows to fetch actual data, without having to know how target data is structured or where it is located.In order to tackle changes in target data sources, the ontologies or the service definitions, we create a Dashboard (3) that can report any changes. The Dashboard reuses some existing tools to perform a series of checks. These tools compare versions of ontologies and databases allowing the Dashboard to report these changes. Once a change has been identified, as series of recommendations can be made, e.g. services can be retired or updated so that data access can continue.ResultsWe used the Mosquito Insecticide Resistance Ontology (MIRO) (5) to define the common lexicon for our data sources and queries. The sources we created are CSV files that use the IRbase (4) schema. With the data defined using we specified several SPARQL queries and the SADI services needed to answer them. These services were designed to enabled access to the data separated in different files using different formats. In order to showcase the capabilities of our Dashboard, we also modified parts of the service definitions, of the ontology and of the data sources. This allowed us to test our change detection capabilities. Once changes where detected, we manually updated the services to comply with a revised ontology and data sources and checked that the changes we proposed where yielding services that gave the right answers. In the future, we plan to make the updating of the services automatic.ConclusionsBeing able to make the relevant information accessible to a surveillance expert in a seamless way is critical in tackling and ultimately curing malaria. In order to achieve this, we used existing ontologies and semantic web services to increase the interoperability of the various sources. The data as well as the ontologies being likely to change frequently, we also designed a tool allowing us to detect and identify the changes and to update the services so that the whole surveillance systems becomes more resilient.

A decentralized gossip based approach for data clustering in peer-to-peer networks

In this paper, a novel distributed approach, named GDSOM-P2P, for clustering distributed data resources is proposed by combining, an improved version of Silhouette algorithm, the dynamic Self-Organizing Map (SOM) neural network, and VICINITY protocol as a generic overlay management framework based on self-organization. The proposed GDSOM-P2P is adapted to the dynamic conditions of these networks. In the proposed GDSOM-P2P algorithm, at first, each node extracts a number of important data through the SOM and Silhouette algorithms. Then each of the nodes chooses one of its neighbors with the help of the VICINITY algorithm, and exchanges their important data with their neighbors. By doing this, over a period, the nodes’ data will be distributed over the entire network and the nodes in the network access the summary data model of the whole data. Finally, each node aggregates its internal data with a summary model and then performs the final clustering to cluster its internal data correctly. Evaluation results over a real P2P environment verify the efficiency of proposed GDSOM-P2P. Furthermore, the proposed GDSOM-P2P is also compared with the existing well-established distributed data clustering techniques. The results show a significant accuracy improvement of the proposed method.

Replica Placement and Predictive Replica Selection Techniques to Improve the Performance of Data Grid

Data grid is a hardware and software infrastructure built on top of Internet and provides sharing and management of distributed data resources. Dynamic data replication techniques are used in data grids to enhance data sharing with reduced access latency and bandwidth consumption. Creating and placing duplicate copies of popular files in data grid is replica placement, whereas, replica selection is selecting the best location to access data for executing job in data grid. Efficient techniques with load balancing and prediction based on SVM and AHL that improve and optimize the replica placement and selection are introduced in this paper. The proposed algorithms reduce the makespan i.e. total job execution time, storage consumption and effective network usage. The proposed techniques consist of two phases. In the first phase, all the nodes in the same region are grouped together and replica is placed in the highest degree and highest frequency node in the region. During second phase, SVM & AHL are used for predicting the behavior of user applications. The jobs are executed in the grid with predicted events. Simulation results reveal that the proposed methodologies perform better in terms of makespan, storage consumption and replication count.

Low bandwidth desktop and video streaming for collaborative tiled display environments

High-resolution display environments built on networked, multi-tile displays have emerged as an enabling tool for collaborative, distributed visualization work. They provide a means to present, compare, and correlate data in a broad range of formats and coming from a multitude of different sources. Visualization of these distributed data resources may be achieved from a variety of clustered processing and display resources for local rendering and may be streamed on demand and in real-time from remotely rendered content. The latter is particularly important when multiple users want to concurrently share content from their personal devices to further augment the shared workspace. This paper presents a high-quality video streaming technique allowing remotely generated content to be acquired and streamed to multi-tile display environments from a range of sources and over a heterogeneous wide area network.The presented technique uses video compression to reduce the entropy and therefore required bandwidth of the video stream. Compressed video delivery poses a series of challenges for display on tiled video walls which are addressed in this paper. These include delivery to the display wall from a variety of devices and localities with synchronized playback, seamless mobility as users move and resize the video streams across the tiled display wall, and low latency video encoding, decoding, and display necessary for interactive applications. The presented technique is able to deliver 1080p resolution, multimedia rich content with bandwidth requirements below 10 Mbps and low enough latency for constant interactivity. A case study is provided, comparing uncompressed and compressed streaming techniques, with performance evaluations for bandwidth use, total latency, maximum frame rate, and visual quality.

A Model of Replication Creation in Distributed Networks

Data Grid provides geographically distributed data resources. However, ensuring efficient and fast access to such huge and widely distributed resources is hindered by the high latencies of the Internet. Replication is adopted to address these problems. Aimed to replica creation, a model to decide replica creation opportune moment is introduced, which is based on queuing theory. Through statistical record of the arrival rate of uses, service period, and replicas already created, calculating length of queue, waiting duration time, busy period, service strength and so on, a reasonable creation opportune moment of replica can be obtained.

MyDas, an Extensible Java DAS Server

A large number of diverse, complex, and distributed data resources are currently available in the Bioinformatics domain. The pace of discovery and the diversity of information means that centralised reference databases like UniProt and Ensembl cannot integrate all potentially relevant information sources. From a user perspective however, centralised access to all relevant information concerning a specific query is essential. The Distributed Annotation System (DAS) defines a communication protocol to exchange annotations on genomic and protein sequences; this standardisation enables clients to retrieve data from a myriad of sources, thus offering centralised access to end-users.We introduce MyDas, a web server that facilitates the publishing of biological annotations according to the DAS specification. It deals with the common functionality requirements of making data available, while also providing an extension mechanism in order to implement the specifics of data store interaction. MyDas allows the user to define where the required information is located along with its structure, and is then responsible for the communication protocol details.

Integrated geoscience databanks for interactive analysis and visualization

There has been a worldwide revolution in geoscientific data availability and access. An effectively infinite and instantaneous free access to geoscientific data from the World Wide System of Geoscience Data Centers and Virtual Observatories is available. In addition, national databanks and commercially available large exploration data-sets also exist. These distributed data resources impose challenges for the future to move toward their objective integration and visualization to discover new knowledge. Such advancements can facilitate meaningful interpretations and decision-making for the benefit of society at global and local scales. This article presents the Digital Earth initiative at a national level to address multiple domains, such as effective management of natural resources, interactive planning of exploration activities and monitoring, mapping and mitigation of natural hazards. It discusses a distributed geospatial data infrastructure and its importance in geoscientific data integration for efficient and interactive data retrieval, analysis and visualization. Some examples are presented to demonstrate the advantages of integrated visualization in geoscientific analysis.

WS-DAI-DM: An Interface Specification for Data Mining in Grid Environments

Providing the appropriate access means for data mining services in Grid Environment is principal for com- bination of Grid and data mining. The transition from cen- tralized data mining process as they are in traditional tools to Grid-compliant and Grid-based data mining services that can coordinate with each other is important to extract useful and potential knowledge/patterns from distributed data resources. But, data mining process has not been integrated with the current Grid architecture, and there are no related specifications or standards available within the Grid com- munity for handling it. The work discusses the interface specification WS-DAI-DM and related issues of providing the consistent web service interfaces to mine meaningful and hidden information/patterns from distributed data re- sources in Grid environments. WS-DAI-DM can facilitate using data mining tools and developing data mining applica- tions. The proposed specification idea is partially imple- mented in the feasibility study and we give an application scenario about how users can access data mining services in Grid environments conveniently via the proposed mecha- nism.

PS1-24: Proof of Principle Demonstrations of a Distributed Research Network: Findings and Lessons Learned

Researchers, policy makers and others commonly use electronic data routinely collected during the delivery of, or payment for, medical care to study the “real world” effectiveness, comparative effectiveness, safety, and costs of medical interventions. However, even very large individual healthcare data resources are often not big enough to adequately conduct post-marketing evidence studies. To improve the ability to use multiple distributed data resources, like the HMORN’s Virtual Data Warehouse, the Developing Evidence to Inform Decisions about Effectiveness (DEcIDE) centers at the HMORN Center for Education and Research on Therapeutics (CERT) and the University of Pennsylvania are developing a design for a scalable distributed research network. Funded by AHRQ’s Effective Health Care Program, the project intends to develop the framework for a distributed research network that will help close the knowledge gap regarding the “real world” effectiveness, comparative effectiveness, and safety of medical technologies. The network will have these important attributes: distributed architecture, strong local control of data uses, and federated querying. Constructing the network is a challenge, even among sites with fully operational local virtual data warehouse (VDW) installations, because of differences in computing environments and information systems, the need for responsible stewardship of clinical records, concerns related to data privacy, software development obstacles, the use of proprietary data, and the need to develop effective governance mechanisms. To investigate some of these issues, the project includes implementation of a proof of concept prototype involving these HMORN sites: Kaiser Permanente Northern California, Kaiser Permanente Colorado, Harvard Pilgrim Health Care, Group Health Cooperative, and Geisinger Health System. The goals of the proof of concept work are to distribute a query to separately stored data resources, execute the query remotely, and present aggregated results to an authorized system user. The proof of concept will be designed with a single network portal that manages the network software, including security, authorization, authentication, messaging, and query aggregation. We will present our findings from the proof of concept, including information on specific architecture and design used, the success of the proof of concept, and lessons learned from implementation.

Services for replica consistency handling in data grids

The grid technology is a popular framework for collaborative, resource sharing computing infrastructure for many scientific domains. As wider scientific communities engage in the use of the grids, the view on the grid technology changes from the traditional high-performance computing perspective to a wider context. Data management is becoming one of the main areas of research interest in the scope of grid systems. In this paper, we present services for replica update propagation and consistency handling, which try to address the absence of software tools for maintaining the consistency among distributed data resources in a grid environment. We describe the architecture of the services as well as several important implementation details. We discuss the possibility of integration consistency handling services with legacy software and grid middleware services. This gives us the possibility to benefit from legacy software functionality and, in the same time, to increase the data availability and fault-tolerance by means of data replication.

A global and parallel file system for grids

Data management is one of the most important problems in grid environments. Most of the efforts in data management in grids have been focused on data replication. Data replication is a practical and effective method to achieve efficient data access in grids. However all data replication schemes lack in providing a grid file system. One important challenge facing grid computing is the design of a grid file system. The Global Grid Forum defines a Grid File System as a human-readable resource namespace for management of heterogeneous distributed data resources, that can span across multiple autonomous administrative domains. This paper describes a new Grid File System according to the Global Grid Forum recommendations that integrates heterogeneous data storage resources in grids using standard grid technologies: GridFTP and the Resource Namespace Services, both defined by the Global Grid Forum. To obtain high performance, we apply the parallel I /O techniques used in traditional parallel file systems.

Relating Requirements and Architectures: A Study of Data-Grids

The requirements and architecture of any complex software system are highly interdependent. We have studied the relationship between these two concerns in several data-Grid systems. Data-Grids are characterized by an infrastructure that focuses on the coordinated management of, and access to distributed data resources. We survey current data-Grid projects to demonstrate that a set of general requirements for data-Grid systems can be identified. Architectural styles are a way of highlighting design and engineering similarities between software systems. We consider the styles that are exhibited by current data-Grids and use a lightweight methodology to analyze how these styles support general requirements. Our conclusions provide guidelines to assist the data-Grid developer in making informed architectural choices. In this paper, we present an extended case study of the relationship between the requirements and architecture of data-Grid systems. The architectures of current data-Grids can be shown to exhibit characteristics of various architectural styles. By analyzing how these styles support the core requirements of the domain, we can identify those styles that offer ‘best-fit’ and provide guidelines for the engineering of dataGrid systems. The relationship between requirements and architectures is not a concern unique to the Grid domain, but an area of active enquiry in the wider software engineering community. By means of this detailed study we hope to contribute to this discussion. Within an informal taxonomy of Grid systems, data-Grids are concerned with the generation of new information from distributed data repositories. DataGrids yield new information in various ways, by making available to scientists an unprecedented volume of useful data. They allow more rigorous statistical analysis, and enable the application of new data-mining techniques and the cross-correlation of sets of data that have not previously been compared. Data-Grids are characterized by an infrastructure that focuses on the coordinated management of distributed data resources and the provision of data access mechanisms.

Scalable networked information processing environment (SNIPE)

Scalable Networked Information Processing Environment (SNIPE) is a metacomputing system that aims to provide a reliable, secure, fault-tolerant environment for long-term distributed computing applications and data stores across the global Internet. This system combines global naming and replication of both processing and data to support large-scale information processing applications leading to better availability and reliability than currently available with typical cluster computing and/or distributed computer environments. To facilitate this the system supports: distributed data collection, distributed computation, distributed control and resource management, distributed output and process migration. The underlying system supports multiple communication paths, media and routing methods to aid performance and robustness across both local and global networks. This paper details the goals, design and an initial implementation of SNIPE, and then demonstrates its usefulness in supporting a middleware project. Initial communications performance is also presented.

Clinical resource auditing and decision support for computerized patient record systems: A mediated architecture approach

This paper has three major goals: (1) Show by demonstration (in a currently implemented CPR) that for this case, the additional information entry required by providers to acquire the information for Clinical Decision Support (CDS) and Resource Auditing (RA) comes as a side effect of patient care documentation. (2) Use simple simulation to explore the practical issues of allowing users database access to this cross-patient information for CDS and RA. (3) Relate by argument the findings in 1 and 2 to the concept of mediated information systems architectures for CDS and RA. The primary finding of this paper is that a mediated architecture is the only practical way to offer distributed knowledge and data resources to the CPR user.