Data Management Requirements Research Articles

The IoT Applications and Consideration for NoSQL Database

A large amount of data are generated every moment by connected objects creating Internet of Things (IoT). These data are difficult to handle using traditional databases leading to use NoSQL databases. These latter have achieved a large popularity thanks to their high performance, flexibility in scaling, and high availability. But, which NoSQL database is the most suitable for IoT applications? In order to answer this question, we need a significant knowledge about the domain requirements. For this reason, we give in the next subsection, a set of IoT data management requirements. In this paper, i first give a set of data management requirements for IoT applications, i propose then a comparative study of five of the most popular NoSQL databases namely, Redis, Cassandra, MongoDB, Couchbase and Neo4j, regarding to data management requirements, in order to find the most suitable NoSQL database for IoT applications.

Extending Data Quality Management for Smart Connected Product Operations

Smart connected product (SCP) operation embodies the concept of the internet of things (IoT). To increase the probability of success of SCP operations for customers, the high quality of the IoT data across operations is imperative. IoT data go beyond sensor data, as integrate some other various type of data such as timestamps, device metadata, business data, and external data through SCP operation processes. Therefore, traditional data-centric approaches that analyze sensor data and correct their errors are not enough to preserve, in long-term basis, adequate levels of quality of IoT data. This research provides and alternative framework of data quality management as a process-centric approach to improve the quality of IoT data. The proposed framework extends the process reference model (PRM) for data quality management (DQM) defined in ISO 8000-61, and tailored to fully adapt to the special requirements of the IoT data management. These involve several adaptations: first, the scope of the SCP operations for data quality management is determined, and the processes required for SCP operations are defined following the process description format of ISO 8000-61. Second, the relationship between the processes and the structure of the processes in the technology stack of the SCP operations are described to cover the actual nature of the IoT data flows. Finally, a new IoT DQM-PRM is proposed by integrating the processes for the SCP operations with DQM-PRM. When these processes are executed in the organization, the quality of IoT data composed of data of various types can be continuously improved and the utilization rate of SCP operations is expected to increase.

Data Management Framework for Internet of Things

The Internet of Things (IoT) is the dynamic and global network of smart interconnected objects connected to each other through the internet. IoT is generating massive volume of heterogenous, continuously streaming data. Database requirements to store and analyze IoT data are unique. This paper focuses on data life cycle, characteristics and the challenges to store IoT data and makes few recommendations on different selection criteria that can be considered for selecting database management technologies. It also explores core IoT components and explains how data management requirements are distinct for different layers of IoT architecture. The paper also touches upon emergence of fog computing and its role in IoT.

Emerging Trends in Hybrid Wireless Communication and Data Management for the Industry 4.0

With the fast-paced realization of the Industry 4.0 paradigm, completely centralized networking solutions will no longer be sufficient to meet the stringent requirements of the related industrial applications. Besides requiring fast response time and increased reliability, they will necessitate computational resources at the edge of the network, which demands advanced communication and data management techniques. In this paper, we provide an overview of the network communications and data management aspects for the Industry 4.0. Our global perspective is to understand the key communication and data management challenges and peculiarities for the effective realization of the fourth industrial revolution. To address these challenges, this paper proposes hybrid communications management and decentralized data distribution solutions supported by a hierarchical and multi-tier network architecture. The proposed solutions combine local and decentralized management with centralized decisions to efficiently use the available network resources and meet the requirements of Industry 4.0 applications. To this end, the distributed management entities interact in order to coordinate their decisions and ensure the correct operation of the whole network. Finally, the use of Radio Access Network (RAN) slicing is proposed to achieve the required flexibility to efficiently meet the stringent and varying communication and data management requirements of industrial applications.

The Italian neuromuscular registry: a coordinated platform where patient organizations and clinicians collaborate for data collection and multiple usage

BackgroundThe worldwide landscape of patient registries in the neuromuscular disease (NMD) field has significantly changed in the last 10 years, with the international TREAT-NMD network acting as strong driver. At the same time, the European Medicines Agency and the large federations of rare disease patient organizations (POs), such as EURORDIS, contributed to a great cultural change, by promoting a paradigm shift from product-registries to patient-centred registries. In Italy, several NMD POs and Fondazione Telethon undertook the development of a TREAT-NMD linked patient registry in 2009, with the referring clinical network providing input and support to this initiative through the years. This article describes the outcome of this joint effort and shares the experience gained.MethodsThe Italian NMD registry is based on an informatics technology platform, structured according to the most rigorous legal national and European requirements for management of patient sensitive data. A user-friendly web interface allows both direct patients and clinicians’ participation. The platform’s design permits expansion to incorporate new modules and new registries, and is suitable of interoperability with other international efforts.ResultsWhen the Italian NMD Registry was initiated, an ad hoc legal entity (NMD Registry Association) was devised to manage registries’ data. Currently, several disease-specific databases are hosted on the platform. They collect molecular and clinical details of individuals affected by Duchenne or Becker muscular dystrophy, Charcot-Marie-Tooth disease, transthyretin type-familial amyloidotic polyneuropathy, muscle glycogen storage disorders, spinal and bulbar muscular atrophy, and spinal muscular atrophy. These disease-specific registries are at different stage of development, and the NMD Registry itself has gone through several implementation steps to fulfil different technical and governance needs. The new governance model is based on the agreement between the NMD Registry Association and the professional societies representing the Italian NMD clinical network. Overall, up to now the NMD registry has collected data on more than 2000 individuals living with a NMD condition.ConclusionsThe Italian NMD Registry is a flexible platform that manages several condition-specific databases and is suitable to upgrade. All stakeholders participate in its management, with clear roles and responsibilities. This governance model has been key to its success. In fact, it favored patient empowerment and their direct participation in research, while also engaging the expert clinicians of the Italian network in the collection of accurate clinical data according to the best clinical practices.

Harnessing Smartphone Technology to Improve Patient Engagement: Codesigning and Testing The SecondEars Consultation Audio Recording App for People With Cancer

Background: Oncology consultations can be overwhelming and information-dense. Consultation audio recordings are an effective method to promote patient participation, improve patients' recall and understanding of medical information, and can increase engagement and support through sharing with family and friends. Innovative m-health technologies such as the SecondEars consultation audio recording mobile app provide a patient-designed and controlled audio recording solution while ensuring security and legal protection for clinicians. To ensure the SecondEars app meets the needs of patients, family, clinicians and hospital administrators, the opinions of stakeholders and end users are integral to the concept and study creation, design, and testing processes. Aim: To use key elements of codesign methodology to develop and test SecondEars, a mobile app that allows patients to audio record their consultations within a clinical oncology setting. Methods: The SecondEars app concept began with consumer suggestions that consultation audio recording be used as standard practice of care using patient-driven (use is controlled/determined by patients) technologies. An app solution was conceived and the concept discussed through engagement of consumers as part of the project and research codesign team. Key health service stakeholders comprising representatives from: legal, information technology (IT), clinical service leads, digital strategy, and health information services (HIS), provided project oversight and core legal and data management requirements. The Scrum management framework was used to structure the codesign process during the development and testing phase. Six workshops were run to facilitate further user input with regard to specific app functionality and design. A wireframe version was developed and tested by the codesign team in the final workshop. Feedback was incorporated into a prototype which was circulated for acceptance testing. Results: Twenty people participated in the stakeholder engagement and workshops, comprising: oncology consumers, researchers, IT, HIS, app developers, and oncology health professionals. Key specifications necessitated that SecondEars be patient-driven, secure and confidential, have clear legal guidelines regarding audio recording sharing, be integrated with medical records, and require minimal upfront and ongoing resources. Seven people tested the prototype app during user acceptance testing and gave positive and constructive feedback. A final version of the app was then made available for clinical testing. Conclusion: The SecondEars consultation audio recording app has been successfully created by and for patients. This app gives patients permission and autonomy to audio record and share their consultations responsibly, while maintaining legal protection for clinicians. This app is currently being tested in a clinical setting prior to conducting an implementation study.

Scalable and flexible management of medical image big data.

Digital imaging plays a critical role for image guided diagnosis and clinical trials, and the amount of image data is fast growing. There are two major requirements for image data management: scalability for massive scales and support of comprehensive queries. Traditional Picture Archiving and Communication Systems (PACS for short) are based on relational data management systems and suffer from limited scalability and query support. Therefore, new systems that support fast, scalable and comprehensive queries on image data are highly demanded. In this paper, we introduce two alternative approaches: DCMRL/XMLStore (RL/XML for short)-a parallel, hybrid relational and XML data management approach, and DCMDocStore (DOC for short)-a NoSQL document store approach. DCMRL/XMLStore manages DICOM images as binary large objects and metadata as relational tables and XML documents based on IBM DB2, which is parallelized through data partitioning. DCMDocStore manages DICOM metadata as JSON objects, and DICOM images as encoded attachments in MongoDB running on multiple nodes. We have delivered two open source systems DCMRL/XMLStore and DCMDocStore. Both systems support scalable data management and comprehensive queries. We also evaluated them with nearly one million DICOM images from National Biomedical Imaging Archive. The results show that, DCMDocStore demonstrates high data loading speed, high scalability and fault tolerance. DCMRL/XMLStore provides efficient queries, but comes with slower data loading. Traditional PACS systems have inherent limitations on flexible queries and scalability for massive amount of images.

A Real-time Linked Dataspace for the Internet of Things: Enabling “Pay-As-You-Go” Data Management in Smart Environments

As smart environments move from a research vision to concrete manifestations in real-world enabled by the Internet of Things, they are encountering a number of very practical challenges in data management in terms of the flexibility needed to bring together contextual and real-time data, the interface between new digital infrastructures and existing information systems, and how to easily share data between stakeholders in the environment. Therefore, data management approaches for smart environments need to support flexibility, dynamicity, incremental change, while keeping costs to a minimum. A Dataspace is an emerging approach to data management that has proved fruitful for personal information and scientific data management. However, their use within smart environments and for real-time data remains largely unexplored.This paper introduces a Real-time Linked Dataspace (RLD) as an enabling platform for data management within smart environments. This paper identifies common data management requirements for smart energy and water environments, details the RLD architecture and the key support services and their tiered support levels, and a principled approach to “Pay-As-You-Go” data management. The paper presents a dataspace query service for real-time data streams and entities to enable unified entity-centric queries across live and historical stream data. The RLD was validated in 5 real-world pilot smart environments following the OODA (Observe, Orient, Decide, and Act) Loop to build real-time analytics, decisions support, and smart apps for energy and water management. The pilots demonstrate that the RLD enables incremental pay-as-you-go data management with support services that simplify the development of applications and analytics for smart environments. Finally, the paper discusses experiences, lessons learnt, and future directions.

Guest Editorial: Higher Capability, Lower Costs With Cloud Subsurface Data Management

Guest editorial Lower oil prices, lack of experienced personnel, and uncertainty about long-term global oil and gas demand are driving petroleum companies to seek new ways to improve operational capability and lower costs. Cloud subsurface data management has the potential to allow standardized data acquisition, improve data integration and quality, and catalyze data analytics—all at a lower cost than traditional methodologies. Many challenges are involved in acquiring, analyzing, and managing subsurface data. Why is subsurface data so different from other domains? Subsurface comprises many complicated data types, including well logs, core analyses, geologic interpretations, and more. Each data type is complex and requires an experienced subject matter expert (SME) to deal with data acquisition, formatting, and managing. In addition, the subsurface data must be aggregated into a business context to be useful to the daily operational business. IT and domain SMEs must collaborate to ensure that data are always correct, integrated, and accessible. To be effective, a subsurface data manager must have a strong blend of domain and technical capabilities. The minimum level of qualification for a position in data management requires a degree in a relevant discipline such as geoscience, engineering, or information technology. Candidates must capably handle multiple operating systems such as Windows and Linux, and should understand company-specific software at a base level. The technical requirements of subsurface data management are significant. Candidates should know how to script in AWK, sed, Perl, or Python, and additional scripting languages can’t hurt. Lastly, candidates must under-stand management from both a technical perspective—in understanding quality control and juggling multiple data bases—and from a people perspective, in possessing excellent communication skills and handling team personalities. With so much required of those in subsurface data management, the result is a complicated landscape that changes from one business unit to another and evolves through time. With the advent of data analytics, the subsurface data manager’s responsibilities have increased. Analytics cannot thrive unless there is good data on which to act. The broadening subsurface workspace, coupled with the large legacy application set and increasing amounts of information, creates a critical problem. Continuing with the same issues and proprietary software is not the answer: A new way of thinking is required. Why Cloud Platforms Work Utilizing a cloud platform solution for subsurface data can lower infrastructure and support costs while simultaneously providing a platform for all structured and unstructured data along with a col-lection of best-in-class applications for managing and acting on the data to create business value.

Measuring Retention in Antiretroviral Therapy Programs—a Synthetic Review of Different Approaches for Field Use in Low- and Middle-Income Settings

Retention measures in antiretroviral therapy programs are important, but there is wide variation in their calculation and relatively little systematic discussion of their relative advantages and limitations. We extracted and compared distinctive approaches to quantifying retention through a systematic search in PubMed and undertook a purposive selection of articles published in peer-reviewed journals and policy documents. We also created a simulated dataset and code examples to help illustrate observations about each metric. Among identified retention approaches, were metrics based only on proportions of either visits alone (constancy) or visits and appointments (visit adherence), which are simple and most accessible in settings using only paper records and registries. However, they are generally appropriate for patients with similar potential follow-up times and do not incorporate all available information. Survival analysis techniques such as Kaplan-Meier and competing risk approaches offer more nuanced retention measures over time, and can combine individuals with different potential follow-up times into one summary, but have trouble capturing the dynamic nature of retention. Newer approaches, including multi-state models and trajectory analyses, enable more nuanced examination of retention but are analytically difficult to carry out and do not yield one single summary. Simple analytical approaches are more widely useable but may miss important gaps in retention. Use of complex analytical approaches might be limited by requirements of electronically available data, data management requirements, and analytic capacity. Overall, efforts to evaluate retention may benefit from informed selection of one or more approaches to meet a range of objectives.

Best practice data life cycle approaches for the life sciences

Throughout history, the life sciences have been revolutionised by technological advances; in our era this is manifested by advances in instrumentation for data generation, and consequently researchers now routinely handle large amounts of heterogeneous data in digital formats. The simultaneous transitions towards biology as a data science and towards a ‘life cycle’ view of research data pose new challenges. Researchers face a bewildering landscape of data management requirements, recommendations and regulations, without necessarily being able to access data management training or possessing a clear understanding of practical approaches that can assist in data management in their particular research domain. Here we provide an overview of best practice data life cycle approaches for researchers in the life sciences/bioinformatics space with a particular focus on ‘omics’ datasets and computer-based data processing and analysis. We discuss the different stages of the data life cycle and provide practical suggestions for useful tools and resources to improve data management practices.

Complex Queries Optimization and Evaluation over Relational and NoSQL Data Stores in Cloud Environments

The production of huge amount of data and the emergence of cloud computing have introduced new requirements for data management. Many applications need to interact with several heterogeneous data stores depending on the type of data they have to manage: relational and NoSQL (i.e., document, graph, key-value, and column) data stores. Interacting with heterogeneous data models via different APIs and query languages imposes challenging tasks to the developers of multiple data stores applications. Indeed, the execution of complex queries over heterogeneous data models cannot, currently, be achieved in a declarative way as it is used to be with single data store application, and therefore requires extra implementation efforts. In this paper we propose a mediation based component to optimize and execute complex queries over multiple data stores in Cloud environments. This component is referred to as virtual data store (VDS). The key ingredients of our solution are (1) a simple global schema describing the different data sources and their relationships, (2) a cost model to evaluate the cost of the operations, (3) an inter data stores parallelism execution model, and (4) a dynamic programming based approach to generate optimal execution plan. Quantitative and qualitative experiments are conducted to validate our approach.

Comparing Data Management Technologies on FSDM Enabled RDBMS: A Literature Study

Relational Database Management Systems (RDBMS) have been used for more than 40 years in many industries to manage structured data. They have known as first, data later approach because users have to build a schema before store and query data. In the era of big data as data from various applications have become increasingly dynamic and contain structured and semi-structured data, they cannot be effectively modeled in the relational model. Therefore, Flexible Schema Data Management (FSDM) will be used to fulfill requirements of data management in the multi-structured data without defining the upfront schema. In this paper, we will compare, by conducting a literature review, several Data Management Technologies for FSDM which supporting structured and semi-structured data to meet the challenges and requirements of Flexible Schema Data (FSD). We found 5 technologies which extend the usability of RDBMS with Structured Query Language (SQL) by supporting FSDM with Not-only Structured Query Language (NoSQL).

The research data management platform (RDMP): A novel, process driven, open-source tool for the management of longitudinal cohorts of clinical data.

BackgroundThe Health Informatics Centre at the University of Dundee provides a service to securely host clinical datasets and extract relevant data for anonymized cohorts to researchers to enable them to answer key research questions. As is common in research using routine healthcare data, the service was historically delivered using ad-hoc processes resulting in the slow provision of data whose provenance was often hidden to the researchers using it. This paper describes the development and evaluation of the Research Data Management Platform (RDMP): an open source tool to load, manage, clean, and curate longitudinal healthcare data for research and provide reproducible and updateable datasets for defined cohorts to researchers.ResultsBetween 2013 and 2017, RDMP tool implementation tripled the productivity of data analysts producing data releases for researchers from 7.1 to 25.3 per month and reduced the error rate from 12.7% to 3.1%. The effort on data management reduced from a mean of 24.6 to 3.0 hours per data release. The waiting time for researchers to receive data after agreeing a specification reduced from approximately 6 months to less than 1 week. The software is scalable and currently manages 163 datasets. A total 1,321 data extracts for research have been produced, with the largest extract linking data from 70 different datasets.ConclusionsThe tools and processes that encompass the RDMP not only fulfil the research data management requirements of researchers but also support the seamless collaboration of data cleaning, data transformation, data summarization and data quality assessment activities by different research groups.

Heterogeneous fuzzy XML data integration based on structural and semantic similarities

Web data integration has become a crucial requirement for Web data management. A considerable number of approaches for integrating Extensible Markup Language (XML) data from heterogeneous data sources have been proposed. Yet these existing approaches are not fit for integration of fuzzy XML data because of their fuzzy characteristics. In this article, we provide a framework to deal with fuzzy XML document integration. Firstly, we propose a new fuzzy XML tree model. Secondly, we present an effective algorithm based on the tree edit distance to identify the structural and semantic similarities between the fuzzy documents represented in the proposed fuzzy XML tree model. Thirdly, we propose an integration strategy that is applied to integrate the fuzzy documents from different data sources. Finally, we conduct experiments to demonstrate that our approach can efficiently integrate fuzzy XML documents.

PENGEMBANGAN SISTEM PENGELOLAAN DATA KUALITAS AIR LIMBAH DOMESTIK PERKOTAAN

One of the activities undertaken within the framework of environmental sanitation management by the government is to publish a variety of regulations governing the wastewater disposal problem. Sources of pollution of the most dominant in urban areas are domestic wastewater. Government institutions through BPLHD have conducted regular monitoring of river water quality in Jakarta. Management of water quality data from domestic waste sources is needed to control river water pollution. This requires data management system of urban domestic waste water quality. In this article will be discussed the development of data management systems for domestic waste water quality with reference to the current applicable regulations. Key words : water quality database, domestic waste water quality

Big Data Management for Cloud-Enabled Geological Information Services

Cloud computing as a powerful technology of performing massive-scale and complex computing plays an important role in implementing geological information services. In the era of big data, data are being collected at an unprecedented scale. Therefore, to ensure successful data processing and analysis in cloud-enabled geological information services (CEGIS), we must address the challenging and time-demanding task of big data processing. This review starts by elaborating the system architecture and the requirements for big data management. This is followed by the analysis of the application requirements and technical challenges of big data management for CEGIS in China. This review also presents the application development opportunities and technical trends of big data management in CEGIS, including collection and preprocessing, storage and management, analysis and mining, parallel computing based cloud platform, and technology applications.

Cyber-Physical Manufacturing Metrology Model (CPM3) – Big Data Analytics Issue

Internet of Things (IoT) is changing the world, and therefore the application of ICT (Information and Communication Technology) in manufacturing. As a paradigm based on the Internet, IoT utilizes the benefits of interrelated technologies/smart devices such as RFID (Radio Frequency Identification) and WSAN (Wireless Sensor and Actuator Networks) for the retrieval and exchange of information thus opening up new possibilities for integration of manufacturing system and its cyber representation through Cyber-Physical Manufacturing (CPM) model. On the other hand, CPM and digital manufacturing represent the key elements for implementation of Industry 4.0 and backbone for “smart factory” generation. Interconnected smart devices generate huge databases (big data), so that Cloud computing becomes indispensable tool to support the CPM. In addition, CPM has an extremely expressed requirement for better control, monitoring and data management. Limitations still exist in storages, networks and computers, as well as in the tools for complex data analysis, detection of its structure and retrieval of useful information.Products, resources, and processes within smart factory are realized and controlled through CPM model. In this context, our recent research efforts in the field of quality control and manufacturing metrology are directed to the development of framework for Cyber-Physical Manufacturing Metrology Model (CPM3). CPM3 framework will be based on: 1) integration of digital product metrology information obtained from big data using BDA (big data analytics) through metrology features recognition, and 2) generation of global/local inspection plan for CMM (Coordinate Measuring Machine) from extracted information. This paper will present recent results of our research on CPM3 – big data analytics issue.

Complex Virtual Instrument Model for Ocean Sensor Networks

Virtual instrument (VI) technology is widely used, because it is more cost effective to perform data processing with computer software than with an additional hardware. Traditional VI developments often suffer from low efficiency due to the complex programming and poor interoperability. In this paper, we analyze the data management and analysis requirements of the ocean sensor networks (OSNs), identify the common characteristics of different systems, and propose complex VI model for OSNs (CVI-OSNs). It consists of an object description model to interpret the observation data, a VI model to enable comprehensive analysis of observation data, and a parameter mapping model to associate observed variables and VIs. The presented design method of CVI-OSN systems improves the development efficiency significantly and enables system interoperability. On this basis, a CVI-OSN composition model is proposed to facilitate data integration among different systems for large-scale research of ocean environment variation. The proposed model has been fully applied in the IEEE standard 2402. Moreover, we have conducted practical experiments to evaluate its performance. The result shows that our proposed model is feasible and efficient.

Design of Data Package Software System for Aerospace Product Quality and Reliability

The quality and reliability data package management of space products is one of the important contents of the fine quality management of the space model, and is also a major basic work throughout the model development process. In order to meet the needs of enterprises, this paper proposes a design of quality and reliability data packet software system, introducing quality intelligence and open architecture. As an exploration of the intelligent direction of quality and reliability data package management platform, the system has launched a new quality management model to meet the quality data management and analysis requirements of the enterprise, and provided intelligent data support for the enterprise.