Data Management Framework Research Articles

Big Data and precision agriculture: a novel spatio-temporal semantic IoT data management framework for improved interoperability

Precision agriculture in the realm of the Internet of Things is characterized by the collection of data from multiple sensors deployed on the farm. These data present a spatial, temporal, and semantic characterization, which further complicates the performance in the management and implementation of models and repositories. In turn, the lack of standards is reflected in insufficient interoperability between management solutions and other non-native services in the framework. In this paper, an innovative system for spatio-temporal semantic data management is proposed. It includes a data query system that allows farmers and users to solve queries daily, as well as feed decision-making, monitoring, and task automation solutions. In the proposal, a solution is provided to ensure service interoperability and is validated against two European smart farming platforms, namely AFarCloud and DEMETER. For the evaluation and validation of the proposed framework, a neural network is implemented, fed through STSDaMaS for training and validation, to provide accurate forecasts for the harvest and baling of forage legume crops for livestock feeding. As a result of the evaluation for the training and execution of neural networks, high performance on complex spatio-temporal semantic queries is exposed. The paper concludes with a distributed framework for managing complex spatio-temporal semantic data by offering service interoperability through data integration to external agricultural data models.Graphical

A Data Management Framework for Nurses Using E-Health as a Service (eHaaS)

The electronic health record (EHR) is a patient care database, which helps doctors or nurses to analyse comprehensive patient healthcare through health-cart (h-cart) assistance. Electronic health (e-Health) services offer efficient sharing of the patient's information based on geo-location in which nurses, doctors, or health care practitioners access the patients, promptly and without time delay in case of emergency. In e-Health services, nurses are considered as the data holder who can store and maintain patient's health records in the cloud h-cart platform to analyses patient's data effectively. Therefore, nurses need to safely share and manage access to data in the healthcare system; this need required prominent solutions. However, data authenticity and response time are considered as challenging characteristics in the e-health care system. Hence, in this paper, an improved e-health service model (IeHSM) has been proposed based on cloud computing technology to improve the data authenticity, reliability, and accessibility time of the healthcare information.

Evolution of research data management in academic libraries: A review of the literature

This study provides insights into the evolution and conceptual framework of research data management (RDM). It also investigates the role of libraries and librarians in offering data management services and the challenges they face in this regard. The study is qualitative in nature and based on an extensive literature review survey. The analysis of the reviewed literature reveals that the idea of RDM has emerged as a new addition to library research support services. The more recent literature clearly established the pivotal role of libraries and librarians in developing and managing RDM services. However, data sharing practices and the development of RDM services in libraries are more prevalent in developed countries. While these trends are still lacking among researchers and libraries in developing countries. Creating awareness among researchers about the benefits of data sharing is a challenging task for libraries. Furthermore, institutional commitment, collaboration, academic engagement, technological infrastructure development, lack of policies, funding, and storage, skills, and competencies required for librarians to offer RDM-based services are some of the other significant challenges highlighted in the literature. Certainly, RDM services are difficult and complicated; therefore, librarians need to master the skills of research data to offer library-based RDM services.

Data sharing governance in sub-Saharan Africa during public health emergencies: Gaps and guidance.

While the COVID-19 pandemic has captured the attention of the global community since the end of 2019, deadly health pandemics are not new to Africa. Tuberculosis (TB), malaria and human immunodeficiency virus (HIV) count amongst other serious diseases that have had a catastrophic impact on the African continent. Effective responses to such pandemics require high-quality, comprehensive data sets that can inform policymaking and enhance healthcare decision-making. While data is driving the information economy in the 21st century, the scarcity in Africa of carefully curated, large epidemiologic data sources and analytical capacity to rapidly identify and understand emerging infectious diseases poses a major challenge to mounting a time-sensitive response to unfolding pandemics. Data access, sharing and transfer between countries are crucial to effectively managing current and future health pandemics. Data access and sharing, however, raises questions about personal privacy, the adequacy of governance mechanisms to regulate cross-border data flows, and ethical issues relating to the collection and use of personal data in the interests of public health. Sub-Saharan Africa's most research-intensive countries are characterised by diverse data management and privacy governance frameworks. Such regional variance can impede time-sensitive data sharing and highlights the need for urgent governance reforms to facilitate effective decision-making in response to rapidly evolving public health threats.

Machine Learning Analytic-Based Two-Staged Data Management Framework for Internet of Things.

In applications of the Internet of Things (IoT), where many devices are connected for a specific purpose, data is continuously collected, communicated, processed, and stored between the nodes. However, all connected nodes have strict constraints, such as battery usage, communication throughput, processing power, processing business, and storage limitations. The high number of constraints and nodes makes the standard methods to regulate them useless. Hence, using machine learning approaches to manage them better is attractive. In this study, a new framework for data management of IoT applications is designed and implemented. The framework is called MLADCF (Machine Learning Analytics-based Data Classification Framework). It is a two-stage framework that combines a regression model and a Hybrid Resource Constrained KNN (HRCKNN). It learns from the analytics of real scenarios of the IoT application. The description of the Framework parameters, the training procedure, and the application in real scenarios are detailed. MLADCF has shown proven efficiency by testing on four different datasets compared to existing approaches. Moreover, it reduced the global energy consumption of the network, leading to an extended battery life of the connected nodes.

Securing Fine-Grained Data Sharing and Erasure in Outsourced Storage Systems

The wide use of internet-connected services makes massive personal data collected by service providers without the need of our consent. Although the archived data may enable them to provide better service experiences for users, it also presents serious risks to individual privacy, especially when active or unexpected data breaches have become commonplace. To mitigate this issue, several acts and regulations (e.g., the European Union general data protection regulation) have been issued and specified a lot of security requirements for personal data management. Among these various requirements, we mainly focus on the requirement of giving back the access control of personal data to data owners themselves and the right to be forgotten for data erasure. In this article, we provide a cryptographic solution of achieving these two requirements in the setting of outsourced storage. Specifically, we introduce a personal data management framework built upon a novel cryptographic primitive dubbed as forward-secure attribute-based puncturable encryption (FS-DABPE). This primitive simultaneously features of system-wide forward secrecy and practical key management as well as fine-grained access control of the encrypted personal data. Consequently, by locally puncturing, updating and erasing system-wide secret keys, it securely realizes fine-grained personal data sharing and data erasure without interactions. Furthermore, to instantiate the proposed framework, we present a concrete FS-DABPE construction, and prove its security under a well-studied complexity assumption. In addition, we provide a prototype implementation of the concrete construction, and present extensive experimental results that illustrate its feasibility and practicability.

An optimized device‐to‐device (D2D) blockchain network for the insurance industry

SummaryBlockchain, a popular technology, remains the decentralized data management framework approved for use by many industries. The application to the insurance industry needs to offer mobility using the wireless network. The wireless network has many limitations to overcome. This paper focuses on such problems and introduces three levels of a solution to the problem. The first level is resolved using the edge computers as storage at the agencies and the partners. The second level of economic operation is solved by introducing a D2D network solution. The third level of high transactions over the network is considered using a two‐stage optimization method. The introduced optimization algorithms are simulated, and results are compared with a classical step‐by‐step calculation method that is not feasible under real‐time application. The optimization methods successfully determine the maximum channel rate with the interferences influencing the operation of such a system.

Data Management Framework for Risk Estimate of Electronic Boards in Drilling and Measurement Tools*

With computer science and technology development in today's world, many traditional industries, such as the oil and gas industry, are beginning to transform to digitalization. In this transformation process, many data-driven models are often necessary; e.g., a data-driven model, based on existing data, is used to estimate the risk associated with drilling tools. Before building this model, the preliminary work needs to assess how much data are available at this stage, what is the quality of the data, whether the existing data are suitable for building the model, and if not, what measures can be taken to improve the data quality. To answer these questions, this paper presents a data management framework that includes data preparation, data quality assessment, and data-based knowledge acquisition. An actual case study result demonstrates that the framework can answer these questions.

A study on the relevance of semantic features extracted using BERT-based language models for enhancing the performance of software defect classifiers

In the context of new research in the software defect prediction (SDP) task using pre-trained language models, the present study aims to analyze the relevance of semantic features extracted using BERT-based language models for the detection of defective source codes. RoBERTa and CodeBERT-MLM language models are used to generate source code embeddings that capture semantic and contextual features which help in solving language understanding tasks such as SDP. The learned representations are then fed to a neural network-based SDP classifier in order to decide which of the learned code embeddings are more informative in discriminating between faulty and non-faulty software entities. Extensive experiments are conducted in a cross-version SDP scenario for Apache Calcite, an open-source framework for data management. The evaluation results of the defect classifiers show a statistically significant improvement when the code representations are learned by the pre-trained models compared with the semantic representations provided by other natural language-based models, doc2vec and LSI.

Scalable Remote Cloud Data Center for Vessel Equipment Predictive Maintenance Service-as-a-Product (SaaP)

Predictive maintenance is gradually replacing conventional preventive maintenance, through an informed decision-making process for a fleet operator to proactively monitor the health status of the sea-based equipment and machinery. As a result, the fleet operator will only need to ask for on-time repair and maintenance services instead of periodic maintenance, which saves both time and cost significantly. While moving into this new maritime business model, which is also known as Service-as-a-Product (SaaP), both fleet operators and maintenance, repair, and overhaul (MRO) companies could also build a stronger collaboration in the maritime industries. With the advent of digital transformation towards Industry 4.0 (I4.0), the Industrial Internet-of-Things (IIoT) renders the massive collection of operational and machinery process data from the vessel equipment through sensorization, where these collected real-time information are useful for advanced analytics to predict equipment failure and to avoid unplanned downtime. Unlike other industries, there are a few challenges in the maritime industries when developing a centralized smart vessel equipment monitoring platform. One of the key challenges, is the lack of a feasible data management system, requires the centralized host must be able to handle and manage the telemetry IoT data that is transmitted over the satellite communication, by complying with all cybersecurity considerations and regulations in maritime sector. In this paper, we present a scalable, hybrid cloud-based data management framework that can connect to multiple edge systems where each system is deployed on a physical vessel, to establish a scalable SaaP business model. Hence, the ship-to-shore sensorized data can be processed and monitored via a real-time visualization dashboard through the centralized cloud-based platform. We argue that our approach, would be a turn-key solution that can be implemented for mostly all types of marine equipment and machinery, and thus to further improve the prediction tools to support advanced decision-making techniques, such as optimal time repair of vessel equipment.

Cloud Mining Pool Aided Blockchain-Enabled Internet of Things: An Evolutionary Game Approach

The past few years have witnessed an exponential growth of diverse Internet of Things (IoT) devices as well as compelling applications ranging from industrial production to medical care. Dramatic advances in IoT technology not only brought enormous economic opportunities but also challenges (e.g., privacy and security vulnerabilities). Recently, with the appearance of blockchain technology, the integration of IoT and blockchain (BCoT) is considered a promising solution to address these issues. Blockchain provides a secure and scalable data management framework for IoT devices. However, the huge computation and energy cost of the consensus process in blockchain prevents it from being directly applied as a generic platform. To overcome this challenge, in this article, we propose a cloud mining pool-aided BCoT architecture, where the IoT devices can rent the computing resources from the cloud mining pools to offload the mining process. Based on this architecture, we study the mining pool selection problem and analyze the colony behaviors of IoT devices with different pooling strategies. We propose a centralized evolutionary game-based pool selection algorithm for the sake of maximizing the system utility. Considering the non-cooperative relationship among multiple miners, we also propose a lightweight distributed reinforcement learning algorithm, named the ‘WoLF-PHC’ algorithm.

Proof of Activity Protocol for IoMT Data Security

The Internet of Medical Things (IoMT) is an online device that senses and transmits medical data from users to physicians within a time interval. In, recent years, IoMT has rapidly grown in the medical field to provide healthcare services without physical appearance. With the use of sensors, IoMT applications are used in healthcare management. In such applications, one of the most important factors is data security, given that its transmission over the network may cause obtrusion. For data security in IoMT systems, blockchain is used due to its numerous blocks for secure data storage. In this study, Blockchain-assisted secure data management framework (BSDMF) and Proof of Activity (PoA) protocol using malicious code detection algorithm is used in the proposed data security for the healthcare system. The main aim is to enhance the data security over the networks. The PoA protocol enhances high security of data from the literature review. By replacing the malicious node from the block, the PoA can provide high security for medical data in the blockchain. Comparison with existing systems shows that the proposed simulation with BSD-Malicious code detection algorithm achieves higher accuracy ratio, precision ratio, security, and efficiency and less response time for Blockchain-enabled healthcare systems.

Privacy Data Management Mechanism Based on Blockchain and Federated Learning

Due to the extensive use of various intelligent terminals and the popularity of network social tools, a large amount of data in the field of medical emerged. How to manage these massive data safely and reliably has become an important challenge for the medical network community. This paper proposes a data management framework of medical network community based on Consortium Blockchain (CB) and Federated learning (FL), which realizes the data security sharing between medical institutions and research institutions. Under this framework, the data security sharing mechanism of medical network community based on smart contract and the data privacy protection mechanism based on FL and alliance chain are designed to ensure the security of data and the privacy of important data in medical network community, respectively. An intelligent contract system based on Keyed-Homomorphic Public Key (KH-PKE) Encryption scheme is designed, so that medical data can be saved in the CB in the form of ciphertext, and the automatic sharing of data is realized. Zero knowledge mechanism is used to ensure the correctness of shared data. Moreover, the zero-knowledge mechanism introduces the dynamic group signature mechanism of chosen ciphertext attack (CCA) anonymity, which makes the scheme more efficient in computing and communication cost. In the end of this paper, the performance of the scheme is analyzed from both asymptotic and practical aspects. Through experimental comparative analysis, the scheme proposed in this paper is more effective and feasible.

Barriers for data management as an enabler of circular economy: an exploratory study of the Norwegian AEC-industry

Effective data management can enable the utilisation of new tools and technologies and, ultimately the creation of circular business models in the building industry. To accomplish this, a targeted mapping and collection of data must take place. However, several challenges hinder the exchange of information in a seamless digital flow through the value chain and building life cycle. This exploratory study contributes to bridging the gap in the research, providing an overview of existing barriers related to mapping, collecting, and storing data about materials and products in existing buildings. The findings are obtained through 18 semi-structured interviews with experts working with circular economy and digitalisation in the Norwegian AEC-industry. Some of the identified barriers, such as the lack of data availability and interoperability, lack of competencies and unwillingness to share data, are strongly interrelated; a collaborative approach will be essential across the value chain. The research sets the basis for developing a framework for data management that can facilitate the reuse of materials and products from a building at the end of life to new construction or refurbishment projects. Ultimately, effective data management opens for developing and implementing innovative circular business models, enhancing strategic data-based asset management.

Knowledge graph construction and Internet of Things optimisation for power grid data knowledge extraction

Abstract Problems exist in power grid data management that have unclear relationships, weak security and low accuracy. By analysing the knowledge graph construction characteristics of smart grid data information and knowledge extraction, the grid data management platform is reshaped architecturally, and the knowledge graph construction technology is embedded in the grid data management framework. For the aforementioned problems, the knowledge graph construction and Internet of Things optimisation framework of power grid data knowledge extraction are proposed in this article. Firstly, the semantic search (KGSS) algorithm based on the knowledge graph is used. The KGSS algorithm extracts knowledge from structured, semi-structured and unstructured grid data through the massively parallel processing acquisition model and Hadoop database, and constructs knowledge entities, attributes and inter-entity relationships. Then, it optimises and predicts through the knowledge graph construction and Internet of Things optimisation framework extracted from power grid data knowledge. Finally, the experimental results show that the accuracy rate of the KGSS algorithm is 92%. The experimental results also show that it provides new ideas and research directions for power grid data under big data in the future.

AI-Assisted Dynamic Modelling for Data Management in a Distributed System

There are many interdependent computers available in distributed networks. In such schemes, overall ownership costs comprise facilities, such as computers, controls, etc.; buying hardware; and running expenses such as wages, electrical charges, etc. Strom use is a large part of operating expenses. AI-assisted dynamic modelling for data management (AI-DM) framework is proposed. The high percentage of power use is connected explicitly to inadequate planning of energy. This research suggests creating a multi-objective method to plan the preparation of multi-criteria software solutions for distributed systems using the fuzzy TOPSIS tool as a comprehensive guide to multi-criteria management. The execution results demonstrate that this strategy could then sacrifice requirements by weight.

An Overview of Data Warehouse and Data Lake in Modern Enterprise Data Management

Data is the lifeblood of any organization. In today’s world, organizations recognize the vital role of data in modern business intelligence systems for making meaningful decisions and staying competitive in the field. Efficient and optimal data analytics provides a competitive edge to its performance and services. Major organizations generate, collect and process vast amounts of data, falling under the category of big data. Managing and analyzing the sheer volume and variety of big data is a cumbersome process. At the same time, proper utilization of the vast collection of an organization’s information can generate meaningful insights into business tactics. In this regard, two of the popular data management systems in the area of big data analytics (i.e., data warehouse and data lake) act as platforms to accumulate the big data generated and used by organizations. Although seemingly similar, both of them differ in terms of their characteristics and applications. This article presents a detailed overview of the roles of data warehouses and data lakes in modern enterprise data management. We detail the definitions, characteristics and related works for the respective data management frameworks. Furthermore, we explain the architecture and design considerations of the current state of the art. Finally, we provide a perspective on the challenges and promising research directions for the future.

A Multi-Objective Clustering for Better Data Management in Connected Environment

Over the past decade, the rapid increase in connected devices has enabled the emergence of new digital ecosystems to provide new opportunities for monitoring and managing systems to optimize overall performance. With these connected environments, data collection and management become increasingly challenging. A significant number of works in the literature have addressed data collection and management based on different contexts (e.g., mobile ad hoc, Peer-2-Peer, and IoT networks). Today, a wired network uses all of these protocols simultaneously, thus highlighting the need to build a standard data collection and management framework that considers all potential user preferences. For this purpose, multi-objective clustering has been utilized as a promising solution to ensure the stability of connected devices during the collection and management of data. In this paper, we introduce a new multi-objective clustering (MOC) technique based on various criteria for cluster construction and head selection in connected environments. More precisely, the proposed solution is based hypergraphs to represent the connected environment and clusters according to similarities between heterogeneous devices. Then, a cross-sectional hypergraph algorithm is applied to select the cluster heads. Experiments conducted show that our solution outperforms the pioneering literature methods in terms of performance and effectiveness.

Advancing exposure data analytics and repositories as part of the European Exposure Science Strategy 2020–2030

High-quality and comprehensive exposure-related data are critical for different decision contexts, including environmental and human health monitoring, and chemicals risk assessment and management. However, exposure-related data are currently scattered, frequently of unclear quality and structure, not readily accessible, and stored in various—partly overlapping—data repositories, leading to inefficient and ineffective data usage in Europe and globally. We propose strategic guidance for an integrated European exposure data production and management framework for use in science and policy, building on current and future data analysis and digitalization trends. We map the existing exposure data landscape to requirements for data analytics and repositories across European policies and regulations. We further identify needs and ways forward for improving data generation, sharing, and usage, and translate identified needs into an operational action plan for European and global advancement of exposure data for policies and regulations. Identified key areas of action are to develop consistent exposure data standards and terminology for data production and reporting, increase data transparency and availability, enhance data storage and related infrastructure, boost automation in data management, increase data integration, and advance tools for innovative data analysis. Improving and streamlining exposure data generation and uptake into science and policy is crucial for the European Chemicals Strategy for Sustainability and European Digital Strategy, in line with EU Data policies on data management and interoperability.

Regional metal pollution risk assessment based on a big data framework: A case study of the eastern Tianshan mining area, China

With the increasing maturity of big data technology, its application in the field of environmental assessment has become an important issue. The mining of mineral resource can affect the balance in the ecological environment surrounding mining areas and the normal life activities of humans through groundwater. To solve this problem, big data-based methods were used to evaluate the pollution risk in mining areas in this article. Based on a spatial big data management framework, in this paper, high-risk areas in eastern Tianshan were quantitatively analyzed and delineated via a new comprehensive heavy metal pollution assessment system. A distributed storage environment, unstructured management method and spatial index coding were used to uniformly manage spatial data in vector and raster formats retrieved from different sources. A system involving 18 geological environment indicators was used to evaluate the risk level in the study area and in delineated areas with a high risk of heavy metal pollution. The results indicated that the proposed framework could efficiently store and process spatial data and realize environmental pollution risk assessment in a big data environment. In addition, some of the high-pollution risk areas identified based on the assessment results were consistent with actual mine locations. Overall, the results show that integrating multi-source geological characteristics through a comprehensive evaluation system based on big data can have a positive effect on improving the accuracy of heavy metal pollution risk assessment. Heavy metal pollution assessment can provide a reference for environmental monitoring and governance of the eastern Tianshan region and offer new solutions to large-scale heavy metal pollution assessment.