Data Management Technology Research Articles

How to manage massive spatiotemporal dataset from stationary and non-stationary sensors in commercial DBMS?

AbstractThe growing diffusion of the latest information and communication technologies in different contexts allowed the constitution of enormous sensing networks that form the underlying texture of smart environments. The amount and the speed at which these environments produce and consume data are starting to challenge current spatial data management technologies. In this work, we report on our experience handling real-world spatiotemporal datasets: a stationary dataset referring to the parking monitoring system and a non-stationary dataset referring to a train-mounted railway monitoring system. In particular, we present the results of an empirical comparison of the retrieval performances achieved by three different off-the-shelf settings to manage spatiotemporal data, namely the well-established combination of PostgreSQL + PostGIS with standard indexing, a clustered version of the same setup, and then a combination of the basic setup with Timescale, a storage extension specialized in handling temporal data. Since the non-stationary dataset has put much pressure on the configurations above, we furtherly investigated the advantages achievable by combining the TSMS setup with state-of-the-art indexing techniques. Results showed that the standard indexing is by far outperformed by the other solutions, which have different trade-offs. This experience may help researchers and practitioners facing similar problems managing these types of data.

PACMMOD Volume 1, Issue 3: Editorial

We are excited to introduce this new issue of PACMMOD (Proceedings of the ACM on Management of Data). PACMMOD is a new journal, concerned with the principles, algorithms, techniques, systems, and applications of database management systems, data management technology, and science and engineering of data. It includes articles reporting cutting-edge data management, data engineering, and data science research. Articles published at PACMMOD address data challenges at various stages of the data lifecycle, from modeling, acquisition, cleaning, integration, indexing, querying, analysis, exploration, visualization, interpretation, and explanation. They focus on dataintensive components of data pipelines, and solve problems in areas of interest to our community (e.g., data curation, optimization, performance, storage, systems), operating within accuracy, privacy, fairness, and diversity constraints. Articles reporting deployed systems and solutions to data science pipelines and/or fundamental experiences and insights from evaluating real-world data engineering problems are especially encouraged.

IoT empowered smart cybersecurity framework for intrusion detection in internet of drones

The emergence of drone-based innovative cyber security solutions integrated with the Internet of Things (IoT) has revolutionized navigational technologies with robust data communication services across multiple platforms. This advancement leverages machine learning and deep learning methods for future progress. In recent years, there has been a significant increase in the utilization of IoT-enabled drone data management technology. Industries ranging from industrial applications to agricultural advancements, as well as the implementation of smart cities for intelligent and efficient monitoring. However, these latest trends and drone-enabled IoT technology developments have also opened doors to malicious exploitation of existing IoT infrastructures. This raises concerns regarding the vulnerability of drone networks and security risks due to inherent design flaws and the lack of cybersecurity solutions and standards. The main objective of this study is to examine the latest privacy and security challenges impacting the network of drones (NoD). The research underscores the significance of establishing a secure and fortified drone network to mitigate interception and intrusion risks. The proposed system effectively detects cyber-attacks in drone networks by leveraging deep learning and machine learning techniques. Furthermore, the model's performance was evaluated using well-known drones’ CICIDS2017, and KDDCup 99 datasets. We have tested the multiple hyperparameter parameters for optimal performance and classify data instances and maximum efficacy in the NoD framework. The model achieved exceptional efficiency and robustness in NoD, specifically while applying B-LSTM and LSTM. The system attains precision values of 89.10% and 90.16%, accuracy rates up to 91.00–91.36%, recall values of 81.13% and 90.11%, and F-measure values of 88.11% and 90.19% for the respective evaluation metrics.

Living without Beth and Craig: Definitions and Interpolants in Description and Modal Logics with Nominals and Role Inclusions

The Craig interpolation property (CIP) states that an interpolant for an implication exists iff it is valid. The projective Beth definability property (PBDP) states that an explicit definition exists iff a formula stating implicit definability is valid. Thus, the CIP and PBDP reduce potentially hard existence problems to entailment in the underlying logic. Description (and modal) logics with nominals and/or role inclusions do not enjoy the CIP nor the PBDP, but interpolants and explicit definitions have many applications, in particular in concept learning, ontology engineering, and ontology-based data management. In this article, we show that, even without Beth and Craig, the existence of interpolants and explicit definitions is decidable in description logics with nominals and/or role inclusions such as 𝒜ℒ𝒞𝒪, 𝒜ℒ𝒞ℋ, and 𝒜ℒ𝒞ℋ𝒪ℐ and corresponding hybrid modal logics. However, living without Beth and Craig makes these problems harder than entailment: the existence problems become 2ExpTime -complete in the presence of an ontology or the universal modality, and coNExpTime -complete otherwise. We also analyze explicit definition existence if all symbols (except the one that is defined) are admitted in the definition. In this case, the complexity depends on whether one considers individual or concept names. Finally, we consider the problem of computing interpolants and explicit definitions if they exist and turn the complexity upper bound proof into an algorithm computing them, at least for description logics with role inclusions.

Machine Learning Applications in Sarcopenia Detection and Management: A Comprehensive Survey.

This extensive review examines sarcopenia, a condition characterized by a loss of muscle mass, stamina, and physical performance, with a particular emphasis on its detection and management using contemporary technologies. It highlights the lack of global agreement or standardization regarding the definition of sarcopenia and the various techniques used to measure muscle mass, stamina, and physical performance. The distinctive criteria employed by the European Working Group on Sarcopenia in Older People (EWGSOP) and the Asian Working Group for Sarcopenia (AWGSOP) for diagnosing sarcopenia are examined, emphasizing potential obstacles in comparing research results across studies. The paper delves into the use of machine learning techniques in sarcopenia detection and diagnosis, noting challenges such as data accessibility, data imbalance, and feature selection. It suggests that wearable devices, like activity trackers and smartwatches, could offer valuable insights into sarcopenia progression and aid individuals in monitoring and managing their condition. Additionally, the paper investigates the potential of blockchain technology and edge computing in healthcare data storage, discussing models and systems that leverage these technologies to secure patient data privacy and enhance personal health information management. However, it acknowledges the limitations of these models and systems, including inefficiencies in handling large volumes of medical data and the lack of dynamic selection capability. In conclusion, the paper provides a comprehensive summary of current sarcopenia research, emphasizing the potential of modern technologies in enhancing the detection and management of the condition while also highlighting the need for further research to address challenges in standardization, data management, and effective technology use.

Using the community-based breeding program (CBBP) model as a collaborative platform to develop the African Goat Improvement Network-Image collection protocol (AGIN-ICP) with mobile technology for data collection and management of livestock phenotypes.

Introduction: The African Goat Improvement Network Image Collection Protocol (AGIN-ICP) is an accessible, easy to use, low-cost procedure to collect phenotypic data via digital images. The AGIN-ICP collects images to extract several phenotype measures including health status indicators (anemia status, age, and weight), body measurements, shapes, and coat color and pattern, from digital images taken with standard digital cameras or mobile devices. This strategy is to quickly survey, record, assess, analyze, and store these data for use in a wide variety of production and sampling conditions. Methods: The work was accomplished as part of the multinational African Goat Improvement Network (AGIN) collaborative and is presented here as a case study in the AGIN collaboration model and working directly with community-based breeding programs (CBBP). It was iteratively developed and tested over 3years, in 12 countries with over 12,000 images taken. Results and discussion: The AGIN-ICP development is described, and field implementation and the quality of the resulting images for use in image analysis and phenotypic data extraction are iteratively assessed. Digital body measures were validated using the PreciseEdge Image Segmentation Algorithm (PE-ISA) and software showing strong manual to digital body measure Pearson correlation coefficients of height, length, and girth measures (0.931, 0.943, 0.893) respectively. It is critical to note that while none of the very detailed tasks in the AGIN-ICP described here is difficult, every single one of them is even easier to accidentally omit, and the impact of such a mistake could render a sample image, a sampling day's images, or even an entire sampling trip's images difficult or unusable for extracting digital phenotypes. Coupled with tissue sampling and genomic testing, it may be useful in the effort to identify and conserve important animal genetic resources and in CBBP genetic improvement programs by providing reliably measured phenotypes with modest cost. Potential users include farmers, animal husbandry officials, veterinarians, regional government or other public health officials, researchers, and others. Based on these results, a final AGIN-ICP is presented, optimizing the costs, ease, and speed of field implementation of the collection method without compromising the quality of the image data collection.

Enhancing Security and Privacy of Patient Data in Healthcare: A SmartPLS Analysis of Blockchain Technology Implementation

Security and privacy of patient data are critical concerns in the healthcare system. This research aims to investigate the impact of implementing blockchain technology on enhancing the security and privacy of patient data in healthcare. Using the SmartPLS analysis method, this study empirically examines the relationships between blockchain technology implementation and data security and data privacy within the healthcare system. The research sample consists of healthcare organizations that have implemented blockchain technology for data management. Data is collected through surveys and analyzed using SmartPLS to assess the effects of blockchain technology on data security and privacy. The findings reveal the positive influence of blockchain technology implementation on enhancing the security and privacy of patient data. The study also identifies challenges, such as scalability and interoperability, that need to be addressed for successful implementation. This research contributes to the existing literature by providing empirical evidence on the benefits and challenges of implementing blockchain technology to safeguard patient data in healthcare systems.

Performance Analysis of Reputation based Proof of Credibility Consensus Mechanism for Blockchain based Applications

Blockchain is a decentralized transaction and data management technology first developed for the Bitcoin cryptocurrency. Blockchain technology is gaining popularity due to its core attributes which provides security, anonymity and data integrity without any involvement of third party. Consensus mechanism is a procedure by which all peers in the blockchain network agrees to a common agreement on the current state of the distributed ledger. It plays vital role in increasing efficiency of any blockchain environment. Though we have many consensus mechanisms working currently in different areas but they still lack in parameters like status of validators, latency, node failure etc. In Our proposed algorithm Proof of credibility, we have tried to incorporate all above factors in it. We have also implemented two or more factors of proposed algorithm and have evaluated and compared with existing consensus algorithm. In future research we aim to implement RPoC in any blockchain network and then we will evaluate it in terms of different evaluation parameters such as performance, security, scalability.

Development of intelligent system operational maintenance of the level of oil and gas production and waterflooding management

This article discusses the development of an intelligent System for the operational maintenance of the level of oil and gas production as part of the implementation of the Strategy for the development of information technologies for data management and the Program for the development of digitalization of fields of JSC «NC Kazmunaigas». The advantage of the system is multitasking and using almost all the data coming from production facilities in real time. The main task of the system is to manage a group of wells taking into account their mutual influence to maximize oil production and reduce the negative impact of uncoordinated well operation without damaging the rational system of field development. A significant feature of the developed system is the creation of complex algorithms for predicting the main development indicators using artificial neural networks based on a combination of CRM (capacity resistance model), FFNN (neural network with direct communication), MBM (material balance model) and BFGS (Broyden-Fletcher-Goldfarb-Shanno algorithm) methods. During the pilot test on 10 wells, the modes were adjusted according to the recommendations issued by the system and the system confirmed its operability and effectiveness of application. Keywords: virtual flow meter; labor productivity; return distribution; machine learning; rational development system; neural networks.

Precision Healthcare Meets DevOps: Secure Data Science Pipelines for Scalable Machine Learning

This paper explores precision healthcare and advanced data management technologies. It focuses on integrating machine learning (ML), DevOps, and DevSecOps to create scalable, secure, and efficient healthcare data pipelines. Precision healthcare uses ML to analyze large datasets. It provides personalized treatment plans and predictive diagnostics, which improve patient outcomes. The huge amount of healthcare data challenges scalability, security, and compliance.

A dataset of water, heat, and carbon fluxes over the winter wheat-summer maize croplands in Luancheng during 2013–2017

The winter wheat-summer maize double-cropping agro-ecosystem holds the largest planting area in the North China Plain. The carbon and water flux observation data of the croplands are crucial to understanding regional carbon and water cycles and their regulation mechanism. Located in the piedmont Plain of Taihang Mountain, Luancheng Agro-ecosystem Experimental Station (Luancheng Station) is the typical groundwater irrigation district. We have been using the eddy covariance technology to carry out the observation work of the winter-summer maize croplands since October 2007. This is the dataset of water, heat, and carbon fluxes over the winter wheat-summer maize croplands in Luancheng from October 2013 to September 2017 (covering four complete growing seasons). The dataset includes latent heat flux (or evapotranspiration), sensible heat flux, net ecosystem exchange of CO2, solar radiation, photosynthetically active radiation, air temperature/humidity, precipitation, soil temperature and volumetric soil water content at the half-hourly, daily, crop-season, and crop-year scales. The network observation, quality control and storage process of this dataset strictly abide by ChinaFLUX data management technology system to ensure data reliability. This dataset can provide solid data support for deeply understanding the processes of water, heat and carbon exchange, and improving the management of water resource to relieve the overdraft of groundwater in North China.

Big data optimisation and management in supply chain management: a systematic literature review

The increasing interest from technology enthusiasts and organisational practitioners in big data applications in the supply chain has encouraged us to review recent research development. This paper proposes a systematic literature review to explore the available peer-reviewed literature on how big data is widely optimised and managed within the supply chain management context. Although big data applications in supply chain management appear to be often studied and reported in the literature, different angles of big data optimisation and management technologies in the supply chain are not clearly identified. This paper adopts the explanatory literature review involving bibliometric analysis as the primary research method to answer two research questions, namely: (1) How to optimise big data in supply chain management? and (2) What tools are most used to manage big data in supply chain management? A total of thirty-seven related papers are reviewed to answer the two research questions using the content analysis method. The paper also reveals some research gaps that lead to prospective future research directions.

“Who Should I Trust with My Data?” Ethical and Legal Challenges for Innovation in New Decentralized Data Management Technologies

News about personal data breaches or data abusive practices, such as Cambridge Analytica, has questioned the trustworthiness of certain actors in the control of personal data. Innovations in the field of personal information management systems to address this issue have regained traction in recent years, also coinciding with the emergence of new decentralized technologies. However, only with ethically and legally responsible developments will the mistakes of the past be avoided. This contribution explores how current data management schemes are insufficient to adequately safeguard data subjects, and in particular, it focuses on making these data flows transparent to provide an adequate level of accountability. To showcase this, and with the goal of enhancing transparency to foster trust, this paper investigates solutions for standardizing machine-readable policies to express personal data processing activities and their application to decentralized personal data stores as an example of ethical, legal, and technical responsible innovation in this field.

Application of «Big Data» technologies for storage, processing and analysis of data in the process of army management

In today's world, the increase in the volume and variety of data used in the process of military management leads to an increase in the requirements for their storage and processing. To effectively manage military operations, it is necessary to have information about various aspects of military activity, such as the location of military units, the availability of resources, the assessment of potential threats, and much more. Modern military operations require the processing and analysis of a large amount of data generated in real time. Data collected from sensors, satellites, probes, sensors, and other sources can be vast in volume and diverse in structure. The increase in the volume and heterogeneity of data used in the process of military management leads to an increase in the requirements for their storage and processing. This presents military organizations with the task of creating effective and reliable data storage and processing systems. The article considers the possibilities of using «Big Data» technology, which allow collecting, storing, processing and analyzing large volumes of various data in real time. The use of modern data management technologies can help increase the efficiency of military activities and ensure fast and accurate decision-making at various levels of management. This can be achieved through the use of artificial intelligence, machine learning and data analysis technologies that can help military structures perform calculations and forecasts, optimize the use of resources and increase the efficiency of solving various tasks. Also, an important component of the effective use of «Big Data» technologies in the military sphere is ensuring the protection of data from unauthorized access, as well as ensuring the reliability and stability of data storage and processing systems in the face of possible cyberattacks. The article analyzes the Hadoop system, which can be used to store, process and analyze «Big Data» in the military sphere.

Distributed Machine Learning Architecture for Security Improvement in Computer Drafting and Writing in Art Asset Identification System

Art asset identification service is becoming increasingly important in the art market, where the value of art assets is constantly changing. The service provides authentication, evaluation, and provenance research for artworks, which helps art collectors and institutions to protect their investments and ensure the authenticity of their collections. The effective management of big data is critical for the art asset identification service, and there are several big data management technologies that can be achieved. To improve security in the big data Management model uses Distributed Associative Rule Mining is implemented with Hashing based Symmetric Key Cryptography. The designed model comprises of Associate Rule Hashing Symmetric Key (ARHSK). The proposed ARHSK model comprises the symmetric key generated with the hashing model to secure art assets. With the ARHSK information is stored and processed for security features. The performance of the ARHSK model is implemented with the machine learning model for classification. Simulation analysis expressed that ARHSK exhibits an improved classification accuracy of 99.67% which is ~13% higher than the CNN and ANN models.

PACMMOD V1 N1 Editorial - Welcome to PACMMOD

We are excited to welcome you to Volume 1 of a brand new journal - Proceedings of the ACM on Management of Data, PACMMOD. This new journal is concerned with the principles, algorithms, techniques, systems, and applications of database management systems, data management technology, and science and engineering of data and will include articles reporting cutting-edge data management, data engineering, and data science research. Articles published at PACMMOD address data challenges at various stages of the data lifecycle, from modeling, acquisition, cleaning, integration, indexing, querying, analysis, exploration, visualization, interpretation, and explanation. They focus on data-intensive components of data pipelines; and solve problems in areas of interest to our community (e.g., data curation, optimization, performance, storage, systems), operating within accuracy, privacy, fairness, and diversity constraints. Articles reporting deployed systems and solutions to data science pipelines and/or fundamental experiences and insights from evaluating real-world data engineering problems are especially encouraged.

Knowledge Management as a New Strategy of Innovative Development

Theoretical framework: The relevance of studying the growing role of knowledge management in the context of ensuring the innovative activity of enterprises is increasing. The results demonstrate a number of important conclusions that have made it possible to expand the existing theoretical and practical aspects of the knowledge management concept. Design/methodology/approach: The conducted study in service companies has revealed a low culture of using this concept in the Ukrainian context. In the course of the research, it has been established similar and different features of the influence of knowledge management on innovation of service companies, direct and indirect relationship between variables, complex structure and indirect impact of production, integration, application of information on the speed, quality and number of innovations in enterprises. Findings: The complex structure of the relationship between knowledge and innovative activity can be a consequence of insufficient level of personnel competencies, available information and data management technologies, methods and practices of production, integration and application. Therefore, the low innovativeness of the service companies under study in the context of system orientation and technological capabilities of the companies, in particular, in the practice of knowledge management, and the low impact of knowledge and information on the innovation of services and processes on the market of Ukraine have been revealed. Research, Practical & Social implications: This is precisely why the strategy of personalization significantly prevails in its use in knowledge management compared to the codification strategy, which is manifested in the limited practice of using systems and technologies for the generation, accumulation, storage, use, dissemination of knowledge. Originality/value: In addition, the use of the personalization strategy is explained by the fact that the service companies under consideration focus more on processes, the optimization of which significantly affects interaction with customers and the efficiency of operations.

Master Data Management Disruptive Modern Architecture

Master Data Management concept came into the mainstream during the mid 2000s, to generate a single Golden profile of the customer. As the Enterprise IT architecture started adopting and integrating various Commercial Products for CRM, ERP, Finance, HR, Supply Change Management etc. along with other homegrown custom apps, the information around the Customer started drifting across the various applications which resulted in various inefficiencies in day-to-day business operation. Master Data Management tools and technologies provide a way to perform identity resolution and survive latest and greatest information. Traditionally, the MDM tools always adopt a lean approach that uses minimal attributions to identify the smallest amount of (master) data with the biggest influence on business outcomes like Name, Phone, SSN, Email and Address. These constitute only less than 1% of the enterprise data, In the age of generative AI, there is a greater need to understand the context and the relationship of ALL data. This paper explores an alternate approach to mitigate the shortcomings of the currently available MDM tools

A Bibliometric Review of the Evolution of Blockchain Technologies.

Bitcoin was created in 2008 as the first decentralized cryptocurrency, providing an innovative data management technology, which was later named blockchain. It ensured data validation without intervention from intermediaries. During its early stages, it was conceived as a financial technology by most researchers. It was not until 2015, when the Ethereum cryptocurrency was officially launched worldwide, along with its revolutionary technology called smart contracts, that researchers began to change their perception of the technology and look for uses outside the financial world. This paper analyzes the literature since 2016, one year after Ethereum, analyzing the evolution of interest in the technology to date. For this purpose, a total of 56,864 documents created between 2016 and 2022 from four major publishers were analyzed, providing answers to the following questions. Q1: How has interest in blockchain technology increased? Q2: What have been the major blockchain research interests? Q3: What have been the most outstanding works of the scientific community? The paper clearly exposes the evolution of blockchain technology, making it clear that, as the years go by, it is becoming a complementary technology instead of the main focus of studies. Finally, we highlight the most popular and recurrent topics discussed in the literature over the analyzed period of time.

CH-Bench: a user-oriented benchmark for systems for efficient distant reading (design, performance, and insights)

Data science deals with the discovery of information from large volumes of data. The data studied by scientists in the humanities include large textual corpora. An important objective is to study the ideas and expectations of a society regarding specific concepts, like “freedom” or “democracy,” both for today’s society and even more for societies of the past. Studying the meaning of words using large corpora requires efficient systems for text analysis, so-called distant reading systems. Making such systems efficient calls for a specification of the necessary functionality and clear expectations regarding typical work loads. But this currently is unclear, and there is no benchmark to evaluate distant reading systems. In this article, we propose such a benchmark, with the following innovations: As a first step, we collect and structure various information needs of the target users. We then formalize the notion of word context to facilitate the analysis of specific concepts. Using this notion, we formulate queries in line with the information needs of users. Finally, based on this, we propose concrete benchmark queries. To demonstrate the benefit of our benchmark, we conduct an evaluation, with two objectives. First, we aim at insights regarding the content of different corpora, i.e., whether and how their size and nature (e.g., popular and broad literature or specific expert literature) affect results. Second, we benchmark different data management technologies. This has allowed us to identify performance bottlenecks.