Traditional Relational Database Research Articles

Efficient Data Management Strategies for NoSQL and Relational Databases in Cloud Environments:An In-Depth Analysis

As data volumes grow exponentially, managing and storing this information efficiently becomes increasingly critical. NoSQL databases have emerged as a preferred solution for handling unstructured data, outperforming traditional relational database management systems (RDBMS) in scalability and flexibility. Techniques such as deduplication (removing duplicate data) and compression, combined with powerful tools like Hadoop and MongoDB, facilitate optimized storage and reduced network usage. However, the challenge of schema evolution in both NoSQL and SQL systems introduces risks of downtime and data inconsistency. This paper reviews current methods, highlights gaps in existing solutions, and proposes an integrated approach to ensure efficient data management across diverse database environments while minimizing downtime and enhancing data integrity.

Using Time-Series Databases for Energy Data Infrastructures

The management of energy market data, such as load, production, forecasts, and prices, is critical for energy market participants, who develop in-house energy data infrastructure services to aggregate data from many sources to support their business operations. Energy data management frequently involves time sensitive operations, including rapid data ingestion, real-time querying, filling in gaps from missing or delayed data, and updating large volumes of timestamped and loosely structured data, all of which demand high processing power. Traditional relational database management systems (RDBMSs) often struggle with these operations, whereas time series databases (TSDBs) appear to be a more efficient solution, providing enhanced scalability, reliability, real-time data availability and superior performance. This paper examines the advantages of TSDBs over RDBMS for energy data management, demonstrating that TSDBs can either replace or complement RDBMSs. We present quantitative improvements in digestion, integration, architecture, and performance, demonstrating that operations such as importing and querying time-series energy data, along with the overall system’s efficiency, can be significantly improved, achieving up to 100 times faster operations compared to relational databases, all without requiring extensive modifications to the existing information system’s architecture.

NoSQL Technologies in Big Data Ecosystems: a Comprehensive Review of Architectural Paradigms and Performance Metrics

The exponential growth of data in volume, velocity, and variety has challenged traditional relational database management systems, leading to the emergence and widespread adoption of NoSQL and other innovative database technologies. This article provides a comprehensive review of NoSQL databases and their role in modern Big Data infrastructure stacks. Through a systematic analysis of current literature and industry case studies, we explore the architectural paradigms, performance metrics, and use cases of various NoSQL database types, including document-based, key-value, column-family, and graph databases. Our findings indicate that NoSQL solutions offer significant advantages in scalability, flexibility, and real-time processing capabilities, particularly for unstructured and semi-structured data. However, challenges persist in areas such as data consistency, security, and interoperability with existing systems. We also examine emerging trends, including NewSQL, time-series databases, and the integration of artificial intelligence in database management. This article contributes to the understanding of how organizations can leverage NoSQL technologies to optimize their Big Data infrastructure, highlighting both the opportunities and considerations for implementation. Our conclusions underscore the transformative impact of NoSQL on data management practices and provide directions for future research in this rapidly evolving field.

GenAI-Assisted Database Deployment for Heterogeneous Indigenous–Native Ethnographic Research Data

In ethnographic research, data collected through surveys, interviews, or questionnaires in the fields of sociology and anthropology often appear in diverse forms and languages. Building a powerful database system to store and process such data, as well as making good and efficient queries, is very challenging. This paper extensively investigates modern database technology to find out what the best technologies to store these varied and heterogeneous datasets are. The study examines several database categories: traditional relational databases, the NoSQL family of key-value databases, graph databases, document databases, object-oriented databases and vector databases, crucial for the latest artificial intelligence solutions. The research proves that when it comes to field data, the NoSQL lineup is the most appropriate, especially document and graph databases. Simplicity and flexibility found in document databases and advanced ability to deal with complex queries and rich data relationships attainable with graph databases make these two types of NoSQL databases the ideal choice if a large amount of data has to be processed. Advancements in vector databases that embed custom metadata offer new possibilities for detailed analysis and retrieval. However, converting contents into vector data remains challenging, especially in regions with unique oral traditions and languages. Constructing such databases is labor-intensive and requires domain experts to define metadata and relationships, posing a significant burden for research teams with extensive data collections. To this end, this paper proposes using Generative AI (GenAI) to help in the data-transformation process, a recommendation that is supported by testing where GenAI has proven itself a strong supplement to document and graph databases. It also discusses two methods of vector database support that are currently viable, although each has drawbacks and benefits.

Integrating NoSQL, Hilbert Curve, and R*-Tree to Efficiently Manage Mobile LiDAR Point Cloud Data

The widespread use of Light Detection and Ranging (LiDAR) technology has led to a surge in three-dimensional point cloud data; although, it also poses challenges in terms of data storage and indexing. Efficient storage and management of LiDAR data are prerequisites for data processing and analysis for various LiDAR-based scientific applications. Traditional relational database management systems and centralized file storage struggle to meet the storage, scaling, and specific query requirements of massive point cloud data. However, NoSQL databases, known for their scalability, speed, and cost-effectiveness, provide a viable solution. In this study, a 3D point cloud indexing strategy for mobile LiDAR point cloud data that integrates Hilbert curves, R*-trees, and B+-trees was proposed to support MongoDB-based point cloud storage and querying from the following aspects: (1) partitioning the point cloud using an adaptive space partitioning strategy to improve the I/O efficiency and ensure data locality; (2) encoding partitions using Hilbert curves to construct global indices; (3) constructing local indexes (R*-trees) for each point cloud partition so that MongoDB can natively support indexing of point cloud data; and (4) a MongoDB-oriented storage structure design based on a hierarchical indexing structure. We evaluated the efficacy of chunked point cloud data storage with MongoDB for spatial querying and found that the proposed storage strategy provides higher data encoding, index construction and retrieval speeds, and more scalable storage structures to support efficient point cloud spatial query processing compared to many mainstream point cloud indexing strategies and database systems.

Analysis for an open-source library in database management systems

AbstractSpatial data management is crucial for applications like urban planning and environmental monitoring. While traditional relational databases are commonly used, they struggle with large and complex spatial data. NoSQL databases provide support for unstructured data and scalability. This article compares the performance and disk space usage of SQL Server (a relational database) and MongoDB (NoSQL database) using an open-source library. Experiments conducted with the OpenStreetMap dataset from Central America show that the MongoDB database outperformed the relational SQL Server database in most cases, offering practical advantages for spatial data management in Geographic Information System applications.

A learned cost model for big data query processing

The efficiency of query processing in the Spark SQL big data processing engine is significantly affected by execution plans and allocated resources. However, existing cost models for Spark SQL rely on hand-crafted rules. While learning-based cost models have been proposed for relational databases, they do not consider available resources. To address this issue, we propose a resource-aware deep learning model capable of automatically predicting query plan execution times based on historical data. To train our model, we embed query execution plans within a query plan tree and extracted features from allocated resources. An adaptive attention mechanism is integrated into the deep learning model to enhance prediction accuracy. Additionally, we extract sufficient features to represent data information and learn the effect of the data on query execution. This approach reduces the need for model retraining owing to data changes. The experimental results demonstrate that our deep cost model outperforms traditional rule-based methods and relational database learning-based optimizers in predicting query plan execution times.

Multidimensional visualization analysis based on large-scale GNSS data

Abstract With the deployment of global navigation systems such as GPS, GLONASS, GALILEO, and BeiDou, and the construction of the global observation network, global navigation satellite system (GNSS) data have greatly expanded. Traditional relational databases and file systems can no longer efficiently store and process GNSS data. In addition, multidimensional, in-depth visual analysis methods for such data are lacking. In this paper, we provide a solution and implement a GNSS data visualization system based on MapV and cesium running on the Hadoop platform, which can effectively solve the problems of the lack of storage and computing resources for these massive multisource heterogeneous data. The proposed system provides multidimensional presentations to show how the space and ground segments of the entire system are organized by analyzing GNSS data.

A Systematic Overview of Fuzzy Extended Entity Relationship (Fuzzy EER) Model Using Practical Examples

Most real-world data are vague and imprecise, so traditional relational databases cannot represent, integrate, and manage them effectively. In this research, we gave an overview of the application of fuzzy logic to the conceptual modeling facet of database design, which is an enhancement of the Extended Entity Relationship (EER) model. This is done to give a clearer picture of how imprecise and vague data are represented and managed, especially in multiple-criteria decision-making database systems. To achieve this, an overview of fuzzy attributes, fuzzy values in the attributes, notation for fuzzy representations, fuzzy degree in each value of an attribute, fuzzy degree in a set of values of different attributes, fuzzy degree with its own meaning and fuzzy degree to the model was carried out with practical examples.

A statistical characteristics preserving watermarking scheme for time series databases

Database watermarking is one of the most effective methods to protect the copyright of databases. However, traditional database watermarking has a potential drawback: watermark embedding will change the distribution of data, which may affect the use and analysis of databases. Considering that most analyses are based on the statistical characteristics of the target database, keeping the consistency of the statistical characteristics is the key to ensuring analyzability. Since statistical characteristics analysis is performed in groups, compared with traditional relational databases, time series databases (TSDBs) have obvious time-grouping characteristics and are more valuable for analysis. Therefore, this paper proposes a robust watermarking algorithm for time series databases, effectively ensuring the consistency of statistical characteristics. Based on the time-group characteristics of TSDBs, we propose a three-step watermarking method, which is based on linear regression, error compensation, and watermark verification, named RCV. According to the properties of the linear regression model and error compensation, the proposed watermark method generates a series of data that have the same statistical characteristics. Then, the verification mechanism is performed to validate the generated data until it conveys the target watermark message. Compared with the existing methods, our method achieves superior robustness and preserves constant statistical properties better.r.

Construction and Application of User Check-in Spatiotemporal Knowledge Graph Based on Neo4j

With the rapid acceleration of urbanization and the widespread adoption of social networks, A vast amount of spatiotemporal check-in data from social media users has been generated. Traditional relational databases have limitations in handling massive spatiotemporal data, whereas graph databases, with their efficiency in data querying and processing, offer a novel solution for the analysis of spatiotemporal data. The paper introduces a method for constructing a user check-in spatiotemporal knowledge graph based on the Neo4j graph database, utilizing a designed graph model to create a visual knowledge graph. By incorporating temporal and spatial characteristics into entities, we realize efficient management and complex query of spatiotemporal data of social media user check-ins. By adding sentiment attributes to relationships through sentiment analysis of check-in text, the application of check-in text is realized. Ultimately, employing Cypher to achieve knowledge querying, we analyze the hot spots of urban activities and categories.

Concurrency versus consistency in NoSQL databases

<p>With the advent of cloud services, the proliferation of data has reached unprecedented levels. The load distribution across multiple servers, driven by web and mobile applications, has become a defining characteristic of contemporary data management. In contrast to this surge in data complexity, traditional relational databases have proven inadequate in handling vast amounts of unstructured data due to their inherent focus on structured data models. Additionally, the concept of clustering, vital for efficient unstructured data management, eluded relational databases, rendering them ill-equipped for customized clustering techniques and the optimal execution of queries. SQL (Structured Query Language) databases earlier emerged as a groundbreaking solution, introducing the relational database model that organized data into structured tables. They employed ACID (atomicity, consistency, isolation, durability) properties to maintain data integrity and enabled intricate querying through SQL. However, as applications grew in complexity, SQL databases encountered hurdles in handling various data types, rapid data expansion, and concurrent workloads. The limitations of SQL databases propelled the rise of NoSQL (Not Only Structured Query Language) databases, which prioritized adaptability, scalability, and performance. NoSQL databases embraced diverse data models such as documents, key-values, column families, and graphs, enabling effective management of structured, semi-structured, and unstructured data. The transition to NoSQL databases was justified by several factors; horizontally scaled across nodes, handling extensive read-write operations effectively, Agile development of accommodating changing data structures without schema constraints, optimization for specific tasks, providing low-latency access and high throughput, dynamic schemas aligned with modern iterative development, promoting adaptability, and adeptly managed diverse data types, spanning text, geospatial, time-series, and multimedia data. These databases are purposefully designed to accommodate the escalating demands of data storage. Notably, this data emanates from diverse nodes and is susceptible to concurrent access by numerous users. However, a critical challenge surfaces as the data present on one node may diverge from its counterpart on another node replica. In this context, the simultaneous execution of database operations, while preserving the integrity of the data, emerges as a pivotal concern. Maintaining data consistency amid concurrent access hinges upon the synchronization of operations across all replica nodes. Achieving this synchronization necessitates the adoption of a robust concurrency control technique. Concurrency control acts as the linchpin for upholding accuracy and reliability within a system where operations unfold concurrently. Hence, the focal point of this investigation lies in examining the assorted concurrency control methodologies employed by NoSQL systems. The objective is to dissect the intricate interplay between concurrency and consistency, shedding light on the strategies these systems employ to strike an optimal balance between the two. In summation, as the landscape of data management witnesses an era of exponential growth catalyzed by cloud services, the dynamics of load distribution and unstructured data have necessitated a departure from traditional relational databases. NoSQL databases have risen to the fore, demonstrating the ability to grapple with these challenges. However, the quest for concurrent data access without compromising data consistency propels the exploration of various concurrency control methods. The aim of this study is to look at some of the different concurrency control approaches employed by NoSQL systems, highlighting how they priorities concurrency and consistency.</p>

Prospects for the use of metaverse tools in the field of public finance in Russia

Purpose: of the study is to identify promising areas for the use of metaverse technologies in the field of public finance in Russia in the context of the current sanctions restrictions and considering the peculiarities of the national budget system.Methods: the research uses such general scientific methods as literature review, comparative analysis, generalization and analysis of factual data. The work uses the method of constructing block diagrams that describe the key relationships of objects and related business processes to illustrate the proposed innovations.Results: the main results of the study include: 1) proposals for further digitalization of public and socially significant services using VR / AR technologies; 2) a scheme for integrating distributed ledger technologies (based on blockchain) into the treasury budget service system; 3) guidelines for the use of smart contract technologies in the field of public procurement, considering exclusive and inclusive access.Conclusions and Relevance: most of the current government information systems are based on traditional relational databases, which have technological limitations compared to modern distributed ledger technologies. At the same time, the development of the sphere of public finance in the context of the digital transformation of economic relations and the active spread of metauniverses requires continuous updating of the technological infrastructure. The study made it possible to conclude that the technological capabilities of the metaverses cover the current needs for the digital transformation of public finances. In addition, for the Russian public finance sector, the current metaverse tools can be assessed as a factor in further digital development under the current sanctions restrictions.

Key-Value Pairs and NoSQL Databases: A Novel Concept to Manage Biologging Data in Data Repositories

Key-Value Pairs and NoSQL Databases: A Novel Concept to Manage Biologging Data in Data Repositories

Multidimensional Data Analysis of Ambient Air Quality Based on Apache Kylin

While the environmental monitoring department increases the number of monitoring points and the frequency of monitoring, it will also bring about a surge in the amount of monitoring data and computational response events. However, the traditional data statistics methods based on relational databases are ineffective in the face of huge environmental monitoring data. Aiming at ambient air quality data analysis, this paper uses Hadoop, Hive, Kylin, and other tools to build a multi-dimensional analysis platform for ambient air quality big data in a distributed environment, which realizes the unified storage, calculation, and analysis of ambient air quality monitoring data. Compared with the traditional relational database statistical analysis scheme, the proposed solution significantly improves the efficiency of statistical analysis of ambient air quality data under the condition of large data. The response time is shortened by 98%, reaching the sub-second level.

TIME OPTIMIZATION OF PROCESS OF DATA CONSISTENCY IN NOSQL

It is emphasized that the performance of a distributed information system depends on many factors, including the size and structure of the database, the hardware used, the number of replicas and their geographical location. A conclusion was made about the perspective of the concept of non-relational NoSQL databases and it was concluded that it was proposed for efficient storage and provision of quick access to large volumes of information, the so-called Big Data, which cannot be achieved using traditional relational database management systems. The essence of the concept of “NoSQL database” is revealed and the currently existing difficulties in using large systems are named. In NoSQL databases, multiple replication (copying) of a record is used to ensure high fault tolerance. But there is a significant drawback for them, because these systems do not support the mode of conducting transactions and blocking, so there is a problem of data reconciliation. It is emphasized that NoSQL databases mainly use two methods of placing and updating the time of the data consistency process in NoSQL: according to the principle of “master-slave” and “ring”. A model of strong data reconciliation in NoSQL is described and an example of time optimization for the healthcare system is considered. It has been established that even with large values of the intensity of receiving requests for reading λ, the waiting time does not exceed 3 ms. However, when performing analytical queries, this time can increase, so with large N and high intensity of incoming read requests, this time can reach 7 ms.

RETRACTED ARTICLE: Design of NoSQL database in oral English teaching based on 5G network and AI recognition

With the continuous development of science and technology, we have fully entered the information age, people's entertainment life is becoming more and more abundant, and the Internet has also provided people with a lot of convenience. The advent of the Internet age means that more and more various kinds of data are appearing, and the situation is becoming more and more abundant. The use of traditional relational databases can no longer store these data, nor can it be queried. With the continuous development of voice technology, database technology based on NoSQL has become a research hotspot. The availability of NoSQL databases is very high, the scalability is also very high, and the efficiency is very high when processing data. Based on the development of 5G networks and the development of big data technology, this research proposes a brand-new business architecture. This architecture can use the network to store data on the basis of massive data. At the same time, we also described the business scenario, providing a new idea for more intelligent education services. We use this model for teaching in schools, and we can transmit some spoken language resources to the school through the Internet for students to use for learning. Nowadays, the application range of AI technology and intelligent technology has become more and more extensive, and these technologies have also been applied in the education field. We can apply this brand-new technology in teaching to promote the development of teaching and improve students’ enthusiasm and learning effect.

LARGE VECTOR SPATIAL DATA STORAGE AND QUERY PROCESSING USING CLICKHOUSE

Abstract. The exponential growth of geospatial data resulting from the development of earth observation technology has created significant challenges for traditional relational databases. While NoSQL databases based on distributed file systems can handle massive data storage, they often struggle to cope with real-time query. Column-storage databases, on other hand, are highly effective at both storage and query processing for large-scale datasets. In this paper, we propose a spatial version of ClickHouse that leverages R-Tree indexing to enable efficient storage and real-time analysis of massive remote sensing data. ClickHouse is a column-oriented, open-source database management system designed for handling large-scale datasets. By integrating R-Tree indexing, we have created a highly efficient system for storing and querying geospatial data. To evaluate the performance of our system, we compare it with HBase, a popular distributed, NoSQL database system. Our experimental results show that ClickHouse outperforms HBase in handling spatial data queries, with a response time approximately three times faster than HBase. We attribute this performance gain to the highly efficient R-Tree indexing used in ClickHouse, which allows for fast spatial data query.

Buildings of Texas

This note describes an ongoing project at the University of Texas Libraries (UTL) that is transforming the way we think about place, people, and events in managing archival collections. This project was developed around a dataset donated by architectural historians, Gerald Moorhead and Mario Sánchez to the Alexander Architectural Archives. This dataset was collected by a team of researchers studying architecturally significant buildings for the two-volume publication, Buildings of Texas. Our team at UT Libraries has used it as a test-bed for geolocating built works in Texas and mapping our Architectural collections. This dataset presented a clear opportunity to develop map-based digital exhibitions and finding aids for archival material, but also posed several challenges due to naming ambiguities, vague building location descriptions, and repeated references to people and architectural firms that were difficult to disambiguate and interconnect. In an attempt to overcome these challenges and develop a set of methods for dealing with similar and related collections, our cross-disciplinary team is evaluating ways to develop a flat spreadsheet into a collection of inter-related datasets. Our goal was a system that could be managed more flexibly, be easily represented through spatial and non-spatial visualizations, and—crucially—contain references to concepts and typologies defined in widely-used ontologies. We have also sought ways to contribute our data as a local authority to both the Getty vocabularies and Wikidata, as a means to broaden representation and allow for multivocality and multiplicity. We have found unique advantages to a team-approach to the cleaning and data normalization process that transformed a single spreadsheet into a graph database. Our experimentation in this process has facilitated our own visualization of the connections between places, people, and events and, in turn, has informed the way we will present this material in our online exhibitions. In addition to the graph database, we explored traditional relational database technologies, which turned out to be much easier to use for mapping the data and managing it with GIS software. Throughout the project, we have been mindful of breaking down traditional modes of archival description, of contributing to but also looking beyond perceived authorities, and about the value of the work we do for architectural artifacts and landscapes, but also any cultural heritage community of practice seeking to meaningfully describe artifacts, events, people, and places over time.

RDBMS to distributed in-memory data grid: A paradigm shift for improved performance

As per available Cisco VNI report [1], global IP traffic will grow more than three times and global network device connections will be more than 28.5 billion. Availability of high-speed network and connectivity will lead to more online business opportunities to enterprises and in turn will generate large data. Sometimes these large datasets need to be processed in near real time to do some critical analysis. Traditional databases are not designed to do real time data analysis and have challenges in terms of performance and throughput. This paper describes various widely used data storage tools and frameworks currently available and how to achieve the best performance while processing large datasets using in memory data grid. It briefly covers the evolution of in-memory data grid from traditional relational database management systems. It also provides a high-level feature comparison across available in-memory database solutions and how they provide the best of solution for processing real time data beyond Hadoop referencing use cases like fraud detection and genetic data processing. It also emphasizes on leveraging power of GPU alongside CPU to achieve high performance and throughput while processing large data with multiple iterations.