NoSQL Database Management Systems Research Articles

Improving web-oriented information systems efficiency using Redis caching mechanisms

The responsiveness of a web application with minimum latency time and maximum web pages loading speed is determined by its overall performance. When dealing with a large number of users and amount of data, the performance of web applications is significantly affected by ways of data processing, storage and access. This paper identifies the significance of data caching process to speed up access to relational database. The study examines approaches to improve the performance of web applications through the joint use of MySQL relational database management system (DBMS) and Redis NoSQL DBMS. The practical part of the study presents a description of a web application built based on Java and Spring Boot framework. The paper proposes the implementation of the caching strategies that take into account the principles of aspect-oriented programming. Made experiments on performance testing of the developed web application with and without caching are presented. The presented results of the study allowed us to conclude that it is possible to improve the performance of web applications by the optimal use of caching strategies when performing database queries.

Performability evaluation of NoSQL-based storage systems

NoSQL database management systems (DBMS) have been commonly adopted in cloud storage applications, as they usually provide better performance and availability than relational DBMSs. Eventual consistency is a remarkable feature of many NoSQL database systems, in which not all redundant nodes have the newest data, but, eventually, such data will be present in all nodes. Distinct consistency levels can be utilized, but they may affect user experience and service level agreements. The assessment of consistency concerning the probability of accessing the newest data is not common, and the jointly evaluation of performance and availability is usually neglected. This work proposes a method based on stochastic Petri nets (SPN) for evaluating the consistency levels of storage systems based on NoSQL DBMS adopting quorum technique. The models take into account distinct consistency levels and redundant nodes for estimating system availability, throughput and the probability of accessing the newest data. Experimental results demonstrate the practical feasibility of our approach.

Strategies for the Storage of Large LiDAR Datasets—A Performance Comparison

The widespread use of LiDAR technologies has led to an ever-increasing volume of captured data that pose a continuous challenge for its storage and organization, so that it can be efficiently processed and analyzed. Although the use of system files in formats such as LAS/LAZ is the most common solution for LiDAR data storage, databases are gaining in popularity due to their evident advantages: centralized and uniform access to a collection of datasets; better support for concurrent retrieval; distributed storage in database engines that allows sharding; and support for metadata or spatial queries by adequately indexing or organizing the data. The present work evaluates the performance of four popular NoSQL and relational database management systems with large LiDAR datasets: Cassandra, MongoDB, MySQL and PostgreSQL. To perform a realistic assessment, we integrate these database engines in a repository implementation with an elaborate data model that enables metadata and spatial queries and progressive/partial data retrieval. Our experimentation concludes that, as expected, NoSQL databases show a modest but significant performance difference in favor of NoSQL databases, and that Cassandra provides the best overall database solution for LiDAR data.

Performance Benchmarking for NoSQL Database Management Systems

NoSQL database management systems are very diverse and are known to evolve very fast. With so many NoSQL database options available nowadays, it is getting harder to make the right choice for certain use cases. Also, even for a given NoSQL database management system, performance may vary significantly between versions. Database performance benchmarking shows the actual performance for different scenarios on different hardware configurations in a straightforward and precise manner. This paper presents a NoSQL database performance study in which two of the most popular NoSQL database management systems (MongoDB and Apache Cassandra) are compared, and the analyzed metric is throughput. Results show that Apache Cassandra outperformes MongoDB in an update heavy scenario only when the number of operations is high. Also, for a read intensive scenario, Apache Cassandra outperformes MongoDB only when both number of operations and degree of parallelism are high.

Smart environmental data management system into a cattle building

The climatic atmosphere in which cattle live is an essential parameter of their environment because of its critical role in their productivity. An adapted cattle building must help to mitigate the effects of climatic stress and allow the farmer to properly control the climatic atmosphere during the production cycle. The most important factors influencing the climatic atmosphere inside a cattle building are temperature, humidity, and greenhouse gas emissions. We propose a case study for a wireless sensor network model placed on a cattle farm, in which each measurement node “mote” collects environmental data (temperature, humidity, and emission gas), in order to control the building's climate, this data is stored and managed in a remote database. We will present HBase, a NoSQL database management system, based on the concept of distributed storage, a column-oriented database that provides the read/write access to data on the HADOOP HDFS file system in real-time. The storage results presented in this paper are obtained via a java code that can connect with the HBase database, in order to store the received data at every second from each node constituting the measurement system via HTTP requests.

Data in the time of COVID-19: a general methodology to select and secure a NoSQL DBMS for medical data

BackgroundAs the COVID-19 crisis endures and the virus continues to spread globally, the need for collecting epidemiological data and patient information also grows exponentially. The race against the clock to find a cure and a vaccine to the disease means researchers require storage of increasingly large and diverse types of information; for doctors following patients, recording symptoms and reactions to treatments, the need for storage flexibility is only surpassed by the necessity of storage security. The volume, variety, and variability of COVID-19 patient data requires storage in NoSQL database management systems (DBMSs). But with a multitude of existing NoSQL DBMSs, there is no straightforward way for institutions to select the most appropriate. And more importantly, they suffer from security flaws that would render them inappropriate for the storage of confidential patient data.MotivationThis paper develops an innovative solution to remedy the aforementioned shortcomings. COVID-19 patients, as well as medical professionals, could be subjected to privacy-related risks, from abuse of their data to community bullying regarding their medical condition. Thus, in addition to being appropriately stored and analyzed, their data must imperatively be highly protected against misuse.MethodsThis paper begins by explaining the five most popular categories of NoSQL databases. It also introduces the most popular NoSQL DBMS types related to each one of them. Moreover, this paper presents a comparative study of the different types of NoSQL DBMS, according to their strengths and weaknesses. This paper then introduces an algorithm that would assist hospitals, and medical and scientific authorities to choose the most appropriate type for storing patients’ information. This paper subsequently presents a set of functions, based on web services, offering a set of endpoints that include authentication, authorization, auditing, and encryption of information. These functions are powerful and effective, making them appropriate to store all the sensitive data related to patients.Results and ContributionsThis paper presents an algorithm to select the most convenient NoSQL DBMS for COVID-19 patients, medical staff, and organizations data. In addition, the paper proposes innovative security solutions that eliminate the barriers to utilizing NoSQL DBMSs to store patients’ data. The proposed solutions resolve several security problems including authentication, authorization, auditing, and encryption. After implementing these security solutions, the use of NoSQL DBMSs will become a much more appropriate, safer, and affordable solution to storing and analyzing patients’ data, which would contribute greatly to the medical and research effort against COVID-19. This solution can be implemented for all types of NoSQL DBMSs; implementing it would result in highly securing patients’ data, and protecting them from any downsides related to data leakage.

A Comparative Study of Unstructured Data with SQL and NO-SQL Database Management Systems

This paper aims to establish a relative study between a relational Microsoft SQL Server database and a non-relational MongoDB database within the unstructured representation of data in JSON format. There is a great amount of work done regarding comparison of multiple database management applications on the basis of their performances, security etc., but we have limited information available where these databases are assessed on the basis of provided data. This study will mainly focus on looking at all the possibilities that both these database types offer us when handling data in JSON. We will accomplish this by implementing a series of experiments while taking into consideration that the subjected data does not require to be normalized; and therefore, evaluate the outcome to conclude the result.

Using Latency Metrics in NoSQL Database Performance Benchmarking

This paper presents an experimental study evaluating the performance of NoSQL database management systems. The study compares two NoSQL database management systems (Cassandra and MongoDB) and considers the following parameters/factors: workload and degree of parallelism. Two different workloads (update heavy and mostly read) were used, and different numbers of threads. The measured results are related to average latency: update latency and read latency. Our study shows that with the only exception of 1000 operations, both latency indicators have a quasi-parabolic behavior, where the minimum (i.e. the best performance) depends mainly on the number of threads and slightly varies with the increase in the number of operations. In the case of 1000 operations, there is also a maximum point (i.e. worst performance) case, after which the latency decreases.

Runtime Performance Benchmarking for NoSQL Databases

An experimental study regarding the performance of NoSQL database management systems is presented in this paper. In this study, two NoSQL database management systems (MongoDB and Cassandra) were compared, and the following factors were considered: degree of parallelism and workload. Two distinct workloads (mostly read and update heavy) were used, and various numbers of client threads. The measured results refer to total runtime and they confirm that MongoDB outperforms Cassandra in almost all tests considered.

K-NEAREST NEIGHBOUR QUERY PERFORMANCE ANALYSES ON A LARGE SCALE TAXI DATASET: POSTGRESQL VS. MONGODB

Abstract. The increasing volume of transport network data necessitates the use of a DataBase Management System (DBMS) to store, query and analyse data. There are two main types of DBMS: relational and non-relational. Many different DBMS are available on the market but only some of them could handle spatial data. Therefore, determining which DBMS to use for operational purposes is of interest to researchers and analysts working in spatial information science. One of the commonly used spatial queries in GIS is the k-Nearest Neighbour (kNN) of a given point. This paper analyses the performance of the kNN query in PostgreSQL and MongoDB, both being a representative of relational and NoSQL DBMS respectively. Two different metrics have been investigated to determine the performance: i) spatial accuracy and ii) run time. Haversine and Vincenty formulas are used to calculate the distance between the point and the determined neighbours, which are then used to determine the spatial accuracy of the DBMS. Sensitivity analysis have been carried out by varying the k value and the execution times are recorded. The experiments are carried out on New York City’s openly available taxi dataset consisting of millions of taxi pickup and dropoff points. The results indicate that MongoDB outperforms Postgres both in terms of execution time and spatial accuracy regardless the value of k. In order to facilitate reproducibility of the results, the developed software is shared on GitHub.

A Study on Join Operations in MongoDB Preserving Collections Data Models for Future Internet Applications

Presently, we are observing an explosion of data that need to be stored and processed over the Internet, and characterized by large volume, velocity and variety. For this reason, software developers have begun to look at NoSQL solutions for data storage. However, operations that are trivial in traditional Relational DataBase Management Systems (DBMSs) can become very complex in NoSQL DBMSs. This is the case of the join operation to establish a connection between two or more DB structures, whose construct is not explicitly available in many NoSQL databases. As a consequence, the data model has to be changed or a set of operations have to be performed to address particular queries on data. Thus, open questions are: how do NoSQL solutions work when they have to perform join operations on data that are not natively supported? What is the quality of NoSQL solutions in such cases? In this paper, we deal with such issues specifically considering one of the major NoSQL document oriented DB available on the market: MongoDB. In particular, we discuss an approach to perform join operations at application layer in MongoDB that allows us to preserve data models. We analyse performance of the proposes approach discussing the introduced overhead in comparison with SQL-like DBs.

ORBDA: An openEHR benchmark dataset for performance assessment of electronic health record servers.

The openEHR specifications are designed to support implementation of flexible and interoperable Electronic Health Record (EHR) systems. Despite the increasing number of solutions based on the openEHR specifications, it is difficult to find publicly available healthcare datasets in the openEHR format that can be used to test, compare and validate different data persistence mechanisms for openEHR. To foster research on openEHR servers, we present the openEHR Benchmark Dataset, ORBDA, a very large healthcare benchmark dataset encoded using the openEHR formalism. To construct ORBDA, we extracted and cleaned a de-identified dataset from the Brazilian National Healthcare System (SUS) containing hospitalisation and high complexity procedures information and formalised it using a set of openEHR archetypes and templates. Then, we implemented a tool to enrich the raw relational data and convert it into the openEHR model using the openEHR Java reference model library. The ORBDA dataset is available in composition, versioned composition and EHR openEHR representations in XML and JSON formats. In total, the dataset contains more than 150 million composition records. We describe the dataset and provide means to access it. Additionally, we demonstrate the usage of ORBDA for evaluating inserting throughput and query latency performances of some NoSQL database management systems. We believe that ORBDA is a valuable asset for assessing storage models for openEHR-based information systems during the software engineering process. It may also be a suitable component in future standardised benchmarking of available openEHR storage platforms.

Migration from relational to NoSQL database

Data generated by various real time applications, social networking sites and sensor devices is of very huge amount and unstructured, which makes it difficult for Relational database management systems to handle the data. Data is very precious component of any application and needs to be analysed after arranging it in some structure. Relational databases are only able to deal with structured data, so there is need of NoSQL Database management System which can deal with semi -structured data also. Relational database provides the easiest way to manage the data but as the use of NoSQL is increasing it is becoming necessary to migrate the data from Relational to NoSQL databases. Various frameworks has been proposed previously which provides mechanisms for migration of data stored at warehouses in SQL, middle layer solutions which can provide facility of data to be stored in NoSQL databases to handle data which is not structured. This paper provides a literature review of some of the recent approaches proposed by various researchers to migrate data from relational to NoSQL databases. Some researchers proposed mechanisms for the co-existence of NoSQL and Relational databases together. This paper provides a summary of mechanisms which can be used for mapping data stored in Relational databases to NoSQL databases. Various techniques for data transformation and middle layer solutions are summarised in the paper.

Towards NoSQL-based Data Warehouse Solutions

Data warehousing is a traditional domain of relational databases, and there are two main reasons for that: (1) data warehouses mostly are used in enterprises with large-scale data sets created in different legacy systems with relational data storages, (2) though rapidly developing non-relational databases are still rather unusual in data processing tasks. This paper discusses the possibilities to create data warehouse solutions by using NoSQL database management systems. The main challenge is to find a good balance between characteristics of classical data warehouses using relational data base management systems and opportunities offered by NoSQL database management systems. The paper describes processes of creation and production of data warehouse using a NoSQL data mart and outlines requirements for technology necessary for such processes. The research is based on practical experience when implementing NoSQL data marts with MongoDB and Clusterpoint DB.

A Scalable Data Access Layer to Manage Structured Heterogeneous Biomedical Data.

This work presents a scalable data access layer, called PyEHR, designed to support the implementation of data management systems for secondary use of structured heterogeneous biomedical and clinical data. PyEHR adopts the openEHR’s formalisms to guarantee the decoupling of data descriptions from implementation details and exploits structure indexing to accelerate searches. Data persistence is guaranteed by a driver layer with a common driver interface. Interfaces for two NoSQL Database Management Systems are already implemented: MongoDB and Elasticsearch. We evaluated the scalability of PyEHR experimentally through two types of tests, called “Constant Load” and “Constant Number of Records”, with queries of increasing complexity on synthetic datasets of ten million records each, containing very complex openEHR archetype structures, distributed on up to ten computing nodes.

PolarDB: An infrastructure for specialized NoSQL datebases and DBMS

This paper presents a new infrastructure for creating specialized databases and database management systems (DBMSs). Some principles of this infrastructure are formulated based on an analysis of various NoSQL solutions. The main features of the proposed approach are: (a) a flexible system of type definitions that allows one to create data structures based on different paradigms and (b) different forms of supporting structured data and mapping them onto the file system. Experiments are carried out to implement RDF graphs, relational tables, name tables, and object-relational mappings. The proposed approach allows one to solve certain problems of developing new technologies for working with Big Data.

Applying Intelligent Adaptation to Remote Cloud Datacenter Backup

HBase and Cassandra are two most commonly used large-scale distributed NoSQL database management systems; especially applicable to a large amount of data processing. Regarding remote data backup, each kind of datacenter has its own backup strategy to prevent the risks of data loss. With Thrift Java, this paper aims to implement in-cloud high efficient remote datacenter backup applied to in-cloud NoSQL databases like HBase and Cassandra. The binary communications protocol technology from Apache Thrift is employed to establish the graphical user interface instead of the command line interface so as to ease data manipulation. In order to control the network traffic flow smoothly, intelligent adaptation using ANFIS and PSO is employed to tune the parameters of NoSQL databases during the remote data backup to improve QoS in the network. The stress test has taken on strictly data reading/writing and remote backup of a huge amount of data to verify the effectiveness. Finally, the performance evaluation of a variety of benchmark databases has been done by performance index. As a result, the proposed HBase approach outperforms the other databases.

CRITÉRIOS DE SELEÇÃO DE SGBD NOSQL EM ORGANIZAÇÕES BRASILEIRAS

The large volume of data generated by Web applications and the requirements of scalability and high degree of availability have contributed to the emergence of new models and technologies related to DBMS (Database Management Systems). In this environment of growing demand for data volume and performance, a new model of DBMS, called NoSQL arises. Within this context, this research aims to identify the main criteria that should be used for selecting a NoSQL DBMS in private organizations. We used a qualitative research based on the Delphi Technique. The application of this technique was carried out between January and March 2013, with the participation of 32 specialists. The research identified nine criteria that should be considered in the selection process of DBMS in private organizations.