NoSQL Data Model Research Articles

SkiQL: A unified schema query language

Most NoSQL systems are schema-on-read: data can be stored without first having to declare a schema that imposes a structure. This schemaless feature offers flexibility to evolve data-intensive applications when data change frequently. However, freeing from declaring schemas does not mean their absence, but rather that they are implicit in data and code. Therefore, diagramming tools similar to those available for relational systems are also needed to help developers and administrators to design and to understand NoSQL schemas.Visualizing diagrams is not practical if schemas contain hundreds of database entities, so exploration or query facilities are then needed. In schemaless NoSQL stores, data of the same entity can be stored with different structure (e.g., non-uniform types and optional fields), which can increase the difficulty of having readable diagrams.NoSQL schema management tools should therefore have three main components: schema extraction, schema visualization, and schema query. As there are four main NoSQL data models, it is convenient for such tools to be built on a generic data model so that they provide platform-independence (of data models and data stores) to query and visualize schemas. With the aim of favoring the creation of generic database tools, the authors of this paper defined the U-Schema unified data model that integrates the four main NoSQL data models as well as the relational model.This paper is focused on querying NoSQL and relational schemas which are represented as U-Schema models. We present the SkiQL language designed on U-Schema to achieve a platform-independent schema query service. SkiQL provides two constructs: schema-query and relationship-query. The former allows to obtain information of entity or relationship types, and the latter that of the aggregations or references (relations among types). We will show how SkiQL was evaluated by calculating well-known metrics for languages as well as using a survey with developers with experience in NoSQL.

Key-value data collection and statistical analysis with local differential privacy

The collection and statistical analysis of simple data types (e.g., categorical, numerical and multi-dimensional data) under local differential privacy has been widely studied. Recently, researchers have focused on the collection of the key-value data, which is one of the main types of NoSQL data model. In the collection and statistical analysis of key-value data under local differential privacy, the frequency and mean of each key must be estimated simultaneously. However, achieving a good utility-privacy tradeoff is difficult, because key-value data has inherent correlation, and some users may have different numbers of key-value pairs. In this paper, we propose an efficient sampling based scheme for collecting and analyzing key-value data. Note that the more valid data collected, the higher the accuracy of statistical data under the same disturbance level and disturbance algorithm. Therefore, we make full use of probability sampling and the inherent correlation of key-value data to improve the probability of users submitting valid key-value data. Moreover, we optimize the budget allocation on key-value data, so that the overall variance of frequency and mean estimation is close to optimal. Detailed theoretical analysis and experimental results show that the proposed scheme is superior to existing schemes in accuracy.

PrivKVM*: Revisiting Key-Value Statistics Estimation With Local Differential Privacy

A key factor in big data analytics and artificial intelligence is the collection of user data from a large population. However, the collection of user data comes at the price of privacy risks, not only for users but also for businesses who are vulnerable to internal and external data breaches. To address privacy issues, local differential privacy (LDP) has been proposed to enable an untrusted collector to obtain accurate statistical estimation on sensitive user data (e.g., location, health, and financial data) without actually accessing the true records. As key-value data is an extremely popular NoSQL data model, there are a few works in the literature that study LDP-based statistical estimation on key-value data. However, these works have some major limitations, including supporting small key space only, fixed key collection range, difficulty in choosing an appropriate padding length, and high communication cost. In this article, we propose a two-phase mechanism <inline-formula><tex-math notation="LaTeX">$PrivKVM^*$</tex-math></inline-formula> as an optimized and highly-complete solution to LDP-based key-value data collection and statistics estimation. We verify its correctness and effectiveness through rigorous theoretical analysis and extensive experimental results.

A Fuzzy Query Method Oriented to Knowledge Graph

With the widespread application of Knowledge Graphs, Knowledge Graph query technology has received widespread attention. The daily use of natural language usually contains fuzzy terms. The use of fuzzy terms in queries is more in line with natural language habits. Therefore, fuzzy questions have been widely discussed and applied in relational data models, and their use in NoSQL data models has also begun to be affected in recent years. RDF (Resource Description Framework) is a representative model of the knowledge graph, and SPARQL is the standard query language of RDF. To realize the fuzzy query of the knowledge graph, this paper introduces the unclear expansion syntax of aggregate function based on summarizing and analyzing the existing SPARQL fuzzy expansion. Supplements the qualitative preference analysis method then proposes and SPARQL1. 1 Corresponding fuzzy SPARQL1.1 (f-SPARQL1.1) simultaneously provides strategies and techniques for rewriting ambiguous queries into classic SPARQL queries. In addition, this article also provides a visual interface for defining the membership functions of fuzzy terms and provides three interactive methods for setting fuzzy membership functions. Compared with the previous fuzzy query methods for RDF models, the method proposed in this paper has more vital fuzzy query ability and more flexible universality. The experimental results show the usability of the method proposed in this paper.

Возможности NOSQL СУБД для обработки пространственных данных

An increasing number of database management systems are expanding their functionality to work with various types of spatial data. This is true for both relational and NoSQL data models. The article pre-sents the main features of those data models for which the functions of storing and processing spatial data are implemented. The methods of artificial intelligence, which are implemented on the basis of a particular data model, are considered. A comparative analysis of the performance of typical spatial queries for database management systems based on various data models, including multi-model ones, is performed. The data set on which the comparison is performed is presented as three blocks of OpenStreetMap vector data for the territory of the Novosibirsk region. Based on the results of the study, recommendations are given on the use of certain data models, depending on the available data and the tasks being solved.

SMAP FRAMEWORK: A NOSQL SOLUTION FOR MANAGING A JSON KNOWLEDGE BASE OF SMART POLITICAL DOCUMENTS

This paper is devoted to developing a MongoDB framework for managing a JSON Knowledge base of “Smart Political Documents” (SPD). For this purpose, we designed a full knowledge base in a JSON Repository that runs under a MongoDB NoSQL server. The study of the experimentally collected data from the United Nations site has shown that our SPD knowledge base will contain various information using different formats (text, image, audio, video, etc.). The representation of these data requires both hierarchical and relational models. In this respect, we are proposing a NoSQL data model based on both JSON data and relational representation. JSON collections can be related to one another to complete information by special JSON properties. We have also defined a JSON Query Language for SMAP, SPQL (Smart Politics Query Language), which provided a powerful environment for integrating these disparate forms of data. In this paper, we will so describe the repository architecture, a Speech Labeling Technique used to associate information to spoken discourses, and finally, the whole SMArt Political documents (SMAP) framework that is based on a 3-tiers architecture and a Big Data environment with automatic deductions from the analysis of these data. The new system's purpose is to offer the end-user an intelligent way to quickly find the desired documents with all semantic links between them—something impossible in the system's initial state. The actual novelty of the system is represented mainly by the combination of the NoSQL feature with a relational model and an intelligent linked data system.

A unified metamodel for NoSQL and relational databases

The Database field is undergoing significant changes. Although relational systems are still predominant, the interest in NoSQL systems is continuously increasing. In this scenario, polyglot persistence is envisioned as the database architecture to be prevalent in the future. Therefore, database tools and systems are evolving to support several data models.Multi-model database tools normally use a generic or unified metamodel to represent schemas of the data model that they support. Such metamodels facilitate developing database utilities, as they can be built on a common representation. Also, the number of mappings required to migrate databases from a data model to another is reduced, and integrability is favored.In this paper, we present the U-Schema unified metamodel able to represent logical schemas for the four most popular NoSQL paradigms (columnar, document, key–value, and graph) as well as relational schemas. We will formally define the mappings between U-Schema and the data model defined for each database paradigm. How these mappings have been implemented and validated will be discussed, and some applications of U-Schema will be shown.To achieve flexibility to respond to data changes, most of NoSQL systems are “schema-on-read,” and the declaration of schemas is not required. Such an absence of schema declaration makes structural variability possible, i.e., stored data of the same entity type can have different structure. Moreover, data relationships supported by each data model are different; For example, document stores have aggregate objects but not relationship types, whereas graph stores offer the opposite. Through the paper, we will show how all these issues have been tackled in our approach.As far as we know, no proposal exists in the literature of a unified metamodel for relational and the NoSQL paradigms which describes how each individual data model is integrated and mapped. Our metamodel goes beyond the existing proposals by distinguishing entity types and relationship types, representing aggregation and reference relationships, and including the notion of structural variability. Our contributions also include developing schema extraction strategies for schemaless systems of each NoSQL data model, and tackling performance and scalability in the implementation for each store.

SQL Versus NoSQL Databases to Assess Their Appropriateness for Big Data Application

Background: Nowadays, the digital world is rising rapidly and becoming very difficult in nature's quantity, diversity, and speed. Recently, there have been two major changes in data management, which are NoSQL databases and Big Data Analytics. While evolving with the diverse reasons, their independent growths balance each other and their convergence would greatly benefit organization to make decisions on-time with the amount of multifaceted data sets that might be semi structured, structured, and unstructured. Though several software solutions have come out to support Big Data analytics on the one hand, on the other hand, there have been several packages of NoSQL database available in the market. Aim and Methods: The main goal of this article is to give comprehension of their perspective and a complete study to associate the future of the emerging several important NoSQL data models. Results: Evaluating NoSQL databases for Big Data analytics with traditional SQL performance shows that NoSQL database is a superior alternative for industry condition need high-performance analytics, adaptability, simplicity, and distributed large data scalability. Conclusion: In this article we conclude with the adoption of NoSQL in various markets.

MetaXplor: an interactive viral and microbial metagenomic data manager.

BackgroundEfficiently managing large, heterogeneous data in a structured yet flexible way is a challenge to research laboratories working with genomic data. Specifically regarding both shotgun- and metabarcoding-based metagenomics, while online reference databases and user-friendly tools exist for running various types of analyses (e.g., Qiime, Mothur, Megan, IMG/VR, Anvi'o, Qiita, MetaVir), scientists lack comprehensive software for easily building scalable, searchable, online data repositories on which they can rely during their ongoing research.ResultsmetaXplor is a scalable, distributable, fully web-interfaced application for managing, sharing, and exploring metagenomic data. Being based on a flexible NoSQL data model, it has few constraints regarding dataset contents and thus proves useful for handling outputs from both shotgun and metabarcoding techniques. By supporting incremental data feeding and providing means to combine filters on all imported fields, it allows for exhaustive content browsing, as well as rapid narrowing to find specific records. The application also features various interactive data visualization tools, ways to query contents by BLASTing external sequences, and an integrated pipeline to enrich assignments with phylogenetic placements. The project home page provides the URL of a live instance allowing users to test the system on public data.ConclusionmetaXplor allows efficient management and exploration of metagenomic data. Its availability as a set of Docker containers, making it easy to deploy on academic servers, on the cloud, or even on personal computers, will facilitate its adoption.

Using a DBMS based on various data models in solving problems of cartography and geoinformatics

An increasing number of database management systems are expanding their functionality to work with various types of spatial data. This is true for both relational and NoSQL data models. The article describes the main features of those data models for which the functions of storing and processing spatial data are implemented. A comparative analysis of the performance of typical spatial queries for database management systems based on various data models, including multi-model ones, is carried out. The dataset on which the comparison is performed is presented in the form of three blocks of OpenStreetMap vector data for the territory of the Novosibirsk region. Based on the results of the study, recommendations are made on the use of certain data models, depending on the available data and the tasks to be solved.

Models and data quality in information systems applicable in the mining industry

The purpose of this article is to present modern approaches to data storage and processing, as well as technologies to achieve the quality of data needed for specific purposes in the mining industry. The data format looks at NoSQL and NewSQL technologies, with the focus shifting from the use of common solutions (traditional RDBMS) to specific ones aimed at integrating data into industrial information systems. The information systems used in the mining industry are characterized by their specificity and diversity, which is a prerequisite for the integration of NoSQL data models in it due to their flexibility. In modern industrial information systems, data is considered high-quality if it actually reflects the described object and serves to make effective management decisions. The article also discusses the criteria for data quality from the point of view of information technology and that of its users. Technologies are also presented, providing an optimal set of necessary functions that ensure the desired quality of data in the information systems applicable in the industry. The format and quality of data in client-server based information systems is of particular importance, especially in the dynamics of data input and processing in information systems used in the mining industry.

Comparison between Relational and NoSQL Databases

Relational databases have not been the only solution for the applications that use them for years. New applications must deal with large volumes of complex, distributed data of different types, whose volume increases very rapidly. In these applications, relational databases are supplemented with several new generic data models referred to as NoSQL. In this article we shall present the main types and characteristics of NoSQL data models and we shall make, both a detailed and a synthetic, comparison between the relational model and the NoSQL data models.

JS4Geo: a canonical JSON Schema for geographic data suitable to NoSQL databases

The large volume and variety of data produced in the current Big Data era lead companies to seek solutions for the efficient data management. Within this context, NoSQL databases rise as a better alternative to the traditional relational databases, mainly in terms of scalability and availability of data. A usual feature of NoSQL databases is to be schemaless, i.e., they do not impose a schema or have a flexible schema. This is interesting for systems that deal with complex data, such as GIS. However, the lack of a schema becomes a problem when applications need to perform processes such as data validation, data integration, or data interoperability, as there is no pattern for schema representation in NoSQL databases. On the other hand, the JSON language stands out as a standard for representing and exchanging data in document NoSQL databases, and JSON Schema is a schema representation language for JSON documents that it is also leading to become a standard. However, it does not include spatial data types. From this limitation, this paper proposes an extension to JSON Schema, called JS4Geo, that allows the definition of schemas for geographic data. We demonstrate that JS4Geo is able to represent schemas of any NoSQL data model, as well as other standards for geographic data, like GML and KML. We also present a case study that shows how a data integration system can benefit of JS4Geo to define local schemas for geographic datasets and generate an integrated global schema.

Bringing SQL databases to key-based NoSQL databases: a canonical approach

Big Data management has brought several challenges to data-centric applications, like the support to data heterogeneity, rapid data growth and huge data volume. NoSQL databases have been proposed to tackle Big Data challenges by offering horizontal scalability, schemaless data storage and high availability, among others. However, NoSQL databases do not have a standard query language, which bring on a steep learning curve for developers. On the other hand, traditional relational databases and SQL are very popular standards for storing and manipulating critical data, but they are not suitable to Big Data management. One solution for relational-based applications to move to NoSQL databases is to offer a way to access NoSQL databases through SQL instructions. Several approaches have been proposed for translating relational database schemata and operations to equivalent ones in NoSQL databases in order to improve scalability and availability. However, these approaches map relational databases only to a single NoSQL data model and, sometimes, to a specific NoSQL database product. This paper presents a canonical approach, called SQLToKeyNoSQL, that translates relational schemata as well as SQL instructions to equivalent schemata and access methods of any key-oriented NoSQL database. We present the architecture of our layer focusing on the mapping strategies as well as experiments that evaluate the benefits of our approach against some state-of-art baselines.

Hybrid approach for big data localization and semantic annotation

SummaryMost of the data concerning business‐oriented systems are still based on either NoSQL or the relational data model. On the other hand, Semantic Web data model Resource Description Framework (RDF) has become the new standard for data modeling and analysis. Due to this situation integration of NoSQL, Relational Database (RDB) and RDF data models are becoming a required feature of the systems. Many solutions like tools and languages are provided in the shape of the transformation of data from RDB to RDF. This research is aimed to compare and map data models used for transformation between NoSQL, RDB, and Semantic Web. This study will help in achieving much better and enhanced technology‐based systems for retrieval and storage of data among Big‐data annotation using Semantic Web. It is aimed to reduce the response time of queries and offer compatibility with the web and semantically enriched data format. A drugs dataset is being used and transformed to have semantical meaning embedded and linked to support big data localization. At the end of this paper, RDF graph and bar chart are used to represent transformed data after passing through the proposed model. Big data localization helps in gaining fast and accurate results.

Analysis of NoSQL Database Vulnerabilities

The NoSQL DBMS offers efficient solutions for storing and analyzing big data. The NoSQL data model provides better flexibility, schema less model, efficient storage and access mechanism. Due to these characteristics many organizations, real time web applications and social networking sites have started focusing towards NoSQL database adoption. As opposed to traditional databases (RDBMS) NoSQL databases more focuses on performance and availability compared to security and consistency. As NoSQL databases consider security with less priority, these databases are susceptible to different security threats. It can be hacked, injected with malicious codes, modified or even destroyed. Thus security is a critical issue for data analysis using NoSQL databases. This paper, discusses about NoSQL data models and outlines their numerous features. After this demonstration, it discusses about the security vulnerabilities, different attack mechanisms on NoSQL databases and mitigation techniques.

A general framework for big data knowledge discovery and integration

SummaryData structure description, conceptual modeling, and logic reasoning for knowledge discovery are three critical factors for the integration of information with heterogeneity. In particular, technologies of NoSQL databases and Internet of Things raise an urgent requirement for a uniform expression of heterogeneous data, and little attention has been paid to researches on the integration of NoSQL databases with traditional data models, as well as the semantic description of big data. To tackle these problems, in this paper, a concept‐and‐relation‐oriented grid data model called GODM model is first proposed based on the definitions of Monad, Compounder, Relation, etc. Then, the GODM model is utilized to uniformly describe traditional data models and NoSQL data models, which eliminates structure differences of heterogeneous data. Next, based on the GODM relation mechanism, an extendable semantic system is built up by choosing SHOIQ(D) description logic as the example to establish the correspondence with GODM grammar subset, providing a fundamental support for semantic integration and knowledge discovery of heterogeneous data. After that, comprehensive comparisons with GODM and other models are made, especially the distinctions between GODM and OWL on the aspects of relation mechanism, hybrid schema, description logic, grammatical constructors, etc. Besides, experimental evaluations and analyses on time and space efficiencies of some primary common data models are conducted after the proposal of a general evaluation model, with the results showing that the GODM model has great advantage on properties of expressiveness, flexibility, etc, particularly time and space efficiency. In summary, the GODM model describes heterogeneous data from both aspects of data structure and semantic relationship and realizes a hybrid schema reconciling the schemaful and schemaless data models, making it especially suitable for dynamic data integration and knowledge discovery from big data models.

NoSQL databases for big data

NoSQL solutions have been created to respond to many issues encountered when dealing with some specific applications, e.g., storage of very large datasets. In fact, traditional RDMS ensure data integrity and transaction consistency. But, this is at the cost of a rigid storage schema and a complex management. Certainly, data integrity and consistency are required in many cases like in financial applications but they are not always needed. The goal of this paper is to establish a precise picture about NoSQL's evolution and mechanisms as well as the advantages and disadvantages of the main NoSQL data models and frameworks. For this purpose, first, a deep comparison between SQL and NoSQL databases is presented. Many criteria are examined such as: scalability, performance, consistency, security, analytical capabilities and fault-tolerance mechanisms. Second, the four major types of NoSQL databases are defined and compared: key-value stores, document databases, column-oriented databases and graph databases. Third, we compare for each NoSQL data model the main available technical solutions.

SecureNoSQL: An approach for secure search of encrypted NoSQL databases in the public cloud

While many schemes have been proposed to search encrypted relational databases, less attention has been paid to NoSQL databases. In this paper we report on the design and the implementation of a security scheme called “SecureNoSQL” for searching encrypted cloud NoSQL databases. Our solution is one of the first efforts covering not only data confidentiality, but also the integrity of the datasets residing on a cloud server. In our system a secure proxy carries out the required transformations and the cloud server is not modified. The construction is applicable to all NoSQL data models and, in our experiments, we present its application to a Document-store data model. The contributions of this paper include: (1) a descriptive language based on a subset of JSON notations; (2) a tool to create and parse a security plan consisting of cryptographic modules, data elements, and mappings of cryptographic modules to the data fields; and (3) a query and data validation mechanism based on the security plan.

A NOVEL TECHNIQUE IN NoSQL DATA EXTRACTION

NoSQL databases (commonly interpreted by developers as „not only SQL databases‟ and not „no SQL‟) is an emerging alternative to the most widely used relational databases. As the name suggests, it does not completely replace SQL but compliments it in such a way that they can co-exist. In this paper we will be discussing the NoSQL data model, types of NoSQL data stores, characteristics and features of each data store, query languages used in NoSQL, advantages and disadvantages of NoSQL over RDBMS and the future prospects of NoSQL.&#x0D; Motivation/Background:NoSQL systems exhibit the ability to store and index arbitrarily big data sets while enabling a large amount of concurrent user requests.&#x0D; Method:Many people think NoSQL is a derogatory term created to poke at SQL. In reality, the term means Not Only SQL. The idea is that both technologies can coexist and each has its place.&#x0D; Results:Large-scale data processing (parallel processing over distributed systems); Embedded IR (basic machine-to-machine information look-up &amp; retrieval); Exploratory analytics on semi-structured data (expert level); Large volume data storage (unstructured, semi-structured, small-packet structured).&#x0D; Conclusions:This study report motivation to provide an independent understanding of the strengths and weaknesses of various NoSQL database approaches to supporting applications that process huge volumes of data; as well as to provide a global overview of this non-relational NoSQL databases.