Key-value Data Research Articles

A MODEL OF THE DIACHRONIC SEMANTIC DIFFERENTIAL: STRATIFICATION OF LEXIS ON FUZZY PRINCIPLES (THE CASE OF OLDER SCOTS ADJECTIVES)

The steps of a unified mathematical model for the study of synonymy in the context of paradigmatic and epidigmatic associative relationships are revealed in the paper. Joint fuzzy associative fields of two antonyms constitute on the one hand the main component of the associative map activated through fuzzification into a fuzzy associative verbal network, whilst on the other hand it represents a fuzzy modification of the Osgood semantic differential with a diachronic stratification of lexis. The connection of a considerable number of such components based on synonymous adjectives makes an episodical adjectival associative map according to the indicated by Kiss two fundamental relations, with its further activation into a fuzzy adjectival associative network. When it becomes possible to cover all language synonymy we can eventually talk about the Global Fuzzy Semantic Network as an efficient part of Artificial Intelligence. Keywords: Older Scots, Fuzzy Associative Fields, Fuzzy Associative Word Structure, associative map, Fuzzy Associative Adjectival Network, semantic Osgood’s differential, the Fuzzy Normalized Diachronic 8 Interval Semantic Differential, fuzzy stratification, Kiss’s associative thesaurus, Global Fuzzy Semantic Network.

Preserving Recomputability of Results from Big Data Transformation Workflows

The ability to recompute results from raw data at any time is important for data-driven companies to ensure data stability and to selectively incorporate new data into an already delivered data product. However, data transformation processes are heterogeneous and it is possible that manual work of domain experts is part of the process to create a deliverable data product. Domain experts and their work are expensive and time consuming, a recomputation process needs the ability of automatically adding former human interactions. It becomes even more challenging when external systems are used or data changes over time. In this paper, we propose a system architecture which ensures recomputability of results from big data transformation workflows on internal and external systems by using distributed key-value data stores. Furthermore, the system architecture will contain the possibility of incorporating human interactions of former data transformation processes. We will describe how our approach significantly relieves external systems and at the same time increases the performance of the big data transformation workflows.

A fast and efficient python library for interfacing with the Biological Magnetic Resonance Data Bank

BackgroundThe Biological Magnetic Resonance Data Bank (BMRB) is a public repository of Nuclear Magnetic Resonance (NMR) spectroscopic data of biological macromolecules. It is an important resource for many researchers using NMR to study structural, biophysical, and biochemical properties of biological macromolecules. It is primarily maintained and accessed in a flat file ASCII format known as NMR-STAR. While the format is human readable, the size of most BMRB entries makes computer readability and explicit representation a practical requirement for almost any rigorous systematic analysis.ResultsTo aid in the use of this public resource, we have developed a package called nmrstarlib in the popular open-source programming language Python. The nmrstarlib’s implementation is very efficient, both in design and execution. The library has facilities for reading and writing both NMR-STAR version 2.1 and 3.1 formatted files, parsing them into usable Python dictionary- and list-based data structures, making access and manipulation of the experimental data very natural within Python programs (i.e. “saveframe” and “loop” records represented as individual Python dictionary data structures). Another major advantage of this design is that data stored in original NMR-STAR can be easily converted into its equivalent JavaScript Object Notation (JSON) format, a lightweight data interchange format, facilitating data access and manipulation using Python and any other programming language that implements a JSON parser/generator (i.e., all popular programming languages). We have also developed tools to visualize assigned chemical shift values and to convert between NMR-STAR and JSONized NMR-STAR formatted files. Full API Reference Documentation, User Guide and Tutorial with code examples are also available.We have tested this new library on all current BMRB entries: 100% of all entries are parsed without any errors for both NMR-STAR version 2.1 and version 3.1 formatted files. We also compared our software to three currently available Python libraries for parsing NMR-STAR formatted files: PyStarLib, NMRPyStar, and PyNMRSTAR.ConclusionsThe nmrstarlib package is a simple, fast, and efficient library for accessing data from the BMRB. The library provides an intuitive dictionary-based interface with which Python programs can read, edit, and write NMR-STAR formatted files and their equivalent JSONized NMR-STAR files. The nmrstarlib package can be used as a library for accessing and manipulating data stored in NMR-STAR files and as a command-line tool to convert from NMR-STAR file format into its equivalent JSON file format and vice versa, and to visualize chemical shift values. Furthermore, the nmrstarlib implementation provides a guide for effectively JSONizing other older scientific formats, improving the FAIRness of data in these formats.

Handling multi-dimensional complex queries in key-value data stores

With the advent of the era of cloud computing and big data, in order to cope with vast amounts of data, a number of key-value databases have emerged. These systems provide the ability of large scale data storage and effective data operations based on primary keys, but they do not efficiently support the range and k-Nearest Neighbor (kNN) queries on multi-dimensional datasets. In this paper, we introduce, SPIKE, a sliced Pyramid-based index system for key-value data stores. SPIKE bridges the gap between the data scale and querying functionality for highly available, scalable distributed key-value data stores. We first present SP-Index, the kernel indexing scheme. The SP-Index is designed as a two-level index mechanism consisting of a sliced pyramid space partition index and a distributed B-Tree index. On the basis of SP-Index, we have designed and implemented SPIKE on Cassandra, which provides efficient multi-dimensional complex query processing. We have conducted a set of comprehensive experiments with three types of datasets including synthetic datasets, TPC-H benchmark datasets and a real-world dataset. The experiment results show that SPIKE can efficiently handle multi-dimensional complex queries on large-scale key-value datasets. Evaluation results in comparison with existing systems demonstrates that SPIKE outperforms the comparing work including the original Pyramid, MySQL Cluster and CCIndex by dozens of times in complex query processing.

Learning network representations

In this review I present several representation learning methods, and discuss the latest advancements with emphasis in applications to network science. Representation learning is a set of techniques that has the goal of efficiently mapping data structures into convenient latent spaces. Either for dimensionality reduction or for gaining semantic content, this type of feature embeddings has demonstrated to be useful, for example, for node classification or link prediction tasks, among many other relevant applications to networks. I provide a description of the state-of-the-art of network representation learning as well as a detailed account of the connections with other fields of study such as continuous word embeddings and deep learning architectures. Finally, I provide a broad view of several applications of these techniques to networks in various domains.

Spatio-Temporal Tolerance of Visuo-Tactile Illusions in Artificial Skin by Recurrent Neural Network with Spike-Timing-Dependent Plasticity

Perceptual illusions across multiple modalities, such as the rubber-hand illusion, show how dynamic the brain is at adapting its body image and at determining what is part of it (the self) and what is not (others). Several research studies showed that redundancy and contingency among sensory signals are essential for perception of the illusion and that a lag of 200–300 ms is the critical limit of the brain to represent one’s own body. In an experimental setup with an artificial skin, we replicate the visuo-tactile illusion within artificial neural networks. Our model is composed of an associative map and a recurrent map of spiking neurons that learn to predict the contingent activity across the visuo-tactile signals. Depending on the temporal delay incidentally added between the visuo-tactile signals or the spatial distance of two distinct stimuli, the two maps detect contingency differently. Spiking neurons organized into complex networks and synchrony detection at different temporal interval can well explain multisensory integration regarding self-body.

ParaView + Alya + D8tree: Integrating High Performance Computing and High Performance Data Analytics

Large scale time-dependent particle simulations can generate massive amounts of data, making it so that storing the results is often the slowest phase and the primary time bottleneck of the simulation. Furthermore, analysing this amount of data with traditional tools has become increasingly challenging, and it is often virtually impossible to have a visual representation of the full set.We propose a novel architecture that integrates an HPC-based multi-physics simulation code, a NoSQL database, and a data analysis and visualisation application. The goals are two: On the one hand, we aim to speed up the simulations taking advantage of the scalability of key-value data stores, while at the same time enabling real-time approximated data visualisation and interactive exploration. On the other hand, we want to make it efficient to explore and analyse the large data base of results produced. Therefore, this work represents a clear example of integrating High Performance Computing with High Performance Data Analytics. Our prototype proves the validity of our approach and shows great performance improvements. Indeed, we reduced by 67.5% the time to store the simulation while we made real-time queries run 52 times faster than alternative solutions.

Improving scalability of ART neural networks

With the increasing amount of available data, the need for classification of large data volumes is permanently growing. In order to cope with this challenge, neural classifiers should be adapted to large-scale data. We present here a well scalable extension to the fuzzy Adaptive Resonance Associative Map (ARAM) neural network, which was specially developed for the quick classification of high-dimensional and large data. This extension aims at increasing the classification speed by adding an extra layer for clustering learned prototypes into large clusters. This enables the activation of only one or a few clusters i.e. a small fraction of all prototypes, reducing the classification time significantly. Further we introduce two methods to adapt this extension to a multi-label classification task.

Dict-Based Energy and Latency Efficient Air Indexing Technique for Full Text Search Over Wireless Broadcast Stream

Wireless data broadcasting seems to be an efficient way for dissemination of data to a large number of mobile users. Because of its scalability and flexibility, the service providers use this technology to resolve request of thousands of users in one single response. Full text search is the latest area of research in wireless data broadcasting. Access time and tuning time are the two metrics for evaluation of an indexing technique. In this paper, we propose an air indexing technique based on the dictionary data structure namely Dict-Based Air Indexing Technique for full text search over wireless broadcast stream that utilizes hash-tables. We also propose algorithms for the implementation of the technique. We analyze, evaluate and compare its performance with existing schemes. The results from simulation experiments demonstrate that the proposed technique is the most latency and energy efficient air indexing technique for full text searches.

Valid Space Description in BIM for 3D Indoor Navigation

The BIM paradigm, supplied by appropriate standards like IFC, became unavoidable in recent construction projects. Several applications (e.g. indoor navigation, energy analysis ...) find in it a source of information on which they can rely. However, practices reveal that BIM models are not always directly reliable for applications and the latter have to ensure the validity of the data by their own. In the case of indoor navigation, the calculations will highly rely on the IfcSpace objects describing the rooms, in addition to their spatial relationships with their surrounding components. Unfortunately, it is common to face IFC models in which IfcSpace objects contain wrong geometric and topological description. In this paper, the authors discuss the issues related to BIM models validation for indoor navigation. Furthermore, they present a method to generate valid indoor spaces in IFC models. The approach relies on the structural components of the building (walls, slabs, etc.) and uses topological operations, supported by the combinatorial map data structure, to produce watertight space volumes.

ForestDB: A Fast Key-Value Storage System for Variable-Length String Keys

Indexing key-value data on persistent storage is an important factor for NoSQL databases. Most key-value storage engines use tree-like structures for data indexing, but their performance and space overhead rapidly get worse as the key length becomes longer. This also affects the merge or compaction cost which is critical to the overall throughput. In this paper, we present ForestDB, a key-value storage engine for a single node of large-scale NoSQL databases. ForestDB uses a new hybrid indexing scheme called HB $^+$ -trie, which is a disk-based trie-like structure combined with B $^+$ -trees. It allows for efficient indexing and retrieval of arbitrary length string keys with relatively low disk accesses over tree-like structures, even though the keys are very long and randomly distributed in the key space. Our evaluation results show that ForestDB significantly outperforms the current key-value storage engine of Couchbase Server [1] , LevelDB [2] , and RocksDB [3] , in terms of both the number of operations per second and the amount of disk writes per update operation.

Supporting Multi Data Stores Applications in Cloud Environments

The production of huge amount of data and the emergence of cloud computing have introduced new requirements for data management. Many applications need to interact with several heterogeneous data stores depending on the type of data they have to manage: traditional data types, documents, graph data from social networks, simple key-value data, etc. Interacting with heterogeneous data models via different APIs, and multiple data store applications imposes challenging tasks to their developers. Indeed, programmers have to be familiar with different APIs. In addition, the execution of complex queries over heterogeneous data models cannot, currently, be achieved in a declarative way as it is used to be with mono-data store application, and therefore requires extra implementation efforts. Moreover, developers need to master and deal with the complex processes of cloud discovery, and application deployment and execution. In this paper we propose an integrated set of models, algorithms and tools aiming at alleviating developers task for developing, deploying and migrating multiple data stores applications in cloud environments. Our approach focuses mainly on three points. First, we provide a unifying data model used by applications developers to interact with heterogeneous relational and NoSQL data stores. Based on that, they express queries using OPEN-PaaS-DataBase API (ODBAPI), a unique REST API allowing programmers to write their applications code independently of the target data stores. Second, we propose virtual data stores, which act as a mediator and interact with integrated data stores wrapped by ODBAPI. This run-time component supports the execution of single and complex queries over heterogeneous data stores. Finally, we present a declarative approach that enables to lighten the burden of the tedious and non-standard tasks of (1) discovering relevant cloud environment and (2) deploying applications on them while letting developers to simply focus on specifying their storage and computing requirements. A prototype of the proposed solution has been developed and is currently used to implement use cases from the OpenPaaS project.

Fast LH $$*$$ ∗

Linear Hashing is an efficient and widely used version of extendible hashing. LH$$*$$ź is its distributed version that stores key-value pairs on up to hundreds of thousands of sites in a distributed system. LH$$*$$ź implements the dictionary data structure efficiently by not using a central component and allows the key-based operations of insertion, deletion, actualization, and retrieval as well as the scan operation. Because it does not use a central addressing component, clients or servers in LH$$*$$ź can commit an addressing error by sending a request to a wrong server. This server then forwards the message to the correct server either directly or in one but never more than one additional forward operation. We discuss here methods to avoid this double forward, which, while rare, still might breach quality of service guarantees. We compare our methods with $$\mathrm{LH}*_{\mathrm{RS}^{\mathrm{\tiny P2P}}}$$LHźRSP2P that pushes information about changes in the file structure to clients, whether they are active or not. A second problem especially relevant in high churn environments such as modern data centers is that sites can suddenly become inaccessible. The various high and scalable reliability versions of LH$$*$$ź then reconstruct the data lost on this site elsewhere. We present a solution to the resulting "wandering bucket" problem that allows clients to find the data at their new location.

Discriminating and simulating actions with the associative self-organising map

We propose a system able to represent others' actions as well as to internally simulate their likely continuation from a partial observation. The approach presented here is the first step towards a more ambitious goal of endowing an artificial agent with the ability to recognise and predict others' intentions. Our approach is based on the associative self-organising map, a variant of the self-organising map capable of learning to associate its activity with different inputs over time, where inputs are processed observations of others' actions. We have evaluated our system in two different experimental scenarios obtaining promising results: the system demonstrated an ability to learn discriminable representations of actions, to recognise novel input, and to simulate the likely continuation of partially seen actions.

SARASOM: a supervised architecture based on the recurrent associative SOM

We present and evaluate a novel supervised recurrent neural network architecture, the SARASOM, based on the associative self-organizing map. The performance of the SARASOM is evaluated and compared with the Elman network as well as with a hidden Markov model (HMM) in a number of prediction tasks using sequences of letters, including some experiments with a reduced lexicon of 15 words. The results were very encouraging with the SARASOM learning better and performing with better accuracy than both the Elman network and the HMM.

VarSim: a high-fidelity simulation and validation framework for high-throughput genome sequencing with cancer applications.

Summary: VarSim is a framework for assessing alignment and variant calling accuracy in high-throughput genome sequencing through simulation or real data. In contrast to simulating a random mutation spectrum, it synthesizes diploid genomes with germline and somatic mutations based on a realistic model. This model leverages information such as previously reported mutations to make the synthetic genomes biologically relevant. VarSim simulates and validates a wide range of variants, including single nucleotide variants, small indels and large structural variants. It is an automated, comprehensive compute framework supporting parallel computation and multiple read simulators. Furthermore, we developed a novel map data structure to validate read alignments, a strategy to compare variants binned in size ranges and a lightweight, interactive, graphical report to visualize validation results with detailed statistics. Thus far, it is the most comprehensive validation tool for secondary analysis in next generation sequencing.Availability and implementation: Code in Java and Python along with instructions to download the reads and variants is at http://bioinform.github.io/varsim.Contact: rd@bina.comSupplementary information: Supplementary data are available at Bioinformatics online.

High dimensional biological data retrieval optimization with NoSQL technology.

BackgroundHigh-throughput transcriptomic data generated by microarray experiments is the most abundant and frequently stored kind of data currently used in translational medicine studies. Although microarray data is supported in data warehouses such as tranSMART, when querying relational databases for hundreds of different patient gene expression records queries are slow due to poor performance. Non-relational data models, such as the key-value model implemented in NoSQL databases, hold promise to be more performant solutions. Our motivation is to improve the performance of the tranSMART data warehouse with a view to supporting Next Generation Sequencing data.ResultsIn this paper we introduce a new data model better suited for high-dimensional data storage and querying, optimized for database scalability and performance. We have designed a key-value pair data model to support faster queries over large-scale microarray data and implemented the model using HBase, an implementation of Google's BigTable storage system. An experimental performance comparison was carried out against the traditional relational data model implemented in both MySQL Cluster and MongoDB, using a large publicly available transcriptomic data set taken from NCBI GEO concerning Multiple Myeloma. Our new key-value data model implemented on HBase exhibits an average 5.24-fold increase in high-dimensional biological data query performance compared to the relational model implemented on MySQL Cluster, and an average 6.47-fold increase on query performance on MongoDB.ConclusionsThe performance evaluation found that the new key-value data model, in particular its implementation in HBase, outperforms the relational model currently implemented in tranSMART. We propose that NoSQL technology holds great promise for large-scale data management, in particular for high-dimensional biological data such as that demonstrated in the performance evaluation described in this paper. We aim to use this new data model as a basis for migrating tranSMART's implementation to a more scalable solution for Big Data.

Simple, compact and robust approximate string dictionary

This paper is concerned with practical implementations of approximate string dictionaries that allow edit errors. In this problem, we have as input a dictionary D of d strings of total length n over an alphabet of size σ. Given a bound k and a pattern x of length m, a query has to return all the strings of the dictionary which are at edit distance at most k from x, where the edit distance between two strings x and y is defined as the minimum-cost sequence of edit operations that transform x into y. The cost of a sequence of operations is defined as the sum of the costs of the operations involved in the sequence. In this paper, we assume that each of these operations has unit cost and consider only three operations: deletion of one character, insertion of one character and substitution of a character by another. We present a practical implementation of the data structure we recently proposed and which works only for one error. We extend the scheme to 2≤k<m.Our implementation has many desirable properties: it has a very fast and space-efficient building algorithm. The dictionary data structure is compact and has fast and robust query time.Finally our data structure is simple to implement as it only uses basic techniques from the literature, mainly hashing (linear probing and hash signatures) and succinct data structures (bitvectors supporting rank queries).

Enhancing Performance of Partitioned Database in Cloud Computing Environment

Cloud computing is a revolutionary technology which has taken more concern than any other before, especially by large companies, huge online services and every compute-data intensive application; not only because it is new, but cloud computing has moved distributed systems from side to another. This paper describes the design and architecture of our self-stabilization cloud DBMS based on an in-depth benchmark model with focusing on read operations. On the contrast of map-Reduce, which is a powerful traditional distributed system working on a dedicated cluster and uses key-value data store with a low-level query language, our architecture benefits from great cloud characteristics, certainly automatic resource provisioning to scale virtual cluster up or down with an easy and straight way, and from a long-term evolution of RDBMS and their indexes including b-tree, hash and also full text to retrieve data; in addition, the SQL is a standard query language. Our system uses a dedicated benchmark server which we have developed to benchmark virtual nodes used in the system to give the broker, the stabilizer in the system, a benchmark matrix containing number of queries, distributed over data object stairs and time stairs, per time cycle, besides a decision tree, so the broker can build up access pattern matrix such the benchmark matrix, but with using a decision tree to estimate query time stair; depending on the two matrices, The broker stabilizes the system by migrating and replicating data objects, while allocating or releasing nodes

A Multi-dimensional Index Structure Based on Improved VA-file and CAN in the Cloud

Currently, cloud computing systems use simple key-value data processing, which cannot support similarity search effectively due to lack of efficient index structures, and with increase of dimensionality, existing tree-like index structures could lead to problem of the curse of dimensionality. In this paper, a novel VF-CAN indexing scheme is proposed. VF-CAN integrates content addressable network (CAN) based routing protocol and improved vector approximation file (VA-file) index. There are two index levels in this scheme: global index and local index. The local index VAK-file is built for data in each storage node. VAK-file is k-means clustering result of VA-file approximation vectors according to their degree of proximity. Each cluster forms a separate local index file and each file stores approximate vectors that are contained in cluster. The vector of each cluster center is stored in cluster center information file of corresponding storage node. In global index, storage nodes are organized into an overlay network CAN, and in order to reduce cost of calculation, only clustering information of local index is issued to entire overlay network through CAN interface. The experimental results show that VF-CAN reduces index storage space and improves query performance effectively.