Query Engine Research Articles

Methods and Techniques for miRNA Data Analysis.

Genomic data analysis consists of techniques to analyze and extract information from genes. In particular, genome sequencing technologies allow to characterize genomic profiles and identify biomarkers and mutations that can be relevant for diagnosis and designing of clinical therapies. Studies often regard identification of genes related to inherited disorders, but recently mutations and phenotypes are considered both in diseases studies and drug designing as well as for biomarkers identification for early detection. Gene mutations are studied by comparing fold changes in a redundancy version of numeric and string representation of analyzed genes starting from macromolecules. This consists of studying often thousands of repetitions of gene representation and signatures identified by biological available instruments that starting from biological samples generate arrays of data representing nucleotides sequences representing known genes in an often not well-known sequence. High-performance platforms and optimized algorithms are required to manipulate gigabytes of raw data that are generated by the so far mentioned biological instruments, such as NGS (standing for Next-Generation Sequencing) as well as for microarray. Also, data analysis requires the use of several tools and databases that store gene targets as well as gene ontologies and gene-disease association. In this chapter we present an overview of available software platforms for genomic data analysis, as well as available databases with their query engines.

Extended Query Pattern Graph and Heuristics - based SPARQL Query Planning

SPARQL query language plays a significant role in developing semantic web and web intelligence. In order to deal with large data query execution over RDF, SPARQL query optimizer is an essential component in the SPARQL query engine for improving query execution performance. This paper proposes an approach for performing query planning and optimization based on an extended query pattern graph and heuristics. First, this paper generalizes SPARQL query statement representation by taking other expressions into account, aiming at overcoming the limitations of only using basic query triple patterns. Second, this paper presents the heuristics for estimating the cost of executing query triple pattern. The proposed query planning methods are implemented within Corese query engine and are evaluated using BSBM benchmark. The results suggest that the proposed methods can optimize effectively the query execution time of SPARQL query engine.

A Decision Support System (DSS) for Colorectal Cancer Follow-Up Program via a Semantic Framework

This paper offers insights into evolving a decision support system (DSS) to aid primary care physicians and/or nurses in the post-surgical care of patients with Colorectal Cancer in a clinical setting. Presently, the oncologists in the cancer center, who are familiar with the Clinical Practice Guidelines (CPGs), are primarily responsible for the provision of follow-up care to their patients on the basis of the CPGs; in contrast, the attending primary care physician and/or nurse assisting the oncologist may be unfamiliar with these guidelines. These caregivers would, therefore, either require hardcopies of the CPGs or can be aided via a DSS for them to be able to provide the appropriate follow-up care for the respective cancer patients. Clearly, the Colorectal Cancer follow-up CPGs have to be analyzed and the ontology representing the knowledge embedded in the guidelines designed prior to realizing such a DSS. The designed ontology is often coded into Web Ontology Language (OWL) as a standard ontology that can be accessed through the Web. The authors' research team designed and presented the semantic framework of the web application, using the designed ontology that combines the current Web technology with database storage to achieve a unified development of the DSS. The authors also designed a user-friendly interface of the Web application to provide medical practitioners the functionality of the CPGs and the flexibility to customize the desired follow-up care schedule. The resulting DSS provides the physicians with follow-up program for the Colorectal Cancer patients based on the CPGs. The system was built using the semantic framework for the follow-up program and queries on the system are executed through SPARQL query engine.

Measuring and Querying Process Performance in Supply Chains: An Approach for Mining Big-Data Cloud Storages

Abstract Survival in today's global environment means continuously improving processes, identifying and eliminating inefficiencies wherever they occur. With so many companies operating as part or all of complex distributed supply chain, gathering, collating and analyzing the necessary data to identify such improvement opportunities is extremely complex and costly. Although few solutions exist to correlate the data, it continues to be generated in vast quantities, rendering the use of highly scalable, cloud-based solutions for process analysis a necessity. In this paper we present an overview of an analytical framework for business activity monitoring and analysis, which has been realized using extremely scalable, cloud-based technologies. It provides a low-latency solution for entire supply chains or individual nodes in such chains to query process data stores in order to deliver business insight. A custom query language has been implemented which allows business analysts to design custom queries on processes and activities based on a standard set of process metrics. Ongoing developments are focused on testing and improving the scalability and latency of the system, as well as extending the query engine to increase its flexibility and performance.

Support patient search on pathology reports with interactive online learning based data extraction

Background: Structural reporting enables semantic understanding and prompt retrieval of clinical findings about patients. While synoptic pathology reporting provides templates for data entries, information in pathology reports remains primarily in narrative free text form. Extracting data of interest from narrative pathology reports could significantly improve the representation of the information and enable complex structured queries. However, manual extraction is tedious and error-prone, and automated tools are often constructed with a fixed training dataset and not easily adaptable. Our goal is to extract data from pathology reports to support advanced patient search with a highly adaptable semi-automated data extraction system, which can adjust and self-improve by learning from a user’s interaction with minimal human effort. Methods: We have developed an online machine learning based information extraction system called IDEAL-X. With its graphical user interface, the system’s data extraction engine automatically annotates values for users to review upon loading each report text. The system analyzes users’ corrections regarding these annotations with online machine learning, and incrementally enhances and refines the learning model as reports are processed. The system also takes advantage of customized controlled vocabularies, which can be adaptively refined during the online learning process to further assist the data extraction. As the accuracy of automatic annotation improves overtime, the effort of human annotation is gradually reduced. After all reports are processed, a built-in query engine can be applied to conveniently define queries based on extracted structured data. Results: We have evaluated the system with a dataset of anatomic pathology reports from 50 patients. Extracted data elements include demographical data, diagnosis, genetic marker, and procedure. The system achieves F-1 scores of around 95% for the majority of tests. Conclusions: Extracting data from pathology reports could enable more accurate knowledge to support biomedical research and clinical diagnosis. IDEAL-X provides a bridge that takes advantage of online machine learning based data extraction and the knowledge from human’s feedback. By combining iterative online learning and adaptive controlled vocabularies, IDEAL-X can deliver highly adaptive and accurate data extraction to support patient search.

Interactive Exploration of Multi-Dimensional Information Spaces with Preference Support

Users access large amounts of information resources mainly through search functions, where they type a few keywords and the web search or query engine returns a linear list of hits. While this is often satisfactory for focalized search, it does not provide enough support for exploratory information needs. Faceted and Dynamic Taxonomies (FDT) is a highly prevalent model for exploratory search, which allows users to get an overview of the information space and offer them various groupings of the results based on their attributes, metadata, or other dynamically mined information, enabling them to gradually restrict their focus through clicks and locate low ranked resources. The enrichment of such mechanisms with preferences could be proven useful for exploratory tasks. However, the current preference-based approaches seem to ignore the fact that users should be acquainted with the information space and the available choices for describing effectively their preferences. In this dissertation we extend the interaction model of FDT with preference actions that allow users to express their preferences interactively, gradually, and in a simple way. We introduce a preference framework appropriate for information spaces comprising resources described by attributes whose values can be hierarchically valued and/or multi-valued. We define the language, its semantics and the required algorithms. The framework supports preference inheritance in the hierarchies, automatic conflict resolution, as well as preference composition. Subsequently, we enrich the FDT model with preference actions and we propose logical optimizations and methods for exploiting the intrinsic characteristics of the FDT-based interaction, aiming at making it applicable to large amounts of information. We present the design and the implementation of the web-based system Hippalus, which realizes the extended interaction model. Regarding user benefits, we theoretically analyze user gain in terms of the number and difficulty of choices, and we describe and analyze three user-based evaluations. The first investigates the degree of effectiveness of preferences when users are not aware of the available choices. We found that only 20% of the users managed to express effective preferences without knowing the available choices. The second comparatively evaluates FDT with other exploratory models, and shows that the majority of users preferred FDT, was more satisfied by FDT and achieved higher rates of task completion with FDT. The last one evaluates the preference-enriched FDT as realized by Hippalus. The results were impressive. Even in a very small dataset, with the preference-enriched FDT all users successfully completed all tasks in 1/3 of the time and with 1/3 of the actions in comparison to the plain FDT. Moreover all of the users preferred the preference-enriched interfac. Available: http://www.ics.forth.gr/_publications/Papadakos_Dissertation.pdf.

Promoting Precision Cancer Medicine through a Community-Driven Knowledgebase

Increasing efforts are being dedicated towards improving cancer care via personalized medicine. These efforts depend to a large degree on the availability of a knowledge foundation. Unfortunately, existing knowledge linking cancer drugs and potential efficacy biomarkers is in its infancy; and where links are known, they are frequently unstructured and poorly documented. We have developed a new open-access knowledgebase for precision cancer medicine (the PCM Wiki and Knowledgebase). This knowledgebase was constructed using an innovative, two-pronged approach involving a structured knowledgebase at the back-end, and an intuitive knowledge-sharing interface and user-friendly query engine in front. The knowledgebase was seeded with several patient case reports and information was mined via text-mining and literature review by human curators. Using our novel Wiki-based platform to present and share knowledge stored in the PCM knowledgebase, users are able to suggest corrections, propose additions or point to errors in the knowledgebase. The result is a community-driven evolving knowledgebase holding integrated and consolidated knowledge of markers and indications for personalized cancer medicine. We suggest that the PCM Knowledgebase and Wiki could serve as an important tool for the advancement of clinical trials and care in the field of precision cancer medicine.

INDREX: In-database relation extraction

The management of text data has a long-standing history in the human mankind. A particular common task is extracting relations from text. Typically, the user performs this task with two separate systems, a relation extraction system and an SQL-based query engine for analytical tasks. During this iterative analytical workflow, the user must frequently ship data between these systems. Worse, the user must learn to manage both systems. Therefore, end users often desire a single system for both analytical and relation extraction tasks.We propose INDREX, a system that provides a single and comprehensive view of the whole process combining both relation extraction and later exploitation with SQL. The system permits a data warehouse style extract-transform-load of generic relations extracted from text documents and can support additional text mining analysis libraries or systems. Once generic relations are loaded, the user can define SQL queries on the extracted relations to discover higher level semantics or to join them with other relational data.For executing this powerful task, our system extends the SQL-based analytical capabilities of a columnar-based massively parallel query processing engine with a broad set of user-defined functions and a data model that supports this task. Our white-box approach permits INDREX to benefit from built-in query optimization and indexing techniques of the underlaying query execution engine.Applications that support both text mining and analytical workflows leverage new analytical platforms based on the MapReduce framework and its open source Hadoop implementation. We compare our system against this base line. We measure execution times for common workflows and demonstrate orders of magnitude improvement in execution time using INDREX.

Helminth.net: expansions to Nematode.net and an introduction to Trematode.net.

Helminth.net (http://www.helminth.net) is the new moniker for a collection of databases: Nematode.net and Trematode.net. Within this collection we provide services and resources for parasitic roundworms (nematodes) and flatworms (trematodes), collectively known as helminths. For over a decade we have provided resources for studying nematodes via our veteran site Nematode.net (http://nematode.net). In this article, (i) we provide an update on the expansions of Nematode.net that hosts omics data from 84 species and provides advanced search tools to the broad scientific community so that data can be mined in a useful and user-friendly manner and (ii) we introduce Trematode.net, a site dedicated to the dissemination of data from flukes, flatworm parasites of the class Trematoda, phylum Platyhelminthes. Trematode.net is an independent component of Helminth.net and currently hosts data from 16 species, with information ranging from genomic, functional genomic data, enzymatic pathway utilization to microbiome changes associated with helminth infections. The databases’ interface, with a sophisticated query engine as a backbone, is intended to allow users to search for multi-factorial combinations of species’ omics properties. This report describes updates to Nematode.net since its last description in NAR, 2012, and also introduces and presents its new sibling site, Trematode.net.

Performance Modeling in CUDA Streams - A Means for High-Throughput Data Processing.

Push-based database management system (DBMS) is a new type of data processing software that streams large volume of data to concurrent query operators. The high data rate of such systems requires large computing power provided by the query engine. In our previous work, we built a push-based DBMS named G-SDMS to harness the unrivaled computational capabilities of modern GPUs. A major design goal of G-SDMS is to support concurrent processing of heterogenous query processing operations and enable resource allocation among such operations. Understanding the performance of operations as a result of resource consumption is thus a premise in the design of G-SDMS. With NVIDIA's CUDA framework as the system implementation platform, we present our recent work on performance modeling of CUDA kernels running concurrently under a runtime mechanism named CUDA stream. Specifically, we explore the connection between performance and resource occupancy of compute-bound kernels and develop a model that can predict the performance of such kernels. Furthermore, we provide an in-depth anatomy of the CUDA stream mechanism and summarize the main kernel scheduling disciplines in it. Our models and derived scheduling disciplines are verified by extensive experiments using synthetic and real-world CUDA kernels.

Adaptive query processing on RAW data

Database systems deliver impressive performance for large classes of workloads as the result of decades of research into optimizing database engines. High performance, however, is achieved at the cost of versatility. In particular, database systems only operate efficiently over loaded data, i.e., data converted from its original raw format into the system's internal data format. At the same time, data volume continues to increase exponentially and data varies increasingly, with an escalating number of new formats. The consequence is a growing impedance mismatch between the original structures holding the data in the raw files and the structures used by query engines for efficient processing. In an ideal scenario, the query engine would seamlessly adapt itself to the data and ensure efficient query processing regardless of the input data formats, optimizing itself to each instance of a file and of a query by leveraging information available at query time. Today's systems, however, force data to adapt to the query engine during data loading. This paper proposes adapting the query engine to the formats of raw data. It presents RAW, a prototype query engine which enables querying heterogeneous data sources transparently. RAW employs Just-In-Time access paths, which efficiently couple heterogeneous raw files to the query engine and reduce the overheads of traditional general-purpose scan operators. There are, however, inherent overheads with accessing raw data directly that cannot be eliminated, such as converting the raw values. Therefore, RAW also uses column shreds, ensuring that we pay these costs only for the subsets of raw data strictly needed by a query. We use RAW in a real-world scenario and achieve a two-order of magnitude speedup against the existing hand-written solution.

An effective encoding scheme for spatial RDF data

The RDF data model has recently been extended to support representation and querying of spatial information (i.e., locations and geometries), which is associated with RDF entities. Still, there are limited efforts towards extending RDF stores to efficiently support spatial queries, such as range selections (e.g., find entities within a given range) and spatial joins (e.g., find pairs of entities whose locations are close to each other). In this paper, we propose an extension for RDF stores that supports efficient spatial data management. Our contributions include an effective encoding scheme for entities having spatial locations, the introduction of on-the-fly spatial filters and spatial join algorithms, and several optimizations that minimize the overhead of geometry and dictionary accesses. We implemented the proposed techniques as an extension to the opensource RDF-3X engine and we experimentally evaluated them using real RDF knowledge bases. The results show that our system offers robust performance for spatial queries, while introducing little overhead to the original query engine.

Errata for "building efficient query engines in a high-level language"

This is in response to recent feedback from our peers that calls for a number of clarifications regarding the experimental section of our paper.

Big linked cancer data: Integrating linked TCGA and PubMed

The amount of bio-medical data available on the Web grows exponentially with time. The resulting large volume of data makes manual exploration very tedious. Moreover, the velocity at which this data changes and the variety of formats in which bio-medical data is published makes it difficult to access them in an integrated form. Finally, the lack of an integrated vocabulary makes querying this data more difficult. In this paper, we advocate the use of Linked Data to integrate, query and visualize bio-medical data. The resulting Big Linked Data allows discovering knowledge distributed across manifold sources, making it viable for the serendipitous discovery of novel knowledge. We present the concept of Big Linked Data by showing how the constant stream of new bio-medical publications can be integrated with the Linked Cancer Genome Atlas dataset (TCGA) within a virtual integration scenario. We ensure the scalability of our approach through the novel TopFed federated query engine, which we evaluate by comparing the query execution time of our system with that of FedX on Linked TCGA. Then, we show how we can harness the value hidden in the underlying integrated data by making it easier to explore through a user-friendly interface. We evaluate the usability of the interface by using the standard system usability questionnaire as well as a customized questionnaire designed for the users of our system. Our overall result of 77 suggests that our interface is easy to use and can thus lead to novel insights.

Concurrent analytical query processing with GPUs

In current databases, GPUs are used as dedicated accelerators to process each individual query. Sharing GPUs among concurrent queries is not supported, causing serious resource underutilization. Based on the profiling of an open-source GPU query engine running commonly used single-query data warehousing workloads, we observe that the utilization of main GPU resources is only up to 25%. The underutilization leads to low system throughput. To address the problem, this paper proposes concurrent query execution as an effective solution. To efficiently share GPUs among concurrent queries for high throughput, the major challenge is to provide software support to control and resolve resource contention incurred by the sharing. Our solution relies on GPU query scheduling and device memory swapping policies to address this challenge. We have implemented a prototype system and evaluated it intensively. The experiment results confirm the effectiveness and performance advantage of our approach. By executing multiple GPU queries concurrently, system throughput can be improved by up to 55% compared with dedicated processing.

Unleashing XQuery for Data-Independent Programming

The XQuery language was initially developed as an SQL equivalent for XML data, but its roots in functional programming make it also a perfect choice for processing almost any kind of structured and semi-structured data. Apart from standard XML processing, however, advanced language features make it hard to efficiently implement the complete language for large data volumes. This work proposes a novel compilation strategy that provides both flexibility and efficiency to unleash XQuery’s potential as data programming language. It combines the simplicity and versatility of a storage-independent data abstraction with the scalability advantages of set-oriented processing. Expensive iterative sections in a query are unrolled to a pipeline of relational-style operators, which is open for optimized join processing, index use, and parallelization. The remaining aspects of the language are processed in a standard fashion, yet can be compiled anytime to more efficient native operations of the actual runtime environment. This hybrid compilation mechanism yields an efficient and highly flexible query engine that is able to drive any computation from simple XML transformation to complex data analysis, even on non-XML data. Experiments with our prototype and state-of-the-art competitors in classic XML query processing and business analytics over relational data attest the generality and efficiency of the design.

Building efficient query engines in a high-level language

In this paper we advocate that it is time for a radical rethinking of database systems design. Developers should be able to leverage high-level programming languages without having to pay a price in efficiency. To realize our vision of abstraction without regret , we present LegoBase, a query engine written in the high-level programming language Scala. The key technique to regain efficiency is to apply generative programming: the Scala code that constitutes the query engine, despite its high-level appearance, is actually a program generator that emits specialized, low-level C code. We show how the combination of high-level and generative programming allows to easily implement a wide spectrum of optimizations that are difficult to achieve with existing low-level query compilers, and how it can continuously optimize the query engine. We evaluate our approach with the TPC-H benchmark and show that: (a) with all optimizations enabled, our architecture significantly outperforms a commercial in-memory database system as well as an existing query compiler, (b) these performance improvements require programming just a few hundred lines of high-level code instead of complicated low-level code that is required by existing query compilers and, finally, that (c) the compilation overhead is low compared to the overall execution time, thus making our approach usable in practice for efficiently compiling query engines.

PrefDB: Supporting Preferences as First-Class Citizens in Relational Databases

In this paper, we argue that preference-aware query processing needs to be pushed closer to the DBMS. We introduce a preference-aware relational data model that extends database tuples with preferences and an extended algebra that captures the essence of processing queries with preferences. Based on a set of algebraic properties and a cost model that we propose, we provide several query optimization strategies for extended query plans. Further, we describe a query execution algorithm that blends preference evaluation with query execution, while making effective use of the native query engine. We have implemented our framework and methods in a prototype system, PrefDB . PrefDB allows transparent and efficient evaluation of preferential queries on top of a relational DBMS. Our extensive experimental evaluation on two real-world datasets demonstrates the feasibility and advantages of our framework.

Application of Ontologies in Identifying Requirements Patterns in Use Cases

Use case specifications have successfully been used for requirements description. They allow joining, in the same modeling space, the expectations of the stakeholders as well as the needs of the software engineer and analyst involved in the process. While use cases are not meant to describe a system's implementation, by formalizing their description we are able to extract implementation relevant information from them. More specifically, we are interested in identifying requirements patterns (common requirements with typical implementation solutions) in support for a requirements based software development approach. In the paper we propose the transformation of Use Case descriptions expressed in a Controlled Natural Language into an ontology expressed in the Web Ontology Language (OWL). OWL's query engines can then be used to identify requirements patterns expressed as queries over the ontology. We describe a tool that we have developed to support the approach and provide an example of usage.

ENGLISH

Nowadays data is available in a single click that is from World Wide Web which is a huge repository of resources. Web search engines have a key role in the discovery of relevant information, but this kind of search is usually performed using keywords and the results into context less results. Result presented to user may contain context irrelevant data. This work is one step towards better understanding of user context from available domain specific resources. The contextualized strategy for information retrieval (IR) can be built around user profile, query expansion and relevance feedback. In this proposed system, new concept called terminology which is good representative of particular domain (or subject) is defined and used to classify resources as relevant or irrelevant (non- relevant). This work focuses on improving results of context sensitive web search engine based on shared resources. Query expansion using shared documents is applied to implement contextual search. These resources are ranked based on query submitted by user which brings most relevant document at the top in hierarchy. From top ranked documents, terms has been weighted to identify most related terms for query expansion. In addition the results of the query engine with and without the contextual information will be evaluated automatically without any interface of user.