Multiple Heterogeneous Data Sources Research Articles

BioTCM-SE: a semantic search engine for the information retrieval of modern biology and traditional Chinese medicine.

Understanding the functional mechanisms of the complex biological system as a whole is drawing more and more attention in global health care management. Traditional Chinese Medicine (TCM), essentially different from Western Medicine (WM), is gaining increasing attention due to its emphasis on individual wellness and natural herbal medicine, which satisfies the goal of integrative medicine. However, with the explosive growth of biomedical data on the Web, biomedical researchers are now confronted with the problem of large-scale data analysis and data query. Besides that, biomedical data also has a wide coverage which usually comes from multiple heterogeneous data sources and has different taxonomies, making it hard to integrate and query the big biomedical data. Embedded with domain knowledge from different disciplines all regarding human biological systems, the heterogeneous data repositories are implicitly connected by human expert knowledge. Traditional search engines cannot provide accurate and comprehensive search results for the semantically associated knowledge since they only support keywords-based searches. In this paper, we present BioTCM-SE, a semantic search engine for the information retrieval of modern biology and TCM, which provides biologists with a comprehensive and accurate associated knowledge query platform to greatly facilitate the implicit knowledge discovery between WM and TCM.

USING SEMANTIC WEB TECHNOLOGIES IN HETEROGENEOUS DISTRIBUTED DATABASE SYSTEM: A CASE STUDY FOR MANAGING ENERGY DATA ON MOBILE DEVICES

Managing energy data from multiple distributed and heterogeneous sources is an important issue worldwide. This study focused on using semantic web technologies in an energy data management system. Specifically, we provided a federated approach a mediator server that allows users to access to multiple heterogeneous data sources, including four typical types of databases in energy data resources: relational database, Triplestore, NoSQL database, and XML. A proposed architecture based on this approach is then presented and our solution can realize the data acquisition and integration without the need to rewrite or transform the local data into a unified data. We further examined our architecture by a case study in a Swiss energy company and tested the system based on

Multiple graph label propagation by sparse integration.

Graph-based approaches have been most successful in semisupervised learning. In this paper, we focus on label propagation in graph-based semisupervised learning. One essential point of label propagation is that the performance is heavily affected by incorporating underlying manifold of given data into the input graph. The other more important point is that in many recent real-world applications, the same instances are represented by multiple heterogeneous data sources. A key challenge under this setting is to integrate different data representations automatically to achieve better predictive performance. In this paper, we address the issue of obtaining the optimal linear combination of multiple different graphs under the label propagation setting. For this problem, we propose a new formulation with the sparsity (in coefficients of graph combination) property which cannot be rightly achieved by any other existing methods. This unique feature provides two important advantages: 1) the improvement of prediction performance by eliminating irrelevant or noisy graphs and 2) the interpretability of results, i.e., easily identifying informative graphs on classification. We propose efficient optimization algorithms for the proposed approach, by which clear interpretations of the mechanism for sparsity is provided. Through various synthetic and two real-world data sets, we empirically demonstrate the advantages of our proposed approach not only in prediction performance but also in graph selection ability.

An adaptable architecture for patient cohort identification from diverse data sources

We define and validate an architecture for systems that identify patient cohorts for clinical trials from multiple heterogeneous data sources. This architecture has an explicit query model capable of supporting temporal reasoning and expressing eligibility criteria independently of the representation of the data used to evaluate them. The architecture has the key feature that queries defined according to the query model are both pre and post-processed and this is used to address both structural and semantic heterogeneity. The process of extracting the relevant clinical facts is separated from the process of reasoning about them. A specific instance of the query model is then defined and implemented. We show that the specific instance of the query model has wide applicability. We then describe how it is used to access three diverse data warehouses to determine patient counts. Although the proposed architecture requires greater effort to implement the query model than would be the case for using just SQL and accessing a data-based management system directly, this effort is justified because it supports both temporal reasoning and heterogeneous data sources. The query model only needs to be implemented once no matter how many data sources are accessed. Each additional source requires only the implementation of a lightweight adaptor. The architecture has been used to implement a specific query model that can express complex eligibility criteria and access three diverse data warehouses thus demonstrating the feasibility of this approach in dealing with temporal reasoning and data heterogeneity.

Information Retrieval Architecture for Heterogeneous Big Data on Situation Awareness

The paper proposes the model of the data integrations systems with which the buyer/user searches and aggregates organizational entity from multiple heterogeneous data sources in order to have a contract with the proper entity for the new project. The requirement for the systems is to retrieve and integrate heterogeneous data which is non- or semi-structured. And, the systems need to be alerted when the data is changed. The proposed model supports services to retrieve and integrate an entity(s) and it also alerts the update of the entities to the user. Besides, the systems store and compute Big Data using Hadoop platform.

A unified structural/terminological interoperability framework based on LexEVS: application to TRANSFoRm.

ObjectiveBiomedical research increasingly relies on the integration of information from multiple heterogeneous data sources. Despite the fact that structural and terminological aspects of interoperability are interdependent and rely on a common set of requirements, current efforts typically address them in isolation. We propose a unified ontology-based knowledge framework to facilitate interoperability between heterogeneous sources, and investigate if using the LexEVS terminology server is a viable implementation method.Materials and methodsWe developed a framework based on an ontology, the general information model (GIM), to unify structural models and terminologies, together with relevant mapping sets. This allowed a uniform access to these resources within LexEVS to facilitate interoperability by various components and data sources from implementing architectures.ResultsOur unified framework has been tested in the context of the EU Framework Program 7 TRANSFoRm project, where it was used to achieve data integration in a retrospective diabetes cohort study. The GIM was successfully instantiated in TRANSFoRm as the clinical data integration model, and necessary mappings were created to support effective information retrieval for software tools in the project.ConclusionsWe present a novel, unifying approach to address interoperability challenges in heterogeneous data sources, by representing structural and semantic models in one framework. Systems using this architecture can rely solely on the GIM that abstracts over both the structure and coding. Information models, terminologies and mappings are all stored in LexEVS and can be accessed in a uniform manner (implementing the HL7 CTS2 service functional model). The system is flexible and should reduce the effort needed from data sources personnel for implementing and managing the integration.

GBC: Gradient boosting consensus model for heterogeneous data†

With the rapid development of database technologies, multiple data sources may be available for a given learning task (e.g. collaborative filtering). However, the data sources may contain different types of features. For example, users' profiles can be used to build recommendation systems. In addition, a model can also use users' historical behaviors and social networks to infer users' interests on related products. We argue that it is desirable to collectively use any available multiple heterogeneous data sources in order to build effective learning models. We call this framework heterogeneous learning. In our proposed setting, data sources can include (i) nonoverlapping features, (ii) nonoverlapping instances, and (iii) multiple networks (i.e. graphs) that connect instances. In this paper, we propose a general optimization framework for heterogeneous learning, and devise a corresponding learning model from gradient boosting. The idea is to minimize the empirical loss with two constraints: (1) there should be consensus among the predictions of overlapping instances (if any) from different data sources; (2) connected instances in graph datasets may have similar predictions. The objective function is solved by stochastic gradient boosting trees. Furthermore, a weighting strategy is designed to emphasize informative data sources, and deemphasize the noisy ones. We formally prove that the proposed strategy leads to a tighter error bound. This approach consistently outperforms a standard concatenation of data sources on movie rating prediction, number recognition, and terrorist attack detection tasks. Furthermore, the approach is evaluated on AT&T's distributed database with over 500 000 instances, 91 different data sources, and over 45 000 000 joined features. We observe that the proposed model can improve out-of-sample error rate substantially.

Query answering under probabilistic uncertainty in Datalog+ / − ontologies

The recently introduced Datalog+�/�� family of ontology languages is especially useful for representing and reasoning over lightweight ontologies, and is set to play a central role in the context of query answering and information extraction for the Semantic Web. Recently, it has become apparent that it is necessary to develop a principled way to handle uncertainty in this domain. In addition to uncertainty as an inherent aspect of the Web, one must also deal with forms of uncertainty due to inconsistency and incompleteness, uncertainty resulting from automatically processing Web data, as well as uncertainty stemming from the integration of multiple heterogeneous data sources. In this paper, we take an important step in this direction by developing a probabilistic extension of Datalog+�/��. This extension uses Markov logic networks as the underlying probabilistic semantics. Here, we focus especially on scalable algorithms for answering threshold queries, which correspond to the question "what is the set of all ground atoms that are inferred from a given probabilistic ontology with a probability of at least p?". These queries are especially relevant to Web information extraction, since uncertain rules lead to uncertain facts, and only information with a certain minimum confidence is desired. We present several algorithms, namely a basic approach, an anytime one, and one based on heuristics, which is guaranteed to return sound results. Furthermore, we also study inconsistency in probabilistic Datalog+�/�� ontologies. We propose two approaches for computing preferred repairs based on two different notions of distance between repairs, namely symmetric and score-based distance. We also study the complexity of the decision problems corresponding to computing such repairs, which turn out to be polynomial and NP-complete in the data complexity, respectively.

Ontology Based Knowledge Processing in Condition Based Maintenance System

Many capital-intensive industries such as iron and steel, energy, refining, petrochemical and manufacturing companies have the pressure of continuous increase in maintenance costs. In recent years, condition based maintenance becomes an important tool for reducing maintenance costs in these industries. It is a very information intensive domain. Good condition based maintenance systems need to integrate multiple heterogeneous data sources. In this paper, we will use ontology to describe the semantics of condition based maintenance concepts. It serves two main purposes: (1) offering a common understanding of the condition based maintenance domain, and (2) the knowledge held in the ontology is machine-readable and explicit, thus making the knowledge easy to be processed and reused.

A Service-Oriented Distributed Semantic Mediator: Integrating Multiscale Biomedical Information

Biomedical research continuously generates large amounts of heterogeneous and multimodal data spread over multiple data sources. These data, if appropriately shared and exploited, could dramatically improve the research practice itself, and ultimately the quality of health care delivered. This paper presents DISMED (DIstributed Semantic MEDiator), an open source semantic mediator that provides a unified view of a federated environment of multiscale biomedical data sources. DISMED is a Web-based software application to query and retrieve information distributed over a set of registered data sources, using semantic technologies. It also offers a userfriendly interface specifically designed to simplify the usage of these technologies by non-expert users. Although the architecture of the software mediator is generic and domain independent, in the context of this paper, DISMED has been evaluated for managing biomedical environments and facilitating research with respect to the handling of scientific data distributed in multiple heterogeneous data sources. As part of this contribution, a quantitative evaluation framework has been developed. It consist of a benchmarking scenario and the definition of five realistic use-cases. This framework, created entirely with public datasets, has been used to compare the performance of DISMED against other available mediators. It is also available to the scientific community in order to evaluate progress in the domain of semantic mediation, in a systematic and comparable manner. The results show an average improvement in the execution time by DISMED of 55% compared to the second best alternative in four out of the five use-cases of the experimental evaluation.

Virtual Drug Screen Schema Based on Multiview Similarity Integration and Ranking Aggregation

The current drug virtual screen (VS) methods mainly include two categories. i.e., ligand/target structure-based virtual screen and that, utilizing protein-ligand interaction fingerprint information based on the large number of complex structures. Since the former one focuses on the one-side information while the later one focuses on the whole complex structure, they are thus complementary and can be boosted by each other. However, a common problem faced here is how to present a comprehensive understanding and evaluation of the various virtual screen results derived from various VS methods. Furthermore, there is still an urgent need for developing an efficient approach to fully integrate various VS methods from a comprehensive multiview perspective. In this study, our virtual screen schema based on multiview similarity integration and ranking aggregation was tested comprehensively with statistical evaluations, providing several novel and useful clues on how to perform drug VS from multiple heterogeneous data sources. (1) 18 complex structures of HIV-1 protease with ligands from the PDB were curated as a test data set and the VS was performed with five different drug representations. Ritonavir ( 1HXW ) was selected as the query in VS and the weighted ranks of the query results were aggregated from multiple views through four similarity integration approaches. (2) Further, one of the ranking aggregation methods was used to integrate the similarity ranks calculated by gene ontology (GO) fingerprint and structural fingerprint on the data set from connectivity map, and two typical HDAC and HSP90 inhibitors were chosen as the queries. The results show that rank aggregation can enhance the result of similarity searching in VS when two or more descriptions are involved and provide a more reasonable similarity rank result. Our study shows that integrated VS based on multiple data fusion can achieve a remarkable better performance compared to that from individual ones and, thus, serves as a promising way for efficient drug screening, taking advantages of the rapidly accumulated molecule representations and heterogeneous data in the pharmacological area.

Access Control Policy Translation, Verification, and Minimization within Heterogeneous Data Federations

Data federations provide seamless access to multiple heterogeneous and autonomous data sources pertaining to a large organization. As each source database defines its own access control policies for a set of local identities, enforcing such policies across the federation becomes a challenge. In this article, we first consider the problem of translating existing access control policies defined over source databases in a manner that allows the original semantics to be observed while becoming applicable across the entire data federation. We show that such a translation is always possible, and provide an algorithm for automating the translation. We show that verifying whether a translated policy obeys the semantics of the original access control policy defined over a source database is intractable, even under restrictive scenarios. We then describe a practical algorithmic framework for translating relational access control policies into their XML equivalent, expressed in the eXtensible Access Control Markup Language. Finally, we examine the difficulty of minimizing translated policies, and contribute a minimization algorithm applicable to nonrecursive translated policies.

A mediator-based approach for integrating heterogeneous multimedia sources

In many applications, the information required by the user cannot be found in just one source, but has to be retrieved from many varying sources. This is true not only of formatted data in database management systems, but also of textual documents and multimedia data, such as images and videos. We propose a mediator system that provides the end-user with a single query interface to an integrated view of multiple heterogeneous data sources. We exploit the capabilities of the MOMIS integration system and the MILOS multimedia data management system. Each multimedia source is managed by an instance of MILOS, in which a collection of multimedia records is made accessible by means of similarity searches employing the query-by-example paradigm. MOMIS provides an integrated virtual view of the underlying multimedia sources, thus offering unified multimedia access services. Two features are that MILOS is flexible--it is not tied to any particular similarity function--and the MOMIS's mediator query processor only exploits the ranks of the local answers.

In Silico Gene Prioritization by Integrating Multiple Data Sources

Identifying disease genes is crucial to the understanding of disease pathogenesis, and to the improvement of disease diagnosis and treatment. In recent years, many researchers have proposed approaches to prioritize candidate genes by considering the relationship of candidate genes and existing known disease genes, reflected in other data sources. In this paper, we propose an expandable framework for gene prioritization that can integrate multiple heterogeneous data sources by taking advantage of a unified graphic representation. Gene-gene relationships and gene-disease relationships are then defined based on the overall topology of each network using a diffusion kernel measure. These relationship measures are in turn normalized to derive an overall measure across all networks, which is utilized to rank all candidate genes. Based on the informativeness of available data sources with respect to each specific disease, we also propose an adaptive threshold score to select a small subset of candidate genes for further validation studies. We performed large scale cross-validation analysis on 110 disease families using three data sources. Results have shown that our approach consistently outperforms other two state of the art programs. A case study using Parkinson disease (PD) has identified four candidate genes (UBB, SEPT5, GPR37 and TH) that ranked higher than our adaptive threshold, all of which are involved in the PD pathway. In particular, a very recent study has observed a deletion of TH in a patient with PD, which supports the importance of the TH gene in PD pathogenesis. A web tool has been implemented to assist scientists in their genetic studies.

Federated Querying Architecture with Clinical & Translational Health IT Application

We present a software architecture that federates data from multiple heterogeneous health informatics data sources owned by multiple organizations. The architecture builds upon state-of-the-art open-source Java and XML frameworks in innovative ways. It consists of (a) federated query engine, which manages federated queries and result set aggregation via a patient identification service; and (b) data source facades, which translate the physical data models into a common model on-the-fly and handle large result set streaming. System modules are connected via reusable Apache Camel integration routes and deployed to an OSGi enterprise service bus. We present an application of our architecture that allows users to construct queries via the i2b2 web front-end, and federates patient data from the University of Utah Enterprise Data Warehouse and the Utah Population database. Our system can be easily adopted, extended and integrated with existing SOA Healthcare and HL7 frameworks such as i2b2 and caGrid.

Gene ontology based transfer learning for protein subcellular localization

BackgroundPrediction of protein subcellular localization generally involves many complex factors, and using only one or two aspects of data information may not tell the true story. For this reason, some recent predictive models are deliberately designed to integrate multiple heterogeneous data sources for exploiting multi-aspect protein feature information. Gene ontology, hereinafter referred to as GO, uses a controlled vocabulary to depict biological molecules or gene products in terms of biological process, molecular function and cellular component. With the rapid expansion of annotated protein sequences, gene ontology has become a general protein feature that can be used to construct predictive models in computational biology. Existing models generally either concatenated the GO terms into a flat binary vector or applied majority-vote based ensemble learning for protein subcellular localization, both of which can not estimate the individual discriminative abilities of the three aspects of gene ontology.ResultsIn this paper, we propose a Gene Ontology Based Transfer Learning Model (GO-TLM) for large-scale protein subcellular localization. The model transfers the signature-based homologous GO terms to the target proteins, and further constructs a reliable learning system to reduce the adverse affect of the potential false GO terms that are resulted from evolutionary divergence. We derive three GO kernels from the three aspects of gene ontology to measure the GO similarity of two proteins, and derive two other spectrum kernels to measure the similarity of two protein sequences. We use simple non-parametric cross validation to explicitly weigh the discriminative abilities of the five kernels, such that the time & space computational complexities are greatly reduced when compared to the complicated semi-definite programming and semi-indefinite linear programming. The five kernels are then linearly merged into one single kernel for protein subcellular localization. We evaluate GO-TLM performance against three baseline models: MultiLoc, MultiLoc-GO and Euk-mPLoc on the benchmark datasets the baseline models adopted. 5-fold cross validation experiments show that GO-TLM achieves substantial accuracy improvement against the baseline models: 80.38% against model Euk-mPLoc 67.40% with 12.98% substantial increase; 96.65% and 96.27% against model MultiLoc-GO 89.60% and 89.60%, with 7.05% and 6.67% accuracy increase on dataset MultiLoc plant and dataset MultiLoc animal, respectively; 97.14%, 95.90% and 96.85% against model MultiLoc-GO 83.70%, 90.10% and 85.70%, with accuracy increase 13.44%, 5.8% and 11.15% on dataset BaCelLoc plant, dataset BaCelLoc fungi and dataset BaCelLoc animal respectively. For BaCelLoc independent sets, GO-TLM achieves 81.25%, 80.45% and 79.46% on dataset BaCelLoc plant holdout, dataset BaCelLoc plant holdout and dataset BaCelLoc animal holdout, respectively, as compared against baseline model MultiLoc-GO 76%, 60.00% and 73.00%, with accuracy increase 5.25%, 20.45% and 6.46%, respectively.ConclusionsSince direct homology-based GO term transfer may be prone to introducing noise and outliers to the target protein, we design an explicitly weighted kernel learning system (called Gene Ontology Based Transfer Learning Model, GO-TLM) to transfer to the target protein the known knowledge about related homologous proteins, which can reduce the risk of outliers and share knowledge between homologous proteins, and thus achieve better predictive performance for protein subcellular localization. Cross validation and independent test experimental results show that the homology-based GO term transfer and explicitly weighing the GO kernels substantially improve the prediction performance.

Efficient Tree Based Distributed Data Mining Algorithms for mining Frequent Patterns

Advancements in the field of wired and wireless network environments have paved route to the advent of many dynamic distributed computing environments. These environments have diverged computing resources and multiple heterogeneous sources of data. Most mining algorithms are designed to mine rules from monolithic non-distributed databases. Even algorithms exclusively designed to operate on distributed databases normally download the relevant data to a centralized location and then perform the data mining operations. This centralized approach does not work well in many of the distributed, ubiquitous, privacy sensitive data mining applications, which opened a new area of research Distributed Data Mining (DDM) under the data mining domain. Out of various methods employed to mine frequent Itemsets, tree based methodology proves some efficiency in distributed environment. So in this paper we study a set of tree based algorithms [DTFIM, PP, LFP and PP] to mine frequent pattern in distributed environment.

European Mathematical Genetics Meeting, Heidelberg, Germany, 12th–13th April 2007

European Mathematical Genetics Meeting, Heidelberg, Germany, 12^th–13^th April 2007

In silico candidate gene mining in livestock species

The identification of genes involved in livestock production and disease is a challenge due to the multi-genic, multifactorial nature of the traits and the complexity of integration of information from different studies. Genome-wide techniques such as microarray analysis, SAGE, linkage analysis and linkage disequilibrium analysis have been extensively used in livestock and have often identified a large number of candidate genes. Selection of the most probable candidate genes for further empirical analysis remains a challenge. Novel extensive biological databases (DB) provide an opportunity for candidate gene mining. Bioinformatic methods and tools to prioritize candidate genes underlying pathways or diseases have been presented mostly for application to human disease candidate gene search. These computational methods employ data from a variety of sources to identify the most likely candidate genes from genes sets. The objectives of the study were: 1. to test a set of existing gene prioritization computational methods on real and simulated livestock traits, namely mastitis susceptibility in cattle, production in cattle, litter size in swine, and tick resistance in cattle; 2. to develop a novel method for candidate prioritization that better suits the characteristics of genomic information of livestock species (lower level of annotation, different experimental designs, etc.). The algorithm performs distinct prioritizations from multiple heterogeneous data sources, which are then integrated into one global ranking using order statistics. Information about a trait or pathway is recorded, ordered and stored from a set of known genes using multiple data sources. Then, the candidate genes are ranked based on similarity with the training properties obtained in the first step, resulting in one prioritized list for each data source. Data for linkage and association analysis, and expression analysis (2 microarray studies) were simulated for a complex trait assigning the largest effects to known, well described genes in the Gene Ontology (GO), InterPro, MEDLINE, and Kegg databases. All the other simulated major genes were assigned to genes described in one of the databases for livestock species. Real data were selected from the literature, mainly from large QTL studies, or obtained from collaborators. Software used for the analysis were: Suspect (http://www.genetics.med.ed.ac.uk/software/prospectr.php), Endevour and COeXpress (http://coxpress.sf.net). Results using simulated data showed that where annotation is missing, the accuracy of the considered algorithms decreased drastically. A new prioritization method applied to simulated data correctly ranked candidate gene only when QTL information had a high level of accuracy. Further work is needed in defining methods of weighting QTL data information.

A compensation-based approach for view maintenance in distributed environments

Data integration over multiple heterogeneous data sources has become increasingly important for modern applications. The integrated data is usually stored as materialized views to allow better access, performance, and high availability. In loosely coupled environments, such as the data grid, the data sources are autonomous. Hence, tie source updates can be concurrent and cause erroneous results during view maintenance. State-of-the-art maintenance strategies apply compensating queries to correct such errors, making the restricting assumption that all source schemata remain static over time. However, in such dynamic environments, the data sources may change not only their data but also their schema. Consequently, either the maintenance queres or the compensating queries may fail. In this paper, we propose a novel framework called DyDa that overcomes these limitations and handles both source data updates and schema changes. We identify three types of maintenance anomalies, caused by either source data updates, data-preserving schema changes, or non-data-preserving schema changes. We propose a compensation algorithm to solve the first two types of anomalies. We show that the third type of anomaly is caused by the violation of dependencies between maintenance processes. Then, we propose dependency detection and correction algorithms to identify and resolve the violations. Put together, DyDa extends prior maintenance solutions to solve all types of view maintenance anomalies. The experimental results show that DyDa imposes a minimal overhead on data update processing while allowing for the extended functionality to handle concurrent schema changes