Structured Data Sources Research Articles

A framework for enriching Data Warehouse analysis with Question Answering systems

Business Intelligence (BI) applications allow their users to query, understand, and analyze existing data within their organizations in order to acquire useful knowledge, thus making better strategic decisions. The core of BI applications is a Data Warehouse (DW), which integrates several heterogeneous structured data sources in a common repository of data. However, there is a common agreement in that the next generation of BI applications should consider data not only from their internal data sources, but also data from different external sources (e.g. Big Data, blogs, social networks, etc.), where relevant update information from competitors may provide crucial information in order to take the right decisions. This external data is usually obtained through traditional Web search engines, with a significant effort from users in analyzing the returned information and in incorporating this information into the BI application. In this paper, we propose to integrate the DW internal structured data, with the external unstructured data obtained with Question Answering (QA) techniques. The integration is achieved seamlessly through the presentation of the data returned by the DW and the QA systems into dashboards that allow the user to handle both types of data. Moreover, the QA results are stored in a persistent way through a new DW repository in order to facilitate comparison of the obtained results with different questions or even the same question with different dates.

PS1-14: Pregnancy Data in Clarity - Stay Tuned for Our Next Episode

Background/Aims For a condition that is so easily defined, pregnancy can be remarkably difficult to locate with any precision in existing structured data sources, especially while a woman is still pregnant. Researchers may require pregnancy information: (a) to avoid recruiting pregnant women to clinical studies; (b) to recruit women in a certain trimester or meeting certain pregnancy-related criteria; (c) to exclude measures taken during pregnancy; (d) to evaluate prenatal care across populations; (e) to provide conditional variables for other analyses, including neonatal outcomes. In 2004, Group Health (GH) adopted Epic’s electronic medical record system. In this poster, we investigate Clarity, Epic’s reporting database, to determine its usefulness as a source of structured pregnancy data.

Dynamic Non-Cooperative Structured Deep Web Selection

Most of structured deep web data sources are non-cooperation, therefore establish an accurate data source content summary by sampling is the core technology of data source selection. The content of deep web data source is updated from time to time, however existing efficient methods of non-cooperation structured data source selection does not consider summary update problem. Unrenewed summary of data source can not accurately characterize the content of the data source that produce a greater impact on data source selection. Base on this, we propose a dynamic data source selection method for non-cooperative structured deep web by combining subject headings sampling technology and subject headings extension technology. The experiment results show that our dynamic structured data source selection method has good recall ratio and precision besides being efficient.

A low redundancy strategy for keyword search in structured and semi-structured data

Keyword Search has been recognised as a viable alternative for information search in semi-structured and structured data sources. Current state-of-the-art keyword-search techniques over relational databases do not take advantage of correlative meta-information included in structured and semi-structured data sources leaving relevant answers out. These techniques are also limited due to scalability, performance and precision issues that are evident when they are implemented on large datasets. Based on an in-depth analysis of issues related to indexing and ranking semi-structured and structured information. We propose a new keyword-search algorithm that takes into account the semantic information extracted from the schemes of the structured and semi-structured data sources and combine it with the textual relevance obtained by a common text retrieval approach. The algorithm is implemented in a keyword-based search engine called KESOSASD (Keyword Search Over Semi-structured and Structured Data), improving its precision and response time. Our approach models the semi-structured and structured information as graphs, and make use of a Virtual Document Structure Aware Inverted Index (VDSAII). This index is created from a set of logical structures called Virtual Documents, which capture and exploit the implicit structural relationships (semantics) depicted in the schemas of the structured and semi-structured data sources. Extensive experiments were conducted to demonstrate that KESOSASD outperforms existing approaches in terms of search efficiency and accuracy. Moreover, KESOSASD is prepared to scale out and manage large databases without degrading its effectiveness.

Improving the Quality of Linked Data Using Statistical Distributions

Linked Data on the Web is either created from structured data sources (such as relational databases), from semi-structured sources (such as Wikipedia), or from unstructured sources (such as text). In the latter two cases, the generated Linked Data will likely be noisy and incomplete. In this paper, we present two algorithms that exploit statistical distributions of properties and types for enhancing the quality of incomplete and noisy Linked Data sets: SDType adds missing type statements, and SDValidate identifies faulty statements. Neither of the algorithms uses external knowledge, i.e., they operate only on the data itself. We evaluate the algorithms on the DBpedia and NELL knowledge bases, showing that they are both accurate as well as scalable. Both algorithms have been used for building the DBpedia 3.9 release: With SDType, 3.4 million missing type statements have been added, while using SDValidate, 13,000 erroneous RDF statements have been removed from the knowledge base.

Development of reusable logic for determination of statin exposure-time from electronic health records

ObjectiveWe aim to quantify HMG-CoA reductase inhibitor (statin) prescriber-intended exposure-time using a generalizable algorithm that interrogates data stored in the electronic health record (EHR). Materials and methodsThis study was conducted using the Marshfield Clinic (MC) Personalized Medicine Research Project (PMRP) a central Wisconsin-based population and biobank with, on average, 30years of electronic health data available in the independently-developed MC Cattails MD EHR. Individuals with evidence of statin exposure were identified from the electronic records, and manual chart abstraction of all mentions of prescribed statins was completed. We then performed electronic chart abstraction of prescriber-intended exposure time for statins, using previously identified logic to capture pill-splitting events, normalizing dosages to atorvastatin-equivalent dose. Four models using iterative training sets were tested to capture statin end-dates. Calculated cumulative provider-intended exposures were compared to manually abstracted gold-standard measures of ordered statin prescriptions, and aggregate model results (totals) for training and validation populations were compared. The most successful model was the one with the smallest discordance between modeled and manually abstracted Atorvastatin 10mg/year Equivalents (AEs). ResultsOf the approximately 20,000 patients enrolled in the PMRP, 6243 were identified with statin exposure during the study period (1997–2011), 59.8% of whom had been prescribed multiple statins over an average of approximately 11years. When the best-fit algorithm was implemented and validated by manual chart review for the statin-ordered population, it was found to capture 95.9% of the correlation between calculated and expected statin provider-intended exposure time for a random validation set, and the best-fit model was able to predict intended statin exposure to within a standard deviation of 2.6 AEs, with a standard error of +0.23 AEs. ConclusionWe demonstrate that normalized provider-intended statin exposure time can be estimated using a combination of structured clinical data sources, including a medications ordering system and a clinical appointment coordination system, supplemented with text data from clinical notes.

DATA WAREHOUSE AND MASTER DATA MANAGEMENT EVOLUTION A META-DATA-VAULT APPROACH

The paper presents a: a) brief overview and analysis of existing approaches to the data warehouse (DW) evolution problem, and b) detailed description of the research idea for the DW evolution problem (primarily intended for structured data sources and realizations with relational database engines). We observe the DW evolution problem as a double issue - from the DW perspective, and from the master data management (MDM) perspective. The proposed general solution will include a Data Vault (DV) model based metadata repository that will integrate the DW data and metadata with the MDM data and metadata. This historicized metadata repository will manage schema versions and support schema changes. We believe this integration will: a) increase the quality of data in the Enterprise Data Warehouse (EDW), b) increase the quality of the metadata in the EDW, and c) increase the simplicity and efficiency of the DW and the MDM schema evolution.

Implementation of Ontology Learning and Population System from Structured Data Sources: Standard-based Approach

This paper proposes an ontology learning and population model from structured data sources. Recently various attempts have been made to harmonize Web 2.0 and the Semantic Web, named as Web 3.0 or Web 4.0. One of the most important issues for realization of the next Web platform is about how to make Web ontology rich as well as to address semantic interoperability between ontologies. To resolve those issues, Web ontology schemas should be precisely defined in semantic aspect and we should also develop a method for learning and population of ontology instances from diverse resources. This paper proposes a model which enables the ontology learning and population from structured data sources. The proposed model in this paper is a standard-based approach. In other words, our proposal is based on ISO/IEC 11179 - Metadata Registry, which is one of the international standards. This standard has been developed to support the interoperability between data by managing standardized common concepts. Therefore, our proposed model enhances the semantic interoperability between ontologies and enables the ontology enrichment from data sources.

Scalable column concept determination for web tables using large knowledge bases

Tabular data on the Web has become a rich source of structured data that is useful for ordinary users to explore. Due to its potential, tables on the Web have recently attracted a number of studies with the goals of understanding the semantics of those Web tables and providing effective search and exploration mechanisms over them. An important part of table understanding and search is column concept determination, i.e., identifying the most appropriate concepts associated with the columns of the tables. The problem becomes especially challenging with the availability of increasingly rich knowledge bases that contain hundreds of millions of entities. In this paper, we focus on an important instantiation of the column concept determination problem, namely, the concepts of a column are determined by fuzzy matching its cell values to the entities within a large knowledge base. We provide an efficient and scalable MapReduce-based solution that is scalable to both the number of tables and the size of the knowledge base and propose two novel techniques: knowledge concept aggregation and knowledge entity partition. We prove that both the problem of finding the optimal aggregation strategy and that of finding the optimal partition strategy are NP-hard, and propose efficient heuristic techniques by leveraging the hierarchy of the knowledge base. Experimental results on real-world datasets show that our method achieves high annotation quality and performance, and scales well.

Structures of Biomolecular Complexes from Heterogeneous Data and Bayesian Data Analysis

To characterize 3D structures of large and often flexible macromolecular complexes, multiple sources of structural data at multiple resolutions need to be combined. Integrating these data into one consistent picture poses particular difficulties: data are much more sparse than in high resolution methods; data sets from heterogeneous sources are of highly different and unknown quality and information content and may be mutually inconsistent; data are in general averaged over large ensembles and long measurement times. Also pre-existing structural knowledge is of different quality, ranging from high-resolution structures over homology models to low resolution models.We will outline a general framework, principally based on Bayesian probability theory. Appropriate models for the major data types used in hybrid approaches (electron microscopy, cross-linking/ mass spectrometry, various spectroscopy techniques, SAXS, ..) need to be developed, as well as representations to include structural knowledge for individual components of the complexes. We are working towards a multi-scale and multi-technique version of the approach we introduced for NMR (Rieping et al., Science 309, 303-305, 2005), implemented in the program ISD. We present examples with data from SAXS and from chemical cross-linking / mass spectrometry.

Antipsychotic Drug Use—Understanding the Recent Attention and Response to the Increased Scrutiny

Increasingly, health care providers are adopting population health management approaches that address the social determinants of health (SDH). However, effectively identifying patients needing services that address a SDH in primary care settings is challenging. The purpose of the current study is to explore how various data sources can identify adult primary care patients that are in need of services that address SDH.A cross-sectional study described patients in need of SDH services offered by a safety-net hospital’s federally qualified health center clinics. SDH services of social work, behavioral health, nutrition counseling, respiratory therapy, financial planning, medical-legal partnership assistance, patient navigation, and pharmacist consultation were offered on a co-located basis and were identified using structured billing and scheduling data, and unstructured electronic health record data. We report the prevalence of the eight different SDH service needs and the patient characteristics associated with service need. Moreover, characteristics of patients with SDH services need documented in structured data sources were compared with those documented by unstructured data sources.More than half (53%) of patients needed SDH services. Those in need of such services tended to be female, older, more medically complex, and higher utilizers of services. Structured and unstructured data sources exhibited poor agreement on patient SDH services need. Patients with SDH services need documented by unstructured data tended to be more complex.The need for SDH services among a safety-net population is high. Identifying patients in need of such services requires multiple data sources with structured and unstructured data.

Exploiting evidence from unstructured data to enhance master data management

Master data management (MDM) integrates data from multiple structured data sources and builds a consolidated 360-degree view of business entities such as customers and products. Today's MDM systems are not prepared to integrate information from unstructured data sources, such as news reports, emails, call-center transcripts, and chat logs. However, those unstructured data sources may contain valuable information about the same entities known to MDM from the structured data sources. Integrating information from unstructured data into MDM is challenging as textual references to existing MDM entities are often incomplete and imprecise and the additional entity information extracted from text should not impact the trustworthiness of MDM data.In this paper, we present an architecture for making MDM text-aware and showcase its implementation as IBM Info-Sphere MDM Extension for Unstructured Text Correlation, an add-on to IBM InfoSphere Master Data Management Standard Edition. We highlight how MDM benefits from additional evidence found in documents when doing entity resolution and relationship discovery. We experimentally demonstrate the feasibility of integrating information from unstructured data sources into MDM.

Receptor Binding by Influenza Virus: Using Computational Techniques To Extend Structural Data

Influenza attaches to host cells via hemagglutinin binding of cell-surface glycans. These relatively low-affinity interactions involving flexible ligands are critical in determining tissue and host specificity, but their dynamic nature complicates structural characterization of hemagglutinin-receptor complexes. Molecular simulation can assist in analyzing glycan and protein flexibility in crystallized complexes, assessing how binding might change under mutation or altered glycosylation patterns, and evaluating how soluble ligands may relate to physiological presentation on the plasma membrane. Molecular dynamics simulation also has the potential to help integrate structural and dynamic data sources. Here we review recent progress from analysis of molecular dynamics simulation and outline challenges for the future.

Dipolar Recoupling in Magic Angle Spinning Solid-State Nuclear Magnetic Resonance

Solid-state nuclear magnetic resonance (SSNMR) magic angle spinning (MAS) can be used to record high-resolution data dominated by site-specific information. Although MAS introduces high resolution by attenuating the anisotropic broadening, it also suppresses the nuclear dipole-dipole distance information that is the source of most structural data in the spectra. Such information can be reintroduced coherently and thus selectively by the application of a carefully chosen sequence of radiofrequency pulses, an approach that was introduced 20 years ago and is referred to as dipolar recoupling. This review presents the establishment of recoupling techniques in SSNMR and recalls the major steps achieved by the community throughout the last two decades. This review also presents emerging techniques and their corresponding new concepts. Finally, we present some recent developments based on second-order recoupling mechanisms and discuss their implications regarding dipolar truncation and the possibility to extract structural constraints in uniformly labeled systems.

Frontiers in Molecular Dynamics Simulations of DNA

It has been known for decades that DNA is extremely flexible and polymorphic, but our knowledge of its accessible conformational space remains limited. Structural data, primarily from X-ray diffraction studies, is sparse in comparison to the manifold configurations possible, and direct experimental examinations of DNA's flexibility still suffer from many limitations. In the face of these shortcomings, molecular dynamics (MD) is now an essential tool in the study of DNA. It affords detailed structural and dynamical insights, which explains its recent transition from a small number of highly specialized laboratories to a large variety of groups dealing with challenging biological problems. MD is now making an irreversible journey to the mainstream of research in biology, with the attendant opportunities and challenges. But given the speed with which MD studies of DNA have spread, the roots remain somewhat shallow: in many cases, there is a lack of deep knowledge about the foundations, strengths, and limits of the technique. In this Account, we discuss how MD has become the most important source of structural and flexibility data on DNA, focusing on advances since 2007 of atomistic MD in the description of DNA under near-physiological conditions and highlighting the possibilities and shortcomings of the technique. The evolution in the field over the past four years is a prelude to the ongoing revolution. The technique has gained in robustness and predictive power, which when coupled with the spectacular improvements in software and hardware has enabled the tackling of systems of increasing complexity. Simulation times of microseconds have now been achieved, with even longer times when specialized hardware is used. As a result, we have seen the first real-time simulation of large conformational transitions, including folding and unfolding of short DNA duplexes. Noteworthy advances have also been made in the study of DNA-ligand interactions, and we predict that a global thermodynamic and kinetic picture of the binding landscape of DNA will become available in a few years. MD will become a crucial tool in areas such as biomolecular engineering and synthetic biology. MD has also been shown to be an excellent source of parameters for mesoscopic models of DNA flexibility. Such models can be refined through atomistic MD simulations on small duplexes and then applied to the study of entire chromosomes. Recent evidence suggests that MD-derived elastic models can successfully predict the position of regulatory regions in DNA and can help advance our understanding of nucleosome positioning and chromatin plasticity. If these results are confirmed, MD simulations can become the ultimate tool to decipher a physical code that can contribute to gene regulation. We are entering the golden age of MD simulations of DNA. Undoubtedly, the expectations are high, but the challenges are also enormous. These include the need for more accurate potential energy functionals and for longer and more complex simulations in more realistic systems. The joint research effort of several groups will be crucial for adapting the technique to the requirements of the coming decade.

Linking Electronic Health Record-Extracted Psychosocial Data in Real-Time to Risk of Readmission for Heart Failure

Knowledge of psychosocial characteristics that helps to identify patients at increased risk for readmission for heart failure (HF) may facilitate timely and targeted care. We hypothesized that certain psychosocial characteristics extracted from the electronic health record (EHR) would be associated with an increased risk for hospital readmission within the next 30 days. We identified 15 psychosocial predictors of readmission. Eleven of these were extracted from the EHR (six from structured data sources and five from unstructured clinical notes). We then analyzed their association with the likelihood of hospital readmission within the next 30 days among 729 patients admitted for HF. Finally, we developed a multivariable predictive model to recognize individuals at high risk for readmission. We found five characteristics-dementia, depression, adherence, declining/refusal of services, and missed clinical appointments-that were associated with an increased risk for hospital readmission: the first four features were captured from unstructured clinical notes, while the last item was captured from a structured data source. Unstructured clinical notes contain important knowledge on the relationship between psychosocial risk factors and an increased risk of readmission for HF that would otherwise have been missed if only structured data were considered. Gathering this EHR-based knowledge can be automated, thus enabling timely and targeted care.

Editorial: data mining in software engineering

Data mining, the automated or semi-automated extraction of useful knowledge from large poorly structured data sources, is a hot topic in all sorts of fields. Potential science applications include, for example, learning from the expected avalanche of data generated by CERN’s Large Hadron Collider. Telecommunications companies apply data mining to detect fraudulent network usage. Companies in many areas of business apply data mining to improve their marketing and advertising. Law enforcement uses data mining to detect various financial crimes. Given the well known complexity of software engineering processes and artifacts, it is perhaps not surprising that data mining can be applied there as well. Developing software intensive systems is difficult in large part because of the complexity of development processes and of the resulting artifacts. Understanding what is to be built and how to build it is a process of discovery driven by large amounts of data. There is experience data from past development projects; there is data derived from abstractions and models of requirements, e.g. by simulation; and there is data derived from the built artifacts themselves, such as code metrics. As the amount of such data grows, it becomes harder and harder for humans to manage it and glean useful and reliable knowledge from it. This points to a growing need for automated tools to help us with this software engineering problem. One of the original driving visions of the ASE Community, perhaps the central one, as laid out in the original KBSA Report and other places, is to apply artificial intelligence techniques in building tools to deal with software engineering’s complexity. This issue of ASEJ brings us three contributions very much in this spirit, applying machine learning to the problem of data mining for software engineering.

NMR Characterization of Long-Range Order in Intrinsically Disordered Proteins

Intrinsically disordered proteins (IDPs) are predicted to represent a significant fraction of the human genome, and the development of meaningful molecular descriptions of these proteins remains a key challenge for contemporary structural biology. In order to describe the conformational behavior of IDPs, a molecular representation of the disordered state based on diverse sources of structural data that often exhibit complex and very different averaging behavior is required. In this study, we propose a combination of paramagnetic relaxation enhancements (PREs) and residual dipolar couplings (RDCs) to define both long-range and local structural features of IDPs in solution. We demonstrate that ASTEROIDS, an ensemble selection algorithm, faithfully reproduces intramolecular contacts, even in the presence of highly diffuse, ill-defined target interactions. We also show that explicit modeling of spin-label mobility significantly improves the reproduction of experimental PRE data, even in the case of highly disordered proteins. Prediction of the effects of transient long-range contacts on RDC profiles reveals that weak intramolecular interactions can induce a severe distortion of the profiles that compromises the description of local conformational sampling if it is not correctly taken into account. We have developed a solution to this problem that involves efficiently combining RDC and PRE data to simultaneously determine long-range and local structure in highly flexible proteins. This combined analysis is shown to be essential for the accurate interpretation of experimental data from alpha-synuclein, an important IDP involved in human neurodegenerative disease, confirming the presence of long-range order between distant regions in the protein.

9ticks – The Web as a Stream

The Internet contains thousands of Frequently Updated, Time-stamped, Structured (FUTS) data sources. This type of information represents a different class of information that is not properly handled by existing data management systems such as databases, data warehouses, search engines, pub-sub, event processing or information retrieval systems. In this position paper, we describe 9ticks, a system we are designing to collect, parse, store, query and disseminate FUTS information. 9ticks is helping us understand that all those steps raise new challenges but also bring new opportunities. In this paper, we summarize the challenges identified and present our vision of an end-to-end FUTS management system.

Data integration for the relational web

The Web contains a vast amount of structured information such as HTML tables, HTML lists and deep-web databases; there is enormous potential in combining and re-purposing this data in creative ways. However, integrating data from this relational web raises several challenges that are not addressed by current data integration systems or mash-up tools. First, the structured data is usually not published cleanly and must be extracted (say, from an HTML list) before it can be used. Second, due to the vastness of the corpus, a user can never know all of the potentially-relevant databases ahead of time (much less write a wrapper or mapping for each one); the source databases must be discovered during the integration process. Third, some of the important information regarding the data is only present in its enclosing web page and needs to be extracted appropriately. This paper describes Octopus, a system that combines search, extraction, data cleaning and integration, and enables users to create new data sets from those found on the Web. The key idea underlying Octopus is to offer the user a set of best-effort operators that automate the most labor-intensive tasks. For example, the Search operator takes a search-style keyword query and returns a set of relevance-ranked and similarity-clustered structured data sources on the Web; the Context operator helps the user specify the semantics of the sources by inferring attribute values that may not appear in the source itself, and the Extend operator helps the user find related sources that can be joined to add new attributes to a table. Octopus executes some of these operators automatically, but always allows the user to provide feedback and correct errors. We describe the algorithms underlying each of these operators and experiments that demonstrate their efficacy.