Areas Of Data Quality Research Articles

Missing Data Imputation with Uncertainty-Driven Network

We study the problem of missing data imputation, which is a fundamental task in the area of data quality that aims to impute the missing data to achieve the completeness of datasets. Though the recent distribution-modeling-based techniques (e.g., distribution generation and distribution matching) can achieve state-of-the-art performance in terms of imputation accuracy, we notice that (1) they deploy a sophisticated deep learning model that tends to be overfitting for missing data imputation; (2) they directly rely on a global data distribution while overlooking the local information. Driven by the inherent variability in both missing data and missing mechanisms, in this paper, we explore the uncertain nature of this task and aim to address the limitations of existing works by proposing an u<u>N</u>certainty-driven netw<u>O</u>rk for <u>M</u>issing data <u>I</u>mputation, termed NOMI. NOMI has three key components, i.e., the retrieval module, the neural network gaussian process imputator (NNGPI) and the uncertainty-based calibration module. NOMI~ runs these components sequentially and in an iterative manner to achieve a better imputation performance. Specifically, in the retrieval module, NOMI~ retrieves local neighbors of the incomplete data samples based on the pre-defined similarity metric. Subsequently, we design NNGPI~ that merges the advantages of both the Gaussian Process and the universal approximation capacity of neural networks. NNGPI~ models the uncertainty by learning the posterior distribution over the data to impute missing values while alleviating the overfitting issue. Moreover, we further propose an uncertainty-based calibration module that utilizes the uncertainty of the imputator on its prediction to help the retrieval module obtain more reliable local information, thereby further enhancing the imputation performance. We also demonstrate that our NOMI~ can be reformulated as an instance of the well-known Expectation Maximization (EM) algorithm, highlighting the strong theoretical foundation of our proposed methods. Extensive experiments are conducted over 12 real-world datasets. The results demonstrate the excellent performance of NOMI in terms of both accuracy and efficiency.

Overcoming diagnostic challenges of artificial intelligence in pathology and radiology: Innovative solutions and strategies

The advent of artificial intelligence (AI) has brought about significant changes in the fields of pathology and radiology, particularly in the area of diagnostic accuracy. Although AI has enormous potential for enhancing the precision and effectiveness of diagnosis, it also presents an array of challenges. This review article examines the diagnostic challenges of AI in pathology and radiology. The article begins by giving a general review of AI and its potential applications in pathology and radiology. It then discusses the challenges posed by AI in the areas of data quality, generalization, interpretability, and hardware limitations. The article also explores the ethical and regulatory implications of AI in diagnostic settings, including issues of bias and transparency. Finally, the article offers potential solutions to address these challenges, such as standardization of AI algorithms, data sharing initiatives, saliency mapping, adversarial training of algorithms, cloud computing, edge computing, hybrid approaches, and increased collaboration between human experts and AI systems. Overall, this review highlights the critical importance of addressing the diagnostic challenges of AI in pathology and radiology to make sure AI is able to achieve its potential to enhance patient care.

Parameterized complexity of completeness reasoning for conjunctive queries

Incompleteness management has become a popular research topic and been viewed in many applications in the area of data quality and data management. Traditional methods for handling incompleteness assume data is totally complete or incomplete. However, in practical applications, data is often partial complete, which means that data is not totally complete but some special parts of the data satisfying given semantic specifications are complete. Intuitively, partial complete data can still give complete answers for queries consistent with the semantic specifications. Therefore, it is highly needed to study the fundamental problems for managing partial complete data. However, as far as known by us, there are only few works focusing on this area. The most important and fundamental problem, completeness reasoning, is studied from the aspect of parameterized complexity by this paper. The completeness reasoning problem, TC-QC (Table Completeness to Query Completeness), is first formally defined and studied by Razniewski et al. [1]. Given completeness statements of data, the goal of the TC-QC problem is to determine whether the result of a special query Q is complete, that is to reason query completeness based on given data completeness. Razniewski et al. have shown that the TC-QC problem is NP-hard even for conjunctive queries, and a natural and interesting question is whether or not TC-QC can be solved efficiently by parameterized algorithms. To answer that, the parameterized complexities of completeness reasoning for conjunctive queries are studied by the paper. First, it is shown that, considering the parameterizations defined by the size of query completeness or table completeness, the parameterized TC-QC problem for conjunctive queries is para-NP-complete, which strongly indicate that the TC-QC problems parameterized by the above two parameters do not admit fixed-parameter tractable algorithms. Then, for more special cases parameterized by different constraints on query structures like degree, tree-width and number of variables, the TC-QC problems are still not fixed parameter tractable. Finally, on the positive side, if each data completeness statement has a constant size bound, the parameterized TC-QC problem defined by the query completeness size can be solved by a fixed-parameter tractable algorithm.

Protocol Update for large-scale genome and gene function analysis with the PANTHER classification system (v.14.0).

The PANTHER classification system ( http://www.pantherdb.org ) is a comprehensive system that combines genomes, gene function classifications, pathways and statistical analysis tools to enable biologists to analyze large-scale genome-wide experimental data. The current system (PANTHER v.14.0) covers 131 complete genomes organized into gene families and subfamilies; evolutionary relationships between genes are represented in phylogenetic trees, multiple sequence alignments and statistical models (hidden Markov models (HMMs)). The families and subfamilies are annotated with Gene Ontology (GO) terms, and sequences are assigned to PANTHER pathways. A suite of tools has been built to allow users to browse and query gene functions and analyze large-scale experimental data with a number of statistical tests. PANTHER is widely used by bench scientists, bioinformaticians, computer scientists and systems biologists. Since the protocol for using this tool (v.8.0) was originally published in 2013, there have been substantial improvements and updates in the areas of data quality, data coverage, statistical algorithms and user experience. This Protocol Update provides detailed instructions on how to analyze genome-wide experimental data in the PANTHER classification system.

Completeness and Accuracy of Local Clinical Registry Data for Children Undergoing Heart Surgery

Data routinely captured in clinical registries may be leveraged to enhance efficiency of prospective research. The quality of registry data for this purpose has not been studied, however. We evaluated the completeness and accuracy of perioperative data within congenital heart centers' local surgical registries. Within 12 Pediatric Heart Network (PHN) sites, we evaluated 31 perioperative variables (and their subcategories, totaling 113 unique fields) collected via sites' local clinical registries for submission to The Society of Thoracic Surgeons Database, compared with chart review by PHN research coordinators. Both used standard STS definitions. Data were collected on 10 subjects for 2 to 5 procedures/site and adjudicated by the study team. Completeness and accuracy (agreement of registry data with medical record review by PHN coordinator, adjudicated by the study team) were evaluated. A total of 56,500 data elements were collected on 500 subjects. With regard to data completeness, 3.1% of data elements were missing from the registry, 0.6% from coordinator-collected data, and 0.4% from both. Overall, registry data accuracy was 98%. In total, 94.7% of data elements were both complete/non-missing and accurate within the registry, although there was variation across data fields and sites. Mean total time for coordinator chart review per site was 49.1 hours versus 7.0 hours for registry query. This study suggests that existing surgical registry data constitute a complete, accurate, and efficient information source for prospective research. Variability across data fields and sites also suggest areas for improvement in some areas of data quality.

Community Engagement among the BioSense 2.0 User Group

This roundtable will provide a forum for the syndromic surveillance Community of Practice (CoP) to learn about activities of the BioSense 2.0 User Group (BUG) workgroups that address priority issues in syndromic surveillance. The goals of the workgroups are to coordinate efforts nationwide, better inform development of BioSense 2.0 to the Governance Group and CDC, and achieve high-quality outcomes for the practice of syndromic surveillance. Representatives from each workgroup will describe their efforts to date so participants can discuss key challenges and best practices in the areas of data quality, data sharing, onboarding, and developing syndrome definitions.

Statistical challenges in systematic evidence generation through analysis of observational healthcare data networks

Observational healthcare databases offer significant potential for generating evidence about the effects of medical products. Recent US and international efforts attempt to coordinate analysis activities across observational data networks for active drug safety surveillance and comparative effectiveness research. Several statistical challenges exist that require further research to inform the appropriate use of observational data for identifying and evaluating temporal relationships between medical product exposures and outcomes. This review highlights current work across the areas of data quality, methods development, and performance evaluation that promise to contribute to our understanding of best practices for observational analyses moving forward.

MULTIPLE REMOVAL SUCCESS IN THE CARNARVON BASIN WITH SRME

In 2003, Santos Ltd revisited a poor data quality area in the northern Carnarvon Basin, offshore Western Australia, where both short and long period multiple energy prohibits imaging of the underlying geology. Previous reprocessing efforts had failed to satisfactorily improve data quality, or reduce the level of multiple contamination. A two-dimensional (2D) reprocessing project was initiated to establish whether any modern variant of Surface-Related Multiple Elimination (SRME) could have success. Consequently, several versions of SRME were tested, with all output diagnostics being imaged with anisotropic Kirchhoff pre-stack time migration (PSTM). The new SRME results are a significant improvement over previous reprocessing efforts, and provide a much better platform for the picking of anisotropic velocity functions, and the application of PSTM imaging. Most of the multiple energy in this location is actually surface-related, with only a small component of internal multiple reverberations. Both long and short period multiple energy was successfully removed, and interpretation can now be pursued with more confidence in a difficult data location. Many outof- the-plane events still appear to contaminate the final 2D result, so a full three-dimensional (3D) production project was then pursued using standard (2D) SRME processing applied to 3D data gathers.Despite many noise challenges existing within the 3D field data, the final data images shed new light on a challenging geological environment, and prove the merits of SRME processing. A new generation of 3D acquisition and processing technology is now required to improve upon existing results, so a brief consideration is also given to the potential applications of 3D SRME processing to 3D seismic data from the North West Shelf. A brief example from offshore Brazil is used to illustrate the potential benefits of 3D SRME.

Report on the Dagstuhl Seminar

Over the past few years, techniques for managing, querying, and integrating data on the Web have significantly matured. Well-founded and practical approaches to assess or even guarantee a required degree of quality of the data in these frameworks, however, are still missing. This can be contributed to the lack of welldefined data quality metrics and assessment techniques, and the difficulty of handling information about data quality during data integration and query processing. Data quality problems arise in many settings, such as the integration of business data, in Web mining, data dissemination, and in querying the Web using search engines. Data quality (DQ) addresses various forms of data, including structured and semistructured data, text documents, multimedia, and streaming data. Different forms of metadata describing the quality of data is becoming increasingly important since they provide applications and users with information about the value and reliability of (integrated) data on the Web. The Dagstuhl Seminar “Data Quality on the Web”, organized by Michael Gertz, Tamer Ozsu, Gunter Saake, and Kai-Uwe Sattler, took place between August 31st and September 5th 2003 at Schloss Dagstuhl, Germany. The objective of the seminar was to (1) foster collaboration among researchers that deal with DQ in different areas, (2) assess existing results in managing the quality of data, and (3) establish a framework for future research in the area of DQ. The application contexts considered during the seminar included in particular (Web-based) data integration and information retrieval scenarios, scientific databases, and application domains in the computational sciences and Bioinformatics. In all these areas, data quality plays a crucial role and therefore different, tailored solutions have been developed. Sharing and exchanging this knowledge could result in significant synergy effects.