Aspects Of Data Quality Research Articles

Web-Based Versus Paper-Pencil Assessment of Behavioral Problems Using the Youth Self-Report

Abstract. This is the first study investigating the use of a web-based version of the Youth Self-Report (YSR) compared to paper-pencil (PP) assessment. Students aged 10–18 years were recruited from Austrian schools and either completed a PP ( N = 841) or a web-based ( N = 2,769) version. Psychometric properties and indicators of data quality were analyzed. Moreover, cost estimations for the web-based and PP assessments were given. Acceptable model fits (RMSEA < .05) of the 8-syndrome model were observed for both versions and measurement invariance testing revealed strong invariance between both formats. While there was no significant difference in the number of missing items, a slightly higher proportion of web-based datasets were analyzable (97.7% vs. 93.7%). A larger amount of information was provided in the open-ended questions in the web-based version ( p < .001). Mean problem scores were equivalent, with the exception of the thought problems syndrome scale in which a slightly higher score was observed in the web-based version ( p < .001). In our study, the web-based format reduced costs by 72% compared to the PP format. Our findings suggest that large web-based epidemiological surveys are cost-efficient, can be applied without the risk of disadvantages compared to PP assessments and might be superior regarding some aspects of data quality.

Collecting egocentric network data with visual tools: A comparative study

AbstractWhen collecting egocentric network data, visual representations of networks can function as a cognitive aid for depicting relationships, helping to maintain an overview of the relationships, and keeping the attention of the interviewees. Additionally, network maps can serve as a narration generator in qualitative and in mixed-methods studies. While varying visual instruments are used for collecting egocentric network data, little is known about differences among visual tools concerning the influence on the resulting network data, the usability for interviewees, and data validity. The article provides an overview of existing visually oriented tools that are used to collect egocentric networks and discusses their functions, advantages, and limitations. Then, we present results of an experimental study where we compare four different visual tools with regard to networks elicited, manageability, and the impact of follow-up questions. In order to assess the manageability of the four tools, we used the thinking aloud method. The results provide evidence that the decision in favor of a specific visual tool (structured vs. unstructured) can affect the size and composition of the elicited networks. Follow-up questions greatly affect the elicited networks and follow-up cues can level out differences among tools. Respondents tend to prefer the concentric circles tool, with some differences in preferences and manageability of tools between participants with low and those with high socioeconomic status. Finally, assets and drawbacks of the four instruments are discussed with regard to data quality and crucial aspects of the data collection process when using visual tools.

Influence of floating car data quality on congestion identification

This paper explores the usability of floating car data (FCD) of mixed quality in congestion analysis on motorways. The specific data quality aspects that we are investigating are the number and density of trajectories, the GPS interval, and the fleet representativeness. We use a dataset provided by the German Automobile Club ADAC covering the Tyrolean road network in 2016. From this dataset, trajectories along the A12 motorway were extracted for congestion analysis. These data are characterized by high GPS time interval, low number of trajectories, and are not representative for total traffic due to overrepresentation of trucks. The influence of these quality parameters on congestion identification is explored by analyzing the parameter distribution among different congestion types. In addition, we validate the results by comparing them with congestion incidents obtained from the stationary detector data (SDD) and examining the impact of quality parameters on the validation results. We find that the given data set does not allow short-term congestion patterns to be identified due to quality flaws. Especially the low number of trajectories proved problematic, whereas the influence of other parameters was less distinct. Despite these flaws, for large-scale congestion incidents, floating car data provide outcomes similar to those derived from stationary detectors.

Inter-observer agreement and reliability assessment for observational studies of clinical work.

Inter-observer agreement (IOA) is a key aspect of data quality in time-and-motion studies of clinical work. To date, such studies have used simple and ad hoc approaches for IOA assessment, often with minimal reporting of methodological details. The main methodological issues are how to align time-stamped task intervals that rarely have agreeing start and end times, and how to assess IOA for multiple nominal variables. We present a combination of methods that simultaneously addresses both these issues and provides a more appropriate measure by which to assess IOA for time-and-motion studies. The issue of alignment is addressed by converting task-level data into small time windows then aligning data from different observers by time. A method applicable to multivariate nominal data, the iota score, is then applied to the time-aligned data. We illustrate our approach by comparing iota scores to the mean of univariate Cohen's kappa scores through application of these measures to existing data from an observational study of emergency department physicians. While the two scores generated very similar results under certain conditions, iota was more resilient to sparse data issues. Our results suggest that iota applied to time windows considerably improves on previous methods used for IOA assessment in time-and-motion studies, and that Cohen's kappa and other univariate measures should not be considered the gold standard. Rather, there is an urgent need for ongoing explicit discussion of methodological issues and solutions to improve the ways in which data quality is assessed in time-and-motion studies in order to ensure the conclusions drawn from such studies are robust.

Research applications of primary biodiversity databases in the digital age.

Our world is in the midst of unprecedented change—climate shifts and sustained, widespread habitat degradation have led to dramatic declines in biodiversity rivaling historical extinction events. At the same time, new approaches to publishing and integrating previously disconnected data resources promise to help provide the evidence needed for more efficient and effective conservation and management. Stakeholders have invested considerable resources to contribute to online databases of species occurrences. However, estimates suggest that only 10% of biocollections are available in digital form. The biocollections community must therefore continue to promote digitization efforts, which in part requires demonstrating compelling applications of the data. Our overarching goal is therefore to determine trends in use of mobilized species occurrence data since 2010, as online systems have grown and now provide over one billion records. To do this, we characterized 501 papers that use openly accessible biodiversity databases. Our standardized tagging protocol was based on key topics of interest, including: database(s) used, taxa addressed, general uses of data, other data types linked to species occurrence data, and data quality issues addressed. We found that the most common uses of online biodiversity databases have been to estimate species distribution and richness, to outline data compilation and publication, and to assist in developing species checklists or describing new species. Only 69% of papers in our dataset addressed one or more aspects of data quality, which is low considering common errors and biases known to exist in opportunistic datasets. Globally, we find that biodiversity databases are still in the initial stages of data compilation. Novel and integrative applications are restricted to certain taxonomic groups and regions with higher numbers of quality records. Continued data digitization, publication, enhancement, and quality control efforts are necessary to make biodiversity science more efficient and relevant in our fast-changing environment.

Data Quality Assessment and Multi-Organizational Reporting: Tools to Enhance Network Knowledge.

Objective:Multi-organizational research requires a multi-organizational data quality assessment (DQA) process that combines and compares data across participating organizations. We demonstrate how such a DQA approach complements traditional checks of internal reliability and validity by allowing for assessments of data consistency and the evaluation of data patterns in the absence of an external “gold standard.”Methods:We describe the DQA process employed by the Data Coordinating Center (DCC) for Kaiser Permanente’s (KP) Center for Effectiveness and Safety Research (CESR). We emphasize the CESR DQA reporting system that compares data summaries from the eight KP organizations in a consistent, standardized manner.Results:We provide examples of multi-organization comparisons from DQA to confirm expectations about different aspects of data quality. These include: 1) comparison of direct data extraction from the electronic health records (EHR) and 2) comparison of non-EHR data from disparate sources.Discussion:The CESR DCC has developed codes and procedures for efficiently implementing and reporting DQA. The CESR DCC approach is to 1) distribute DQA tools to empower data managers at each organization to assess their data quality at any time, 2) summarize and disseminate findings to address data shortfalls or document idiosyncrasies, and 3) engage data managers and end-users in an exchange of knowledge about the quality and its fitness for use.Conclusion:The KP CESR DQA model is applicable to networks hoping to improve data quality. The multi-organizational reporting system promotes transparency of DQA, adds to network knowledge about data quality, and informs research.

Item Nonresponse Rates and Panel Conditioning in a Longitudinal Survey among Youth

The article examines two important aspects of data quality in self-completion surveys of young people, taking advantage of a unique data source: Understanding Society: the United Kingdom Household Longitudinal Study. Young persons aged 10–15 are asked to complete a self-administered paper questionnaire at annual intervals. The number of completed interviews varies over waves from 4,049 to 5,020. Data are also collected from parents, providing important explanatory covariates for our analysis. Stronger parent–child relationship and higher mother’s involvement in education were associated with lower item nonresponse rate and lower inconsistency throughout waves. We also found some evidence for a negative panel conditioning effect with an increase of social desirability bias and measurement errors in the subsequent waves. There was a higher level of inconsistent responses and a higher probability of social desirability bias throughout waves in more sensitive items.

The evaluation of IRNSS/NavIC system’s performance in its primary and secondary service areas–data quality, usability and single point positioning

The Navigation with Indian Constellation (NavIC), also known as Indian regional navigation satellite system, is a regional navigation satellite system recently developed by India. Its service area covers from 30°E to 130°E and from 30°S to 50°N. In this contribution, an assessment of NavIC from aspects of data quality, usability and single point positioning (SPP) performance is carried out using real measured data collected from four sites both within NavIC’s primary and secondary service areas. Data quality of NavIC’s signal is assessed measuring its carrier-to-noise-density ratio and each satellite’s orbital period is calculated using its broadcast ephemeris. Visible satellite number and DOPs values of each site in modes of NavIC-only, GPS-only and GPS/NavIC are counted and calculated respectively. SPP solutions in modes of NavIC-only, GPS-only and GPS/NavIC are also carried out for these four sites. The results show that: the signal strength of NavIC’s L5 frequency generally equals that of GPS in site IISC (Fig. 1); mean orbital periods of IGSO and GEO satellites are 86160.70 s and 86152.03 s respectively; in site IISC (within the primary service area), currently NavIC system can provide an independent positioning service with an accuracy of less than 1 m in the east direction and less than 2 m in north and up directions respectively; the usability and SPP performance can be improved significantly in the mode of GPS/NavIC compared with those in either single mode.

Don’t Get Lost in the Crowd: Best Practices for Using Amazon’s Mechanical Turk in Behavioral Research

The use of Amazon’s Mechanical Turk (MTurk) to conduct academic research has steadily grown since its inception in 2005. The ability to control every aspect of a study, from sampling to collection, is extremely appealing to researchers. Unfortunately, the additional control offered through MTurk can also lead to poor data quality if researchers are not careful. Despite research on various aspects of data quality, participant compensation, and participant demographics, the academic literature still lacks a practical guide to the effective use of settings and features in MTurk for survey and experimental research. Therefore, the purpose of this tutorial is to provide researchers with a recommended set of best practices to follow before, during, and after collecting data via MTurk to ensure that responses are of the highest possible quality. We also recommend that editors and reviewers place more emphasis on the collection methods employed by researchers, rather than assume that all samples collected using a given online platform are of equal quality. We also recommend that editors and reviewers place more emphasis on the collection methods employed by researchers, rather than assuming that all samples collected using a given online platform are of equal quality.

What can millions of laboratory test results tell us about the temporal aspect of data quality? Study of data spanning 17 years in a clinical data warehouse

ObjectiveTo identify common temporal evolution profiles in biological data and propose a semi-automated method to these patterns in a clinical data warehouse (CDW). Materials and MethodsWe leveraged the CDW of the European Hospital Georges Pompidou and tracked the evolution of 192 biological parameters over a period of 17 years (for 445,000 + patients, and 131 million laboratory test results). ResultsWe identified three common profiles of evolution: discretization, breakpoints, and trends. We developed computational and statistical methods to identify these profiles in the CDW. Overall, of the 192 observed biological parameters (87,814,136 values), 135 presented at least one evolution. We identified breakpoints in 30 distinct parameters, discretizations in 32, and trends in 79. Discussion and conclusionour method allowed the identification of several temporal events in the data. Considering the distribution over time of these events, we identified probable causes for the observed profiles: instruments or software upgrades and changes in computation formulas. We evaluated the potential impact for data reuse. Finally, we formulated recommendations to enable safe use and sharing of biological data collection to limit the impact of data evolution in retrospective and federated studies (e.g. the annotation of laboratory parameters presenting breakpoints or trends).

Continuity Trumps? The Impact of Interviewer Change on Item Nonresponse

Allocating the same interviewer to each respondent over multiple waves is typically recommended for panel surveys. While some studies have investigated the effect of this strategy on unit nonresponse, there is scarce empirical evidence on how interviewer (dis-)continuity affects item nonresponse. This is surprising, given that the amount and pattern of item nonresponse is a crucial aspect of data quality. Using the first seven waves of the German Family Panel pairfam, we investigate whether interviewer continuity indeed influences item nonresponse. Our analysis differentiates between “I don’t know” responses and the complete refusal to answer, both with respect to the specific question of household income as well as the entire question program. By applying respondent fixed effects methods we can base our results on a within-person comparison, controlling for time-constant unobserved characteristics. Our analysis shows no detrimental effect of interviewer change over the panel; in fact, being assigned to a new interviewer even leads to lower refusal percentages. Only temporary dropouts showed lower percentages of “I don’t know” answers if an interviewer change occurred. Changes in social distance with respect to age and gender do not prove to be relevant mechanisms for this effect. However, both female and male respondents exhibit a higher likelihood of “I don’t know” responses over all questions when reassigned to a female interviewer. Reassignment to a more experienced interviewer produces less “I don’t know” answers.

Consistency of Open Data as Prerequisite for Usability in Agriculture

Abstract Benefits of open data are diverse and range from improved efficiency of public administrations, economic growth in the private sector. Agriculture is also an inseparable part of the private sector. These data can stimulate economic growth. Economy can benefit from an easier access to information, content and knowledge in turn contributing to the development of innovative services and creation of new business models (European Data Portal, 2016). The prerequisite of this stimulation is consistency of open data. Both of these features have to be a part of these data quality. However, this aspect of open data quality has not yet been satisfactorily resolved in the framework of international standardization of quality (Systems and Software Quality Requirements and Evaluation (SQuaRE)). The issue of possible evaluation of open data consistency in agriculture is discussed. Results suggest that open data consistency may be achieved by consistent application of the technique of data normalization of relevant data sets. Consistent application of data normalization technique of open data sets can reduce the risk of inconsistency of the open data. That is the only way to guarantee that the open data will be the benefit to private sector.

Desenvolvimento de ontologia ciente de qualidade de informações para o domínio de gerenciamento de emergências

Para a comunidade de avaliação e gerenciamento de situações de emergências, determinar um processo de análise informacional para o desenvolvimento da Consciência da Situação (SAW) é um desafio, devido à limitada qualidade e complexidade dos dados, além da dinamicidade de ambientes reais monitorados. Operadores de sistemas de avaliação de situações de emergência, expostos a dados de qualidade incerta, estão suscetíveis a erros de SAW, diminuindo suas chances de uma tomada de decisão assertiva. Neste contexto, representar corretamente informações sobre tais situações, à medida que são coletadas e processadas, vai contribuir para melhor orientar humanos e sistemas sobre o que de fato acontece num ambiente de interesse. O presente trabalho visa contribuir com o problema de produção de melhores insumos na avaliação de situações de incêndios florestais, no âmbito do projeto DF100Fogo, apresentando o desenvolvimento de uma ontologia que agrega informações sobre a qualidade dos dados, promovendo ao operador de sistemas de emergência, uma melhor representação de dados sobre os incêndios. Um estudo de caso envolvendo um atendimento a uma situação de incêndio florestal será mostrado para demonstrar a aplicabilidade da ontologia junto à gestão da qualidade. Resultados apresentam um modelo de representação semântica, capaz de receber instâncias extraídas de dados de inteligência humana, com origem em sensoriamento participativo, e avaliadas quantitativamente sob aspectos de qualidade da informação.

Accuracy of parameter identification using the dispersion of surface waves and the role of data quality for inhomogeneous concrete

We study the influence of the data quality on the accuracy of parameter identification for inhomogeneous concrete. The surface of the concrete may differ as a result of either intentional action (decoration or repair) or the interaction of initially homogeneous concrete with the environment. Non-destructive evaluation of the concrete surface layer is vital to monitoring the integrity of concrete structures and preventing their irreversible damage. We use the methodology (also used for ground structure recovery) of multichannel analysis of surface waves as the non-destructive testing (NDT) tool in the civil engineering domain to characterize the concrete surface layer. The procedure consists of the generation and reception of surface waves within the required frequency band. Then, the phase velocity dispersion characteristic is extracted and used as input data for solving the inverse problem to obtain the variation of the shear wave velocity as a function of the sample depth.Two aspects of data quality are considered: completeness in the frequency or wavelength domain and accuracy. In the former case, the efficiency of identification is studied for variable ranges (in the frequency or wavelength domains) of the dispersion curve of the fundamental mode of the surface wave. The role of data accuracy is evaluated through comparison of the identified parameters using experimental and appropriate synthetic data.To perform these studies, cuboidal models of inhomogeneous concrete composed of two layers of different classes of concrete are prepared. During experiments, surface wave signals are acquired within a frequency range of 35 kHz–160 kHz using a non-contact ultrasonic automated scanner, which enables quick and precise signal recording, avoiding concrete surface modification through the use of the coupling agent. Experimental dispersion curves are obtained using the Slant-Stack transformation. The synthetic data are obtained from the solution of the Thomson-Haskell model. The inversions are performed using the in-house software CLOUD and tested and validated on the multilayer model samples. The criteria for the optimum wavelength intervals of the dispersion curves are proposed: the lower wavelength limit λL ≤ d, and the upper wavelength limit λU ≥ 3.5d, where d is the thickness of the top layer. The consequences of narrowing the intervals are thoroughly studied, and upper and lower limits are determined both for experimental and synthetic data. It is shown that the efficiency of inversion depends on both wavelength limits and accuracy of the data.

Efficient global correlation measures for a collaborative filtering dataset

Recommender systems based on collaborative filtering (CF) rely on datasets containing users’ taste preferences for various items. Accuracy of various prediction approaches depends on the amount of similarity between users and items in a dataset. As a heuristic estimate of this data quality aspect, which could serve as an indicator of the prediction ability, we define the Global User Correlation Measure (GUCM) and the Global Item Correlation Measure (GICM) of a dataset containing known user–item ratings. The proposed measures range from 0 to 1 and describe the quality of the dataset regarding the user–user and item–item similarities: a higher measure indicates more similar pairs and better prediction ability. The experiments show a correlation between the proposed measures and the accuracy of standard prediction models. The measures can be used to quickly estimate whether a dataset is suitable for collaborative filtering and whether we can expect high prediction accuracy of user-based or item-based CF approaches.

The Crowd is the Territory: Assessing Quality in Peer-Produced Spatial Data During Disasters

ABSTRACT Today, disaster events are mobilizing digital volunteers to meet the data needs of those on the ground. One form of this crowd work is Volunteered Geographic Information. This peer-produced spatial data creates the most up-to-date map of the affected region; maintaining the accuracy of these data is therefore a critical task. Accuracy is one aspect of data quality, a relative concept requiring standards to measure against. The field of Geographic Information Sciences has developed standards for this comparison, achieving widespread acceptance. However, the peer production model of spatial data presents new opportunities—and challenges—to traditional methods of quality assessment. Through analysis of the OpenStreetMap database, we show that temporal editing patterns and contributor characteristics can provide additional means of understanding spatial data quality. Drawing upon experiences from Wikipedia, we offer and evaluate three intrinsic quality metrics of peer-produced spatial data to assess the quality of contributions to OpenStreetMap for crisis response.

Measuring Spatial Data Fitness-for-Use through Multiple Criteria Decision Making

This article presents a new methodology for data fitness-for-use assessment. Most current measures of data quality rely on metadata and other data producer-derived information. This creates a void of options for a user-driven assessment of data quality when metadata are sparse or unavailable, as is often the case with citizen science and volunteered geographic information. This article puts forward data fitness-for-use (DaFFU), a method that can be adapted for a wide range of data uses. Using the mathematical framework of multiple criteria decision making (MCDM), we create a method to select the best data set from multiple options using a select set of user criteria. The DaFFU methodology is demonstrated with both a simple exemplar and a detailed case study for watershed management. The simple exemplar illustrates how varying parameters and weights influence the outcome. The case study on watershed management considers four possible data sets and six data quality criteria for wetland delineation and an application toward watershed nitrogen retention, each of which has a claim on being of the “best” quality, depending on which data quality aspect the user evaluates. The DaFFU methodology allows the user to consider these data in terms of how they will be used and to use selected data quality measures. Case study results show this methodology is a robust and flexible approach to quantitatively assessing multiple data sets in terms of their intended use.

Methodological Choices and Data Quality Issues for Official Poverty Measures: Evidences from Italy

The plurality of the official poverty estimates in Italy covers both absolute and relative approaches, ranges from consumption to income-based measures, follows different methodologies and uses several data sources. We can therefore expect that each measure gives a somewhat different picture of poverty, in its level as well as in its change across subgroups of the population. This paper investigates the effect of methodological choices together with the effect of different data quality aspects on the official poverty estimates. Usually, methodological issues attract much attention both in literature and empirical studies. However, the results of the sensitivity analysis suggest that more specific attention should be paid to data quality issues and to the definition of the variables. Our main conclusion is that an improvement in the quality as well as the inclusion of some items in the definition of the variable may result in large changes in poverty indicators. This finding signals that the data quality aspects have a higher impact on poverty estimates than some methodological issues.

The relevance of flood hazards and impacts in Turkey: What can be learned from different disaster loss databases?

Turkey has been severely affected by many natural hazards, in particular earthquakes and floods. Especially over the last two decades, these natural hazards have caused enormous human and economic damage. Although there is a large body of literature on earthquake hazards and risks in Turkey, comparatively little is known about flood hazards and risks. Therefore, this study aims to investigate the severity of flooding in comparison with other natural hazards in Turkey and to analyse the flood patterns by providing an overview of the temporal and spatial distribution of flood losses. These will act as a metric for the societal and economic impacts of flood hazards in Turkey. For this purpose, Turkey Disaster Database (TABB) was used for the years 1960–2014. As input for more detailed event analyses, the most severe flood events in Turkey for the same time interval will also be retrieved. Sufficiency of the TABB database to achieve the main aim of the study in terms of data quality and accuracy was also discussed. The TABB database was analysed and reviewed through comparison, mainly with the Emergency Events Database (EM-DAT), the Global Active Archive of Large Flood Events—Dartmouth Flood Observatory database, news archives and the scientific literature, with a focus on listing the most severe flood event. The comparative review of these data sources reveals big mismatches in the flood data, i.e. the reported number of events, number of affected people and economic loss all differ dramatically. Owing to the fact that the TABB is the only disaster loss database for Turkey, it is important to explore the reasons for the mismatches between TABB and the other sources with regard to aspects of accuracy and data quality. Therefore, biases and fallacies in the TABB loss data are also discussed. The comparative TABB database analyses show that large mismatches between global and national databases can occur. Current global and national databases for monitoring losses from national hazards suffer from a number of limitations, which in turn could lead to misinterpretations of the loss data. Since loss data collection is gaining more and more attention, e.g. in the Sendai Framework for Disaster Risk Reduction 2015–2030, this study offers a framework for developing guidelines for the Turkey Disaster Database (TABB), implications on how to standardize national loss databases and implement across the other hazard events in Turkey.

The administration of patient-reported outcome questionnaires in cancer trials: Interviews with trial coordinators regarding their roles, experiences, challenges and training.

AimsTo explore cancer trial coordinators' roles and challenges in administering patient-reported outcome (PRO) questionnaires, and establish what PRO-specific training and guidance they received and needed.MethodsEligible cancer trial coordinators experienced with PRO assessment from approved Australian sites participated in an audio-recorded, semi-structured interview (transcribed verbatim). Recruitment continued until data saturation. Transcripts underwent content analysis.ResultsTwenty coordinators participated (professional training: nursing (n = 12), science/research (n = 4), both (n = 4)). PRO administration formed a minor component of most (85%) coordinators' roles. PRO administration challenges included managing ‘English second language’ participants, participants' companions who attempted to complete questionnaires, burdensome questionnaires, and balancing their duty of care against trial requirements. Coordinators reported inconsistencies in PRO administration, which appeared to arise as a result of confusion and inconsistent or contradictory PRO training. Inconsistencies concerned whether/when they explained the purpose of PRO assessment, which participants they approached to complete PROs, and whether they used PRO trial data to inform care.Coordinators received PRO training from various sources; most commonly study-specific start-up meetings (45%) or from colleagues (30%). Two received no PRO-specific training. Despite the challenges reported, many (55%) felt they did not need further PRO training.ConclusionTrial coordinators receive inconsistent PRO-specific training and are often unclear how to prioritise different aspects of data quality when faced with everyday challenges, leading to inconsistent methods, missing data, poor quality data, and even bias. Agreement on how coordinators should prioritise the requirements of PRO studies is a necessary pre-requisite for the development of much-needed, consensus-based PRO administration guidelines.