Influence Data Quality Research Articles

Applying Persuasive Design Techniques to Influence Data-Entry Behaviors in Primary Care: Repeated Measures Evaluation Using Statistical Process Control.

BackgroundPersuasive design is an approach that seeks to change the behaviors of users. In primary care, clinician behaviors and attitudes are important precursors to structured data entry, and there is an impact on overall data quality. We hypothesized that persuasive design changes data-entry behaviors in clinicians and thus improves data quality.ObjectiveThe objective of this study was to use persuasive design principles to change clinician data-entry behaviors in a primary care environment and to increase data quality of data held in a family health team’s reporting system.MethodsWe used the persuasive systems design framework to describe the persuasion context. Afterward, we designed and implemented new features into a summary screen that leveraged several persuasive design principles. We tested the influence of the new features by measuring its impact on 3 data quality measures (same-day entry, record completeness, and data validity). We also measured the impacts of the new features with a paired pre-post t test and generated XmR charts to contextualize the results. Survey responses were also collected from users.ResultsA total of 53 users used the updated system that incorporated the new features over the course of 8 weeks. Based on a pre-post analysis, the new summary screen successfully encouraged users to enter more of their data on the same day as their encounter. On average, the percentage of same-day entries rose by 10.3% for each user (P<.001). During the first month of the postimplementation period, users compensated by sacrificing aspects of data completeness before returning to normal in the second month. Improvements to record validity were marginal over the study period (P=.05). Statistical process control techniques allowed us to study the XmR charts to contextualize our results and understand trends throughout the study period.ConclusionsBy conducting a detailed systems analysis and introducing new persuasive design elements into a data-entry system, we demonstrated that it was possible to change data-entry behavior and influence data quality in a reporting system. The results show that using persuasive design concepts may be effective in influencing data-entry behaviors in clinicians. There may be opportunities to continue improving this approach, and further work is required to perfect and test additional designs. Persuasive design is a viable approach to encourage clinician user change and could support better data capture in the field of medical informatics.

Comprehensive Improvement of Sample Preparation Methodologies Facilitates Dynamic Metabolomics of Aspergillus niger.

Metabolomics is an essential discipline in industrial biotechnology. Sample preparation approaches dramatically influence data quality and, ultimately, interpretation and conclusions from metabolomic experiments. However, standardized protocols for highly reproducible metabolic datasets are limited, especially for the fungal cell factory Aspergillus niger. Here, an improved liquid chromatography-tandem mass spectrometry-based pipeline for A. niger metabolomics is developed. It is found that fast filtration with liquid nitrogen is more suitable for cell quenching, causing minimal disruption to cell integrity, and improved intracellular metabolite recovery when compared to cold methanol quenching approaches. Seven solutions are evaluated for intracellular metabolite extraction, and found acetonitrile/water (1:1, v/v) at -20 °C, combined with boiling ethanol extraction protocols, showed unbiased metabolite profiling. This improved methodology is applied to unveil the dynamic metabolite profile of one citrate over-producing A. niger isolate under citrate fermentation. Citrate precursors, especially pyruvate, oxaloacetate, and malate, are maintained at a relatively high intracellular level, which can be necessary for high citrate synthesis flux. Glutamine shows a similar trend compared to citrate production, suggesting glutamine may be involved in intracellular pH homeostasis. Taken together, this study delivers a highly standardized and improved metabolomics methodology and paves the way for systems metabolic engineering in biotechnologically important fungi.

Evaluation of Automated Online Quality Checker Implementation for the Brucellosis Surveillance system, Egypt 2016

Background: National Egyptian Disease Surveillance System collects data on 40 diseases and conditions from 584 nominated reporting sites out of 27 governorates. Brucellosis is a weekly notifiable disease. Reporting levels of completeness and timeliness may differ that may influence data quality; these attributes needed to be assessed prior further manipulation of data. In 2016, Surveillance Online Checker (SOC) was launched to facilitate quality check to give feedback about timeliness and completeness of optional (to online entry) variables to the reporting sites (Completeness of mandatory variables is 100%). Objective: We aimed to measure the effect of Surveillance Online Checker on brucellosis surveillance data completeness and timeliness after one year of implementation. Methods: Brucellosis data completeness and timeliness of selected optional variables from the case reporting form were compared before and after one year (2015-2016) of launching SOC using Chi square and student t-test. Results: It is found that completeness of optional variables were increased from 67.3 % to 78.4% as follows; Detailed Address (67.3% to 78.9%, P<0.0001), Patient Telephone (22.1% to 59.9%, P<0.0001), National ID (2.8% to 23.8%, P<0.0001), Lab Test (81.3% to 86.2%, P=0.043) Final Diagnosis (68.3 to 91.8, P<0.0001) and Investigation Forms (52.3% to 75.2%, P<0.0001). While other variables had no significant change such as Occupation (92.7% to 93.0%, P=0.946). The reporting during predefined Timeliness increased from 69.1% to 89.8%, P<0.0001, with Average Data Entry Time per Case since its classification improved to be 3.8 days instead of 8.8 days. The average time to retrieve previous calculations shortened from 2 working days /month to just seconds after running SOC. Conclusions: Completeness and timeliness of brucellosis surveillance improved after SOC. It is recommended to utilize SOC results by intermediate and peripheral levels after adapting SOC for these levels and adding additional surveillance attributes to SOC.

Optimized targeted sequencing of cell-free plasma DNA from bladder cancer patients

Analysis of plasma cell-free DNA (cfDNA) may provide important information in cancer research, though the often small fraction of DNA originating from tumor cells makes the analysis technically challenging. Digital droplet PCR (ddPCR) has been utilized extensively as sufficient technical performance is easily achieved, but analysis is restricted to few mutations. Next generation sequencing (NGS) approaches have been optimized to provide comparable technical performance, especially with the introduction of unique identifiers (UIDs). However, the parameters influencing data quality when utilizing UIDs are not fully understood. In this study, we applied a targeted NGS approach to 65 plasma samples from bladder cancer patients. Laboratory and bioinformatics parameters were found to influence data quality when using UIDs. We successfully sequenced 249 unique DNA fragments on average per genomic position of interest using a 225 kb gene panel. Validation identified 24 of 38 mutations originally identified using ddPCR across several plasma samples. In addition, four mutations detected in associated tumor samples were detected using NGS, but not using ddPCR. CfDNA analysis of consecutively collected plasma samples from a bladder cancer patient indicated earlier detection of recurrence compared to radiographic imaging. The insights presented here may further the technical advancement of NGS mediated cfDNA analysis.

Content Analysis by the Crowd: Assessing the Usability of Crowdsourcing for Coding Latent Constructs

ABSTRACTCrowdsourcing platforms are commonly used for research in the humanities, social sciences and informatics, including the use of crowdworkers to annotate textual material or visuals. Utilizing two empirical studies, this article systematically assesses the potential of crowdcoding for less manifest contents of news texts, here focusing on political actor evaluations. Specifically, Study 1 compares the reliability and validity of crowdcoded data to that of manual content analyses; Study 2 proceeds to investigate the effects of material presentation, different types of coding instructions and answer option formats on data quality. We find that the performance of the crowd recommends crowdcoded data as a reliable and valid alternative to manually coded data, also for less manifest contents. While scale manipulations affected the results, minor modifications of the coding instructions or material presentation did not significantly influence data quality. In sum, crowdcoding appears a robust instrument to collect quantitative content data.

Improving Data Quality in Mass-Gatherings Health Research.

Mass gatherings attract large crowds and can strain the planning and health resources of the community, city, or nation hosting an event. Mass-Gatherings Health (MGH) is an evolving niche of prehospital care rooted in emergency medicine, emergency management, public health, and disaster medicine. To explore front-line issues related to data quality in the context of mass gatherings, the authors draw on five years of management experience with an online, mass-gathering event and patient registry, as well as clinical and operational experience amassed over several decades. Here the authors propose underlying human, environmental, and logistical factors that may contribute to poor data quality at mass gatherings, and make specific recommendations for improvement through pre-event planning, on-site actions, and post-event follow-up. The advancement of MGH research will rely on addressing factors that influence data quality and developing strategies to mitigate or enhance those factors. This is an exciting time for MGH research as higher order questions are beginning to be addressed; however, quality research must start from the ground up to ensure optimal primary data capture and quality. Guy A , Prager R , Turris S , Lund A . Improving data quality in mass-gatherings health research. Prehosp Disaster Med. 2017;32(3):329-332.

Reusability of coded data in the primary care electronic medical record: A dynamic cohort study concerning cancer diagnoses

ObjectivesTo assess quality and reusability of coded cancer diagnoses in routine primary care data. To identify factors that influence data quality and areas for improvement. MethodsA dynamic cohort study in a Dutch network database containing 250,000 anonymized electronic medical records (EMRs) from 52 general practices was performed. Coded data from 2000 to 2011 for the three most common cancer types (breast, colon and prostate cancer) was compared to the Netherlands Cancer Registry. MeasurementsData quality is expressed in Standard Incidence Ratios (SIRs): the ratio between the number of coded cases observed in the primary care network database and the expected number of cases based on the Netherlands Cancer Registry. Ratios were multiplied by 100% for readability. ResultsThe overall SIR was 91.5% (95%CI 88.5–94.5) and showed improvement over the years. SIRs differ between cancer types: from 71.5% for colon cancer in males to 103.9% for breast cancer. There are differences in data quality (SIRs 76.2% − 99.7%) depending on the EMR system used, with SIRs up to 232.9% for breast cancer. Frequently observed errors in routine healthcare data can be classified as: lack of integrity checks, inaccurate use and/or lack of codes, and lack of EMR system functionality. ConclusionsRe-users of coded routine primary care Electronic Medical Record data should be aware that 30% of cancer cases can be missed. Up to 130% of cancer cases found in the EMR data can be false-positive. The type of EMR system and the type of cancer influence the quality of coded diagnosis registry. While data quality can be improved (e.g. through improving system design and by training EMR system users), re-use should only be taken care of by appropriately trained experts.

The impact of visual design and response formats on data quality in a web survey of MOOC students

The objective of this paper is to gain more evidence regarding how the design of the rating scales and open-ended questions influence data quality in Web surveys of MOOC (Massive Open Online Course) students. We present the results of four full-factorial randomized experiments that investigate the impact of the following factors: 1) order of response options; 2) user interface for rating questions 3) layout of question's options; and 4) size of answer boxes in open-ended questions. We found that responses to scalar questions with ascending (from negative to positive) or descending (from positive to negative) order of response options do not differ substantially. The use of the radio button format allows a reduction in the percentage of respondents who choose the “Don't know” option and makes responding to questions less challenging in comparison with slider and text box interfaces. There are no significant differences in the answers of respondents who completed questionnaires with a vertical or horizontal orientation of the questions' options. In addition, respondents who answer the questions with larger answer boxes are more likely to write longer comments.

Use of a miniature diamond-anvil cell in high-pressure single-crystal neutron Laue diffraction.

The first high-pressure neutron diffraction study in a miniature diamond-anvil cell of a single crystal of size typical for X-ray diffraction is reported. This is made possible by modern Laue diffraction using a large solid-angle image-plate detector. An unexpected finding is that even reflections whose diffracted beams pass through the cell body are reliably observed, albeit with some attenuation. The cell body does limit the range of usable incident angles, but the crystallographic completeness for a high-symmetry unit cell is only slightly less than for a data collection without the cell. Data collections for two sizes of hexamine single crystals, with and without the pressure cell, and at 300 and 150 K, show that sample size and temperature are the most important factors that influence data quality. Despite the smaller crystal size and dominant parasitic scattering from the diamond-anvil cell, the data collected allow a full anisotropic refinement of hexamine with bond lengths and angles that agree with literature data within experimental error. This technique is shown to be suitable for low-symmetry crystals, and in these cases the transmission of diffracted beams through the cell body results in much higher completeness values than are possible with X-rays. The way is now open for joint X-ray and neutron studies on the same sample under identical conditions.

Abstract TP417: Improving Stroke Rehabilitation Intensity Data Collection: Collaborative Implementation of a Quality Assurance Framework

Background and Issues: A minimum of 3 hours of rehabilitation intensity (RI) is a stroke best practice standard of care that has been established in Ontario. As collection of RI time has been mandated in Ontario, the Ontario Stroke Network has been working with stakeholders to capture RI data using workload measurement systems (WMS). Given that this is a new indicator and variability exists in use of WMS across Toronto, there is a need for standardization in RI data collection. Purpose: To develop a quality assurance (QA) framework for RI data collection in Toronto that can be used locally and provincially to ensure consistent and accurate data collection and reporting. Methods: The Toronto Stroke Networks formed a working group comprised of clinical and decision support leads from 6 rehabilitation centres. This group developed standard approaches to capturing RI data. They also examined factors that influence the quality of the data, ranked these factors by level of influence on data quality, and established mitigation strategies for the top ranked factors. Additionally, a reporting and monitoring plan was determined. Results: Through collaborative sharing of information, a QA framework was developed and adopted by the group. The top ranked factors affecting data quality included: 1) When the RI data field is left blank; 2) Inaccurate data entry by staff; 3) Variations in service interruptions and alternate level of care; and 4) Timeliness in entering data. Mitigation strategies included use of lock out periods, clear definitions, processes for staff feedback and education, and establishment of a quarterly reporting structure for organizational comparison. This QA framework has been shared with provincial stakeholders to inform system planning in other regions. Conclusions: As data accuracy in capturing RI data is important for current state analysis and benchmarking, a comprehensive QA framework was developed to support data accuracy and has been used for local monitoring and reporting of RI data. Identified factors that influence data quality related to processes for data capture. As mitigation strategies will further support these processes, this QA framework ensures accurate data collection and confidence in using RI data to support system planning.

Developing a Workflow to Identify Inconsistencies in Volunteered Geographic Information: A Phenological Case Study.

Recent improvements in online information communication and mobile location-aware technologies have led to the production of large volumes of volunteered geographic information. Widespread, large-scale efforts by volunteers to collect data can inform and drive scientific advances in diverse fields, including ecology and climatology. Traditional workflows to check the quality of such volunteered information can be costly and time consuming as they heavily rely on human interventions. However, identifying factors that can influence data quality, such as inconsistency, is crucial when these data are used in modeling and decision-making frameworks. Recently developed workflows use simple statistical approaches that assume that the majority of the information is consistent. However, this assumption is not generalizable, and ignores underlying geographic and environmental contextual variability that may explain apparent inconsistencies. Here we describe an automated workflow to check inconsistency based on the availability of contextual environmental information for sampling locations. The workflow consists of three steps: (1) dimensionality reduction to facilitate further analysis and interpretation of results, (2) model-based clustering to group observations according to their contextual conditions, and (3) identification of inconsistent observations within each cluster. The workflow was applied to volunteered observations of flowering in common and cloned lilac plants (Syringa vulgaris and Syringa x chinensis) in the United States for the period 1980 to 2013. About 97% of the observations for both common and cloned lilacs were flagged as consistent, indicating that volunteers provided reliable information for this case study. Relative to the original dataset, the exclusion of inconsistent observations changed the apparent rate of change in lilac bloom dates by two days per decade, indicating the importance of inconsistency checking as a key step in data quality assessment for volunteered geographic information. Initiatives that leverage volunteered geographic information can adapt this workflow to improve the quality of their datasets and the robustness of their scientific analyses.

Interviewer effects in real and falsified interviews: Results from a large scale experiment

Interviewers influence data quality in surveys unintentionally or intentionally. Within the framework of theories on interviewers' motivation, we analyze influences of interviewers' characteristics and payment schemes on falsified and real data. The empirical analysis is based on data of a large scale experimental study, which includes both real and falsified interviews. For this experimental study, the interviewers' payment was subject to two different conditions both for real and falsified inter- views, namely payment per completed interview and payment per hour. The impact of payment, gender, and some measures of interviewers' attitudes is analyzed with regard to duration of interviews and some meta-indicators used previously to identify potential falsifications in survey data. Empirical results are presented, and a conclusion is drawn regarding the impact of payment scheme and interviewers' characteristics on data quality.

Secondary use of electronic health records for building cohort studies through top-down information extraction

Controlled clinical trials are usually supported with an in-front data aggregation system, which supports the storage of relevant information according to the trial context within a highly structured environment. In contrast to the documentation of clinical trials, daily routine documentation has many characteristics that influence data quality. One such characteristic is the use of non-standardized text, which is an indispensable part of information representation in clinical information systems. Based on a cohort study we highlight challenges for mining electronic health records targeting free text entry fields within semi-structured data sources. Our prototypical information extraction system achieved an F-measure of 0.91 (precision=0.90, recall=0.93) for the training set and an F-measure of 0.90 (precision=0.89, recall=0.92) for the test set. We analyze the obtained results in detail and highlight challenges and future directions for the secondary use of routine data in general.

Does use of computer technology for perinatal data collection influence data quality?

Population health data, collected worldwide in an effort to monitor mortality and morbidity of mothers and babies, namely, perinatal data, are mandated at a federal level within Australia. The data are used to monitor patterns in midwifery, obstetric and neonatal practice, health outcomes, used for research purposes, funding allocation and education. Accuracy in perinatal data is most often reported via quantitative validation studies of perinatal data collections both internationally and in Australia. These studies report varying levels of accuracy and suggest researchers need to be more aware of the quality of data they use. This article presents findings regarding issues of concern identified by midwives relating to their perceptions of how technology affects the accuracy of perinatal data records. Perinatal data records are perceived to be more complete when completed electronically. However, issues regarding system functionality, the inconsistent use of terminology, lack of data standards and the absence of clear, written records contribute to midwives' perceptions of the negative influence of technology on the quality of perinatal data.

Mass accuracy and isotopic abundance measurements for HR-MS instrumentation: capabilities for non-targeted analyses.

The development of automated non-targeted workflows for small molecule analyses is highly desirable in many areas of research and diagnostics. Sufficient mass and chromatographic resolution is necessary for the detectability of compounds and subsequent componentization and interpretation of ions. The mass accuracy and relative isotopic abundance are critical in correct molecular formulae generation for unknown compounds. While high-resolution instrumentation provides accurate mass information, sample complexity can greatly influence data quality and the measurement of compounds of interest. Two high-resolution instruments, an Orbitrap and a Q-TOF, were evaluated for mass accuracy and relative isotopic abundance with various concentrations of a standard mixture in four complex sample matrices. The overall average ± standard deviation of the mass accuracy was 1.06 ± 0.76ppm and 1.62 ± 1.88ppm for the Orbitrap and the Q-TOF, respectively; however, individual measurements were ± 5ppm for the Orbitrap and greater than 10ppm for the Q-TOF. Relative isotopic abundance measurements for A + 1 were within 5% of the theoretical value if the intensity of the monoisotopic peak was greater than 1E7 for the Orbitrap and 1E5 for the Q-TOF, where an increase in error is observed with a decrease in intensity. Furthermore, complicating factors were found in the data that would impact automated data analysis strategies, including coeluting species that interfere with detectability and relative isotopic abundance measurements. The implications of these findings will be discussed with an emphasis on reasonable expectations from these instruments, guidelines for experimental workflows, data analysis considerations, and software design for non-targeted analyses.

On measurements and their quality. Paper 4: Verbal anchors and the number of response options in rating scales

This is the last in a short series of papers on measurement theory and practice with particular relevance to intervention research in nursing, midwifery, and healthcare. Understanding how it is that people respond to the questions posed by researchers is fundamental to progress in the social and health sciences. For decades methodologists in psychology, marketing, education, and survey research have studied this issue. In this paper I review this diverse empirical literature to synthesize basic principles for creating rating scales which can reduce measurement error and increase the quality of resulting data. After introducing a theoretical framework known as the cognitive aspects of survey methods (CASM), I review the fundamentals of psychological scaling theory and discuss how it has been used to study the meanings of verbal response options and provide an illustration of how the quality of measurements may be influenced by our choice of the verbal phrases we present as response options. Next, I review the research on the optimal number of response options to use in various measurement situations and how verbal and numeric anchors can combine to influence data quality. Finally, I summarize the issues covered and present recommendations for best practice when creating and using rating scales in research.

The influence of calibration method and eye physiology on eyetracking data quality

Recording eye movement data with high quality is often a prerequisite for producing valid and replicable results and for drawing well-founded conclusions about the oculomotor system. Today, many aspects of data quality are often informally discussed among researchers but are very seldom measured, quantified, and reported. Here we systematically investigated how the calibration method, aspects of participants' eye physiologies, the influences of recording time and gaze direction, and the experience of operators affect the quality of data recorded with a common tower-mounted, video-based eyetracker. We quantified accuracy, precision, and the amount of valid data, and found an increase in data quality when the participant indicated that he or she was looking at a calibration target, as compared to leaving this decision to the operator or the eyetracker software. Moreover, our results provide statistical evidence of how factors such as glasses, contact lenses, eye color, eyelashes, and mascara influence data quality. This method and the results provide eye movement researchers with an understanding of what is required to record high-quality data, as well as providing manufacturers with the knowledge to build better eyetrackers.

Quantitative and qualitative verification of data quality in the childbirth registers of two rural district hospitals in Western Kenya

Objectiveto verify the data quality of childbirth registers and explore factors that influence quality at two rural district hospitals in Western Kenya. Designa retrospective comparative case study for data quality of the 2006 childbirth registers by quantitative and qualitative methods. SettingSiaya and Bondo District Hospitals. Methodsafter confirming the physical condition and availability of childbirth registers, the total number of births; number of complete/incomplete data; and number of complete data that were illegible, incorrectly coded, inappropriate and unrecognised were verified quantitatively to evaluate accuracy and completeness. Data categories and instructions were examined qualitatively to assess the relevance, completeness and accuracy of the data. Semi-structured interviews were conducted with key informants to capture their views and factors that influence data quality. Findingsthe childbirth registers used by the two hospitals were not developed by the Ministry of Health, and their supply to Bondo was interrupted. Of the 30 data categories in the registers, five for Siaya and 23 for Bondo were more than 20% incomplete. Data for number of antenatal consultations and use of human immunodeficiency virus drugs were at least 50% incomplete for both hospitals. The percentage of illegible, incorrectly coded and inappropriate data was relatively low, and only the place of residence had unrecognised data. Data categories in the registers did not correspond well with those of monthly reports, and inappropriate instructions suggested hidden inaccuracy among apparently valid data. Organisational impediments of the health information system in general, perinatal and intrapartum contexts were identified. Key conclusionsdata quality of the childbirth registers was unsatisfactory. Influential factors were primarily organisational and technical, which may have had an adverse effect on midwives' record keeping behaviour. Implications for practicedata quality of the registers can be improved by re-examining technical challenges and organisational impediments at different levels. Midwives' awareness of data quality needs to be increased by sharing the purpose of the childbirth registers. Strong political commitment is also indispensable for putting these findings into action.

Are Internet surveys an alternative to face-to-face interviews in contingent valuation?

Internet is an increasingly popular data collection mode for stated preference research in environmental economics. However, little is known about how this survey mode may influence data quality and welfare estimates. As part of a national contingent valuation (CV) survey estimating willingness to pay (WTP) for biodiversity protection plans, we assign two groups of respondents either to an Internet or face-to-face (in-home) interview mode. Our design aims to better isolate measurement effects from sample composition effects by drawing both samples from the same sample frame. We find little evidence of social desirability bias in the interview setting or satisficing (shortcutting the response process) in the Internet survey. The share of “don't knows”, zeros and protest responses to the WTP question with a payment card is very similar between modes and equality of mean WTP cannot be rejected. Results are fairly encouraging for the use of Internet in CV as stated preferences do not seem to be significantly different or biased compared to face-to-face interviews.

Factors That Influence Data Quality in Caries Experience Detection: A Multilevel Modeling Approach

Caries experience detection is prone to misclassification. For this reason, calibration exercises which aim at assessing and improving the scoring behavior of dental raters are organized. During a calibration exercise, a sample of children is examined by the benchmark scorer and the dental examiners. This produces a 2 × 2 contingency table with the true and possibly misclassified responses. The entries in this misclassification table allow to estimate the sensitivity and the specificity of the raters. However, in many dental studies, the uncertainty with which sensitivity and specificity are estimated is not expressed. Further, caries experience data have a hierarchical structure since the data are recorded for the surfaces nested in the teeth within the mouth. Therefore, it is important to report the uncertainty using confidence intervals and to take the clustering into account. Here we apply a Bayesian logistic multilevel model for estimating the sensitivity and specificity. The main goal of this research is to find the factors that influence the true scoring of caries experience accounting for the hierarchical structure in the data. In our analysis, we show that the dentition type and tooth or surface type affect the quality of caries experience detection.