Quality Assessment Framework Research Articles

Automated Quality Assessment of Metadata across Open Data Portals

The Open Data movement has become a driver for publicly available data on the Web. More and more data—from governments and public institutions but also from the private sector—are made available online and are mainly published in so-called Open Data portals. However, with the increasing number of published resources, there is a number of concerns with regards to the quality of the data sources and the corresponding metadata, which compromise the searchability, discoverability, and usability of resources. In order to get a more complete picture of the severity of these issues, the present work aims at developing a generic metadata quality assessment framework for various Open Data portals: We treat data portals independently from the portal software frameworks by mapping the specific metadata of three widely used portal software frameworks (CKAN, Socrata, OpenDataSoft) to the standardized Data Catalog Vocabulary metadata schema. We subsequently define several quality metrics, which can be evaluated automatically and in an efficient manner. Finally, we report findings based on monitoring a set of over 260 Open Data portals with 1.1M datasets. This includes the discussion of general quality issues, for example, the retrievability of data, and the analysis of our specific quality metrics.

Quantifying blur in colour images using higher order singular values

This Letter introduces a novel framework for blind blur assessment in colour images using higher order singular values. The RGB colour image is seen as a third-order tensor to exploit the spatial and inter-channel correlations, so that blurring effects are captured more robustly. The tensor is decomposed into different two-dimensional matrices, also called unfoldings. The conventional singular value decomposition is carried out for these unfoldings instead of computing it for the luminance component alone. The experiments were performed on several publicly available databases and the results validate the superiority of the proposed metric among different state-of-the-art blind blur assessment metrics. The proposed framework for image quality assessment (IQA) from colour images fits well with the current trends and research efforts put in enhancing the quality of experience for different multimedia applications and in benchmarking new imaging and sensing technologies including camera and other vision systems with IQA capabilities.

Learning Blind Quality Evaluator for Stereoscopic Images Using Joint Sparse Representation

Perceptual quality prediction for stereoscopic images is of fundamental importance in determining the level of quality perceived by humans in terms of the 3D viewing experience. However, the existing no-reference quality assessment (NR-IQA) framework has its limitation in addressing binocular combination for stereoscopic images. In this paper, we propose a new NR-IQA for stereoscopic images using joint sparse representation. We analyze the relationship between left and right quality predictors, and formulate stereoscopic quality prediction as a combination of feature-prior and feature-distribution. Based on this finding, we extract feature vector that handles different features to be interacted by joint sparse representation, and use support vector regression to characterize feature-prior. Meanwhile, we implement feature-distribution using sparsity regularization as the basis of weights for binocular combination to derive the overall quality score. Experimental results on five public 3D IQA databases demonstrate that in comparison with the existing methods, the devised algorithm achieves high consistent alignment with subjective assessment.

A Harmonized Data Quality Assessment Terminology and Framework for the Secondary Use of Electronic Health Record Data.

Objective:Harmonized data quality (DQ) assessment terms, methods, and reporting practices can establish a common understanding of the strengths and limitations of electronic health record (EHR) data for operational analytics, quality improvement, and research. Existing published DQ terms were harmonized to a comprehensive unified terminology with definitions and examples and organized into a conceptual framework to support a common approach to defining whether EHR data is ‘fit’ for specific uses.Materials and Methods:DQ publications, informatics and analytics experts, managers of established DQ programs, and operational manuals from several mature EHR-based research networks were reviewed to identify potential DQ terms and categories. Two face-to-face stakeholder meetings were used to vet an initial set of DQ terms and definitions that were grouped into an overall conceptual framework. Feedback received from data producers and users was used to construct a draft set of harmonized DQ terms and categories. Multiple rounds of iterative refinement resulted in a set of terms and organizing framework consisting of DQ categories, subcategories, terms, definitions, and examples. The harmonized terminology and logical framework’s inclusiveness was evaluated against ten published DQ terminologies.Results:Existing DQ terms were harmonized and organized into a framework by defining three DQ categories: (1) Conformance (2) Completeness and (3) Plausibility and two DQ assessment contexts: (1) Verification and (2) Validation. Conformance and Plausibility categories were further divided into subcategories. Each category and subcategory was defined with respect to whether the data may be verified with organizational data, or validated against an accepted gold standard, depending on proposed context and uses. The coverage of the harmonized DQ terminology was validated by successfully aligning to multiple published DQ terminologies.Discussion:Existing DQ concepts, community input, and expert review informed the development of a distinct set of terms, organized into categories and subcategories. The resulting DQ terms successfully encompassed a wide range of disparate DQ terminologies. Operational definitions were developed to provide guidance for implementing DQ assessment procedures. The resulting structure is an inclusive DQ framework for standardizing DQ assessment and reporting. While our analysis focused on the DQ issues often found in EHR data, the new terminology may be applicable to a wide range of electronic health data such as administrative, research, and patient-reported data.Conclusion:A consistent, common DQ terminology, organized into a logical framework, is an initial step in enabling data owners and users, patients, and policy makers to evaluate and communicate data quality findings in a well-defined manner with a shared vocabulary. Future work will leverage the framework and terminology to develop reusable data quality assessment and reporting methods.

Developing information quality assessment framework of presentation slides

Computerized presentation slides have become essential for many occasions such as business meetings, classroom discussions, multipurpose talks and public events. Given the tremendous increases in online resources and materials, locating high-quality slides relevant to a given task is often a formidable challenge, particularly when a user looks for superior quality slides. This study proposes a new, comprehensive framework for information quality (IQ) developed specifically for computerized presentation slides and explores the possibility of automatically detecting the IQ of slides. To determine slide-specific IQ criteria as well as their relative importances, we carried out a user study, involving 60 participants from two universities, and conducted extensive coding analysis. Further, we subsequently conducted a series of multiple experiments to examine the validity of the IQ features developed on the basis of the selected criteria from the user study. The study findings contribute to identifying key dimensions and related features that can improve effective IQ assessments of computerized presentation slides.

Patient-centric quality assessment framework for healthcare services

In this study we propose a quality assessment framework for healthcare services. The proposed assessment framework is patient centric as it is based on patient expectations and perceived quality of service through their personal experience with the healthcare provider, across the various touchpoints during patient care. The framework generates a quality score which is a measure of the degree to which the patient's expectations were met or exceeded. We model the patient's perceived experiential value as a combination of extrinsic and intrinsic values. The extrinsic value includes the functional or utilitarian value, and to some extent, the social value. The intrinsic value includes emotional, epistemic and intrinsic social value. Using this generic framework for healthcare providers, and using a computerized system, appropriate instruments, rubrics or metrics can be designed for specific types of healthcare services. We show how this framework can be utilized for creating an assessment instrument for a specific healthcare facility in Korea – the Childhood Asthma Atopy Center in a general hospital located in Korea.

Using primary care electronic health record data for comparative effectiveness research: experience of data quality assessment and preprocessing in The Netherlands

Details of data quality and how quality issues were solved have not been reported in published comparative effectiveness studies using electronic health record data. We developed a conceptual framework of data quality assessment and preprocessing and apply it to a study comparing angiotensin-converting enzyme inhibitors with angiotensin receptor blockerss on renal function decline in diabetes patients. The framework establishes a line of thought to identify and act on data issues. The core concept is to evaluate whether data are fit-for-use for research tasks. Possible quality problems are listed through specific signal detections, and verified whether they are true problems. Optimal solutions are selected for the identified problems. This framework can be used in observational studies to improve validity of results.

No-reference image quality assessment using statistical wavelet-packet features

In this paper an efficient no-reference (NR) image quality assessment (IQA) method is presented based on the statistical features of subband coefficients in the wavelet-packet domain. The proposed method is based on the hypothesis that potential distortions may alter the statistical characteristics of natural un-distorted images. Hence, by characterizing the statistical properties of a given distorted image one can identify the distortion and its strength in the distorted image. For this purpose, several statistical features of a given gray-scale image as well as the magnitude of its gradient and its Laplacian are extracted in the wavelet-packet domain. The extracted features are then mapped to quality scores within a two-stage quality assessment framework. The proposed method is general-purpose, and is able to assess the image quality across various distortion categories. Experimental results indicate that the proposed method achieves high accuracy in image quality prediction as compared to several prominent and state-of-the-art full-reference and no-reference IQA methods.

No-Reference Video Quality Assessment With 3D Shearlet Transform and Convolutional Neural Networks

In this paper, we propose an efficient general-purpose no-reference (NR) video quality assessment (VQA) framework that is based on 3D shearlet transform and convolutional neural network (CNN). Taking video blocks as input, simple and efficient primary spatiotemporal features are extracted by 3D shearlet transform, which are capable of capturing natural scene statistics properties. Then, CNN and logistic regression are concatenated to exaggerate the discriminative parts of the primary features and predict a perceptual quality score. The resulting algorithm, which we name shearlet- and CNN-based NR VQA (SACONVA), is tested on well-known VQA databases of Laboratory for Image & Video Engineering, Image & Video Processing Laboratory, and CSIQ. The testing results have demonstrated that SACONVA performs well in predicting video quality and is competitive with current state-of-the-art full-reference VQA methods and general-purpose NR-VQA algorithms. Besides, SACONVA is extended to classify different video distortion types in these three databases and achieves excellent classification accuracy. In addition, we also demonstrate that SACONVA can be directly applied in real applications such as blind video denoising.

The effect of depth map up-sampling on the overall quality of stereopairs

The quality of depth maps affects the quality of generated 3D content. Practically, the depth maps often have lower resolution than that of color images, thus, Depth map Up-sampling (DU) is needed in various 3D applications. DU can yield specific artifacts which can degrade the quality of depth maps as well as constructed stereoscopic (color plus depth map) images. This paper investigates the effect of DU on 3D perception. The depth maps were up-sampled using seven approaches and the quality of stereoscopic images obtained from up-sampled depth maps was estimated through subjective and objective tests. The objective quality prediction was performed using a depth map quality assessment framework. The method is able to predict the quality of stereoscopic images through evaluation of their corresponding up-sampled depth maps using 2D Image Quality Metrics (IQMs). In order to improve the quality estimation, the framework selects the 2D IQMs with highest correlation to subjective test. Furthermore, motivated by previous researches on multiple metrics combination, a new metric fusion method is proposed. Experimental results show that the combined metric delivers higher performance than single metrics in 3D quality prediction.

Data quality assessment framework to assess electronic medical record data for use in research

IntroductionThe proliferation and use of electronic medical records (EMR) in the clinical setting now provide a rich source of clinical data that can be leveraged to support research on patient outcomes, comparative effectiveness, and health systems research. Once the large volume and variety of data that robust clinical EMRs provide is aggregated, the suitability of the data for research purposes must be addressed. Therefore, the purpose of this paper is two-fold. First, we present a stepwise framework capable of guiding initial data quality assessment when matching multiple data sources regardless of context or application. Then, we demonstrate a use case of initial analysis of a longitudinal data repository of electronic health record data that illustrates the first four steps of the framework, and report results. MethodsA six-step data quality assessment framework is proposed and described that includes the following data quality assessment steps: (1) preliminary analysis, (2) documentation–longitudinal concordance, (3) breadth, (4) data element presence, (5) density, and (6) prediction. The six-step framework was applied to the Transport Data Mart—a data repository that contains over 28,000 records for patients that underwent interhospital transfer that includes EMRs from the sending hospitalization, transport, and receiving hospitalization. ResultsThere were a total of 9557 log entries of which 8139 were successfully matched to corresponding hospital encounters. 2832 were successfully mapped to both the sending and receiving hospital encounters (resulting in a 93% automatic matching rate), with 590 including air medical transport EMR data representing a complete case for testing. Results from Step 2 indicate that once records are identified and matched, there appears to be relatively limited drop-off of additional records when the criteria for matching increases, indicating the a proportion of records consistently contain nearly complete data. Measures of central tendency used in Step 3 and 4 exhibit a right skewness suggesting that a small proportion of records contain the highest number of repeated measures for the measured variables. ConclusionsThe proposed six-step data quality assessment framework is useful in establishing the metadata for a longitudinal data repository that can be replicated by other studies. There are practical issues that need to be addressed including the data quality assessments—with the most prescient being the need to establish data quality metrics for benchmarking acceptable levels of EMR data inclusiveness through testing and application.

A framework for quality assessment of just-in-time requirements: the case of open source feature requests

Until now, quality assessment of requirements has focused on traditional up-front requirements. Contrasting these traditional requirements are just-in-time (JIT) requirements, which are by definition incomplete, not specific and might be ambiguous when initially specified, indicating a different notion of correctness. We analyze how the assessment of JIT requirements quality should be performed based on the literature of traditional and JIT requirements. Based on that analysis, we have designed a quality framework for JIT requirements and instantiated it for feature requests in open source projects. We also indicate how the framework can be instantiated for other types of JIT requirements. We have performed an initial evaluation of our framework for feature requests with eight practitioners from the Dutch agile community, receiving overall positive feedback. Subsequently, we have used our framework to assess 550 feature requests originating from three Open Source Software systems (Netbeans, ArgoUML and Mylyn Tasks). In doing so, we obtain a view on the feature request quality for the three open source projects. The value of our framework is threefold: (1) it gives an overview of quality criteria that are applicable to feature requests (at creation time or JIT); (2) it serves as a structured basis for teams that need to assess the quality of their JIT requirements; and (3) it provides a way to get an insight into the quality of JIT requirements in existing projects.

A color intensity invariant low-level feature optimization framework for image quality assessment

Image Quality Assessment (IQA) algorithms evaluate the perceptual quality of an image using evaluation scores that assess the similarity or difference between two images. We propose a new low-level feature based IQA technique, which applies filter-bank decomposition and center-surround methodology. Differing from existing methods, our model incorporates color intensity adaptation and frequency scaling optimization at each filter-bank level and spatial orientation to extract and enhance perceptually significant features. Our computational model exploits the concept of object detection and encapsulates characteristics proposed in other IQA algorithms in a unified architecture. We also propose a systematic approach to review the evolution of IQA algorithms using unbiased test datasets, instead of looking at individual scores in isolation. Experimental results demonstrate the feasibility of our approach.

Evaluating the impact of working memory training programmes on children – a systematic review

Background:An increasing body of evidence suggests that working memory can have a significant effect on the academic success of children in school (Gathercole & Alloway, 2008). As such a number of working memory training programmes have been developed and marketed commercially to schools and parents as tools for helping children to boost their working memory and as a result, their learning ability.Aim:The reliability and validity of this body of evidence has been challenged in recent years with questions raised as to whether such programmes actually do boost working memory or show any subsequent impact upon learning (Apter, 2012). This paper aims to examine eight studies published within the last 10 years that explore the effects of commercially available working memory training on working memory and various other factors including ADHD, literacy and numeracy.Rationale/Approach:Drawing on systematic literature review methodology and using study quality assessment frameworks, the paper examines the evidence for the impact of working memory training programmes upon working memory and other neuro-behavioural and neuro-cognitive factors relevant to teaching and learning.Findings/Conclusion:The studies examined used samples from a range of ages (5 to 12 years) and with varied demographic, behavioural and cognitive characteristics. The review indicated that although significant near and far transfer effects for improvements to working memory were found, there were some differences in the pattern of these improvements, with different studies finding different sub-components of working memory benefiting over others. The review also found that the evidence that working memory training leads to improvements in other areas such as literacy and numeracy is inconclusive at best, and that further research will be required for us to better understand the mechanism through which working memory training may improve working memory and academic performance in children.

Garbage in, Garbage Out: Data Collection, Quality Assessment and Reporting Standards for Social Media Data Use in Health Research, Infodemiology and Digital Disease Detection.

BackgroundSocial media have transformed the communications landscape. People increasingly obtain news and health information online and via social media. Social media platforms also serve as novel sources of rich observational data for health research (including infodemiology, infoveillance, and digital disease detection detection). While the number of studies using social data is growing rapidly, very few of these studies transparently outline their methods for collecting, filtering, and reporting those data. Keywords and search filters applied to social data form the lens through which researchers may observe what and how people communicate about a given topic. Without a properly focused lens, research conclusions may be biased or misleading. Standards of reporting data sources and quality are needed so that data scientists and consumers of social media research can evaluate and compare methods and findings across studies.ObjectiveWe aimed to develop and apply a framework of social media data collection and quality assessment and to propose a reporting standard, which researchers and reviewers may use to evaluate and compare the quality of social data across studies.MethodsWe propose a conceptual framework consisting of three major steps in collecting social media data: develop, apply, and validate search filters. This framework is based on two criteria: retrieval precision (how much of retrieved data is relevant) and retrieval recall (how much of the relevant data is retrieved). We then discuss two conditions that estimation of retrieval precision and recall rely on—accurate human coding and full data collection—and how to calculate these statistics in cases that deviate from the two ideal conditions. We then apply the framework on a real-world example using approximately 4 million tobacco-related tweets collected from the Twitter firehose.ResultsWe developed and applied a search filter to retrieve e-cigarette–related tweets from the archive based on three keyword categories: devices, brands, and behavior. The search filter retrieved 82,205 e-cigarette–related tweets from the archive and was validated. Retrieval precision was calculated above 95% in all cases. Retrieval recall was 86% assuming ideal conditions (no human coding errors and full data collection), 75% when unretrieved messages could not be archived, 86% assuming no false negative errors by coders, and 93% allowing both false negative and false positive errors by human coders.ConclusionsThis paper sets forth a conceptual framework for the filtering and quality evaluation of social data that addresses several common challenges and moves toward establishing a standard of reporting social data. Researchers should clearly delineate data sources, how data were accessed and collected, and the search filter building process and how retrieval precision and recall were calculated. The proposed framework can be adapted to other public social media platforms.

Client Satisfaction with Delivery Care Service and Associated Factors in the Public Health Facilities of Gamo Gofa Zone, Southwest Ethiopia: In a Resource Limited Setting.

Background. Ensuring patient satisfaction is an important means of secondary prevention of maternal mortality. This study presents findings from a multidimensional study of client satisfaction from the Gamo Gofa Zone in Southwest Ethiopia. Methods. A facility based cross-sectional study using exit interviews was conducted from 2014. Client satisfaction was measured using a survey adopted from the Donabedian quality assessment framework. Thirteen health institutions were randomly sampled of 66 institutions in Gamo Gofa Zone. Logistic regression was used to determine predictors of client satisfaction. Results. The overall satisfaction level of the clients in this study was 79.1% with (95% CI; 75–82). Women attending health centres were more likely satisfied than women attending hospitals (χ 2 = 83.7, df = 12, P < 0.001). The proportion of women who complained about an unfriendly attitude or unresentful care from health workers was higher in the hospitals (χ 2 = 27.4, df = 1, P < 0.001). The presence of support persons during child birth improved client satisfaction (AOR = 6.23 95% CI; 2.75–14.1) and women who delivered with caesarean section are four times more likely satisfied than those who deliver vaginally (AOR 3.6 95% CI; 1.44–9.06). Client satisfaction was reduced if the women had to pay for the services (AOR = 0.27 95% CI; 0.09–81). Conclusions. The study shows that overall satisfaction level is good. More emphasis should be put on giving women friendly care, particularly at the hospitals.

Study on the Effects of Human Intention on Spatial Data Quality

GIScience 2016 Short Paper Proceedings Study on the Effects of Human Intention on Spatial Data Quality Bo Zhao College of Earth, Ocean, and Atmospheric Sciences, Oregon State University, 101 SW 26th St, Corvallis, OR 97331 Email: zhao2@oregonstate.edu Abstract The GIScience community has devoted considerable research to locational uncertainty rather than other types of spatial data quality issues. Today, the flood of spatial big data has brought about new concerns, such as location spoofing, GPS jamming, or AIS (Automatic Identification System) hacking. Yet, the current data quality assessment framework falls short in defining, interpreting and analyzing these critical issues. By examining the reasons for measuring the geographic world, I suggest a modification of the hypothesis of the rational geographer in this paper, and further to analyze the distinctions among mistakes, spoofing and uncertainties, with the goal of placing the identified types of locational inconsistencies into a more holistic theoretical framework for spatial data quality. I call on GIScientists to pay more attention to the role of human intentions and advocate for a human-centric assessment of spatial data generation. Only then can we more effectively handle the emerging quality issues in the era of big data. 1. Introduction The GIScience community frequently focuses on uncertainty with regard to spatial data quality (Devillers et al., 2010). In the era of big data, the advent of mobile, social and geospatial technologies has created a considerable degree of heterogeneous, real-time and geo-referenced data. A large percentage of such data, especially VGI (Volunteered Geographic Information), geo- tagged social media, or location based service feeds, may be generated by ones’ mistakes and/or created deliberately rather than being merely affected by inherent uncertainties. Although geographers have recognized the significance of human intention, the motivations of the data generator are seldom examined, and the recent popularity of spoofings, especially those in the form of location spoofing (Zhao, 2015), GPS jamming (Grant et al., 2009) and AIS hacking (McCauley et al., 2016) are often dismissed by GIScientists. In order to more effectively address this critical issue, I will investigate the role of human intentions in the process of spatial data generation and clearly distinguish among uncertainty, mistakes and spoofing. In the remainder of this paper, I will review the concepts of error, accuracy, precision, uncertainty, mistakes, and spoofing in the context of geography. Then, I will examine the role of human motivation with regard to spatial data quality, and end with a brief concluding remark. 2. Uncertainty, mistakes, and spoofing Since human beings are forced to view the world through a fuzzy and distorting lens, the measured data are inevitably generalized, approximated, and subject to uncertainties (Zhang & Goodchild, 2002). In other words, the way we observe the world has invoked an inevitable locational (or positional) inconsistency between any observed object in the geographic world and the data that it produces (value of the object being measured). When referring to such locational inconsistencies, geographers usually consider them to be underlying uncertainties. One older and simpler term to describe uncertainty is error. By definition, error is the difference between the measured value and the “true” value of the object being measured. It is represented as an estimation of the range of values within which the true value is likely to be found. There are two types of errors: systematic

Resting blood pressure reductions following isometric handgrip exercise training and the impact of age and sex: protocol for a systematic review.

BackgroundThe risk of developing cardiovascular disease is directly correlated to one’s resting blood pressure (BP), age, and biological sex. Resting BP can be reduced using handgrip exercise training, but the impact of age and sex on the effectiveness of training is not well documented.Methods/designA systematic search of the literature will be conducted for all experimental studies (including randomized controlled trials and prospective experiments) that report the influence of isometric handgrip exercise training on resting systolic blood pressure. The databases Medline, Embase, Cochrane Reviews, Cumulative Index to Nursing and Allied Health Literature (CINAHL), SPORTDiscus, Web of Science, Allied and Complementary Medicine (AMED), PubMed, and Scopus will be searched until 1 December 2015. Screening of potential articles, data abstraction, and quality appraisal will be completed in duplicate independently. When necessary, corresponding authors will be contacted in order to facilitate the separation of pooled data into age and sex categories. Methodological quality will be determined using the Quality Assessment Framework developed by the Cochrane Collaboration and the Newcastle-Ottawa Quality Assessment Scale as appropriate. Any discrepancies will be resolved by a third author. Findings will be presented in accordance with the preferred reporting items for systematic reviews and meta-analysis (PRISMA) guidelines.DiscussionThis systematic review will determine the overall effectiveness of handgrip exercise training in improving resting blood pressure. A novel, focused assessment will contrast effectiveness of handgrip training based on the age (younger 18–54 years, older >55 years) and the sex (men, women) of study participants. This information is essential to consolidate before moving forward with the development and implementation of handgrip exercise training programmes which are designed to best meet the needs of particular cohorts.Systematic review registrationPROSPERO CRD42015019792Electronic supplementary materialThe online version of this article (doi:10.1186/s13643-015-0164-6) contains supplementary material, which is available to authorized users.

Quality of antenatal and childbirth care in rural health facilities in Burkina Faso, Ghana and Tanzania: an intervention study

To assess the impact of an intervention consisting of a computer-assisted clinical decision support system and performance-based incentives, aiming at improving quality of antenatal and childbirth care. Intervention study in rural primary healthcare (PHC) facilities in Burkina Faso, Ghana and Tanzania. In each country, six intervention and six non-intervention PHC facilities, located in one intervention and one non-intervention rural districts, were selected. Quality was assessed in each facility by health facility surveys, direct observation of antenatal and childbirth care, exit interviews, and reviews of patient records and maternal and child health registers. Findings of pre- and post-intervention and of intervention and non-intervention health facility quality assessments were analysed and assessed for significant (P < 0.05) quality of care differences. Post-intervention quality scores do not show a clear difference to pre-intervention scores and scores at non-intervention facilities. Only a few variables had a statistically significant better post-intervention quality score and when this is the case this is mostly observed in only one study-arm, being pre-/post-intervention or intervention/non-intervention. Post-intervention care shows similar deficiencies in quality of antenatal and childbirth care and in detection, prevention, and management of obstetric complications as at baseline and non-intervention study facilities. Our intervention study did not show a significant improvement in quality of care during the study period. However, the use of new technology seems acceptable and feasible in rural PHC facilities in resource-constrained settings, creating the opportunity to use this technology to improve quality of care.

Defining and assessing research quality in a transdisciplinary context

Research increasingly seeks both to generate knowledge and to contribute to real-world solutions, with strong emphasis on context and social engagement. As boundaries between disciplines are crossed, and as research engages more with stakeholders in complex systems, traditional academic definitions and criteria of research quality are no longer sufficient—there is a need for a parallel evolution of principles and criteria to define and evaluate research quality in a transdisciplinary research (TDR) context. We conducted a systematic review to help answer the question: What are appropriate principles and criteria for defining and assessing TDR quality? Articles were selected and reviewed seeking: arguments for or against expanding definitions of research quality, purposes for research quality evaluation, proposed principles of research quality, proposed criteria for research quality assessment, proposed indicators and measures of research quality, and proposed processes for evaluating TDR. We used the information from the review and our own experience in two research organizations that employ TDR approaches to develop a prototype TDR quality assessment framework, organized as an evaluation rubric. We provide an overview of the relevant literature and summarize the main aspects of TDR quality identified there. Four main principles emerge: relevance, including social significance and applicability; credibility, including criteria of integration and reflexivity, added to traditional criteria of scientific rigor; legitimacy, including criteria of inclusion and fair representation of stakeholder interests, and; effectiveness, with criteria that assess actual or potential contributions to problem solving and social change.