Use Of Quality Scores Research Articles

Comparative Analysis of the Clustering Quality in Self-Organizing Maps for Human Posture Classification.

The objective of this article is to develop a methodology for selecting the appropriate number of clusters to group and identify human postures using neural networks with unsupervised self-organizing maps. Although unsupervised clustering algorithms have proven effective in recognizing human postures, many works are limited to testing which data are correctly or incorrectly recognized. They often neglect the task of selecting the appropriate number of groups (where the number of clusters corresponds to the number of output neurons, i.e., the number of postures) using clustering quality assessments. The use of quality scores to determine the number of clusters frees the expert to make subjective decisions about the number of postures, enabling the use of unsupervised learning. Due to high dimensionality and data variability, expert decisions (referred to as data labeling) can be difficult and time-consuming. In our case, there is no manual labeling step. We introduce a new clustering quality score: the discriminant score (DS). We describe the process of selecting the most suitable number of postures using human activity records captured by RGB-D cameras. Comparative studies on the usefulness of popular clustering quality scores-such as the silhouette coefficient, Dunn index, Calinski-Harabasz index, Davies-Bouldin index, and DS-for posture classification tasks are presented, along with graphical illustrations of the results produced by DS. The findings show that DS offers good quality in posture recognition, effectively following postural transitions and similarities.

Finger Vein Image Quality Evaluation based on Support Vector Regression

It has been found that poor quality images decrease the performance of finger vein recognition system, due to missing, vague or spurious features. Therefore, it is important for a finger vein recognition system to evaluate the quality of finger vein images. In this paper, a new method based on Support Vector Regression (SVR) is proposed for finger vein image quality evaluation. In our method, we first manually annotate quality scores for finger vein images in training set and extract five quality features of these images. Then quality scores and quality features are used to build a SVR model, which will be applied to evaluate quality for testing images. In addition, we explore the use of quality score and ascertain that quality score can be used as ancillary information to enhance recognition accuracy for finger vein. Experimental results show that our proposed method is effective for finger vein image quality evaluation.

Evidence Synthesis for Medical Decision Making and the Appropriate Use of Quality Scores

Meta-analyses today continue to be run using conventional random-effects models that ignore tangible information from studies such as the quality of the studies involved, despite the expectation that results of better quality studies reflect more valid results. Previous research has suggested that quality scores derived from such quality appraisals are unlikely to be useful in meta-analysis, because they would produce biased estimates of effects that are unlikely to be offset by a variance reduction within the studied models. However, previous discussions took place in the context of such scores viewed in terms of their ability to maximize their association with both the magnitude and direction of bias. In this review, another look is taken at this concept, this time asserting that probabilistic bias quantification is not possible or even required of quality scores when used in meta-analysis for redistribution of weights. The use of such a model is contrasted with the conventional random effects model of meta-analysis to demonstrate why the latter is inadequate in the face of a properly specified quality score weighting method.

Updates to the RMAP short-read mapping software

Summary: We report on a major new version of the RMAP software for mapping reads from short-read sequencing technology. General improvements to accuracy and space requirements are included, along with novel functionality. Included in the RMAP software package are tools for mapping paired-end reads, mapping using more sophisticated use of quality scores, collecting ambiguous mapping locations and mapping bisulfite-treated reads.Availability: The applications described in this note are available for download at http://www.cmb.usc.edu/people/andrewds/rmap and are distributed as Open Source software under the GPLv3.0. The software has been tested on Linux and OS X platforms.Contact: andrewds@usc.edu; mzhang@cshl.eduThe RMAP algorithm was introduced by (Smith et al., 2008) as one of the earliest available programs for mapping reads from the Illumina second-generation sequencing technology. One important contribution of RMAP was to incorporate the use of quality scores directly into the mapping process: read positions with too low a quality score were not considered while mapping, and that quality score cutoff could be adjusted by the user. Subsequently, numerous mapping algorithm have appeared (Langmead et al., 2009; Li,H. et al., 2008; Li,R. et al., 2008; Lin et al., 2008; Schatz, 2009; Yanovsky et al., 2008), with improvements in both efficiency and breadth of functionality (e.g. ability to map paired-end reads; integrated SNP calling). Investigators requiring solutions to mapping problems now have many options. As new applications of short-read sequencing emerge, many variations on the analysis task of read mapping emerge. Diversity in performance characteristics of existing mapping tools becomes potentially valuable.We report the first major update to RMAP. The basic algorithmic framework in RMAP is still to preprocess reads and scan the genome, but several modifications have been made and much additional functionality has been included. Importantly, RMAP has a memory footprint that depends on the number of reads being mapped. This feature allows RMAP to be used effectively in cluster environments with commodity nodes, because partitioning the reads allows natural parallelizations with linear reduction in memory requirements per processor core used.Included in this release of the RMAP software package is functionality for mapping paired-end reads, making more sophisticated use of quality scores, collecting mapping locations for ambiguously mapping reads and mapping bisulfite-treated reads.

The development and purpose of the FREDERICA radiation effects database

Any system for assessing the impact of a contaminant on the environment requires an analysis of the possible effects on the organisms and ecosystems concerned. To facilitate this, the FREDERICA radiation effects database has been developed to provide an online search of the known effects of ionising radiation on non-human species, taken from papers in the scientific peer reviewed literature. The FREDERICA radiation effects database has been produced by merging the work done on radiation effects under two European funded projects (FASSET and EPIC) and making the database available online. This paper highlights applications for the database, gaps in the available data and explains the use of quality scores to help users of the database determine which papers may benefit their research in terms of techniques and reproducibility.

Using quality scores and longer reads improves accuracy of Solexa read mapping

BackgroundSecond-generation sequencing has the potential to revolutionize genomics and impact all areas of biomedical science. New technologies will make re-sequencing widely available for such applications as identifying genome variations or interrogating the oligonucleotide content of a large sample (e.g. ChIP-sequencing). The increase in speed, sensitivity and availability of sequencing technology brings demand for advances in computational technology to perform associated analysis tasks. The Solexa/Illumina 1G sequencer can produce tens of millions of reads, ranging in length from ~25–50 nt, in a single experiment. Accurately mapping the reads back to a reference genome is a critical task in almost all applications. Two sources of information that are often ignored when mapping reads from the Solexa technology are the 3' ends of longer reads, which contain a much higher frequency of sequencing errors, and the base-call quality scores.ResultsTo investigate whether these sources of information can be used to improve accuracy when mapping reads, we developed the RMAP tool, which can map reads having a wide range of lengths and allows base-call quality scores to determine which positions in each read are more important when mapping. We applied RMAP to analyze data re-sequenced from two human BAC regions for varying read lengths, and varying criteria for use of quality scores. RMAP is freely available for downloading at .ConclusionOur results indicate that significant gains in Solexa read mapping performance can be achieved by considering the information in 3' ends of longer reads, and appropriately using the base-call quality scores. The RMAP tool we have developed will enable researchers to effectively exploit this information in targeted re-sequencing projects.

Reply

Drs Max, Murad, and Limburg nicely summarized the results of our meta-analysis and raised some issues concerning search and selection strategy and the use of quality scores. Our meta-analysis was a small part of the major WCRF/AICR systematic literature review project. Therefore, the search strategy and the first 3 selection steps have been performed to identify all human studies on the association between food, nutrition, and physical activity and CRC. The total number of potentially eligible studies was 43,191. Of these, 3193 references were selected based on title and keywords and subsequently 1703 based on the abstract. After reading the full papers, we ended up with 715 papers concerning 164 exposures. Of these exposures, fish consumption and/or n-3 polyunsaturated fatty acids (PUFAs) were evaluated in 24 publications involving 20 independent cohort studies on CRC incidence or mortality. As mentioned by Max et al, the use of quality scores is controversial. They have been poor predictors of study results, are hard to extract from published information, and using them as weighting factor might produce biased effect estimates (Biostatistics 2001;2:463–471). We performed our meta-analysis according to the WCRF/AICR methodology, advised by a methodology taskforce of experts (J Nutr 2003;133:3837S–3842S), which states that studies should not be excluded on the basis of perceived quality but study characteristics that may influence results should be defined. We did not provide quality scores because a score assumes that the relative importance of the separate characteristics of the study is known. In an analysis, it is more informative if the separate study characteristics can be studied, to explore quality differences as an explanation for heterogeneity in study results. Therefore, we reviewed qualitative characteristics of each study, such as aspects of study design, methods of exposure assessment, adjustment for potential confounding variables, and so on. However, contrary to the STROBE guidelines for reporting observational studies, most publications did not provide reasons for inclusion or exclusion of potential confounding variables in multivariate analyses (Lancet 2007;370:1453–1457). More standardized information in papers on observational studies would be very helpful. Our meta-analysis on observational studies suggests that there may be a “catch,” but to be sure randomized controlled trials are essential. We are currently involved in a trial on the effects of fatty and lean fish on markers of CRC, which will make it possible to answer the question whether n-3 PUFA and/or other nutrients like selenium or vitamin D are responsible for the chemopreventive action of fish. This study is part of the integrated, EU-funded research project SEAFOODplus (www.seafoodplus.org). Results are expected in 2008; let’s hope we will catch something. Fish Consumption and Colorectal Cancer Prevention: Is There a Catch?GastroenterologyVol. 134Issue 2PreviewGeelen A, Schouten JM, Kamphuis C, et al. (Division of Human Nutrition, Wageningen University, Wageningen, The Netherlands.) Fish consumption, n-3 fatty acids, and colorectal cancer: a meta-analysis of prospective cohort studies. Am J Epidemiol 2007;166:1116–1125. Full-Text PDF

Echinacea, Vitamin C, the Common Cold, and Blinding

Sir—Caruso and Gwaltney [1] reviewed the effect of echinacea treatment on the common cold, identifying 9 placebo-controlled trials. A quality score was calculated for each trial, and the main focus was put on 2 trials that received maximum scores and reported negative results. However, the use of quality scores in literature reviews has been strongly discouraged. For example, Greenland [2] commented that quality scoring “introduces an unnecessary and somewhat arbitrary subjective element into the analysis via the scoring scheme. Quality scoring can and should be replaced by direct categorical and regression analyses of the impact of each quality item. Such item-specific analyses let the data, rather than the investigator, indicate the importance of each item in determining the estimated effect” (p. 672). Also, the Cochrane Handbook [3] states that “reviewers should avoid the use of quality scores and undue reliance on detailed quality assessments. It is not supported by empirical evidence, it can be time-consuming, and it is potentially misleading” (p. 66).

No role for quality scores in systematic reviews of diagnostic accuracy studies.

BackgroundThere is a lack of consensus regarding the use of quality scores in diagnostic systematic reviews. The objective of this study was to use different methods of weighting items included in a quality assessment tool for diagnostic accuracy studies (QUADAS) to produce an overall quality score, and to examine the effects of incorporating these into a systematic review.MethodsWe developed five schemes for weighting QUADAS to produce quality scores. We used three methods to investigate the effects of quality scores on test performance. We used a set of 28 studies that assessed the accuracy of ultrasound for the diagnosis of vesico-ureteral reflux in children.ResultsThe different methods of weighting individual items from the same quality assessment tool produced different quality scores. The different scoring schemes ranked different studies in different orders; this was especially evident for the intermediate quality studies. Comparing the results of studies stratified as "high" and "low" quality based on quality scores resulted in different conclusions regarding the effects of quality on estimates of diagnostic accuracy depending on the method used to produce the quality score. A similar effect was observed when quality scores were included in meta-regression analysis as continuous variables, although the differences were less apparent.ConclusionQuality scores should not be incorporated into diagnostic systematic reviews. Incorporation of the results of the quality assessment into the systematic review should involve investigation of the association of individual quality items with estimates of diagnostic accuracy, rather than using a combined quality score.

On the bias produced by quality scores in meta-analysis, and a hierarchical view of proposed solutions.

Results from better quality studies should in some sense be more valid or more accurate than results from other studies, and as a consequence should tend to be distributed differently from results of other studies. To date, however, quality scores have been poor predictors of study results. We discuss possible reasons and remedies for this problem. It appears that 'quality' (whatever leads to more valid results) is of fairly high dimension and possibly non-additive and nonlinear, and that quality dimensions are highly application-specific and hard to measure from published information. Unfortunately, quality scores are often used to contrast, model, or modify meta-analysis results without regard to the aforementioned problems, as when used to directly modify weights or contributions of individual studies in an ad hoc manner. Even if quality would be captured in one dimension, use of quality scores in summarization weights would produce biased estimates of effect. Only if this bias were more than offset by variance reduction would such use be justified. From this perspective, quality weighting should be evaluated against formal bias-variance trade-off methods such as hierarchical (random-coefficient) meta-regression. Because it is unlikely that a low-dimensional appraisal will ever be adequate (especially over different applications), we argue that response-surface estimation based on quality items is preferable to quality weighting. Quality scores may be useful in the second stage of a hierarchical response-surface model, but only if the scores are reconstructed to maximize their correlation with bias.

Incorporating variations in the quality of individual randomized trials into meta-analysis

Meta-analysis is a method of synthesizing evidence from multiple sources. It has been increasingly applied to combine results from randomized trials of therapeutic strategies. Unfortunately there is often variation in the quality of the trials that are included in meta-analyses, limiting the value of combining the results in an overview. This variation in quality can lead to both bias and reduction in precision of the estimate of the therapy's effectiveness. There are a number of methods for quantifying the quality of trials including the detailed Chalmers system and simple scales. The nature of the relationship between these quality scores and the true estimate of effectiveness is unknown at this time. We discuss four methods of incorporating quality into meta-analysis: threshold score as inclusion/exclusion criterion, use of quality score as a weight in statistical pooling, visual plot of effect size against quality score and sequential combination of trial results based on quality score. The last method permits an examination of the relation between quality and both bias and precision on the pooled estimates. We conclude that while it is possible to incorporate the effect of variation of quality of individual trials into overviews, this issue requires more study.