Receiver Operating Characteristics Technique Research Articles

Principal Component Analysis in Linear Regression Survival Model with Microarray Data

As a useful alternative to the Cox proportional hazards model, the linear regression survival model assumes a linear relationship between the covariates and a known monotone transformation, for example logarithm, of an event time of interest. In this article, we study the linear regression survival model with right censored survival data, when high-dimensional microarray measurements are present. Such data may arise in studies in vestigating the statistical influence of molecular features on survival risk. We propose using the principal component regression (PCR) technique for model reduction based on the weight least squared Stute estimate. Com pared with other model reduction techniques, the PCR approach is relatively insensitive to the number of covariates and hence suitable for high dimen sional microarray data. Component selection based on the nonparametric bootstrap, and model evaluation using the time-dependent ROC (receiver operating characteristic) technique are investigated. We demonstrate the proposed approach with datasets from two microarray gene expression pro filing studies of lymphoma cancers

A new concordant partial AUC and partial c statistic for imbalanced data in the evaluation of machine learning algorithms

BackgroundIn classification and diagnostic testing, the receiver-operator characteristic (ROC) plot and the area under the ROC curve (AUC) describe how an adjustable threshold causes changes in two types of error: false positives and false negatives. Only part of the ROC curve and AUC are informative however when they are used with imbalanced data. Hence, alternatives to the AUC have been proposed, such as the partial AUC and the area under the precision-recall curve. However, these alternatives cannot be as fully interpreted as the AUC, in part because they ignore some information about actual negatives.MethodsWe derive and propose a new concordant partial AUC and a new partial c statistic for ROC data—as foundational measures and methods to help understand and explain parts of the ROC plot and AUC. Our partial measures are continuous and discrete versions of the same measure, are derived from the AUC and c statistic respectively, are validated as equal to each other, and validated as equal in summation to whole measures where expected. Our partial measures are tested for validity on a classic ROC example from Fawcett, a variation thereof, and two real-life benchmark data sets in breast cancer: the Wisconsin and Ljubljana data sets. Interpretation of an example is then provided.ResultsResults show the expected equalities between our new partial measures and the existing whole measures. The example interpretation illustrates the need for our newly derived partial measures.ConclusionsThe concordant partial area under the ROC curve was proposed and unlike previous partial measure alternatives, it maintains the characteristics of the AUC. The first partial c statistic for ROC plots was also proposed as an unbiased interpretation for part of an ROC curve. The expected equalities among and between our newly derived partial measures and their existing full measure counterparts are confirmed. These measures may be used with any data set but this paper focuses on imbalanced data with low prevalence.Future workFuture work with our proposed measures may: demonstrate their value for imbalanced data with high prevalence, compare them to other measures not based on areas; and combine them with other ROC measures and techniques.

Application of fuzzy logic and geometric average: A Cu sulfide deposits potential mapping case study from Kapsan Basin, DPR Korea

In this paper, two kinds of knowledge-driven methods, one using the fuzzy logic and another using geometric average, were applied to create the mineral potential maps for Cu sulfide deposits in the greenfield Kapsan Basin, DPR Korea. The ore geology studies for the study area have revealed that Cu sulfide deposits of hydrothermal genesis in Kapsan Basin are closely associated with Jurassic intrusions and faulting tectonics. Based on the conceptual model of Cu sulfide deposits and the available spatial datasets in the study area, we used five independent evidential maps for Cu sulfide deposits potential mapping. They include: (1) faults; (2) aeromagnetic anomaly; (3) Cu geochemical data; (4) Pb geochemical data; and (5) Zn geochemical data. The evidential map values were transformed into continuous values of the [0, 1] range using the non-linear fuzzy membership functions; logistic sigmoid and fuzzy Gaussian functions. Because the fuzzy logic and geometric average methods can use the same fuzzification methodology based on suitable membership functions, it is very economic and efficient to simultaneously apply two predictive models for mineral potential mapping of the study area. The preparation of these evidential layers were performed using spatial analyses supported in ArcGIS 10.4 GIS platform based on geological, geophysical and geochemical spatial datasets.The validation and comparative analysis results for the two predictive models demonstrated that most of known mineral occurrences are distributed in areas with high potential values. The target areas classified by the fuzzy logic occupy 15% of the study area and contain 78% of the total number of known mineral occurrences. Compared with the fuzzy logic, the resulting areas by the geometric average occupy 13% of the study area, but contain 93% of the total number of known mineral occurrences. Although the total number of known mineral occurrences is relatively low for the application of ROC (receiver operating characteristics) technique, the areas under the ROC curve (AUC) obtained by two predictive models were greater than 0.5, suggesting that both predictive models and their resulting potential maps are useful for evaluating the prospectivity of Cu sulfide deposits in Kapsan Basin.

The Impact of Fatigue on Satisfaction of Search in Chest Radiography

To assess the nature of the satisfaction of search (SOS) effect in chest radiography when observers are fatigued; determine if we could replicate recent findings that have documented the nature of the SOS effect to be due to a threshold shift rather than a change in diagnostic accuracy as in earlier film-based studies. Nearing or at the end of a clinical workday, 20 radiologists read 64 chest images twice, once with and once without the addition of a simulated pulmonary nodule. Half of the images had different types of "test" abnormalities. Decision thresholds were analyzed using the center of the range of false-positive (FP) and true-positive (TP) fractions associated with each receiver operating characteristic (ROC) point for reporting test abnormalities. Detection accuracy was assessed with ROC technique and inspection time was recorded. The SOS effect was confirmed to be a reduction in willingness to respond (threshold shift). The center of the FP range was significantly reduced (FP = 0.10 without added nodules, FP = 0.05 with added nodules, F(1,18) = 19.85, P = 0.0003). The center of the TP range was significantly reduced (TP = 0.39 without added nodules, TP = 0.33 with added nodules, F(1,18) = 10.81, P = 0.004). This study suggests that fatigue does not change the nature of the SOS effect, but rather may be additive with the SOS effect. SOS reduces both TP and FP responses, whereas fatigue reduces TPs more than FPs.

Receiver operating characteristics (ROC) as validation tool for prospectivity models — A magmatic Ni–Cu case study from the Central Lapland Greenstone Belt, Northern Finland

A prospectivity model for magmatic Ni–Cu deposits was created by integrating spatially referenced geophysical and geochemical datasets based on a simple and practical exploration model. The study area is the Central Lapland Greenstone Belt, Northern Fennoscandian Shield, Finland. Magmatic nickel deposits are related to rock types that are typically characterized by local magnetic and gravity anomalies. These deposit types can also be a source of nickel, copper and cobalt anomalies in the overlying glacial till cover. This straightforward exploration criterion was translated into a fuzzy logic prospectivity model. The model validation is an essential step in justifying the validity of the prospectivity model. This was accomplished by using receiver operating characteristics (ROC) technique. We used the known Ni–Cu occurrences and deposits as true positive cases and other deposit type locations or random points as true negative cases in the validation process. It appears that the ROC technique provides a robust model validation and optimization technique, providing that suitable validation data exists.

Comparison of ROC Feature Selection Method for the Detection of Decay in Citrus Fruit Using Hyperspectral Images

Hyperspectral imaging systems allow to detect the initial stages of decay caused by fungi in citrus fruit automatically, instead of doing it manually under dangerous ultraviolet illumination, thus preventing the fungal infestation of other sound fruit and, consequently, the enormous economical losses generated. However, these systems present the disadvantage of generating a huge amount of data, which is necessary to select for achieving some result useful for the sector. There are numerous feature selection methods to reduce dimensionality of hyperspectral images. This work compares a feature selection method using the area under the receiver operating characteristic (ROC) curve with other common feature selection techniques, in order to select an optimal set of wavelengths effective in the detection of decay in a citrus fruit using hyperspectral images. This comparative study is done using images of mandarins with the pixels labelled in five different classes: two types of healthy skin, two types of decay and scars, ensuring that the ROC technique generally provides better results than the other methods.

What is a good patient reported outcome after total hip replacement?

There is an increasing movement to collect and report patient reported outcome measures (PROM's) following total hip replacement (THR). In the UK, the procedure specific PROM of choice is the Oxford Hip Score (OHS). It is currently unclear how to use this information to determine outcome following surgery. The aim of this study was to define a threshold for the OHS that is correlated with patient satisfaction. Prospective cohort study. A district general hospital (St. Helier Hospital, Carshalton, UK). 799 patients receiving THR from 1995 to 2004. At 12 and 24 months after surgery patients were asked if they were satisfied with surgery and completed the OHS. Receiver operating characteristic (ROC) analyses were used to identify thresholds of follow-up OHS, which best discriminated patient satisfaction. Analyses were stratified by age, sex, body mass index (BMI), baseline OHS and patient expectations. 91.9% of patients were satisfied with THR at 12 months (92.8% at 24 months). Using the ROC technique, the OHS at 12 months associated with patient satisfaction was 38 and at 24 months 33. The OHS at 24 months associated with satisfaction was higher in those with highest tertile of baseline OHS (30, 33, 43 respectively), and lowest tertile of BMI. We have identified a value of the OHS that predicts patient satisfaction 12-24 months following THR within a standard clinical setting. However, this threshold is markedly influenced by pre-operative OHS and should be stratified accordingly.

Principles and practical application of the receiver-operating characteristic analysis for diagnostic tests

We review the principles and practical application of receiver-operating characteristic (ROC) analysis for diagnostic tests. ROC analysis can be used for diagnostic tests with outcomes measured on ordinal, interval or ratio scales. The dependence of the diagnostic sensitivity and specificity on the selected cut-off value must be considered for a full test evaluation and for test comparison. All possible combinations of sensitivity and specificity that can be achieved by changing the test’s cut-off value can be summarised using a single parameter; the area under the ROC curve. The ROC technique can also be used to optimise cut-off values with regard to a given prevalence in the target population and cost ratio of false-positive and false-negative results. However, plots of optimisation parameters against the selected cut-off value provide a more-direct method for cut-off selection. Candidates for such optimisation parameters are linear combinations of sensitivity and specificity (with weights selected to reflect the decision-making situation), odds ratio, chance-corrected measures of association (e.g. kappa) and likelihood ratios. We discuss some recent developments in ROC analysis, including meta-analysis of diagnostic tests, correlated ROC curves (paired-sample design) and chance- and prevalence-corrected ROC curves.

Advanced Multiple Beam Equalization Radiography: Receiver Operating Characteristic Comparison With Screen-Film Chest Radiography

To test the hypothesis that the advanced multiple beam equalization radiography (AMBER) imaging system is superior to conventional chest radiography in the demonstration of diffuse infiltrative lung disease, emphysema, pulmonary nodules, calcification within nodules, and mediastinal or hilar masses and lymphadenopathy. The study involved 115 patients, each of whom underwent chest computed tomography (CT), AMBER, posteroanterior chest radiography, and conventional posteroanterior stereoscopic chest radiography (two films). All radiographs were obtained with the InSight Thoracic Imaging System. Four chest radiologists independently analyzed the 115 AMBER studies, 115 unpaired single conventional radiographs (a single film from a stereoscopic pair), and 115 stereoscopic conventional radiographs (2 films) for the presence of diffuse infiltrative lung disease, emphysema, pulmonary nodules, calcification within nodules, and mediastinal or hilar masses and lymphadenopathy. For each abnormality detected, the radiologists described their level of confidence based on a scale of 1 to 5. The 115 CT examinations were interpreted by consensus among 3 of the chest radiologists. The CT results were considered the standard. Receiver operating characteristic (ROC) techniques were used for statistical analysis. No statistically significant differences were found with ROC techniques between the AMBER system and single or stereoscopic conventional screen-film radiography for the abnormalities studied. We noted no clinically significant difference between AMBER and either single or stereoscopic conventional screen-film radiography in this prospective study of 115 patients in which CT (performed within 1 week of both radiographic examinations) was the standard.