Continuous Diagnostic Tests Research Articles

Multivariate reference and tolerance regions based on conditional transformation models: Application to glycemic markers.

The reference interval is the most widely used medical decision-making, constituting a central tool in determining whether an individual is healthy or not. When the results of several continuous diagnostic tests are available for the same patient, their clinical interpretation is more reliable if a multivariate reference region (MVR) is available rather than multiple univariate reference intervals. MVRs, defined as regions containing 95% of the results of healthy subjects, extend the concept of the reference interval to the multivariate setting. However, they are rarely used in clinical practice owing to difficulties associated with their interpretability and the restrictions inherent to the assumption of a Gaussian distribution. Further statistical research is thus needed to make MVRs more applicable and easier for physicians to interpret. Since the joint distribution of diagnostic test results may well change with patient characteristics independent of disease status, MVRs adjusted for covariates are desirable. The present work introduces a novel formulation for MVRs based on multivariate conditional transformation models (MCTMs). Additionally, we take into account the estimation uncertainty of such MVRs by means of tolerance regions. These conditional MVRs imply no parametric restriction on the response, and potentially nonlinear continuous covariate effects can be estimated. MCTMs allow the estimation of the effects of covariates on the joint distribution of multivariate response variables and on these variables' marginal distributions, via the use of most likely transformation estimation. Our contributions proved reliable when tested with simulated data and for a real data application with two glycemicmarkers.

Bayesian latent class models for identifying canine visceral leishmaniosis using diagnostic tests in the absence of a gold standard.

Like many infectious diseases, there is no practical gold standard for diagnosing clinical visceral leishmaniasis (VL). Latent class modeling has been proposed to estimate a latent gold standard for identifying disease. These proposed models for VL have leveraged information from diagnostic tests with dichotomous serological and PCR assays, but have not employed continuous diagnostic test information. In this paper, we employ Bayesian latent class models to improve the identification of canine visceral leishmaniasis using the dichotomous PCR assay and the Dual Path Platform (DPP) serology test. The DPP test has historically been used as a dichotomous assay, but can also yield numerical information via the DPP reader. Using data collected from a cohort of hunting dogs across the United States, which were identified as having either negative or symptomatic disease, we evaluate the impact of including numerical DPP reader information as a proxy for immune response. We find that inclusion of DPP reader information allows us to illustrate changes in immune response as a function of age. Utilization of continuous DPP reader information can improve the correct discrimination between individuals that are negative for disease and those with clinical VL. These models provide a promising avenue for diagnostic testing in contexts with multiple, imperfect diagnostic tests. Specifically, they can easily be applied to human visceral leishmaniasis when diagnostic test results are available. Also, appropriate diagnosis of canine visceral leishmaniasis has important consequences for curtailing spread of disease to humans.

Potential anticancer effect of aspirin and 2'-hydroxy-2,3,5'-trimethoxychalcone-linked polymeric micelles against cervical cancer through apoptosis.

Although early diagnosis and treatment of cancers in women are achievable through continuous diagnostic tests, cervical cancer (CVC) still has a high mortality rate. In the present study, we investigated whether certain nanoparticles (NPs), comprising aspirin conjugated 2′-hydroxy-2,3,5′-trimethoxychalcone chemicals, could induce the apoptosis of cancer cells. HeLa cells were treated with NPs and the cell viability was evaluated using WST-1 assay. Protein expression of Ki-67 was measured using immunocytochemistry. In addition, the apoptotic effect of NPs was determined using TUNEL assay. To investigate the apoptosis signaling pathways, reverse transcription quantitative PCR was performed and lipid accumulation was observed via holotomographic microscopy. The IC50 value of the NPs was 4.172 µM in HeLa cells. Furthermore, 10 µM NPs significantly inhibited the cell proliferation and stimulated the apoptosis of HeLa cells. In addition, apoptosis and mitochondrial dysfunction were induced by the NPs through lipid accumulation in HeLa cells, leading to apoptotic signaling cascades. Taken together, the results from the present study demonstrated that the NPs developed promoted apoptosis though efficient lipid accumulation in HeLa cells, suggesting that they may provide a novel way to improve the efficacy of CVC anticancer treatment.

Modeling conditional reference regions: Application to glycemic markers.

Many clinical decisions are taken based on the results of continuous diagnostic tests. Usually, only the results of one single test is taken into consideration, the interpretation of which requires a reference range for the healthy population. However, the use of two different tests, can be necessary in the diagnosis of certain diseases. This obliges a bivariate reference region be available for their interpretation. It should also be remembered that reference regions may depend on patient variables (eg, age and sex) independent of the suspected disease. However, few proposals have been made regarding the statistical modeling of such reference regions, and those put forward have always assumed a Gaussian distribution, which can be rather restrictive. The present work describes a new statistical method that allows such reference regions to be estimated with no insistence on the results being normally distributed. The proposed method is based on a bivariate location-scale model that provides probabilistic regions covering a specific percentage of the bivariate data, dependent on certain covariates. The reference region is estimated nonparametrically and the nonlinear effects of continuous covariates via polynomial kernel smoothers in additive models. The bivariate model is estimated using a backfitting algorithm, and the optimal smoothing parameters of the kernel smoothers selected by cross-validation. The model performed satisfactorily in simulation studies under the assumption of non-Gaussian conditions. Finally, the proposed methodology was found to be useful in estimating a reference region for two continuous diagnostic tests for diabetes (fasting plasma glucose and glycated hemoglobin), taking into account the age of the patient.

On the Principles of Believe the Positive and Believe the Negative for Diagnosis Using Two Continuous Tests

Believe the Positive (BP) and Believe the Negative (BN) rules for combining two continuous diagnostic tests are compared with proce dures based on likelihood ratio and linear combination of the two tests. The sensitivity-specificity relationship for BP/BN is illustrated through a graph ical presentation of a ”ROC surface”, which leads to a natural approach of choosing between BP and BN. With a bivariate normal model, it is shown that the discriminating power of this approach is higher when the correla tion between the two tests has different signs for non-diseased and diseased population, given the location and variations of the two distributions are fixed. The idea is illustrated through an example.

Covariate-adjusted Bayesian estimation of the performance of a continuous diagnostic test with a limit of detection in the absence of a reference standard: a simulation study

A receiver operating characteristic (ROC) curve is often used to assess the discriminatory ability of a continuous diagnostic test, which is summarized by its corresponding area under the curve (AUC). For tests that are subject to a limit of detection and especially when a reference standard is not available, unquantifiable test scores beyond the limit of detection introduces bias in AUC estimation. We propose a Bayesian approach for estimating AUC that are facilitated by the presence of covariates for a no reference standard scenario. We develop methods for AUC estimation where both an absence of a reference standard and a limit of detection co-exist for both a normally- and gamma-distributed test scores. Our work extends previous developments to the scenarios involving the presence of a limit of detection for the test and also varying degrees of censoring due to the extent of the detection limit that could affect AUC estimation. Simulations suggested that AUC estimates are accurate except when covariates information is ignored, when there is a high percentage of unquantifiable scores (e.g., 60%), or when substantial overlap exits between diseased and non-diseased populations (e.g., true AUC = 0.6) that could result in marginally less precise AUC estimates. When a continuous test is subject to a limit of detection and in the absence of a reference standard, covariate information could help with a more accurate distinction of the latent disease status for individuals.

Estimation of the volume under the receiver-operating characteristic surface adjusting for non-ignorable verification bias.

The receiver-operating characteristic surface is frequently used for presenting the accuracy of a diagnostic test for three-category classification problems. One common problem that can complicate the estimation of the volume under receiver-operating characteristic surface is that not all subjects receive the verification of the true disease status. Estimation based only on data from subjects with verified disease status may be biased, which is referred to as verification bias. In this article, we propose new verification bias correction methods to estimate the volume under receiver-operating characteristic surface for a continuous diagnostic test. We assume the verification process is missing not at random, which means the missingness might be related to unobserved clinical characteristics. Three classes of estimators are proposed, namely, inverse probability weighted, imputation-based, and doubly robust estimators. A jackknife estimator of variance is derived for all the proposed volume under receiver-operating characteristic surface estimators. The finite sample properties of the new estimators are examined via simulation studies. We illustrate our methods with data collected from Alzheimer's disease research.

Bayesian bootstrap inference for the receiver operating characteristic surface

Accurate diagnosis of disease is of great importance in clinical practice and medical research. The receiver operating characteristic (ROC) surface is a popular tool for evaluating the discriminatory ability of continuous diagnostic test outcomes when there exist three‐ordered disease classes (e.g., no disease, mild disease, and advanced disease). We propose the Bayesian bootstrap, a fully nonparametric method, for conducting inference about the ROC surface and its functionals, such as the volume under the ROC surface (VUS). The proposed method is based on a simple, yet interesting, representation of the ROC surface in terms of placement variables and has the appealing feature of producing point and interval estimates for the ROC surface and its corresponding VUS in a single integrated framework. Results from a simulation study demonstrate the ability of our method to successfully recover the true ROC surface and to produce valid inferences in a variety of complex scenarios. An application to data from the Trail Making Test to assess cognitive impairment in Parkinson's disease patients is provided.

Bootstrap-based procedures for inference in nonparametric receiver-operating characteristic curve regression analysis.

Prior to using a diagnostic test in a routine clinical setting, the rigorous evaluation of its diagnostic accuracy is essential. The receiver-operating characteristic curve is the measure of accuracy most widely used for continuous diagnostic tests. However, the possible impact of extra information about the patient (or even the environment) on diagnostic accuracy also needs to be assessed. In this paper, we focus on an estimator for the covariate-specific receiver-operating characteristic curve based on direct regression modelling and nonparametric smoothing techniques. This approach defines the class of generalised additive models for the receiver-operating characteristic curve. The main aim of the paper is to offer new inferential procedures for testing the effect of covariates on the conditional receiver-operating characteristic curve within the above-mentioned class. Specifically, two different bootstrap-based tests are suggested to check (a) the possible effect of continuous covariates on the receiver-operating characteristic curve and (b) the presence of factor-by-curve interaction terms. The validity of the proposed bootstrap-based procedures is supported by simulations. To facilitate the application of these new procedures in practice, an R-package, known as npROCRegression, is provided and briefly described. Finally, data derived from a computer-aided diagnostic system for the automatic detection of tumour masses in breast cancer is analysed.

BcROCsurface: an R package for correcting verification bias in estimation of the ROC surface and its volume for continuous diagnostic tests

BackgroundReceiver operating characteristic (ROC) surface analysis is usually employed to assess the accuracy of a medical diagnostic test when there are three ordered disease status (e.g. non-diseased, intermediate, diseased). In practice, verification bias can occur due to missingness of the true disease status and can lead to a distorted conclusion on diagnostic accuracy. In such situations, bias–corrected inference tools are required.ResultsThis paper introduce an R package, named bcROCsurface, which provides utility functions for verification bias–corrected ROC surface analysis. The shiny web application of the correction for verification bias in estimation of the ROC surface analysis is also developed.ConclusionbcROCsurface may become an important tool for the statistical evaluation of three–class diagnostic markers in presence of verification bias. The R package, readme and example data are available on CRAN. The web interface enables users less familiar with R to evaluate the accuracy of diagnostic tests, and can be found at http://khanhtoduc.shinyapps.io/bcROCsurface_shiny/.

GsymPoint: An R Package to Estimate the Generalized Symmetry Point, an Optimal Cut-off Point for Binary Classification in Continuous Diagnostic Tests

GsymPoint: An R Package to Estimate the Generalized Symmetry Point, an Optimal Cut-off Point for Binary Classification in Continuous Diagnostic Tests

PlotROC: A Tool for Plotting ROC Curves.

Plots of the receiver operating characteristic (ROC) curve are ubiquitous in medical research. Designed to simultaneously display the operating characteristics at every possible value of a continuous diagnostic test, ROC curves are used in oncology to evaluate screening, diagnostic, prognostic and predictive biomarkers. I reviewed a sample of ROC curve plots from the major oncology journals in order to assess current trends in usage and design elements. My review suggests that ROC curve plots are often ineffective as statistical charts and that poor design obscures the relevant information the chart is intended to display. I describe my new R package that was created to address the shortcomings of existing tools. The package has functions to create informative ROC curve plots, with sensible defaults and a simple interface, for use in print or as an interactive web-based plot. A web application was developed to reach a broader audience of scientists who do not use R.

Validación de la Escala de Parentalidad Positiva (EPP) para evaluar programas presenciales y online de apoyo parental || Validation of the Positive Parenting Scale (PPS) for evaluating face-to-face and online parenting support programs

Como continuación del estudio donde se presentó la Escala de Apoyo Parental Online (EAPO), como parte de la evaluación del programa “Educar en Positivo” (http://educarenpositivo.es), el presente estudio describe la validación de una nueva escala para evaluar los principios de la parentalidad positiva en usuarios de programas presenciales y online. Para la validación de la Escala de Parentalidad Positiva (EPP) se contó con 323 padres y madres españoles y latinoamericanos, registrados en dicho programa. Para obtener la estructura factorial de la escala, se utilizó el modelo de ecuaciones estructurales exploratorio (MESE) con rotación Oblimin y para su confirmación el método de estimación de ponderación de mínimos cuadrados ajustados por la media y varianza (WLSMW). Se realizó también una prueba de la validez discriminante de la escala por medio del análisis del área bajo la curva ROC (ABC) y su comprobación mediante un análisis multivariado de covarianza (MANCOVA). Los resultados muestran una factorización óptima en el modelo de cuatro factores: Implicación familiar, Afecto y reconocimiento, Comunicación y control del estrés y Actividades compartidas con una fiabilidad adecuada. Asimismo, se demostró la capacidad discriminativa de la escala en función del perfil de experiencia y uso educativo de Internet, una vez controlada la posible influencia del nivel educativo de los participantes. La escala muestra unas propiedades psicométricas adecuadas y su contenido recoge los aspectos clave del ejercicio de la parentalidad positiva, lo que resulta muy útil para evaluar la efectividad de los programas basados en dicho enfoque.

Thresholding of a Continuous Companion Diagnostic Test Confident of Efficacy in Targeted Population

ABSTRACTIn personalized medicine, continuous biomarker values are often dichotomized to classify patients into target and nontarget populations. For example, baseline hemoglobin A1c (A1C) and Positive and Negative Syndrome Scale (PANSS) measurements are potential biomarkers in diabetes and psychiatry, respectively. For Alzheimer's Disease (AD), currently Mild versus Moderate classification is used as a potential binary biomarker for treatment effect prediction. However, Normal, Mild, Moderate, and Severe staging of AD is a discretization of the mini-mental state examination (MMSE), which is measured on a scale of 0–30. Therefore, MMSE is potentially a continuously valued biomarker for AD treatment. For such scenarios, we provide simultaneous confidence intervals for efficacy corresponding to all candidate thresholds. Our method allows for rigorous and flexible decision making, taking into consideration medical impact based on both the size of the target population and efficacy in the target population.

Inverse probability weighting estimation of the volume under the ROC surface in the presence of verification bias.

In diagnostic medicine, the volume under the receiver operating characteristic (ROC) surface (VUS) is a commonly used index to quantify the ability of a continuous diagnostic test to discriminate between three disease states. In practice, verification of the true disease status may be performed only for a subset of subjects under study since the verification procedure is invasive, risky, or expensive. The selection for disease examination might depend on the results of the diagnostic test and other clinical characteristics of the patients, which in turn can cause bias in estimates of the VUS. This bias is referred to as verification bias. Existing verification bias correction in three-way ROC analysis focuses on ordinal tests. We propose verification bias-correction methods to construct ROC surface and estimate the VUS for a continuous diagnostic test, based on inverse probability weighting. By applying U-statistics theory, we develop asymptotic properties for the estimator. A Jackknife estimator of variance is also derived. Extensive simulation studies are performed to evaluate the performance of the new estimators in terms of bias correction and variance. The proposed methods are used to assess the ability of a biomarker to accurately identify stages of Alzheimer's disease.

Confidence intervals for the symmetry point: an optimal cutpoint in continuous diagnostic tests.

Continuous diagnostic tests are often used for discriminating between healthy and diseased populations. For this reason, it is useful to select an appropriate discrimination threshold. There are several optimality criteria: the North-West corner, the Youden index, the concordance probability and the symmetry point, among others. In this paper, we focus on the symmetry point that maximizes simultaneously the two types of correct classifications. We construct confidence intervals for this optimal cutpoint and its associated specificity and sensitivity indexes using two approaches: one based on the generalized pivotal quantity and the other on empirical likelihood. We perform a simulation study to check the practical behaviour of both methods and illustrate their use by means of three real biomedical datasets on melanoma, prostate cancer and coronary artery disease.

Bias–corrected methods for estimating the receiver operating characteristic surface of continuous diagnostic tests

Verification bias is a well-known problem that may occur in the evaluation of predictive ability of diagnostic tests. When a binary disease status is considered, various solutions can be found in the literature to correct inference based on usual measures of test accuracy, such as the receiver operating characteristic (ROC) curve or the area underneath. Evaluation of the predictive ability of continuous diagnostic tests in the presence of verification bias for a three-class disease status is here discussed. In particular, several verification bias-corrected estimators of the ROC surface and of the volume underneath are proposed. Consistency and asymptotic normality of the proposed estimators are established and their finite sample behavior is investigated by means of Monte Carlo simulation studies. Two illustrations are also given.

An enhancement of ROC curves made them clinically relevant for diagnostic-test comparison and optimal-threshold determination

ObjectivesThe receiver operating characteristic curves (ROC curves) are often used to compare continuous diagnostic tests or determine the optimal threshold of a test; however, they do not consider the costs of misclassifications or the disease prevalence. The ROC graph was extended to allow for these aspects. Study Design and SettingTwo new lines are added to the ROC graph: a sensitivity line and a specificity line. Their slopes depend on the disease prevalence and on the ratio of the net benefit of treating a diseased subject to the net cost of treating a nondiseased one. First, these lines help researchers determine the range of specificities within which test comparisons of partial areas under the curves is clinically relevant. Second, the ROC curve point the farthest from the specificity line is shown to be the optimal threshold in terms of expected utility. ResultsThis method was applied: (1) to determine the optimal threshold of ratio specific immunoglobulin G (IgG)/total IgG for the diagnosis of congenital toxoplasmosis and (2) to select, among two markers, the most accurate for the diagnosis of left ventricular hypertrophy in hypertensive subjects. ConclusionThe two additional lines transform the statistically valid ROC graph into a clinically relevant tool for test selection and threshold determination.

Basics of diagnostic evaluation part 2

Despite a long tradition of reporting diagnostic accuracy in terms of sensitivity and specificity, only a minority of physicians correctly interprets and applies this information. Likelihood ratios (LRs) are more intelligible and more meaningful because it connects the preprobability of a person to postprobability that guide the physician to make the right decision about further procedures. Using Fagan Nomogram, makes the application of the LRs more customized to each case. In case of testing a continuous diagnostic test, receiver operating curve is used to set a cut-off value that will be the line between disease and nondisease persons. When determining the cut-off, the trade-offs between increasing the sensitivity with lowering specificity and vise-versa depends on the nature of the disease, as well as whether this test will be used as screening or as diagnostic.

OptimalCutpoints: AnRPackage for Selecting Optimal Cutpoints in Diagnostic Tests

Continuous diagnostic tests are often used for discriminating between healthy and diseased populations. For the clinical application of such tests, it is useful to select a cutpoint or discrimination value c that defines positive and negative test results. In general, individuals with a diagnostic test value of c or higher are classified as diseased. Several search strategies have been proposed for choosing optimal cutpoints in diagnostic tests, depending on the underlying reason for this choice. This paper introduces an R package, known as OptimalCutpoints, for selecting optimal cutpoints in diagnostic tests. It incorporates criteria that take the costs of the different diagnostic decisions into account, as well as the prevalence of the target disease and several methods based on measures of diagnostic test accuracy. Moreover, it enables optimal levels to be calculated according to levels of given (categorical) covariates. While the numerical output includes the optimal cutpoint values and associated accuracy measures with their confidence intervals, the graphical output includes the receiver operating characteristic (ROC) and predictive ROC curves. An illustration of the use of OptimalCutpoints is provided, using a real biomedical dataset.