Binormal Model Research Articles

The shape of an ROC curve in the evaluation of credit scoring models

The AUC, i.e. the area under the receiver operating characteristic (ROC) curve, or its scaled version, the Gini coefficient, are the standard measures of the discriminatory power of credit scoring. Using binormal ROC curve models, we show how the shape of the curves affects the economic benefits of using scoring models with the same AUC. Based on the results, we propose that the shape parameter of the fitted ROC curve is reported alongside its AUC/Gini whenever the quality of a scorecard is discussed.

Relative economic value of global ensemble prediction system of NCMRWF, India, for extreme weather events

AbstractAn ensemble prediction system quantifies the uncertainty in the forecast by a numerical weather prediction model and helps in decision‐making. The global ensemble prediction system (NEPS‐G) of NCMRWF, India, has 1 control and 22 perturbed ensemble members. In the present study, the relative economic value of NEPS‐G forecast has been investigated based on a simple decision‐analytic model. Temperature at 850 hPa (T850), maximum surface temperature (Tmax) and precipitation (rf) over the Indian domain have been studied. Ensemble mean forecast offers a larger value to the users than the control forecast. For rare extreme events, the area under the curve (T‐AUC) of a relative operating characteristic (ROC) obtained using the trapezoidal approximation for 22‐member NEPS‐G (NEPS‐22) is significantly smaller than the area under the curve (Z‐AUC) of the full ROC obtained using binormal model and Z‐transformation. As a result, the discrimination ability and relative economic value of NEPS‐G are underestimated. A method (SUB‐EM) which subdivides the lowest probability category into several subcategories and uses ensemble mean forecast as the secondary decision variable to enhance T‐AUC and improve relative economic value of NEPS‐G has been investigated. Tmax > 45°C and rf > 65 mm·day−1 have been considered as extreme events. The impact of increasing the number of perturbed ensemble members from 22 to 33 (NEPS‐33) by running additional 11 perturbed members from 0000 UTC initial condition has also been investigated and compared with the results obtained using SUB‐EM. SUB‐EM offers larger economic value to the users with small cost–loss ratio (α) without engaging any extra computational resources. An experiment is also carried out to study the impact of using different ensemble summary statistics as a secondary decision variable. Ensemble Maximum provides maximum value and benefits the largest range of users.

Likelihood-type confidence regions for optimal sensitivity and specificity of a diagnostic test

New methods are proposed that provide approximate joint confidence regions for the optimal sensitivity and specificity of a diagnostic test, i.e., sensitivity and specificity corresponding to the optimal cutpoint as defined by the Youden index criterion. Such methods are semi-parametric or non-parametric and attempt to overcome the limitations of alternative approaches. The proposed methods are based on empirical likelihood pivots, giving rise to likelihood-type regions with no predetermined constraints on the shape and automatically range-respecting. The proposal covers three situations: the binormal model, the binormal model after the use of Box-Cox transformations and the fully non-parametric model. In the second case, it is also shown how to use two different transformations, for the healthy and the diseased subjects. The finite sample behaviour of our methods is investigated using simulation experiments. The simulation results also show the advantages offered by our methods when compared with existing competitors. Illustrative examples, involving three real datasets, are also provided.

Resilience Family of Receiver Operating Characteristic Curves

A new semiparameteric model of the receiver operating characteristic (ROC) curve based on the resilience family or proportional reversed hazard family is proposed, which is an alternative to the existing models. The resulting ROC curve and its summary indices (such as area under the curve and Youden index) have simple analytic forms. The partial likelihood method is applied to estimate the ROC curve. Moreover, the estimation methodologies of the resilience family of the ROC curve have been developed based on area under the curve estimators exploiting Mann–Whitney statistics and the Rojo approach. A simulation study has been carried out to assess the performance of all considered estimators. Real data from the American National Health and Nutrition Examination Survey has been analyzed in detail based on the proposed model and the usual binormal model prevalent in the literature. Real data in the context of brain injury-related biomarkers are also analyzed in order to compare our model with the Lehmann family of the ROC curves. Finally, we show that the proposed model may be applicable in the misspecification scenario through the Ducheme muscular dystrophy data.

Optical recursional binormal optical visco Landau–Lifshitz electromagnetic optical density

In this manuscript, we study a new version of the optical recursional binormal microbeam model for a flexible binormal microscale beam in terms of a binormal normalized operator. Also, we give new explanations for the optical recursional visco Landau–Lifshitz binormal electromagnetic binormal microscale beam. Finally, we obtain an optical application for the normalized visco Landau–Lifshitz electromagnetic binormal optimistic density with an optical binormal resonator.

ROC Analyses Based on Measuring Evidence Using the Relative Belief Ratio.

ROC (Receiver Operating Characteristic) analyses are considered under a variety of assumptions concerning the distributions of a measurement X in two populations. These include the binormal model as well as nonparametric models where little is assumed about the form of distributions. The methodology is based on a characterization of statistical evidence which is dependent on the specification of prior distributions for the unknown population distributions as well as for the relevant prevalence w of the disease in a given population. In all cases, elicitation algorithms are provided to guide the selection of the priors. Inferences are derived for the AUC (Area Under the Curve), the cutoff c used for classification as well as the error characteristics used to assess the quality of the classification.

Multi-Class Classification using Mixtures of Univariate and Multivariate ROC Curves

Introduction: Receiver Operating Characteristic (ROC) curve is one of the widely used supervised classification technique to allocate/classify the individuals and also instrumental in comparing diagnostic tests. Generally, to deal with classification problems we need to have knowledge on class labels. In most of the medical scenarios, most of data sets exhibit multi-model patterns in class labels which leads to multi-class classification problems. The main aim of this study is to address on the issue of constructing ROC models when there exists multimodel patterns in the class labels further, to classify the individuals for better diagnosis and also to reduce the complexity of graphical representation of ROC curves in such classification problems. Methods: A new version of univariate and multivariate ROC models are proposed in the framework of Finite Mixtures, due to the flexibility of identifying and modelling the subcomponents in the heterogeneous populations. Results: Oral Glucose Tolerance Test and Disk Hernia datasets are used and simulation studies are also performed. Results show that the proposed models possess better accuracy when compared with Bi-Normal and MROC models with reasonable low 1-Specificity and higher Sensitivity. The ROC curves are depicted in a 2D space rather than higher dimension for multi-class classification problem. Conclusion: It is suggested that before one proceeds to model ROC curves, it is better to take a look at the density patterns of the study variable(s), which in turn help in explaining the true information between the classes and also provides good amount of “true” accuracy.

Estimation of the area under the ROC curve for non-normal data

The Receiver Operating Characteristic curve is one of the widely used classification tools that helps in assessing the performance of the diagnostic test as well as accommodates for comparing two diagnostic tests/statistical procedures using its intrinsic and accuracy measures, such as, sensitivity; specificity, and the Area under the Curve. The conventional and standard ROC model is the Bi-normal ROC model which is based on the assumption that the test scores/marker values underlie Normal distributions. Over the years, several researchers have developed various bi-distributional ROC models where the data possess the pattern of Exponential, Gamma, the combination of Half Normal and Rayleigh, etc. However, there are many practical situations, particularly in the field of medicine, where these available distributions may not be of fit for the data at hand. In this article, we attempted to propose two new ROC models and showed that these models have a better fit and explain better accuracy than that of the existing ROC models. The work is supported by a real dataset and simulated datasets.

On the ROC Area of Ensemble Forecasts for Rare Events

Abstract The relative operating characteristic (ROC) curve is a popular diagnostic tool in forecast verification, with the area under the ROC curve (AUC) used as a verification metric measuring the discrimination ability of a forecast. Along with calibration, discrimination is deemed as a fundamental probabilistic forecast attribute. In particular, in ensemble forecast verification, AUC provides a basis for the comparison of potential predictive skill of competing forecasts. While this approach is straightforward when dealing with forecasts of common events (e.g., probability of precipitation), the AUC interpretation can turn out to be oversimplistic or misleading when focusing on rare events (e.g., precipitation exceeding some warning criterion). How should we interpret AUC of ensemble forecasts when focusing on rare events? How can changes in the way probability forecasts are derived from the ensemble forecast affect AUC results? How can we detect a genuine improvement in terms of predictive skill? Based on verification experiments, a critical eye is cast on the AUC interpretation to answer these questions. As well as the traditional trapezoidal approximation and the well-known binormal fitting model, we discuss a new approach that embraces the concept of imprecise probabilities and relies on the subdivision of the lowest ensemble probability category.

Identical-test Roe and Metz simulation model for validating multi-reader methods of analysis for comparing different radiologic imaging modalities.

The most frequently used model for simulating multireader multicase (MRMC) data that emulate confidence-of-disease ratings from diagnostic imaging studies has been the Roe and Metz model, proposed by Roe and Metz in 1997 and later generalized by Hillis (2012), Abbey et al (2013) and Gallas and Hillis (2014). All of these models generate continuous confidence-of-disease ratings based on an underlying binormal model for each reader, with the separation between the normal and abnormal rating distributions varying across readers. Numerous studies have used these models for evaluating MRMC analysis and sample size methods. The models suggested in these papers for assessing type I error have been "null" models, where the expected AUC across readers is the same for each test. However, for the null models that have been suggested, there are other differences that would not exist if the two tests were identical. None of the papers cited above discuss how to formulate a null model that is also an "identical-test" model, where the two tests are identical in all respects. The purpose of this paper is to show how to formulate an identical-test model and to discuss the importance of this model. Using the identical-test model, I show through simulations the importance of the Obuchowski-Rockette model constraints to avoid a negative variance estimate, a result which had not previously been empirically demonstrated.

Trammel net size selectivity and determination of a minimum legal size (MLS) for the haffara seabream, Rhabdosargus haffara in the Gulf of Suez

The size selectivity of trammel nets used in the Suez Bay in small-scale fishing operations was calculated for the haffara seabream Rhabdosargus haffara using length-frequency distributions of the catch from three different mesh sizes (34, 37 and 40 mm) of the inner layer of the trammel net. Three experimental fishing trials were conducted in the Suez Bay during October 2018, andJanuary and September 2019. The SELECT method has been used to estimate selectivity parameters using five major models (log-normal model, normal location model, normal scale model, gamma model, and bi-normal model). The bi-normal was assessed as the most appropriate model for the haffara seabream. Modal sizes were estimated as 11.8, 12.1, 16.1 cm for 34, 37 and 40 mm mesh sizes, respectively. The calculated lengths at first maturity (Lm) were 13.67 cm for males and 14.15 cm for females. Our findings revealed that the fish lengths caught by the inner layer 34 and 37 mm mesh sizes were less than the length at first maturity, while, those caught with 40 mm were more than the length at first maturity. The minimum legal size (MLS) for R. haffara was fixed at 14.15 cm. Finally, the results revealed that increasing the mesh size of the trammel nets, especially the inner layer mesh, can lead to significant improvements in size selectivity and allow the protection of this valuable fish resource.

Application of the skew exponential power distribution to ROC curves

The bi-Normal ROC model and corresponding metrics are commonly used in medical studies to evaluate the discriminatory ability of a biomarker. However, in practice, many clinical biomarkers tend to have skewed or other non-Normal distributions. And while the bi-Normal ROC model’s AUC tends to be unbiased in this setting, providing a reasonable measure of global performance, the corresponding decision thresholds tend to be biased. To correct this bias, we propose using an ROC model based on the skew exponential power (SEP) distribution, whose additional parameters can accommodate skewed, heavy tailed, or other non-Normal distributions. Additionally, the SEP distribution can be used to evaluate whether the bi-Normal model would be appropriate. The performance of these ROC models and the non-parametric approach are evaluated via a simulation study and applied to a real data set involving infections from Klebsiella pneumoniae. The SEP based ROC-model provides some efficiency gains with respect to estimation of the AUC and provides cut-points with improved classification rates. As such, in the presence non-Normal data, we suggest using the proposed SEP ROC model.

Comparison of the binormal and Lehman receiver operating characteristic curves

In this article, we compared the binormal and the Lehmann receiver operating characteristic (ROC) curves using extensive simulation studies and real data sets related to HIV. We ran a large set of simulation studies with a wide range of distributions, varying sample sizes, and different degrees of overlap between the diseased and non-diseased population distributions. Simulation results suggest that the binormal ROC model performs better for normal data. However, for non-normal data, the Lehman model outperforms the binormal model. The discrepancies between the performances of the two models were more apparent for larger sample sizes or larger degree of separation between the distributions of the diseased and non-diseased subjects. Like the binormal model, the Lehman model provides smooth estimates of ROC curves and a closed-form expression for the area under the ROC curve. Moreover, the Lehmann model can also be used to obtain covariate-adjusted ROC curves and accommodates inference of correlated and longitudinal biomarkers. Thus, the Lehman model should be considered as an alternative when the restrictive distributional assumptions of the binormal model are not met.

Receiver operating characteristic (ROC) curves: equivalences, beta model, and minimum distance estimation

Receiver operating characteristic (ROC) curves are used ubiquitously to evaluate scores, features, covariates or markers as potential predictors in binary problems. We characterize ROC curves from a probabilistic perspective and establish an equivalence between ROC curves and cumulative distribution functions (CDFs). These results support a subtle shift of paradigms in the statistical modelling of ROC curves, which we view as curve fitting. We propose the flexible two-parameter beta family for fitting CDFs to empirical ROC curves and derive the large sample distribution of minimum distance estimators in general parametric settings. In a range of empirical examples the beta family fits better than the classical binormal model, particularly under the vital constraint of the fitted curve being concave.

Inference on the overlap coefficient: The binormal approach and alternatives.

The overlap coefficient () measures the similarity between two distributions through the overlapping area of their distribution functions. Given its intuitive description and ease of visual representation by the straightforward depiction of the amount of overlap between the two corresponding histograms based on samples of measurements from each one of the two distributions, the development of accurate methods for confidence interval construction can be useful for applied researchers. The overlap coefficient has received scant attention in the literature since it lacks readily available software for its implementation, while inferential procedures that can cover the whole range of distributional scenarios for the two underlying distributions are missing. Such methods, both parametric and non-parametric are developed in this article, while R-code is provided for their implementation. Parametric approaches based on the binormal model show better performance and are appropriate for use in a wide range of distributional scenarios. Methods are assessed through a large simulation study and are illustrated using a dataset from a study on human immunodeficiency virus-related cognitive function assessment.

Using Multi-Antenna Trajectory Constraint to Analyze BeiDou Carrier-Phase Observation Error of Dynamic Receivers.

Appropriate cycle-slip and measurement-error models are essential for BeiDou carrier-phase-based integrity risk calculation. To establish the receiver’s measurement-error model, an accurate position reference of the GNSS antenna is fundamental for calculating the measurement error. However, it is still a challenge to acquire position references for dynamic BeiDou receivers, resulting in improper GNSS measurement-error models and unreliable integrity monitoring. This paper proposes an improved precise relative positioning scheme by adopting multi-antenna trajectory constraints for dynamic BeiDou receivers. The dynamic experiments show an obvious decline of 78.7%, at most, in the positioning failure rate of the proposed method, as compared with the traditional method. The position solutions obtained from the proposed approach are used as the reference to analyze the cycle-slip and measurement-error characteristics of the dynamic receiver. The field test results indicate that the cycle-slip rate decreases with the increase of signal-to-noise ratio (SNR), and cycle slipping obeys a positively skewed distribution that could be fitted by the Gaussian mixture model (GMM). On the other hand, the standard deviation of the carrier-phase measurement error is inversely proportional to SNR, and its distribution is characteristically fat-tailed, which could be fitted by the bi-normal model.

Radiation-Induced Pneumonitis in the Era of the COVID-19 Pandemic: Artificial Intelligence for Differential Diagnosis.

Simple SummaryRadiation-induced pneumonitis and severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) interstitial pneumonia show overlapping clinical features. As we are facing the COVID-19 pandemic, the discrimination between these two entities is of paramount importance. In fact, lung cancer patients are at higher risk of complications from SARS-CoV-2. In this study, we aimed to investigate if a deep learning algorithm was able to discriminate between COVID-19 and radiation therapy-related pneumonitis (RP). The algorithm showed high sensitivity but low specificity in the detection of RP against COVID-19 pneumonia (sensitivity = 97.0%, specificity = 2%, area under the curve (AUC = 0.72). The specificity increased when an estimated COVID-19 risk probability cut-off of 30% was applied (sensitivity 76%, specificity 63%, AUC = 0.84).(1) Aim: To test the performance of a deep learning algorithm in discriminating radiation therapy-related pneumonitis (RP) from COVID-19 pneumonia. (2) Methods: In this retrospective study, we enrolled three groups of subjects: pneumonia-free (control group), COVID-19 pneumonia and RP patients. CT images were analyzed by mean of an artificial intelligence (AI) algorithm based on a novel deep convolutional neural network structure. The cut-off value of risk probability of COVID-19 was 30%; values higher than 30% were classified as COVID-19 High Risk, and values below 30% as COVID-19 Low Risk. The statistical analysis included the Mann–Whitney U test (significance threshold at p < 0.05) and receiver operating characteristic (ROC) curve, with fitting performed using the maximum likelihood fit of a binormal model. (3) Results: Most patients presenting RP (66.7%) were classified by the algorithm as COVID-19 Low Risk. The algorithm showed high sensitivity but low specificity in the detection of RP against COVID-19 pneumonia (sensitivity = 97.0%, specificity = 2%, area under the curve (AUC = 0.72). The specificity increased when an estimated COVID-19 risk probability cut-off of 30% was applied (sensitivity 76%, specificity 63%, AUC = 0.84). (4) Conclusions: The deep learning algorithm was able to discriminate RP from COVID-19 pneumonia, classifying most RP cases as COVID-19 Low Risk.

Improving the size-selectivity of trammel net for Pomadasys stridens and Gerres oyena in Suez Bay, the Gulf of Suez, Red Sea

Size selectivity of the trammel net for Pomadasys stridens and Gerres oyena in Suez Bay, Gulf of Suez, Red sea, was estimated by comparing three mesh sizes 34, 37 and 40 mm for the inner layer of the trammel net. The SELECT model was used to estimate the selectivity curve. Length frequency data were fitted by five models: normal scale, gamma distribution model, normal location, lognormal distributions model, and bi-normal model. The results revealed that the best fit model of selectivity curve of the trammel nets data for Gerres oyena was obtained by the and bi-normal model, whilst the Log-normal model gave the best fit for Pomadasys stridens. The length-weight relationship equations were estimated to be W = 0.027 × L2.735 with r2 = 0.97 for Pomadasys stridens and W = 0.050 × L2.470, r2 = 0.93 for Gerres oyena. The study concluded that the mesh size of the trammel nets used in Suez Bay fishery should not be less than 40 mm especially for Pomadasys stridens and Gerres oyena to protect their stocks from overfishing.

Biomarker assessment in ROC curve analysis using the length of the curve as an index of diagnostic accuracy: the binormal model framework

In receiver operating characteristic (ROC) curve analysis, the area under the curve (AUC) is undoubtedly the most widely used index of diagnostic accuracy for the assessment of the utility of a biomarker or for the comparison of competing biomarkers. Along with the AUC, the maximum of the Youden index, J, is often used both as an index of diagnostic accuracy and as a tool useful for the estimation of an optimal cutoff point that can be used for diagnostic purposes based on the biomarker under consideration. In this work, we study the utility of the length of the binormal model-based ROC curve (LoC) as an index of diagnostic accuracy for biomarker evaluation. Estimation procedures for LoC, described in this article, are based either on normality assumptions or on the same assumptions after a Box–Cox transformation to normality. Two simulation studies are considered. In the first, the estimation procedures for LoC are compared in terms of bias and root mean squared error, while in the second one, the performance of LoC is compared with approaches based on AUC and J, both for the case of the assessment of a single biomarker and for the comparison of two biomarkers, in a parametric framework. We provide an interpretation for the proposed index and illustrate with an application on biomarkers from a colorectal cancer study.

On the Binormal Predictive Receiver Operating Characteristic Curve for the Joint Assessment of Positive and Negative Predictive Values.

The predictive receiver operating characteristic (PROC) curve is a diagrammatic format with application in the statistical evaluation of probabilistic disease forecasts. The PROC curve differs from the more well-known receiver operating characteristic (ROC) curve in that it provides a basis for evaluation using metrics defined conditionally on the outcome of the forecast rather than metrics defined conditionally on the actual disease status. Starting from the binormal ROC curve formulation, an overview of some previously published binormal PROC curves is presented in order to place the PROC curve in the context of other methods used in statistical evaluation of probabilistic disease forecasts based on the analysis of predictive values; in particular, the index of separation (PSEP) and the leaf plot. An information theoretic perspective on evaluation is also outlined. Five straightforward recommendations are made with a view to aiding understanding and interpretation of the sometimes-complex patterns generated by PROC curve analysis. The PROC curve and related analyses augment the perspective provided by traditional ROC curve analysis. Here, the binormal ROC model provides the exemplar for investigation of the PROC curve, but potential application extends to analysis based on other distributional models as well as to empirical analysis.