Statistical Measures Of Accuracy Research Articles

The Limited English Proficient Population: Describing Medicare, Medicaid, and Dual Beneficiaries.

Purpose: The Limited English Proficient (LEP) population experiences well-documented suboptimal health outcomes and substandard provider experiences. The lack of national estimates on the size of the LEP population relative to the healthcare setting makes examining health outcomes for this population very difficult. This analysis addresses this limitation by publishing population estimates for LEP persons enrolled in Medicare, Medicaid, and Duals (enrolled in Medicare and Medicaid). Focusing on the Medicare and Medicaid programs provides an important foundation as these programs are not only the largest insurers in the United States but are also governed by legislation that requires LEP persons to receive equitable access to care.Methods: Data from the 2014 American Community Survey Public Use Microdata Sample (ACS PUMS) were used to produce national estimates and measures of statistical accuracy for the LEP population enrolled in Medicare and/or Medicaid (LEPMM).Results: In 2014, there were approximately 8.7 million LEP persons enrolled in Medicare, Medicaid, or both programs (Duals). The LEPMM was concentrated along the western and eastern coastlines and the southwestern region, with California and New York each containing more than 1 million LEPMMs. The LEPMM was also highly diverse with varying disability status, and most were racial or ethnic minorities and elderly.Conclusion: These findings provide a foundation for measuring an understudied and at-risk population that will enable population health professionals to develop effective culturally and linguistically, and appropriate services and policies that address health disparities in the LEPMM.

Enhancing of accuracy assessment for forest above-ground biomass estimates obtained from remote sensing via hypothesis testing and overfitting evaluation

The evaluation of accuracy is essential for assuring the reliability of ecological models. Usually, the accuracy of above-ground biomass (AGB) predictions obtained from remote sensing is assessed by the mean differences (MD), the root mean squared differences (RMSD), and the coefficient of determination (R2) between observed and predicted values. In this article we propose a more thorough analysis of accuracy, including a hypothesis test to evaluate the agreement between observed and predicted values, and an assessment of the degree of overfitting to the sample employed for model training. Using the estimation of forest AGB from LIDAR and spectral sensors as a case study, we compared alternative prediction and variable selection methods using several statistical measures to evaluate their accuracy. We showed that the hypothesis tests provide an objective method to infer the statistical significance of agreement. We also observed that overfitting can be assessed by comparing the inflation in residual sums of squares experienced when carrying out a cross-validation. Our results suggest that this method may be more effective than analysing the deflation in R2. We proved that overfitting needs to be specifically addressed since, in light of MD, RMSD and R2 alone, predictions may apparently seem reliable even in clearly unrealistic circumstances, for instance when including too many predictor variables. Moreover, Theil’s partial inequality coefficients, which are employed to resolve the proportions of the total errors due to the unexplained variance, the slope and the bias, may become useful to detect averaging effects common in remote sensing predictions of AGB. We concluded that statistical measures of accuracy, precision and agreement are necessary but insufficient for model evaluation. We therefore advocate for incorporating evaluation measures specifically devoted to testing observed-versus-predicted fit, and to assessing the degree of overfitting.

Potential of four different machine-learning algorithms in modeling daily global solar radiation

In this study, the potential of different machine-learning algorithms in modeling global horizontal solar irradiation is examined. Multi-layer perceptron (MLP), adaptive neuro-fuzzy inference system (ANFIS) and Support Vector Machines (SVM) algorithms are adopted, beside a newly suggested algorithm: decision trees. All models are grouped in four categories: sunshine-, temperature-, meteorological parameters- and day number-based models. All models have been trained, optimized, validated and compared with each other and with old and newly suggested regression models, using high-resolution, highly accurate measured data recorded over Cairo, Egypt, throughout five years, as a case study. Models with best statistical measures of accuracy and best generalization abilities have been recommended after being tested using an independent dataset. The results show that MLP models excel in estimating global irradiation with root mean square error lower than that of best corresponding regression models by 4.75–31.69%, depending on the model category. Followed by ANFIS models (if carefully validated) and SVM models. In addition, the study assesses the ability of decision trees in modeling solar radiation. Despite of their simplicity, the merits of temperature- and day number-based models are demonstrated, with coefficients of determination greater than 85%, to be used in case of unavailability of sunshine records.

Excess Stormwater Quantification in Ungauged Watersheds Using an Event-Based Modified NRCS Model

Quantifying runoff from a storm event is a crucial part of rainfall-runoff model development. The objective of this study is to illustrate inconsistencies in the initial abstraction (Ia) and curve number (CN) in the Natural Resources Conservation Service (NRCS) model for ungauged steep slope watersheds. Five alternatives to the NRCS model were employed to estimate stormwater runoff in 39 forest-dominated mountainous watersheds. The change to the parameterization (slope-adjusted CN and Ia) leads to more efficient modified NRCS models. The model evaluations based on root mean square error (RMSE), Nash-Sutcliffe coefficient E, coefficient of determination (R2), and percent bias (PB) indicated that our proposed model with modified Ia, consistently performed better than the other four models and the original NRCS model, in reproducing the runoff. In addition to the quantitative statistical accuracy measures, the proposed Ia modification in the NRCS model showed very encouraging results in the scatter plots of the combined 1799 storm events, compared to other alternatives. This study’s findings support modifications to the CN and the Ia in the NRCS model for steep slope ungauged watersheds and suggest additional changes for more accurate runoff estimations.

Modified Kalman Filter for GPS Position Estimation over the Indian Sub Continent

Position accuracy is the measure of a system's capability to provide quality estimates, which in turn depends mainly on measurement noise and the type of algorithm employed. Though many positioning algorithms have emerged, due to its exceptional performance in a wide range of real time applications, the Kalman Filter Estimator (KFE) is used often for their implementation. So this paper concentrates in improving its accuracy further while introducing a new observation matrix in its estimation process and succeeds in performing the same. This paper also extends the modified algorithm for GPS receiver position estimation over the Indian subcontinent. Real time data collected from GPS receiver located at IISC, Bangalore (Lat/Lon: 13.010N /77.560E), India is used to evaluate the performance of this developed algorithm called Modified Kalman Filter Estimator (MKFE). GPS Statistical Accuracy Measures (SAM) such as Circular Error Probability (CEP), Spherical Error Probability (SEP) etc. are used for performance evaluation. From the results it is observed that the proposed MKFE has faster convergence rate with high accuracy and is suitable for real time defence applications over the Indian subcontinent.

Benchmarking Compound Methods (CBS-QB3, CBS-APNO, G3, G4, W1BD) against the Active Thermochemical Tables: A Litmus Test for Cost-Effective Molecular Formation Enthalpies.

The theoretical atomization energies of some 45 CxHyOz molecules present in the Active Thermochemical Tables compilation and of particular interest to the combustion chemistry community have been computed using five composite model chemistries as titled. The species contain between 1-8 "heavy" atoms, and a few are conformationally diverse with up to nine conformers. The enthalpies of formation at 0 and 298.15 K are then derived via the atomization method and compared against the recommended values. In general, there is very good agreement between our averaged computed values and those in the ATcT; those for 1,3-cyclopentadiene exceptionally differ considerably, and we show from isodesmic reactions that the true value for 1,3-cyclopentadiene is closer to 134 kJ mol(-1) than the reported 101 kJ mol(-1). If one is restricted to using a single method, statistical measures indicate that the best methods are in the rank order G3 ≈ G4 > W1BD > CBS-APNO > CBS-QB3. The CBS-x methods do on average predict ΔfH(⊖)(298.15 K) within ≈5 kJ mol(-1) but are prone to occasional lapses. There are statistical advantages to be gained from using a number of methods in tandem, and all possible combinations have been tested. We find that the average formation enthalpy coming from using CBS-APNO/G4, CBS-APNO/G3, and G3/G4 show lower mean signed and mean unsigned errors, and lower standard and root-mean-squared deviations, than any of these methods in isolation. Combining these methods also leads to the added benefit of providing an uncertainty rooted in the chemical species under investigation. In general, CBS-APNO and W1BD tend to underestimate the formation enthalpies of target species, whereas CBS-QB3, G3, and G4 have a tendency to overestimate the same. Thus, combining CBS-APNO with a G3/G4 combination leads to an improvement in all statistical measures of accuracy and precision, predicting the ATcT values to within 0.14 ± 4.21 kJ mol(-1), thus rivalling "chemical accuracy" (±4.184 kJ mol(-1)) without the excessive cost associated with higher-level methods such as W1BD.

Ultrasound Alongside with Mammogram in Women with Physically Dense Breast

We report usefulness of ultrasound used as an adjunct diagnostic tool to mammogram in routine annual checkup for women breasts of certain ages and breast mass. The purpose of breast imaging is to detect areas of tissue distortion and breast cancers. A mammogram is the common diagnostic imaging modality used to find breast diseases but sometimes the mammogram might not give the doctor enough information especially in women with dense breasts. As a result, the patient may be asked to undergo ultrasound or magnetic resonance imaging as a better mean of judgment to the case. Because ultrasound is widely used, simple and safe to patients we were encouraged to emphasis on exploring its role adjunct to mammogram. A retrospective observation study was done at the diagnostic radiology department at King Abdulaziz University Hospital (KAUH) in the period from January 2012 to June 2012; we covered all women with dense breasts in mammography and ultrasound units. The study group was 40 patients. All patients were imaged with both mammography and ultrasound. The statistical measures of accuracy, sensitivity and specificity were calculated using the SPSS program. The results we obtained suggest that age and the physical density of breast potentially affect mammogram images of women with 41 years or smaller with sensitivity 66% and specificity 68%. Therefore, we recommend using ultrasound alongside the mammogram in women with dense breast for better diagnosis of small cancers that were not identified on mammography or clinical breast examination alone.

Assessing the Auto Associative Network Approach for Prediction in Civil Engineering Databases

Auto-associative networks are a type of Artificial Neural Network (ANN) architectures that has been used in a variety of engineering areas for the past two decades. In auto-associative networks, the knowledge to be extracted from a database is the identity function. In other words, this particular network is trained to reproduce its inputs and output(s). Due to the fact that the network is optimized on inputs, as well as outputs, obtaining highly accurate results can be challenging. In this study, auto-associative network was explored using seven civil engineering databases from various applications and with a range of data types. The architecture of the auto-associative networks was developed with only three layers - input, hidden, and output layers - in order to maintain the generalization capabilities. Only the output was considered when assessing the statistical accuracy measures. A traditional ANN model was developed for each database to provide an initial estimate of the output. Then these estimates and the inputs were used to develop the auto-associative network. The auto-associative network improved the statistical accuracy measures for some databases relative to the traditional ANN approach. Overall, the auto-associative network yielded promising results and can be applicable to civil engineering databases.

Coherent uncertainty analysis of aerosol measurements from multiple satellite sensors

Abstract. Aerosol retrievals from multiple spaceborne sensors, including MODIS (on Terra and Aqua), MISR, OMI, POLDER, CALIOP, and SeaWiFS – altogether, a total of 11 different aerosol products – were comparatively analyzed using data collocated with ground-based aerosol observations from the Aerosol Robotic Network (AERONET) stations within the Multi-sensor Aerosol Products Sampling System (MAPSS, http://giovanni.gsfc.nasa.gov/mapss/ and http://giovanni.gsfc.nasa.gov/aerostat/. The analysis was performed by comparing quality-screened satellite aerosol optical depth or thickness (AOD or AOT) retrievals during 2006–2010 to available collocated AERONET measurements globally, regionally, and seasonally, and deriving a number of statistical measures of accuracy. We used a robust statistical approach to detect and remove possible outliers in the collocated data that can bias the results of the analysis. Overall, the proportion of outliers in each of the quality-screened AOD products was within 7%. Squared correlation coefficient (R2) values of the satellite AOD retrievals relative to AERONET exceeded 0.8 for many of the analyzed products, while root mean square error (RMSE) values for most of the AOD products were within 0.15 over land and 0.07 over ocean. We have been able to generate global maps showing regions where the different products present advantages over the others, as well as the relative performance of each product over different land cover types. It was observed that while MODIS, MISR, and SeaWiFS provide accurate retrievals over most of the land cover types, multi-angle capabilities make MISR the only sensor to retrieve reliable AOD over barren and snow/ice surfaces. Likewise, active sensing enables CALIOP to retrieve aerosol properties over bright-surface closed shrublands more accurately than the other sensors, while POLDER, which is the only one of the sensors capable of measuring polarized aerosols, outperforms other sensors in certain smoke-dominated regions, including broadleaf evergreens in Brazil and South-East Asia.

How useful are no-arbitrage restrictions for forecasting the term structure of interest rates?

We develop a general framework for analyzing the usefulness of imposing parameter restrictions on a forecasting model. We propose a measure of the usefulness of the restrictions that depends on the forecaster’s loss function and that could be time varying. We show how to conduct inference about this measure. The application of our methodology to analyzing the usefulness of no-arbitrage restrictions for forecasting the term structure of interest rates reveals that: (1) the restrictions have become less useful over time; (2) when using a statistical measure of accuracy, the restrictions are a useful way to reduce parameter estimation uncertainty, but are dominated by restrictions that do the same without using any theory; (3) when using an economic measure of accuracy, the no-arbitrage restrictions are no longer dominated by atheoretical restrictions, but for this to be true it is important that the restrictions incorporate a time-varying risk premium.

Development of a Flow Number Predictive Model

The NCHRP 9-19 panel recommended the repeated load permanent deformation test as a laboratory procedure that could be used to evaluate the resistance of a hot-mix asphalt (HMA) to tertiary flow. No standard test protocol addresses the required laboratory stress to be applied. The test can take several hours until tertiary flow is reached and in many cases the sample may never fail. A model capable of predicting or providing general guidance on the flow number characteristics of a mix can be of great value. The model can be ideally used as a guideline to determine the stress–temperature combination that will yield tertiary flow within a reasonable testing time. In this study, an effort was undertaken to develop a flow number predictive model. The model uses HMA mixture volumetric properties and stress–temperature testing conditions as predictor variables. The laboratory test data used are a combination of two valuable databases. The first one included tests conducted at Arizona State University; the second one included tests conducted by the FHWA Mobile Asphalt Material Testing Laboratory. Ninety-four mixtures were evaluated, and 1,759 flow number test results were available. Various regression models were evaluated by combining several independent variables. The final model selected had fair statistical measures of accuracy, and it covered a wide range of mixtures, gradations, and binder properties, as well as laboratory-applied stress. As more testing data become available, the model could be refined and recalibrated for better accuracy.

Modeling churn using customer lifetime value

The definition and modeling of customer loyalty have been central issues in customer relationship management since many years. Recent papers propose solutions to detect customers that are becoming less loyal, also called churners. The churner status is then defined as a function of the volume of commercial transactions. In the context of a Belgian retail financial service company, our first contribution is to redefine the notion of customer loyalty by considering it from a customer-centric viewpoint instead of a product-centric one. We hereby use the customer lifetime value (CLV) defined as the discounted value of future marginal earnings, based on the customer’s activity. Hence, a churner is defined as someone whose CLV, thus the related marginal profit, is decreasing. As a second contribution, the loss incurred by the CLV decrease is used to appraise the cost to misclassify a customer by introducing a new loss function. In the empirical study, we compare the accuracy of various classification techniques commonly used in the domain of churn prediction, including two cost-sensitive classifiers. Our final conclusion is that since profit is what really matters in a commercial environment, standard statistical accuracy measures for prediction need to be revised and a more profit oriented focus may be desirable.

Forecasting with DSGE Models: The Role of Nonlinearities

This paper studies whether the out-of-sample forecasting performance of a dynamic stochastic general equilibrium (DSGE) model improves by taking its nonlinear rather than its linear approximation to the data. We address this question within a New Keynesian monetary economy, considering both environments of simulated and real data. Precisely, we estimate our model based on its linear respectively quadratic approximate solution, generate out-of-sample forecasts for three observables (output, inflation, and the nominal interest rate), and compare the quality of forecasts by several statistical measures of accuracy. We find that the value of nonlinearities in terms of predictive power depends crucially on whether the model is well specified. For simulated data, the nonlinear model indeed forecasts noticeably better as compared to its linearized counterpart, whereas for real data, we find no substantial differences in predictive abilities.

Evaluation of Accuracy in Proband-Reported Family History and Its Determinants: The Genes in Myopia Family Study

Proband-reported family histories are widely used in epidemiological and genetic studies. The accuracy of these reports may have significant effects on the intended outcome, particularly in genetic studies. This study aims to determine the accuracy of proband-reported family history of myopia and to assess whether demographic or clinical factors are predictive of an accurate history. In 2004 to 2005, the study recruited 120 myopic probands (< or = -0.50 D spherical equivalent in both eyes) aged 18 to 72 years and 358 nuclear family members residing within Victoria, Australia as part of the Genes in Myopia (GEM) family study. Data collection used an examiner-administered questionnaire with an ocular examination. Proband-reported family history of myopia was evaluated for agreement with ophthalmic examination results of family members. The statistical measures of accuracy used in this report were sensitivity, specificity, positive predictive value, and negative predictive value. Sensitivity varied from 85 to 98%, specificity from 84 to 96%, positive predictive value from 83 to 97%, and negative predictive value from 84 to 97%. Following multivariate analysis, an evaluation of demographic and clinical factors indicated that the highest predictive accuracy was obtained from proband reporting of their children [odds ratio (OR), 0.38; 95% confidence interval (CI), 0.15 to 0.94] whereas the most inaccurate reporting of a proband was when there was less-severe maternal myopia (per 0.50 D less myopic) (OR, 1.23; 95% CI, 1.06 to 1.43) or for increase in total education of the proband (per 1 year increase) (OR, 1.22; 95% CI, 1.04 to 1.42). Several variables influence the accuracy of obtaining a family history of myopia. A questionnaire-based approach alone will introduce some error into the study and this should be taken into account when designing and undertaking family-based epidemiological or genetic studies of myopia.

The Impact of Operative Complexity on Patient Risk Factors

The VA National Surgical Quality Improvement Program (NSQIP) formula for risk factors was applied to the University of Texas Health Science Center at San Antonio (UTHSCSA)/University Hospital (UH) database. Its applicability to a civilian organization was established. Logistic regression analysis of the UH database revealed that operative complexity was significantly related to mortality only at high complexity levels. Patient risk factors were the major determinants of operative mortality for most civilian surgical cases. Since 1994, the NSQIP has collected preoperative risk factors, intraoperative data, 30-day morbidity, and mortality within the VA health system. The VA formula to predict 30-day postoperative mortality was applied to our UH patients (N = 8593). The c-index of .907, a statistical measure of accuracy, compared favorably to the VA patient c-index of .89. The UH database did not include a surrogate for operative complexity. We were elated by the predictive accuracy but had concern that operative complexity needed further evaluation. Operative complexity was ascribed to each of the 8593 UH cases, and logistic regression analyses were compared with and without operative complexity. Operative complexity was graded on a scale of 1 to 5; 5 was the most complex. Without operative complexity, a c-index was .915. With operative complexity: an even higher c-index of .941 was reached. The large volume of level 2-3 operative cases obscured to a degree the effect of operative difficulty on mortality. Operative complexity played a major role in risk estimation, but only at the extreme. The dominance of cases of midlevel complexity masked the effect of higher complexity cases on mortality. In any individual case, operative complexity must be added to estimate operative mortality accurately. Patient risk factors alone accounted for operative mortality for operations less than level 4 (95% of patients).

A Guide to Tax Policy Analysis: Problems with Distributional Tax Tables

The analysis of tax policy and tax legislation can be highly conjectural and consequently more art than science. Tables and figures detailing revenue effects and distribution of burdens associated with projected outcomes of proposed tax legislation are often presented in ways that distort or fail to disclose information regarding the economic outcomes. Additionally, some of these tables are based on data sources that are statistically compromised and for which statistical measures of accuracy are impossible to calculate. Furthermore, the public is often not informed as to the limitations inherent in the information. Members of Congress, students of tax analysis, the media and ordinary citizens seeking to understand the economic effects of proposed tax legislation are inundated with revenue estimates and distributional tables that often obscure the economic issues and hinder the policy process.

Application of Bootstrap Methods to Reduced Major Axis Line Fitting

The bootstrap method developed by Efron allows the estimation of measures of statistical accuracy, such as the standard errors and confidence limits, in cases where the un- derlying distribution is unknown or where sample sizes are small. This is frequently the case for statistics and data sets of systematic interest, especially in paleontology. The utility of the bootstrap is demonstrated for the reduced major axis (RMA) method for fitting the allometric equation. The results are in good agreement with statistical properties determined by available analytical methods and are superior to many of these methods in capturing the detailed distri- bution of the parameters. The bootstrap also allows the simultaneous comparison of slope and intercept estimates between taxa. Bootstrap methods should be a valuable tool in morphometrics. (Allometry; bootstrap; arthropods.)

Toward an empirical approach to evaluating the knowledge base of an expert system

A general approach is described to the test and evaluation of an expert system knowledge base that involves the use of a representative set of randomly selected test problems and the application of well-defined statistical measures of accuracy and bias for each node in an inference network. Using the analogy that each node can be treated as a signal detector, some elements of a mathematics consistent with the approach are developed. >

Recent advances in fine particle magnetism: Implications for paleomagnetism

It is becoming increasingly obvious in paleomagnetism that the older practice of using large numbers of samples to improve statistical measures of accuracy is often not adequate to reveal the signal of greatest geological importance. “Blanket” demagnetization treatments are now largely outdated, except for the most ideally behaved samples (a rapidly vanishing phenomenon in many locales). Variations in characteristic remanent directions between neighboring (and supposedly contemporaneous) but lithogically distinct rock units can no longer be simply treated as “noise”. We now recognize that such differences may actually reflect rock magnetic effects rather than such factors as the short‐term behavior of the geomagnetic field or local variations in structure. Multiple overprints, often the result of moderate reheating and/or chemical alteration, can result in an uncertain relationship between the timing of emplacement and the acquisition of remanence. In the past, rocks which presented such problems could often be avoided in favor of more straightforward units. Much of that geological “cream” has been skimmed, however. Further progress in many areas, particularly the determination of the APW path and reversal stratigraphy in the Paleozoic, will require a better understanding of magnetization processes in order to resolve the above‐mentioned problems and produce accurate and reliable results.

Estimating Brand Positioning Maps Using Supermarket Scanning Data

Hauser and Shugan's DEFENDER model is estimated with data obtained from supermarket scanning devices. As a result of that estimation, per dollar multidimensional brand maps are obtained from aggregate sales and price time series data. The estimation procedure employs least squares regression analysis. The maps enable a researcher or brand manager to (1) derive a map from observed choice behavior, (2) study the direct effect of both a brand's price and a brand's position on that brand's sales, (3) obtain statistical measures of accuracy for the estimated brand coordinates, and (4) integrate the traditional cross-price elasticity interpretation of substitutability with perceptual mapping. Actual maps are estimated for three product categories (toothpaste, mouthwash, and dishwashing liquid). Preliminary findings are presented. For example, brands produced by the same manufacturer are often in proximity.