Proportion Of Misclassifications Research Articles

How well does NamSor perform in predicting the country of origin and ethnicity of individuals based on their first and last names?

We aimed to evaluate NamSor's performance in predicting the country of origin and ethnicity of individuals based on their first/last names. We retrieved the name and country of affiliation of all authors of PubMed publications in 2021, affiliated with universities in the twenty-two countries whose researchers authored ≥1,000 medical publications and whose percentage of migrants was <2.5% (N = 88,699). We estimated with NamSor their most likely "continent of origin" (Asia/Africa/Europe), "country of origin" and "ethnicity". We also examined two other variables that we created: "continent#2" ("Europe" replaced by "Europe/America/Oceania") and "country#2" ("Spain" replaced by "Spain/Hispanic American country" and "Portugal" replaced by "Portugal/Brazil"). Using "country of affiliation" as a proxy for "country of origin", we calculated for these five variables the proportion of misclassifications (= errorCodedWithoutNA) and the proportion of non-classifications (= naCoded). We repeated the analyses with a subsample consisting of all results with inference accuracy ≥50%. For the full sample and the subsample, errorCodedWithoutNA was 16.0% and 12.6% for "continent", 6.3% and 3.3% for "continent#2", 27.3% and 19.5% for "country", 19.7% and 11.4% for "country#2", and 20.2% and 14.8% for "ethnicity"; naCoded was zero and 18.0% for all variables, except for "ethnicity" (zero and 10.7%). NamSor is accurate in determining the continent of origin, especially when using the modified variable (continent#2) and/or restricting the analysis to names with accuracy ≥50%. The risk of misclassification is higher with country of origin or ethnicity, but decreases, as with continent of origin, when using the modified variable (country#2) and/or the subsample.

Using genderize.io to infer the gender of first names: how to improve the accuracy of the inference

Objective:We recently showed that genderize.io is not a sufficiently powerful gender detection tool due to a large number of nonclassifications. In the present study, we aimed to assess whether the accuracy of inference by genderize.io can be improved by manipulating the first names in the database.Methods:We used a database containing the first names, surnames, and gender of 6,131 physicians practicing in a multicultural country (Switzerland). We uploaded the original CSV file (file #1), the file obtained after removing all diacritic marks, such as accents and cedilla (file #2), and the file obtained after removing all diacritic marks and retaining only the first term of the compound first names (file #3). For each file, we computed three performance metrics: proportion of misclassifications (errorCodedWithoutNA), proportion of nonclassifications (naCoded), and proportion of misclassifications and nonclassifications (errorCoded).Results:naCoded, which was high for file #1 (16.4%), was reduced after data manipulation (file #2: 11.7%, file #3: 0.4%). As the increase in the number of misclassifications was small, the overall performance of genderize.io (i.e., errorCoded) improved, especially for file #3 (file #1: 17.7%, file #2: 13.0%, and file #3: 2.3%).Conclusions:A relatively simple manipulation of the data improved the accuracy of gender inference by genderize.io. We recommend using genderize.io only with files that were modified in this way.

How accurate are gender detection tools in predicting the gender for Chinese names? A study with 20,000 given names in Pinyin format.

Objective:We recently showed that the gender detection tools NamSor, Gender API, and Wiki-Gendersort accurately predicted the gender of individuals with Western given names. Here, we aimed to evaluate the performance of these tools with Chinese given names in Pinyin format.Methods:We constructed two datasets for the purpose of the study. File #1 was created by randomly drawing 20,000 names from a gender-labeled database of 52,414 Chinese given names in Pinyin format. File #2, which contained 9,077 names, was created by removing from File #1 all unisex names that we were able to identify (i.e., those that were listed in the database as both male and female names). We recorded for both files the number of correct classifications (correct gender assigned to a name), misclassifications (wrong gender assigned to a name), and nonclassifications (no gender assigned). We then calculated the proportion of misclassifications and nonclassifications (errorCoded).Results:For File #1, errorCoded was 53% for NamSor, 65% for Gender API, and 90% for Wiki-Gendersort. For File #2, errorCoded was 43% for NamSor, 66% for Gender API, and 94% for Wiki-Gendersort.Conclusion:We found that all three gender detection tools inaccurately predicted the gender of individuals with Chinese given names in Pinyin format and therefore should not be used in this population.

Performance of gender detection tools: a comparative study of name-to-gender inference services.

Objective:To evaluate the performance of gender detection tools that allow the uploading of files (e.g., Excel or CSV files) containing first names, are usable by researchers without advanced computer skills, and are at least partially free of charge.Methods:The study was conducted using four physician datasets (total number of physicians: 6,131; 50.3% female) from Switzerland, a multilingual country. Four gender detection tools met the inclusion criteria: three partially free (Gender API, NamSor, and genderize.io) and one completely free (Wiki-Gendersort). For each tool, we recorded the number of correct classifications (i.e., correct gender assigned to a name), misclassifications (i.e., wrong gender assigned to a name), and nonclassifications (i.e., no gender assigned). We computed three metrics: the proportion of misclassifications excluding nonclassifications (errorCodedWithoutNA), the proportion of nonclassifications (naCoded), and the proportion of misclassifications and nonclassifications (errorCoded).Results:The proportion of misclassifications was low for all four gender detection tools (errorCodedWithoutNA between 1.5 and 2.2%). By contrast, the proportion of unrecognized names (naCoded) varied: 0% for NamSor, 0.3% for Gender API, 4.5% for Wiki-Gendersort, and 16.4% for genderize.io. Using errorCoded, which penalizes both types of error equally, we obtained the following results: Gender API 1.8%, NamSor 2.0%, Wiki-Gendersort 6.6%, and genderize.io 17.7%.Conclusions:Gender API and NamSor were the most accurate tools. Genderize.io led to a high number of nonclassifications. Wiki-Gendersort may be a good compromise for researchers wishing to use a completely free tool. Other studies would be useful to evaluate the performance of these tools in other populations (e.g., Asian).

The fallacy of indexed effective orifice area charts to predict prosthesis-patient mismatch after prosthesis implantation.

Aims Indexed effective orifice area (EOAi) charts are used to determine the likelihood of prosthesis–patient mismatch (PPM) after aortic valve replacement (AVR). The aim of this study is to validate whether these EOAi charts, based on echocardiographic normal reference values, can accurately predict PPM.Methods and results In the PERIcardial SurGical AOrtic Valve ReplacemeNt (PERIGON) Pivotal Trial, 986 patients with aortic valve stenosis/regurgitation underwent AVR with an Avalus valve. Patients were randomly split (50:50) into training and test sets. The mean measured EOAs for each valve size from the training set were used to create an Avalus EOAi chart. This chart was subsequently used to predict PPM in the test set and measures of diagnostic accuracy (sensitivity, specificity, and negative and positive predictive value) were assessed. PPM was defined by an EOAi ≤0.85 cm2/m2, and severe PPM was defined as EOAi ≤0.65 cm2/m2. The reference values obtained from the training set ranged from 1.27 cm2 for size 19 mm up to 1.81 cm2 for size 27 mm. The test set had an incidence of 66% of PPM and 24% of severe PPM. The EOAi chart inaccurately predicted PPM in 30% of patients and severe PPM in 22% of patients. For the prediction of PPM, the sensitivity was 87% and the specificity 37%. For the prediction of severe PPM, the sensitivity was 13% and the specificity 98%.Conclusion The use of echocardiographic normal reference values for EOAi charts to predict PPM is unreliable due to the large proportion of misclassifications.

Gamblers' (mis-)interpretations of Problem Gambling Severity Index items: Ambiguities in qualitative accounts from the Swedish Longitudinal Gambling Study.

The Problem Gambling Severity Index (PGSI) is a screening instrument frequently used to identify risk and problem gambling. Even though the PGSI has good psychometric properties, it still produces a large proportion of misclassifications.Aims:To explore possible reasons for misclassifications in problem gambling level by analysing previously classified moderate-risk gamblers’ answers to the PGSI items, in relation to their own current and past gambling behaviours.Methods:Semi-structured telephone interviews were conducted with 19 participants reporting no negative consequences from gambling. They were asked the PGSI questions within an eight-year time frame (2008 to 2016). Ambiguous answers to PGSI items were subject to content analysis.Results:Several answers to the PGSI items contained ambiguities and misinterpretations, making it difficult to assess to what extent their answers actually indicated any problematic gambling over time. The item about feelings of guilt generated accounts rather reflecting self-recrimination over wasting money or regretting gambling as a meaningless or immoral activity. The item concerning critique involved mild interpretations such as being ridiculed for buying lottery tickets or getting comments for being boring. Similar accounts were given by the participants irrespective of initial endorsement of the items. Other possible reasons for misclassifications were related to recall bias, language difficulties, selective memory, and a tendency to answer one part of the question without taking the whole question into account.Conclusions:Answers to the PGSI can contain a variety of meanings based on the respondents’ subjective interpretations. Reports of lower levels of harm in the population should thus be interpreted with caution. In clinical settings it is important to combine use of screening instruments with interviews, to be able to better understand gamblers’ perceptions of the gambling behaviour and its negative consequences.

Arbre à nœuds multivariés : le modèle rop (régression optimisée)

Introduction In a classic tree of the model CART, knots used a unique attribute. Moreover, decision trees are hierarchical method, the pathway between the root and leaves are one-way and the classification of the observations in a leaf is definitive. We propose a new concept of decision tree, including multivariate knots and non-hierarchical pathway. Methods A risk score, including all factors, is calculated for each knot, using a combinatorial approach. Terminal leaves are exclusively constituted by negative observations (“not at risk”) and by the last positive leaf (“at risk”) of the tree. An algorithm based on 5 criterions, selects one combination among those maximize the sum of sensitivity + specificity: identical observations, the minimization of the sum of the absolute values of coefficients, the maximization of the AUC, the minimization of the sum of the absolute values of coefficients, and random selection. The data of the sinking of Titanica (n = 1316) were used to compare the performance and the internal validation (bootstrap, n = 1000) obtained with this new tree and the logistic regression. Results The sensibilities estimated were equal to 84.75% for the logistic model versus 84.76% for the new tree. The specificities estimated were equal to 67.72% for the logistic model versus 67.69% for the new tree. The new tree model was slightly better (21.62% versus 21.69%) than the logistic regression for the criteria of proportion of misclassifications estimated. Conclusion We described a new method of regression-classification including several methodological innovations in methods of classification trees. Applications of this new tree were already published. The research is continuing on the development of random forest built with these trees. The first results are very promising. The first version of the program will be available as a package for R and could be requested.

Differential diagnosis of pleural mesothelioma using Logic Learning Machine.

BackgroundTumour markers are standard tools for the differential diagnosis of cancer. However, the occurrence of nonspecific symptoms and different malignancies involving the same cancer site may lead to a high proportion of misclassifications.Classification accuracy can be improved by combining information from different markers using standard data mining techniques, like Decision Tree (DT), Artificial Neural Network (ANN), and k-Nearest Neighbour (KNN) classifier. Unfortunately, each method suffers from some unavoidable limitations. DT, in general, tends to show a low classification performance, whereas ANN and KNN produce a "black-box" classification that does not provide biological information useful for clinical purposes.MethodsLogic Learning Machine (LLM) is an innovative method of supervised data analysis capable of building classifiers described by a set of intelligible rules including simple conditions in their antecedent part. It is essentially an efficient implementation of the Switching Neural Network model and reaches excellent classification accuracy while keeping low the computational demand.LLM was applied to data from a consecutive cohort of 169 patients admitted for diagnosis to two pulmonary departments in Northern Italy from 2009 to 2011. Patients included 52 malignant pleural mesotheliomas (MPM), 62 pleural metastases (MTX) from other tumours and 55 benign diseases (BD) associated with pleurisies. Concentration of three tumour markers (CEA, CYFRA 21-1 and SMRP) was measured in the pleural fluid of each patient and a cytological examination was also carried out.The performance of LLM and that of three competing methods (DT, KNN and ANN) was assessed by leave-one-out cross-validation.ResultsLLM outperformed all other considered methods. Global accuracy was 77.5% for LLM, 72.8% for DT, 54.4% for KNN, and 63.9% for ANN, respectively. In more details, LLM correctly classified 79% of MPM, 66% of MTX and 89% of BD. The corresponding figures for DT were: MPM = 83%, MTX = 55% and BD = 84%; for KNN: MPM = 58%, MTX = 45%, BD = 62%; for ANN: MPM = 71%, MTX = 47%, BD = 76%.Finally, LLM provided classification rules in a very good agreement with a priori knowledge about the biological role of the considered tumour markers.ConclusionsLLM is a new flexible tool potentially useful for the differential diagnosis of pleural mesothelioma.

Dementia in patients with atrial fibrillation and the value of the Hachinski ischemic score

To assess the prevalence of vascular dementia, mixed dementia and Alzheimer's disease in patients with atrial fibrillation, and to evaluate the accuracy of the Hachinski ischemic score for these subtypes of dementia. A nested case-control study was carried out. A total of 103 of 784 consecutive patients evaluated for cognitive status at the Ambulatory Geriatric Clinic had a diagnosis of atrial fibrillation. Controls without atrial fibrillation were randomly selected from the remaining 681 patients using a 1:2 matching for sex, age and education. The prevalence of vascular dementia was twofold in patients with atrial fibrillation compared with controls (21.4% vs 10.7%, P = 0.024). Alzheimer's disease was also more frequent in the group with atrial fibrillation (12.6% vs 7.3%, P = 0.046), whereas mixed dementia had a similar distribution. The Hachinski ischemic score poorly discriminated between dementia subtypes, with misclassification rates between 46% (95% CI 28-66) and 70% (95% CI 55-83). In patients with atrial fibrillation, these rates ranged from 55% (95% CI 32-77) to 69% (95% CI 39-91%). In patients in whom the diagnosis of dementia was excluded, the Hachinski ischemic score suggested the presence of vascular dementia in 11% and mixed dementia in 30%. Vascular dementia and Alzheimer's disease, but not mixed dementia, are more prevalent in patients with atrial fibrillation. The discriminative accuracy of the Hachinski ischemic score for dementia subtypes in atrial fibrillation is poor, with a significant proportion of misclassifications.

Do Postconcussive Symptoms Discriminate Injury Severity in Pediatric Mild Traumatic Brain Injury?

To assess whether postconcussive symptoms (PCS) can be used to discriminate injury severity among children with mild traumatic brain injury (TBI). One hundred eighty-six children with mild TBI, divided into high and low injury severity depending on whether the injury was associated with a loss of consciousness (LOC), and a comparison group of 99 children with orthopedic injuries (OI), all aged 8 to 15 years at the time of injury. Parent-rated frequency and severity of PCS at initial assessment within 2 weeks postinjury and again at 3 and 12 months postinjury. Ratings of PCS obtained at the initial and 3-month assessments differentiated children with mild TBI from OI, although only ratings at the initial assessment discriminated among all 3 groups. Somatic PCS accounted for most of the discriminatory power. Overall, the accuracy of group classification was relatively modest, with a large proportion of misclassifications of children in the mild-TBI groups. Although children with mild TBI have more PCS than children with OI, PCS do not permit sufficiently accurate discrimination of mild TBI and injury severity to warrant diagnostic decisions at this time.

Improving the Discriminatory Power of a Near-Infrared Microscopy Spectral Library with a Support Vector Machine Classifier

A multi-group classifier based on the support vector machine (SVM) has been developed for use with a library of 48,456 spectra measured by near-infrared reflection microscopy (NIRM) on 227 samples representing 26 animal feed ingredients and 4 possible contaminants of animal origin. The performance of the classifier was assessed by a five-fold cross-validation, dividing at the sample level. Although the overall proportion of misclassifications was 27%, almost all of these involved the confusion of pairs of similar ingredients of vegetable origin. Such confusions are unimportant in the context of the intended use of the library, which is the detection of banned ingredients in animal feed. The error rate in discrimination between permitted and banned ingredients was just 0.17%. The performance of the SVM classifier was substantially better than that of the K-nearest-neighbors method employed in previous work with the same library, for which the comparable error rates are 36% overall and 0.39% for permitted versus banned ingredients.

Assessment of Sleep in Ventilator-Supported Critically Ill Patients

In critically ill patients, sleep derangements are reported to be severe using Rechtschaffen and Kales (R&K) methodology; however, whether such methodology can reliably assess sleep during critical illness is unknown. We set out to determine the reproducibility of 4 different sleep-assessment methods (3 manual and 1 computer-based) for ventilator-supported critically ill patients and also to quantify the extent to which the reproducibility of the manual methods for measuring sleep differed between critically ill and ambulatory (control) patients. Observational methodologic study. Academic center. Critically ill patients receiving mechanical ventilation and age-matched controls underwent polysomnography. None. Reproducibility for the computer-based method (spectral analysis of electroencephalography [EEG]) was better than that for the manual methods: R&K methodology and sleep-wakefulness organization pattern (P = 0.03). In critically ill patients, the proportion of misclassifications for measurements using spectral analysis, sleep-wakefulness organization pattern, and R&K methodology were 0%, 36%, and 53%, respectively (P < 0.0001). The EEG pattern of burst suppression was not observed. Interobserver and intraobserver reliability of the manual sleep-assessment methods for critically ill patients (kappa = 0.52 +/- 0.23) was worse than that for control patients (kappa = 0.89 +/- 0.13; P = 0.03). In critically ill patients, the overall reliability of the R&K methodology was relatively low for assessing sleep (kappa = 0.19), but detection of rapid eye movement sleep revealed good agreement (kappa = 0.70). Reproducibility for spectral analysis of EEG was better than that for the manual methods: R&K methodology and sleep-wakefulness organization pattern. For assessment of sleep in critically ill patients, the use of spectral analysis, sleep-wakefulness organization state, or rapid eye movement sleep alone may be preferred over the R&K methodology.

Prediction of cancer outcome with microarrays: a multiple random validation strategy

General studies of microarray gene-expression profiling have been undertaken to predict cancer outcome. Knowledge of this gene-expression profile or molecular signature should improve treatment of patients by allowing treatment to be tailored to the severity of the disease. We reanalysed data from the seven largest published studies that have attempted to predict prognosis of cancer patients on the basis of DNA microarray analysis. The standard strategy is to identify a molecular signature (ie, the subset of genes most differentially expressed in patients with different outcomes) in a training set of patients and to estimate the proportion of misclassifications with this signature on an independent validation set of patients. We expanded this strategy (based on unique training and validation sets) by using multiple random sets, to study the stability of the molecular signature and the proportion of misclassifications. The list of genes identified as predictors of prognosis was highly unstable; molecular signatures strongly depended on the selection of patients in the training sets. For all but one study, the proportion misclassified decreased as the number of patients in the training set increased. Because of inadequate validation, our chosen studies published overoptimistic results compared with those from our own analyses. Five of the seven studies did not classify patients better than chance. The prognostic value of published microarray results in cancer studies should be considered with caution. We advocate the use of validation by repeated random sampling.

An integrity constraint for database systems containing embedded static neural networks

Static neural networks are used in some database systems to classify objects, but like traditional statistical classifiers they often misclassify. For some applications, it is necessary to bound the proportion of misclassified objects. This is clearly an integrity problem. We describe a new integrity constraint for database systems with embedded static neural networks, with which a database administrator can enforce a bound on the proportion of misclassifications in a class. The approach is based upon mapping probabilities generated by a probabilistic neural network to the likely percentage of misclassifications. © 2001 John Wiley & Sons, Inc.

Diagnosis of canine hip dysplasia and selection against this trait.

N, the Norberg-Olsson value, when regarded as the single predictor of FS in the total research subpopulation, leads to an unacceptably high proportion of misclassifications. These misclassifications are found to be much reduced in number on elimination of all animals suffering from bone deformities (BD) as perceivable on the radiographs. Discriminant analysis, applied to the findings in 915 Rottweilers, yields the combination of a suitable high N with the absence of perceivable bone deformities as a more acceptable predictor for FS, specially applicable in borderline cases.