Cohen's Kappa Measure Research Articles

Every crisis does matter: Comparing the databases of financial crisis events

AbstractIn this paper we examine the consistency in the timing of crisis events of the most prominent databases of banking and fiscal crises. We calculate Cohen's kappa measure, utilize signaling approach and panel logit models with random effects on selected early warning indicators and identify the most influential crisis observations using the Pregibon's delta‐beta influence statistics. Our results confirm that the degree of commonality across databases is indeed relatively high, especially if introducing a 1‐year lag. However, there is still a significant role played by a few influential observations. This problem is more pronounced in the banking crisis literature.

Deep learning and Boosted trees for injuries prediction in power infrastructure projects

Electrical injury impacts are substantial and massive. Investments in electricity will continue to increase, leading to construction project complexities, which undoubtedly contribute to injuries and associated effects. Machine learning (ML) algorithms are used to quantify and model causes of injuries; however, conventional ML techniques do not produce optimal results since they require careful engineering to transform data into suitable forms. In this study, we develop and compare state-of-the-art ML algorithms (deep learning and boosted trees) with other conventional methods to overcome this problem by analyzing incident cases obtained from a leading UK power infrastructure company. The predictive performance of the developed models was benchmarked using a statistical comparison between observations and predicted values. Results showed that the implementation of deep feedforward neural networks achieved the best predictions (accuracy = 0.967 and Cohen Kappa measure = 0.964). Furthermore, we conduct a sensitivity analysis to determine the effect of input parameters and data sizes on the modes’ behavior. The sensitivity analysis results showed strong generalization abilities of the deep learning and boosted tree models and their effectiveness for safety risks management.

Interobserver concordance in visual assessment of Ki67 immunohistochemistry in surgical excision specimens from patients with lymph node-negative breast cancer.

This study aimed to determine the interobserver concordance of two methods for proliferation assessment in breast cancer using Ki67 immunohistochemistry. Ki67 was independently assessed in randomly selected tumour samples from patients with lymph node-negative breast cancer using two different methods: either cell counting or visual estimation of hot spot areas. For hot spot cell counting, positive and negative cell numbers were recorded for total cell counts of 300-500, 500-800 and 800-1000 cells. Visual estimation involved allocation of a score from 1 to 5 using a visual scale to estimate percentage positivity. Interobserver agreement for hot spot counting was calculated using a two-way fixed effects intraclass correlation model, and by using Cohen's kappa measure for visual assessment. Prognostic concordance between the two methods was also calculated using Cohen's kappa. Samples from 96 patients were included in this analysis. Interobserver agreement for hot spot cell counting was excellent (> 0.75) across all three cell count ranges, with correlation coefficients of 0.88 (95% CI 0.84-0.92), 0.87 (95% CI 0.82-0.91) and 0.89 (95% CI 0.85-0.92), respectively. Interobserver agreement with visual estimation was greatest for hot spots compared with areas of intermediate or low proliferation, with kappa scores of 0.49, 0.42 and 0.40, respectively. Both assessment methods demonstrated excellent prognostic agreement. Interobserver and prognostic concordance in Ki67 immunohistochemistry assessments was high using either hot spot cell counting or visual estimation, further supporting the utility and reproducibility of these cost-efficient methods to assess proliferation.

Alcohol Advertising on Social Media: Examining the Content of Popular Alcohol Brands on Instagram

ABSTRACTBackground: There is considerable evidence that exposure to alcohol marketing increases the likelihood of adolescents initiating and engaging in alcohol consumption. There is a paucity of research, however, specifically examining industry generated alcohol marketing occurring on social media/networking platforms. Objective: The purpose of this investigation was to analyze the content of promotional advertisements by alcohol brands on Instagram. Methods: For a 30-day period, Instagram profiles of 15 distinct alcohol brands were examined. Pictorial posts/updates from each profile were screen captured and individually documented. Approximately 184 distinct posts constituted our final sample. The Content Appealing to Youth Index was independently employed by two raters to assess each post. For each characteristic, Cohen's Kappa measures, and associated 95% confidence intervals, were calculated. Descriptive statistics were performed. Results: Posts increased throughout the week and peaked on Thursday and Friday. The production value of the posts examined was generally high, frequently featuring color, texture, shine, contrast, faces, and action. Character appeals and use of youth-oriented genres were uncommon. Many of the posts used product appeals and physical benefits to consumption. The posts also emphasized the following rewarding appeal characteristics: positive emotional experiences, achievement, individuality, and camaraderie. The most commonly coded risk-related feature was inappropriate use. Conclusions/Importance: This investigation represents an initial attempt to provide insights into the content alcohol brands are including in their promotional materials on social networking sites.

Anomaly Detection Techniques Based on Kappa-Pruned Ensembles

Ensemble-based anomaly detection systems (ADSs), using Boolean combination, have been shown to reduce the false alarm rate over that of a single detector. However, the existing Boolean combination methods rely on an exponential number of combinations making them impractical, even for a small number of detectors. In this paper, we propose weighted pruning-based Boolean combination, an efficient approach for selecting and combining accurate and diverse anomaly detectors. It works in three phases. The first phase selects a subset of the available base diverse soft detectors by pruning all the redundant soft detectors based on a weighted version of Cohen's kappa measure of agreement. The second phase selects a subset of diverse and accurate crisp detectors from the base soft detectors (selected in Phase1) based on the unweighted kappa measure. The selected complementary crisp detectors are then combined in the final phase using Boolean combinations. The results on two large scale datasets show that the proposed weighted pruning approach is able to maintain and even improve the accuracy of existing Boolean combination techniques, while significantly reducing the combination time and the number of detectors selected for combination.

Evaluation of an automated pipeline for large-scale EEG spectral analysis: the National Sleep Research Resource

Study objectivesWe present an automated sleep electroencephalogram (EEG) spectral analysis pipeline that includes an automated artifact detection step, and we test the hypothesis that spectral power density estimates computed with this pipeline are comparable to those computed with a commercial method preceded by visual artifact detection by a sleep expert (standard approach). MethodsEEG data were analyzed from the C3-A2 lead in a sample of polysomnograms from 161 older women participants in a community-based cohort study. We calculated the sensitivity, specificity, accuracy, and Cohen's kappa measures from epoch-by-epoch comparisons of automated to visual-based artifact detection results; then we computed the average EEG spectral power densities in six commonly used EEG frequency bands and compared results from the two methods using correlation analysis and Bland–Altman plots. ResultsAssessment of automated artifact detection showed high specificity [96.8%–99.4% in non-rapid eye movement (NREM), 96.9%–99.1% in rapid eye movement (REM) sleep] but low sensitivity (26.7%–38.1% in NREM, 9.1–27.4% in REM sleep). However, large artifacts (total power > 99th percentile) were removed with sensitivity up to 87.7% in NREM and 90.9% in REM, with specificities of 96.9% and 96.6%, respectively. Mean power densities computed with the two approaches for all EEG frequency bands showed very high correlation (≥0.99). The automated pipeline allowed for a 100-fold reduction in analysis time with regard to the standard approach. ConclusionDespite low sensitivity for artifact rejection, the automated pipeline generated results comparable to those obtained with a standard method that included manual artifact detection. Automated pipelines can enable practical analyses of recordings from thousands of individuals, allowing for use in genetics and epidemiological research requiring large samples.

Detecting left ventricular impaired relaxation in cardiac MRI using moving mesh correspondences

Anatomical cine cardiovascular magnetic resonance (CMR) imaging is widely used to assess the systolic cardiac function because of its high soft tissue contrast. Assessment of diastolic LV function has not regularly been performed due the complex and time consuming procedures. This study presents a semi-automated assessment of the left ventricular (LV) diastolic function using anatomical short-axis cine CMR images. The proposed method is based on three main steps: (1) non-rigid registration, which yields a sequence of endocardial boundary points over the cardiac cycle based on a user-provided contour on the first frame; (2) LV volume and filling rate computations over the cardiac cycle; and (3) automated detection of the peak values of early (E) and late ventricular (A) filling waves. In 47 patients cine CMR imaging and Doppler-echocardiographic imaging were performed. CMR measurements of peak values of the E and A waves as well as the deceleration time were compared with the corresponding values obtained in Doppler-Echocardiography. For the E/A ratio the proposed algorithm for CMR yielded a Cohen's kappa measure of 0.70 and a Gwet's AC1 coefficient of 0.70. Conclusion: Semi-automated assessment of the left ventricular (LV) diastolic function using anatomical short-axis cine CMR images provides mitral inflow measurements comparable to Doppler-Echocardiography.

A Machine Learning Framework for Detecting Landslides on Earthen Levees Using Spaceborne SAR Imagery

Earthen levees have a significant role in protecting large areas of inhabited and cultivated land in the United States from flooding. Failure of the levees can result in loss of life and property. Slough slides are among the problems which can lead to complete levee failure during a high water event. In this paper, we develop a method to detect such slides using X-band synthetic aperture radar (SAR) data. Our proposed methodology includes: radiometric normalization of the TerraSAR image using high-resolution digital elevation map (DEM) data; extraction of features including backscatter and texture features from the levee; a feature selection method based on minimum redundancy maximum relevance (mRMR); and training a support vector machine (SVM) classifier and testing on the area of interest. To validate the proposed methodology, ground-truth data are collected from slides and healthy areas of the levee. The study area is part of the levee system along the lower Mississippi River in the United States. The output classes are healthy and slide areas of the levee. The results show the average classification accuracies of approximately 0.92 and Cohen's kappa measures of 0.85 for both healthy and slide pixels using ten optimal features selected by mRMR with a sigmoid SVM. A comparison of the SVM performance to the maximum likelihood (ML) and back propagation neural network (BPNN) shows that the average accuracy of the SVM is superior to that of the BPNN and ML classifiers.

Unraveling the two entities of endometrioid ovarian cancer: A single center clinical experience

ObjectiveDue to the increasing prevalence of the benign condition, ovarian carcinoma arising from endometriosis is emerging as a relevant clinical entity with an unclear biological signature. We have investigated clinical and histologic features of endometriosis-associated endometrioid ovarian cancer using an institutional retrospective database. MethodsPatients diagnosed with endometrioid ovarian cancer at our institution were divided into two groups according to the fulfillment or not of Sampson's and Scott's criteria for the detection of endometriosis-associated ovarian cancer. Clinical and histological data were reported and compared. Survival analysis was obtained using the log-rank test in an unadjusted Kaplan–Meier method. Multivariate analysis was performed using the Cox proportional hazards regression model to establish independent factors associated with endometriosis-associated endometrioid ovarian cancer and to identify predictors of survival. The degree of concordance was evaluated by Cohen's Kappa measures. ResultsPatients with endometriosis-associated endometrioid ovarian cancer were significantly younger, had a lower disease stage (62% vs 23%; p=0.003), a less prevalent high grade tumor (38% vs 82%; p=0.002) and a higher prevalence of squamous and mucinous metaplasia. The rate of endometrial cancer diagnosis was significantly higher in women with endometriosis-associated endometrioid ovarian cancer (33%) than in other patients (11%) (p=0.04) with a 92% concordance between ovarian and endometrial histologic tumor grade. A significant difference in survival rate could not be demonstrated between patients with or without endometriosis. ConclusionsThe analysis of a retrospective endometrioid ovarian cancer database may allow to suggest a 40 molecular, morphological and clinical parallelism between endometrial and endometrioid ovarian cancers.

Validity and Reliability of the Rapid Assessment for Adolescent Preventive Services Adolescent Health Risk Assessment

Seventy-five percent of adolescent morbidity and mortality is a result of risky behaviors. The Rapid Assessment for Adolescent Preventive Services (RAAPS) is a 21-question adolescent risk screening tool developed to identify the behaviors contributing most to adolescent morbidity, mortality, and social problems. The aim of this study was to evaluate the validity and reliability of the RAAPS. This study used psychometric methods to establish face-, content-, and criterion-related validity and inter-rater and equivalence reliability. Focus groups with adolescents (n = 21) and health care professionals (n = 7) were facilitated to establish face validity. Adolescent expert review (n = 10) was gathered to establish content validity and reliability. A retrospective chart audit of adolescents (n = 263) who completed both the RAAPS and the Guidelines for Adolescent Preventive Services risk questionnaire was conducted to assess criterion-related validity and equivalence between paired question responses using Cohen kappa measure, percent agreement, and Fisher exact test. Face validity was established by consensus of focus group. Adolescent expert content validity index scores ranged from 0.825 to 1.0, and inter-rater content agreement ranged from 0.9 to 1.0. Cohen kappa ranged from 0.44 to 0.99; percent agreement ranged from 0.71 to .99; Fisher exact test resulted in all p > 0.05 establishing criterion-related validity and equivalence. Validity and reliability of the RAAPS as a measure of adolescent risk behaviors was established. Results indicate RAAPS is an acceptable screening tool in identifying adolescent risk behaviors, contributing most to morbidity, mortality, and social problems.

Comparison of the Flotac-400 Dual Technique and the Formalin-Ether Concentration Technique for Diagnosis of Human Intestinal Protozoon Infection

There is a need for accurate diagnosis of intestinal parasite infections in humans, but currently available copromicroscopic techniques have shortcomings, such as low sensitivity. We compared the diagnostic accuracy of a further modified version of the recently developed Flotac technique with that of the widely used formalin-ether concentration technique (FECT) for the detection of intestinal protozoa in human stool samples. Formaldehyde-preserved stool samples from 108 individuals in Côte d'Ivoire were subjected to the Flotac-400 dual technique, using two different flotation solutions (FS), and to the FECT. Stool samples were examined according to computer-generated random lists by an experienced laboratory technician blinded for the results of each method. Both methods detected the same eight intestinal protozoon species. While the Flotac-400 dual technique (results from both FS combined) found higher prevalences of Entamoeba coli (77.8% versus 71.3%, P < 0.001), Blastocystis hominis (20.4% versus 13.0%, P = 0.458), and Giardia intestinalis (8.3% versus 6.5%, P < 0.001), the FECT detected higher prevalences of Entamoeba histolytica/Entamoeba dispar (27.8% versus 20.4%, P = 0.019) and four species of nonpathogenic intestinal protozoa. The diagnostic agreement between the two methods differed considerably depending on the intestinal protozoon investigated (Cohen's kappa measures; range, 0.01 to 0.46). Our study confirmed that the Flotac-400 dual technique can be utilized for the diagnosis of intestinal protozoon infections in humans. Since Flotac is a sensitive technique for the detection of soil-transmitted helminths and Schistosoma mansoni, it might become a viable copromicroscopic technique for the concurrent diagnosis of helminths and intestinal protozoon infections.

Correlation of Endoscopic and Histological Grading in Acute Graft-Versus-Host Disease (GVHD) after Bone Marrow Transplantation

Purpose: Graft-versus-host disease (GVHD) of the gastrointestinal tract is a major cause of morbidity and mortality after allogeneic bone marrow or stem cell transplantation. Whether macroscopic endoscopic grading predicts the histological staging of GVHD remains controversial. We reviewed the grading of endoscopic images in patients who were reported to have definite histological diagnosis of GVHD. This early endoscopic diagnosis could direct us to start specific therapy earlier and avoid the complication of endoscopic biopsies in severely thrombocytopenic patients. Methods: 41 patients, who had clinical and definite histological diagnosis of acute GVHD, had undergone endoscopic biopsies from gastric (34), duodenum (7) and Sigmoid (32). Endoscopic images of these patients were reviewed retrospectively since 2003 by 2 expert endoscopists. The endoscopic appearances of stomach, duodenum and Sigmoid were staged in four grades, according to the severity of macroscopic pictures. Correlations of four endoscopic grading (Brand RE et al, 1998) with four histological grading (McDonald and Sale, 1984) were compared, to determine the diagnostic accuracy in assessing the severity of GVHD. Statistical analysis of relationship was measured by Cohen's Kappa measure of agreement. Results: As the measure of agreement, grading between two techniques, endoscopic and histological, the Gastric measure was 0.579, moderate agreement (P < 0.0001) and sigmoid was 0.219, fair agreement (P < 0.010). Duodenal biopsies numbers were too small to get a significant relationship. Conclusion: Quite accurate early diagnosis and grading of acute GVHD could be obtained by the endoscopic images especially from the gastric and sigmoid, but histological evaluation is mandatory to exclude other causes of inflammation which give similar endoscopic images before committing for the final diagnosis of GVHD.

Reproducibility of grading scales of pelvic muscle strength: standard versus tampon scale.

The purpose of this study was to demonstrate the reproducibility of an objective scale utilizing a tampon to assess pelvic muscle strength and to compare it to a digital evaluation of muscle strength [1]. Thirty women presenting to the office were enrolled in the study. Their pelvic floor muscle strength was separately assessed by both examiners using both scales. The reproducibility of the scales was then analyzed by Cohen's kappa measure of interrater agreement. The tampon scale was found to have a measure of agreement of 0.76 (excellent interrater agreement if >0.75). In contrast, the digital scale was found to have a measure of agreement of 0.68 (fair agreement if 0.40-0.75). Therefore, the tampon scale is an easy and reproducible approach for assessing pelvic floor muscle strength.