Statistical Measures Of Agreement Research Articles

Alternative skeletal muscle index for sarcopenia diagnosis in elderly patients with type 2 diabetes mellitus: A pilot study.

To determine an alternative skeletal muscle index (a-SMI), easy diagnosis of sarcopenia in elderly patients with type 2 diabetes mellitus (T2DM). This cross-sectional study included 223 inpatients with T2DM (100 males, age range 60-89; 123 females, age range 60-87). Screening for grip strength and gait speed, measuring SMI by dual-energy X-ray absorptiometry (d-SMI) for sarcopenia diagnosis, according to the Asian Working Group for Sarcopenia (AWGS) 2019 consensus. The a-SMI was established by binary logistic regression analysis with positive screening population. To assess the conformance of the new diagnostic approach with the AWGS 2019. Sarcopenia was present in 36.3% of the study population. 59 had normal d-SMI and 81 had low d-SMI in screening patients with probable sarcopenia. In univariate analyses for all positive screening population, body mass index (BMI), 25-hydroxyvitamin D (25 - (OH) VitD), high density lipoprotein cholesterol (HDL-C), hypertension (HTN), and gender were correlates of d-SMI. Binary logistic regression analysis revealed that male (B = 2.463, 95%CI: 3.640 ~ 37.883, p = 0.000), HTN (B = 1.404, 95%CI: 1.599 ~ 10.371, p = 0.003), BMI (B = -0.344, 95%CI: 0.598 ~ 0.839, p = 0.000), 25-(OH) VitD (B = -0.058, 95%CI: 0.907 ~ 0.982, p = 0.004) were independent factors for d-SMI detection. Based on the extracted four correlates, the a-SMI was determined. The area under receiver operating characteristic (ROC) curve was 0.842, sensitivity and specificity for the new diagnostic approach were 84.0% and 84.5%. In a statistical measure of agreement between the AWGS 2019 and the new diagnostic approach, the kappa coefficient was 0.669 (p < 0.001). The a-SMI - based on gender, obesity status, 25-(OH) VitD, and HTN history - can be used in the absence of the d-SMI to supplement the algorithm for sarcopenia diagnosis in elderly patients with T2DM.

Practising prioritisation: exploring variation in applying a clinical pharmacy risk stratification tool

ObjectivesTo evaluate the use of a risk stratification tool and explore the contributing factors to variation in practice by clinical pharmacists.MethodsThe quantitative phase was a prospective evaluation of adherence to...

Left ventricular volumes on positron emission tomography: validation against cardiovascular magnetic resonance

Abstract Introduction Cardiovascular magnetic resonance (CMR) is the gold standard for determination of left ventricular volumes (LVV), including end diastolic volume (LVEDV) and end systolic volume (LVESV). With incorporation of ECG gating, the quantitative analysis of ventricular function can be achieved in Positron Emission Tomography (PET) studies. However, the accuracy of LVV quantification using PET has not been shown in previous studies. Purpose We aimed to assess the inter-study variability of LVEDV and LVESV between CMR and PET. Methods Consecutive patients who underwent both PET and CMR within 1 year were identified from prospective institutional registries. LVV on PET was compared against CMR as a reference standard using several statistical measures of agreement. Results The study population consisted of 233 patients (mean age 60.8±13.6 years, 64.8% male). The median (IQR) duration between CMR and PET was 41 (11–96) days. The median (IQR) LVEDV values were 144 (107–196) on PET and 178 (138–236) on CMR (mean difference 36.09±2.36, p&amp;lt;0.001). As shown in Figure 1, there was a good correlation (Spearman rho=0.88, p&amp;lt;0.001; Intraclass Correlation Coefficient 0.88, 95% CI 0.84–0.90, P&amp;lt;0.001) with moderate limits of agreement (95% limits of agreement −108.17–35.99). A total of 0% and 3.9% of the cohort had LVEDV &amp;gt;150 ml/m2 by CMR and PET respectively. Overall results were similar with LVESV values and similar in subgroup analysis of patients with a low EF (&amp;lt;40%), a coronary artery disease scar, LV hypertrophy and of patients with defibrillators. Conclusion Our analysis has shown a good correlation of LVV between PET and CMR, indicating PET derived LVV can accurately be used in the diagnosis and management of patients. Funding Acknowledgement Type of funding sources: None.

Left ventricular mass on positron emission tomography: Validation against cardiovascular magnetic resonance

BackgroundLeft ventricular hypertrophy (LVH) is an important clinical finding that is independently associated with mortality and cardiovascular events. We aimed to assess the interstudy variability of LV mass quantitation between PET and CMR. MethodsPatients who underwent both PET and CMR within 1 year were identified from prospective institutional registries. LV mass on PET was compared against LV mass on CMR using several statistical measures of agreement. ResultsA total of 105 patients (mean age 60 ± 14 years, 67.6% male) were included. The median (interquartile range, IQR) duration between CMR and PET was 47 (11-154) days. The median (IQR) LV mass values were 168.0 g (126.0-202.0) on CMR and 174.0 g (150.0-212.0) with PET (absolute mean difference 29.42 ± 25.3). There was a good correlation (Spearman ρ = 0.81, P < 0.001; Intraclass Correlation Coefficient 0.78, 95% CI 0.70–0.85, P < 0.001) with moderate limits of agreement (95% limits of agreement − 63.78 to 83.7.) Results were consistent, albeit with moderate correlation, in subgroups of patients with LVH, in patients with myocardial infarction, in patients with LV ejection fraction < 50%, and those with limited image quality. LV mass on PET tended to be underestimated at high values compared to CMR. ConclusionWe demonstrate good correlation and reproducibility of LV mass quantitation by PET against the reference standard of CMR across a wide range of normal and diseased hearts with a tendency of PET to underestimate mass at higher mass values.

Occupational Therapists' Perspective on the International Classification of Functioning, Disability and Health Core Sets (ICF-CS) for Schizophrenia.

The International Classification of Functioning, Disability and Health (ICF) Core Sets (CSs) for schizophrenia are sets of ICF categories that are relevant specifically to the functioning and disability of people with schizophrenia. To identify the problems occupational therapists commonly encounter when treating people with schizophrenia and to validate the ICF-CSs for schizophrenia from their perspective. Three-round Delphi study using online surveys distributed to occupational therapists worldwide. Participants were asked which problems with functioning they considered most relevant when treating people with schizophrenia. Occupational therapists experienced in the treatment of people with schizophrenia. Responses were linked to the ICF categories by two trained health professionals, and a statistical measure of agreement and κ coefficient were calculated. Ninety-two occupational therapists from 29 countries in all six World Health Organization regions named 2,527 meaningful concepts. After the linking process, 121 ICF categories and 31 Personal Factors were presented to the expert panel, who reached consensus (agreement of ≥75%) on 97 ICF categories and 27 Personal Factors. Consensus was reached on the 25 categories in the Brief ICF-CS for schizophrenia and 89 of the 97 categories in the Comprehensive ICF-CS for schizophrenia. The ICF Core Sets for schizophrenia were validated from the perspective of occupational therapists and are potentially useful tools for clinical practice because they cover a wide variety of problems that occupational therapists deal with in interventions with people with schizophrenia. The ICF-CSs for schizophrenia are useful guides for describing and classifying functioning, disability, and health to aid occupational therapy intervention with people with schizophrenia. Occupational therapists are essential in the rehabilitation of this client population, and their perspective has contributed to the development and enrichment of the ICF-CSs for schizophrenia.

Bayesian network-based methodology for selecting a cost-effective sewer asset management model.

This paper presents a methodology based on Bayesian networks (BN) to prioritize and select the minimal number of variables that allows predicting the structural condition of sewer assets to support the strategies in proactive management. The integration of BN models, statistical measures of agreement (Cohen's Kappa coefficient) and a statistical test (Wilcoxon test) were useful for a robust and straightforward selection of a minimum number of variables (qualitative and quantitative) that ensure a suitable prediction level of the structural conditions of sewer pipes. According to the application of the methodology to a specific case study (Bogotás sewer network, Colombia), it found that with only two variables (age and diameter) the model could achieve the same capacity of prediction (Cohen's Kappa coefficient = 0.43) as a model considering several variables. Furthermore, the methodology allows finding the calibration and validation percentage subsets that best fit (80% for calibration and 20% for validation data in the case study) in the model to increase the capacity of prediction with low variations. Furthermore, it found that a model, considering only pipes in critical and excellent conditions, increases the capacity of successful predictions (Cohen's Kappa coefficient from 0.2 to 0.43) for the proposed case study.

Half-body MRI volumetry of abdominal adipose tissue in patients with obesity

BackgroundThe purpose of this study was to determine to what extent the whole volumes of abdominal subcutaneous (ASAT) and visceral adipose tissue (VAT) of patients with obesity can be predicted by using data of one body half only. Such a workaround has already been reported for dual-energy x-ray absorption (DEXA) scans and becomes feasible whenever the field of view of an imaging technique is not large enough.MethodsFull-body abdominal MRI data of 26 patients from an obesity treatment center (13 females and 13 males, BMI range 30.8–41.2 kg/m2, 32.6–61.5 years old) were used as reference (REF). MRI was performed with IRB approval on a clinical 1.5 T MRI (Achieva dStream, Philips Healthcare, Best, Netherlands). Segmentation of adipose tissue was performed with a custom-made Matlab software tool. Statistical measures of agreement were the coefficient of determination R2 of a linear fit.ResultsMean ASATREF was 12,976 (7812–24,161) cm3 and mean VATREF was 4068 (1137–7518) cm3. Mean half-body volumes relative to the whole-body values were 50.8% (48.2–53.7%) for ASATL and 49.2% (46.3–51.8%) for ASATR. Corresponding volume fractions were 56.4% (51.4–65.9%) for VATL and 43.6% (34.1–48.6%) for VATR. Correlations of ASATREF with ASATL as well as with ASATR were both excellent (R2 > 0.99, p < 0.01). Corresponding correlations of VATREF were marginally lower (R2 = 0.98 for VATL, p < 0.01, and R2 = 0.97 for VATR, p < 0.01).ConclusionsIn conclusion, abdominal fat volumes can be reliably assessed by half-body MRI data, in particular the subcutaneous fat compartment.

Español

Introduction: Dynamic systems theory provides quantitative measures of evolution of systems by building attractors. Spatial occupation measures of cardiac attractors in fractal Box Counting space differentiated normality and chronic disease from acute illness. Objective: To apply the developed methodology to evaluate mathematically the cardiac status of Holter with different pathologies, confirming the applicability of this methodology for the detection of acute dynamic by statistical measures of agreement regarding the Gold Standard. Methodology: 170 Holter, including normal, chronic and in acute states were evaluated. Simulations were constructed the entire dynamic based on the number of beats and the minimum and maximum frequencies every hour for 21 hours, to build attractors in the phase space. The fractal dimension of attractors is calculated, evaluating the spatial occupation in the Box Counting space, establishing which corresponds to normal setting and acute disease in accordance with previous mathematical results. Mathematical diagnosis was compared with conventional diagnostic Holter, taken as the Gold Standard, setting sensitivity, specificity, positive and negative predictive value and Kappa coefficient. Results: The fractal dimension failed to show quantitative differences while the methodology detected in all cases normal dynamics and acute state independently of the disease, achieving sensitivity, specificity, positive and negative predictive value of 100% and a Kappa 1. Conclusions: the ability of the physical-mathematical methodology to detect acute dynamic regardless of the associated pathology was confirmed, as well as an acausal self-organization of the system dynamics, which allows for assessment of clinical applicability measures.

Agreement between different sleep states and behaviour indicators in dairy cows

Conclusive data regarding behavioural indicators of different sleep states in adult dairy cows are lacking, i.e. agreement between behavioural indicators of sleep and corresponding electrophysiological measures. Behavioural estimates for quantifying total sleep time in calves have been developed, so this study examined whether these behavioural estimates also apply for adult cows.Behaviour observations and electrophysiological readings were recorded for a total of 13 cows during one recording session per cow lasting on average 4h 22min. Recording started when the cow was fully awake and finished when at least one probable sleep bout had been recorded. The behavioural estimates used in the study were: ‘lying with head lifted and still’ for non-rapid eye movement (NREM) sleep, ‘lying with head resting’ for rapid eye movement (REM) sleep and ‘lying with head lifted and moving’ for awakeness. As statistical measures of agreement between behavioural estimates and electrophysiological status (both recorded at 30s intervals), Cohen's kappa as well as sensitivity and specificity measures were calculated. Additionally, misclassifications were evaluated to better understand agreement between the behaviour and electrophysiological sleep classification. Since interval length might have affected the agreement, the output data were also aggregated into 60, 90 and 120s intervals and analysed using Wilcoxon sign-rank test to determine the most appropriate interval length.It was found that the behavioural estimates for assessing total sleep time in calves could not be applied to adult cows as they markedly overestimated NREM and REM sleep time. Behavioural estimates for NREM and REM sleep time were on average 124±17 and 14±4min per cow, respectively, while the electrophysiological estimate for NREM and REM was on average 20±5 and 10±3min per cow, respectively. Using the behavioural estimate ‘lying with head resting’, REM sleep could be identified with moderate precision, but this indicator alone likely underestimates total duration of REM sleep. Behavioural estimates for NREM sleep showed high sensitivity (81%) but low specificity (6%) while the behavioural estimates for REM sleep showed high sensitivity (70%) and moderate specificity (41%). For both categories, both sensitivity and specificity increased with increasing measurement interval length. Drowsing as identified from electrophysiological data was present mainly when the cows were lying with head lifted and still, whereas awakeness was present when the cows were lying with head lifted and moving.

Impact of Specific Training in Detecting Osteoporotic Vertebral Fractures on Routine Chest Radiographs

Vertebral fractures are the most common complication of osteoporosis. Routine chest radiographs are a potential screening method, but a significant under-reporting has been described previously. The purpose of this study was to evaluate the effect of a specific training on the detection rate of vertebral fractures of a radiology resident. 936 routine lateral chest radiographs of postmenopausal women were evaluated by a radiology resident (R1) during clinical routine. After the evaluation of 470 radiographs (pre-training group), R1 underwent a specific training based on the teaching initiative of the IOF/ESSR. Afterwards the remaining 466 radiographs were evaluated (post-training group). As a standard of reference, all radiographs were reviewed by two radiologists in consensus (R2 + 3). A semi-quantitative method (spinal fracture index, SFI) was used to assess vertebral fractures. Kappa-values as statistical measure of agreement between R1 and R2 + 3 for the detection of vertebral fractures (Genant Severity > 0) increased from κ = 0.311 (95 % CI: 0.217 - 0.405; "fair agreement") in the pre-training group to κ = 0,882 (95 % CI: 0,835 - 0,929; "almost perfect agreement") in the post-training group. Similar results were observed for severe fractures (Genant Severity > 1). Especially fractures with Genant Severity 1 were not detected by R1 before training. A brief training is essential to increase the awareness of radiologists to correctly report osteoporotic vertebral fractures and may help to initiate appropriate therapy in patients with vertebral fractures.

Critical discussion of evaluation parameters for inter-observer variability in target definition for radiation therapy

Inter-observer studies represent a valid method for the evaluation of target definition uncertainties and contouring guidelines. However, data from the literature do not yet give clear guidelines for reporting contouring variability. Thus, the purpose of this work was to compare and discuss various methods to determine variability on the basis of clinical cases and a literature review. In this study, 7 prostate and 8 lung cases were contoured on CT images by 8 experienced observers. Analysis of variability included descriptive statistics, calculation of overlap measures, and statistical measures of agreement. Cross tables with ratios and correlations were established for overlap parameters. It was shown that the minimal set of parameters to be reported should include at least one of three volume overlap measures (i.e., generalized conformity index, Jaccard coefficient, or conformation number). High correlation between these parameters and scatter of the results was observed. A combination of descriptive statistics, overlap measure, and statistical measure of agreement or reliability analysis is required to fully report the interrater variability in delineation.

The Illusion Of Quality: Controlling Subjective Inspection

An inspection process that depends on subjective evaluation may give a misleading image of product quality. An experiment was conducted at an electronics company to measure the accuracy of inspectors judgements on the acceptability/rejectability of electrical connections. Using a statistical measure of agreement, it was found that inspectors were accepting a high proportion of electrical connections that should have been rejected. After identifying management deficiencies, recommendations are given for improving the accuracy of the inspection process.

The Comprehensive ICF core sets for spinal cord injury from the perspective of physical therapists: a worldwide validation study using the Delphi technique

To validate the International Classification of Functioning, Disability and Health (ICF) Core Set for individuals with spinal cord injury (SCI) in the early post-acute and long-term context from the perspective of physical therapists. International. Physical therapists experienced in the treatment of SCI were asked about problems, resources and aspects of the environment treated by them, in a three-round electronic mail survey using the Delphi technique. Responses were linked to the ICF by two researches; κ-coefficient was calculated as statistical measure of agreement. In all, 81 experts from 27 countries named 3694 concepts. They were linked to 187 ICF categories for the early post-acute context. Three ICF categories from the component body function, five ICF categories from the component body structures and two ICF categories from the component activities and participation were not represented in the ICF Core Set for the early post-acute context. In all, 207 ICF categories were linked for the long-term context. Four ICF categories from the component body function, five ICF categories from the component body structures and two ICF categories from the component activities and participation were not represented in the ICF Core Set for the long-term context. Physical therapists addressed a vast variety of problems that they take care of in their interventions in patients with SCI. The Comprehensive ICF Core Sets covered a high percentage of these problems. Further research is necessary on several responses not covered in the ICF.

Relationships between statistical measures of agreement: sensitivity, specificity and kappa

Cohen's kappa coefficient is presently a standard tool for the analysis of agreement on a binary outcome between two tests. In view of the ubiquity of the use of sensitivity, specificity, raw agreement and kappa in clinical studies, clearly it is advantageous to have a useful analytic relation connecting these measures of agreement. We elaborate on previous work, comment on other results appearing in the literature and discuss analytic formulas relevant to various problems connecting specificity, sensitivity and kappa. For selected values of kappa that range from good to excellent, a graph of the curves representing minimal pairs of sensitivity and specificity is provided. The analytic formulas and graph could be potentially useful to clinicians and biostatisticians in better interpreting the outcomes of an alternative diagnostic test whenever the measures sensitivity, specificity and kappa are employed together.

Efficacy of Contrast-Enhanced CT in Assessing the Endometrium

The purpose of our study was to determine the efficacy of contrast-enhanced CT in detecting a thickened endometrium. We used transvaginal sonography as the reference standard. Between March 2005 and January 2007, data from 259 patients (mean age, 47 years; age range, 18-90 years) who underwent transvaginal sonography and contrast-enhanced CT of the pelvis were analyzed retrospectively. The endometrium was quantitatively measured in millimeters on sonography. On CT it was qualitatively categorized as normal, thickened, indeterminate, or not visualized and compared with the sonography findings and original radiology reports. When the endometrium was indeterminate (thickened or triangular in shape on axial images), sagittal reconstructions were performed for final categorization. Two reviewers evaluated the CT scans and sonograms jointly with differences resolved by consensus. Kappa, Wilcoxon's rank sum test, and intraclass correlation statistics were derived. The overall sensitivity and specificity of CT in detecting the thickened endometrium was 53.1% and 93.5%, respectively, relative to transvaginal sonography. The positive and negative predictive values were 66.7% and 89.1%, respectively. Kappa, the statistical measure of agreement between CT and sonography data, was 0.5049. All cases of a triangular endometrium were normal in size on sagittal reconstruction images. Routine pelvic CT correctly identifies a normal endometrium in most patients. Sagittal reconstruction images are helpful to further evaluate the endometrium on CT in cases with a prominent or triangular endometrium because these are often related to uterine version. CT is relatively insensitive in detecting the thickened endometrium but better able to identify gross rather than subtle thickening, which must be further characterized by transvaginal sonography.

Evaluating the present‐day simulation of clouds, precipitation, and radiation in climate models

This paper describes a set of metrics for evaluating the simulation of clouds, radiation, and precipitation in the present‐day climate. As with the skill scores used to measure the accuracy of short‐term weather forecasts, these metrics are low‐order statistical measures of agreement with relevant, well‐observed physical quantities. The metrics encompass five statistical summaries computed for five physical quantities (longwave, shortwave, and net cloud radiative effect, projected cloud fraction, and surface precipitation rate) over the global climatological annual cycle. Agreement is measured against two independent observational data sets. The metrics are computed for the models that participated in the Coupled Model Intercomparison Project phase 3, which formed the basis for the Fourth Assessment of the IPCC. Model skill does not depend strongly on the data set used for verification, indicating that observational uncertainty does not limit the ability to assess model simulations of these fields. No individual model excels in all scores though the “IPCC mean model,” constructed by averaging the fields produced by all the CMIP models, performs particularly well across the board. This skill is due primarily to the individual model errors being distributed on both sides of the observations, and to a lesser degree to the models having greater skill at simulating large‐scale features than those near the grid scale. No measure of model skill considered here is a good predictor of the strength of cloud feedbacks under climate change. The model climatologies, observational data sets, and metric scores are available on‐line.

Critical points for certain statistical measures of agreement

This paper further explores the analytic connections between three commonly used statistical measures of agreement for 2 × 2 contingency tables: sensitivity, specificity, and the kappa coefficient, which are often employed in evaluating diagnostic tests. In particular, for a given fixed kappa the corresponding locus of minimum values of sensitivity and specificity is rigorously derived.

The kappa statistic as a function of sensitivity and specificity

This paper explores analytically the connections between three commonly used statistical measures of agreement: sensitivity, specificity, and Cohen's (unweighed) kappa, which are employed often in clinical studies. In particular, we show that if both sensitivity and specificity are below 7/8, then a kappa of 3/4 or greater (which is considered to indicate good to excellent agreement) can never be achieved even for raw agreement approaching 100%. In addition we derive some of the known properties of kappa for two-by-two tables which apparently are not readily available in the literature.

Consensus Methods: A Bridge Between Clinical Reasoning and Clinical Research?

Evidence-Based Practice does head the "hierarchy of evidence" upon which developments in clinical practice should be based. There are, however, situations where evidence is either unavailable, unclear, or results between studies are at variance. Consensus is a reliable contingency, and approaches to reaching consensus have acceptable construct validity (Nominal Group Technique, Delphi, and Consensus Development Conference). Consensus is reached when: (i) the method of investigation tightly controls communication to reduce the obscuring "noise" of divergent discussion; (ii) statistical measures of agreement or dissent screen out the bias that would otherwise be produced by the dictate of vociferous minorities or coalitions that may represent vested interests; (iii) all participants contribute equally to the product of the investigation.

Telephone vs Face-to-Face Interviews for Quantitative Food Frequency Assessment

Objective To develop and test a quantitative food frequency method for administration by telephone. Design A comparison study of telephone and face-to-face interviews was conducted among a representative sample of the five major ethnic groups in Oahu, Hawaii. Two interviews were administered 4 to 6 months apart by trained interviewers using identical questionnaires and color photographs of food items showing three different portion sizes. The order of the interviewing methods was randomly assigned. The questionnaire included 115 food items selected to estimate 80% or more of usual dietary intakes. Frequencies and quantities of each item consumed during the past year were obtained. Subjects/setting Subjects were recruited from the Health Surveillance Program of the Hawaii State Department of Health and consisted of 167 men and 158 women, aged 45 to 74 years, who provided a telephone number. Eighty percent of the face-to-face interviews were conducted in the subjects’ homes and 20% were conducted at the workplace or the University of Hawaii Cancer Research Center, if requested. Statistical analyses The paired t test was used to compare the mean daily intakes obtained by the telephone and face-to-face methods. Agreement was measured by the intraclass correlation coefficient (ICC), Pearson correlation coefficient, and weighted κ statistic. Results The means of energy and each nutrient were slightly higher in the first interview than the second, regardless of the interviewing method. Because of close correspondence among all 3 statistical measures of agreement, only the ICCs are reported. The ICCs ranged from .61 for protein and vitamin A to .69 for dietary cholesterol among men, and from .61 for vitamin C to .74 for saturated fat among women. Agreement was not significantly affected by age, gender, ethnicity, order of interview, or educational level. Applications Telephone interviews to obtain quantitative food frequencies are cost-efficient methods for estimating usual dietary intakes among persons in widely scattered geographic areas. Photographs of the foods in 3 portion sizes mailed in advance help the respondents estimate amounts eaten. J Am Diet Assoc. 1998;98:44-48.