Summary Measure Of Performance Research Articles

Measuring and improving performance of clinicians: an application of patient-based records

BackgoundEfforts to measure performance and identify its driving factors among clinicians are needed for building a high-quality clinician workforce. The availability of data is the most challenging thing. This paper presented a summary performance measure for clinicians and its application on examining factors that influence performance using routine patient-based records.MethodsPerfomance indicators and difficulty score were extracted from electronic medical records (EMRs). Difficulty adjustment and standardized processing were used to obtain indicators which were comparable between specialties. Principal component analysis (PCA) was used to estimate the summary performance measure. The performance measure was then used to examine the influence of person-job fit and burnout through a mediator effect model and cluster analysis.ResultsA valid sample of 404 clinicians were included in this study, and 244 of them had valid response in the questionnaire. PCA explained 79.37% of the total variance presented by the four adjusted performance indicators. Non-performance attributes and performance driving factors help distinguish different clusters of clinicians. Burnout mediates the relationship between person-job fit and performance in a specific group of clinicians (β = 0.120, p = 0.008).ConclusionsWe demonstrated the analytical steps to estimate clinicians’ performance and its practical application using EMRs. Our findings provide insight into personnel classified management. Such practice can be applied in countries where electronic medical record systems are relatively less developed to continuously improve the application of performance management.

Evaluation of Range Estimation Performance of FLIR with Field Requirements Criteria

Thermal imaging performance depends on many variables, ranging from the properties of the imaged object to atmospheric transmittance and system parameters. After clarification of the functional needs in system design or procurement, system parameters of the design that can meet these needs should be determined. Diagnosis/recognition from a distance is one of the foremost of these needs. The following briefly introduces the Forward Looking Infrared (FLIR) systems, followed by explanations for calculating the theoretical diagnostic range. After the theoretical information, sample systems are given, and high-performance FLIR systems are presented. To accurately analyze, measure and predict the performance of FLIR systems, a model should calculate summary performance measures of the system in the form of Minimum Resolvable Temperature (MRT) and Modulation Transfer Function (MTF) between a target and its background and estimate range for a given scenario electro-optical required for the performance evaluation of the system. The accuracy of these calculations will ultimately determine the accuracy of the model by which the performance of the FLIR system is evaluated.

A Critical Evaluation of Validation and Clinical Experience Studies in Non-Invasive Prenatal Testing for Trisomies 21, 18, and 13 and Monosomy X.

Non-invasive prenatal testing (NIPT) for trisomies 21, 18, 13 and monosomy X is widely utilized with massively parallel shotgun sequencing (MPSS), digital analysis of selected regions (DANSR), and single nucleotide polymorphism (SNP) analyses being the most widely reported methods. We searched the literature to find all NIPT clinical validation and clinical experience studies between January 2011 and January 2022. Meta-analyses were performed using bivariate random-effects and univariate regression models for estimating summary performance measures across studies. Bivariate meta-regression was performed to explore the influence of testing method and study design. Subgroup and sensitivity analyses evaluated factors that may have led to heterogeneity. Based on 55 validation studies, the detection rate (DR) was significantly higher for retrospective studies, while the false positive rate (FPR) was significantly lower for prospective studies. Comparing the performance of NIPT methods for trisomies 21, 18, and 13 combined, the SNP method had a higher DR and lower FPR than other methods, significantly so for MPSS, though not for DANSR. The performance of the different methods in the 84 clinical experience studies was consistent with validation studies. Clinical positive predictive values of all NIPT methods improved over the last decade. We conclude that all NIPT methods are highly effective for fetal aneuploidy screening, with performance differences across methodologies.

Gastric Juice-Based Genotypic Methods for Diagnosis of Helicobacter pylori Infection and Antibiotic Resistance Testing: A Systematic Review and Meta-analysis.

To evaluate the diagnostic efficacy of gastric juice-based genotypic methods for Helicobacter pylori detection and antibiotic resistance testing. We used electronic databases including Medline, Embase, Web of Science and the Cochrane Central Register of Controlled Trial for literature survey using keywords such as "gastric juice", "Helicobacter pylori" and their synonyms. The quality of the studies was assessed using QUADAS-2. Summary performance measures (sensitivity, specificity, positive predictive values, negative predictive values, diagnostic odds ratio, and area under the summary receiver operating characteristic curve) and HSROC curves were produced. In addition, fagan plots were applied to illustrate the relationship among the prior test probability, PLR/NLR, and posterior test probability. Our study cohort comprised eight studies with 1235 participants (617 participants of H. pylori infection and 618 participants of non-H. pylori infection). Pooled sensitivity and specificity with a corresponding 95% confidence interval (CI) of gastric juice-based genotypic methods reflected values of 94% (95%CI, 86% - 98%) and 98% (95%CI, 85% - 100%), respectively. The global sensitivity and specificity of clarithromycin resistance were 92% (95%CI, 85% - 96%) and 90% (95%CI, 80% - 95%), respectively. Gastric juice-based genotypic methods can be used for diagnostic prediction of H. pylori infection as well as clarithromycin resistance testing.

Serial order in perception, memory, and action.

This article asks whether serial order phenomena in perception, memory, and action are manifestations of a single underlying serial order process. The question is addressed empirically in two experiments that compare performance in whole report tasks that tap perception, serial recall tasks that tap memory, and copy typing tasks that tap action, using the same materials and participants. The data show similar effects across tasks that differ in magnitude, which is consistent with a single process operating under different constraints. The question is addressed theoretically by developing a Context Retrieval and Updating (CRU) theory of serial order, fitting it to the data from the two experiments, and generating predictions for 7 different summary measures of performance: list accuracy, serial position effects, transposition gradients, contiguity effects, error magnitudes, error types, and error ratios. Versions of the model that allowed sensitivity in perception and memory to decrease with serial position fit the data best and produced reasonably accurate predictions for everything but error ratios. Together, the theoretical and empirical results suggest a positive answer to the question: Serial order in perception, memory, and action may be governed by the same underlying mechanism. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

When Deploying Predictive Algorithms, Are Summary Performance Measures Sufficient?

When Deploying Predictive Algorithms, Are Summary Performance Measures Sufficient?

Multivariate analysis of covariance with potentially singular covariance matrices and non-normal responses

In applied research, it is often sensible to account for one or several covariates when testing for differences between multivariate means of several groups. However, the “classical” parametric multivariate analysis of covariance (MANCOVA) tests (e.g., Wilks’ Lambda) are based on quite restrictive assumptions (homoscedasticity and normality of the errors), which might be difficult to justify. Furthermore, existing potential remedies (e.g., heteroskedasticity-robust approaches) become inappropriate in cases where the covariance matrices are singular or close to singular. Nevertheless, such scenarios are frequently encountered in the life sciences and other fields, when, for example, in the context of standardized assessments, a summary performance measure as well as its corresponding subscales are analyzed. Moreover, computational issues may also lead to singular covariance structures. In the present manuscript, we consider a general MANCOVA model, allowing for potentially heteroskedastic and even singular covariance matrices as well as non-normal errors. We combine heteroskedasticity-consistent covariance matrix estimation methods with our proposed modified MANCOVA ANOVA-type statistic (MANCATS) and apply two different bootstrap approaches. We provide the proofs of the asymptotic validity of the respective testing procedures as well as the results from an extensive simulation study, which indicate that especially the parametric bootstrap version of the MANCATS outperforms its competitors in most small-sample scenarios, both in terms of type I error rates and power. These considerations are further illustrated and substantiated by examining real-life data from standardized achievement tests. Yet, for large sample sizes, Wald-type approaches are an attractive alternative option. Moreover, the choice between Wald-type and MANCATS-based tests might depend on the particular setting (e.g., the covariance structure, or the alternative under consideration).

Moving beyond summary scores: Decomposing free recall performance to understand episodic memory deficits in schizotypy.

Negative symptom schizophrenia and negative schizotypy are associated with deficits in episodic memory, which may reflect deficits in context processing. However, studies that rely on summary performance measures such as mean accuracy or latency are limited in the extent that they can examine processes underlying memory impairment. The present study decomposed free recall performance by examining serial position functions, first response probability, temporal contiguity effect, cumulative recall functions, and interresponse times in high-positive schizotypy, high-negative schizotypy, and control groups. The negative schizotypy group exhibited not only impaired overall free recall performance but also a pattern of deficits consistent with impaired context processing on the underlying measures. Specifically, the negative schizotypy group was less likely than the other groups to initiate recall with the first item in the list, suggesting impaired encoding or reinstatement of context, and also showed reduced temporal contiguity compared with the other groups, suggesting diminished temporal organization. The cumulative recall function indicated that the negative schizotypy group experienced disruptions in both the sampling and recovery stages of retrieval. Finally, the negative schizotypy group experienced greater slowing between the responses during retrieval, consistent with the finding of reduced temporal contiguity and indicating that it likely terminated memory search before the remaining groups. The positive schizotypy and control groups did not differ on any of the measures. The finding that context-processing deficits occur in both subclinical negative schizotypy and negative symptom schizophrenia suggests that they may represent core areas of impairment in the schizophrenia spectrum. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Use of risk assessment instruments to predict violence in forensic psychiatric hospitals: a systematic review and meta-analysis

Background and AimsViolent behaviour by forensic psychiatric inpatients is common. We aimed to systematically review the performance of structured risk assessment tools for violence in these settings. MethodsThe nine most commonly used violence risk assessment instruments used in psychiatric hospitals were examined. A systematic search of five databases (CINAHL, Embase, Global Health, PsycINFO and PubMed) was conducted to identify studies examining the predictive accuracy of these tools in forensic psychiatric inpatient settings. Risk assessment instruments were separated into those designed for imminent (within 24 hours) violence prediction and those designed for longer-term prediction. A range of accuracy measures and descriptive variables were extracted. A quality assessment was performed for each eligible study using the QUADAS-2. Summary performance measures (sensitivity, specificity, positive and negative predictive values, diagnostic odds ratio, and area under the curve value) and HSROC curves were produced. In addition, meta-regression analyses investigated study and sample effects on tool performance. ResultsFifty-two eligible publications were identified, of which 43 provided information on tool accuracy in the form of AUC statistics. These provided data on 78 individual samples, with information on 6,840 patients. Of these, 35 samples (3,306 patients from 19 publications) provided data on all performance measures. The median AUC value for the wider group of 78 samples was higher for imminent tools (AUC 0.83; IQR: 0.71–0.85) compared with longer-term tools (AUC 0.68; IQR: 0.62-0.75). Other performance measures indicated variable accuracy for imminent and longer-term tools. Meta-regression indicated that no study or sample-related characteristics were associated with between-study differences in AUCs. InterpretationThe performance of current tools in predicting risk of violence beyond the first few days is variable, and the selection of which tool to use in clinical practice should consider accuracy estimates. For more imminent violence, however, there is evidence in support of brief scalable assessment tools.

Breast Cancer Screening in a Multimodality Environment—The Need for a Simple Summary Measure of Marginal Value

In a rapidly changing clinical environment, assessment of imaging-based technologies and practices for periodic screening for the early detection of breast cancer is constrained by cost, complexity, and professional resources, particularly concerning supplementary imaging of subgroups constituting a large fraction of the screened population. Relatively high survival rates after detection make it extremely difficult to adequately assess marginal values of proposed approaches either before the technology in question being widely accepted and used or before it becomes largely obsolete. The author discusses several issues related to the assessment process and proposes the use of a surrogate summary measure of performance for this purpose, namely the number of recalled cases for the diagnostic workup of suspicious findings during repeat examinations, per one additional screen detected cancer that is invasive, node-negative, and classified grade 2 or above.

Farmers, bandits and soldiers: a generic system for addressing peace agendas

AbstractWe propose a generic model that explains why political systems tend toward certain outcomes. The model identifies possible economic and psychological paths toward change in a metaphorical political economy consisting of farmers, bandits and soldiers. In addition to economic factors, we also consider how two psychological factors, broadly categorized as group identity and exposure to violence, affect the behavior of metaphorical agents. We find that though outcomes tend to be similar with and without the psychological influences, the psychological influences accelerate the adjustment process and create additional policy space for interventions. A methodological contribution of the paper also is the use of summary performance measures represented as phase plots that have the potential to be used with advantage in system dynamics analyses. Copyright © 2014 System Dynamics Society

Nonfinancial Performance Measures and the Incentive Intensity of CEO Bonus Contracts

Firms add nonfinancial measures to their CEO bonus contracts when a sole reliance on financial measures would not be informative about managerial effort. In this paper, we argue that adding nonfinancial measures to CEO contracts may not resolve the congruence problems that would originate from a sole reliance on financial measures. Thus, this paper investigates the assertion that companies optimally provide weak incentives if they lack undistorted performance measures (Baker, 2002). Using a sample of 164 Dutch firms listed on the Amsterdam Stock Exchange and explicitly taking into account the joint nature of decisions to assign a weight to nonfinancial measures and to select the appropriate incentive intensity through a 2SLS model, we find that firms that seek to provide strong incentives decrease the relative weight of nonfinancial measures in CEO bonus contracts. This coincides with prior literature that suggests that firms adjust for items in their reported earnings to improve their financial summary performance measure.

Which Performance Measures Do Investors Around the World Value the Most—and Why?

ABSTRACT: We examine the value relevance of a comprehensive set of summary performance measures including sales, earnings, comprehensive income, and operating cash flows. We find that, while value relevance peaks for measures “above the line,” no single measure dominates around the world. Instead, a measure is more relevant when it captures, directly and quickly, information about firms’ cash flows. Specifically, for each performance measure by country, we estimate eight attributes commonly used to assess earnings quality. We find these attributes highly correlated—most of their variance is explained by only two principal factors. A factor capturing articulation with cash flows is positively associated with a measure’s value relevance; a factor reflecting the measure’s persistence, predictability, smoothness, and conservatism is negatively associated. Our results suggest that, when it comes to equity valuation, accounting researchers and standard-setters should focus not on what performance measure is “best” at a given point in time, but on the underlying attributes that investors find most relevant.

MRI and Mammography Surveillance of Women at Increased Risk for Breast Cancer: Recommendations Using an Evidence-based Approach

To evaluate breast cancer screening with mammography and magnetic resonance imaging (MRI) in high-risk populations, including women with the BRCA mutation, using an evidence-based approach. The MEDLINE, PubMed, EBM Reviews, ACP Journal Club, Cochrane Database MEDSEARCH, and SCOPUS databases were accessed and searched for articles up to August 2007. Articles were collected using the following terms and medical subject headings (MeSH) that applied to the focused clinical question: "BRCA1" and "BRCA2" with "mammography," "MRI," "prevention," "screening," and "surveillance." References from retrieved articles were also used to identify relevant papers. Abstracts were screened and relevant papers retrieved. Retrieved papers were graded for quality. Summary performance measures were obtained by random effects modeling of study-specific performance estimates and standard errors derived from the multiple 2 x 2 tables. Additionally, studies meeting the Centre for Evidence-Based Medicine level 2b quality were reviewed. In women with an increased risk without the BRCA gene, cancer detection rates by MRI were 0.011 (95% confidence interval [CI] 0.003-0.019), by mammography 0.005 (95% CI 0.002-0.008), and by a combination of both, 0.012 (95% CI 0.004-0.020). False-positive rates by MRI, mammography, or a combination of both were 0.10 (95% CI 0.03-0.18), 0.05 (95% CI 0.03-0.06), and 0.14 (95% CI 0.04-0.24). In BRCA positive women, cancer detection rates by MRI were 0.027 (95% CI 0.015-0.040), by mammography 0.010 (95% CI 0.005-0.016), and by a combination of both 0.031 (95% CI 0.018-0.045). False-positive rates by MRI, mammography, or a combination of both were 0.10 (95% CI 0.01-0.19), 0.05 (95% CI 0.03-0.07), and 0.14 (95% CI 0.04-0.24), respectively. The data support an essential role for screening MRI in women with an increased risk for breast cancer.

Which Performance Measures do Investors Value the Most - and Why?

We examine the value relevance of a comprehensive set of summary performance measures including sales, earnings, comprehensive income, and operating cash flows. We find that, while value relevance peaks for measures above the line, no single measure dominates for firms across the world. Instead, we find a performance measure more relevant when it captures in a more direct and timely fashion information about firms' cash flows. Specifically, for each performance measure by country, we estimate eight attributes commonly used by accounting researchers to assess earnings quality. We find these attributes highly correlated-most of their variance can be explained by only two principal factors. A factor capturing nearness to cash flows is positively associated with a performance measure's value relevance; a factor reflecting the measure's persistence, predictability, smoothness and conservatism is negatively associated. Together, our results suggest that, when it comes to equity valuation, accounting researchers and standard setters should focus not on what performance measure is best at one point in time, but on the underlying attributes that investors find most relevant.

Quality Measurement in Adult Cardiac Surgery: Part 2—Statistical Considerations in Composite Measure Scoring and Provider Rating

There is increasing interest among payers, patients, regulators, and providers to measure and compare cardiac surgery quality. The Society of Thoracic Surgeons (STS) Quality Measurement Task Force (QMTF) was established to develop comprehensive, summary performance measures encompassing multiple domains of quality. This report describes statistical considerations relevant to combining multiple measures into an overall composite score and then using such scores to rate providers.

Comparing the Quality of Accruals for Alternative Summary Performance Measures in the Real Estate Investment Trust (REIT) Industry

This study examines Funds from Operations (FFO), a non-GAAP summary performance measure used extensively by the Real Estate Investment Trust (REIT) industry, by comparing the quality of accruals in FFO and in net income. Accrual quality is empirically evaluated from regressions relating accruals to past, present, future cash flows and changes in revenue, with larger residuals implying lower accrual quality. Results show that the accruals component of FFO is of higher quality than the accruals component of net income. The study then considers how the individual accrual items excluded from FFO, but included in net income contribute to the accrual quality differences of the two measures. I show that all these accrual items (including depreciation and some non-recurring accruals) are of low quality and are insignificant in predicting future cash flows. Since accounting depreciation is supposed to help match revenue with expenses, it is surprising to find that depreciation accruals are of low quality for the REIT industry. Further analysis on depreciation shows that the low quality of depreciation accruals for REIT firms is due to estimation errors and managerial manipulation and, to some extent, extreme changes in market inflation.

Do Depressive Symptoms Predict Declines in Physical Performance in an Elderly, Biracial Population?

We investigated whether depressive symptoms, assessed by the 10-item Center for Epidemiological Studies Depression Scale (CES-D), predicted change in physical function in elderly adults. Participants were from a biracial, population-based sample of adults aged 65 and older (N: 4069; 61% black; 61% female). Physical function was assessed as a summary performance measure of tandem stand, measured walk, and repeated chair stand (mean [standard deviation], 10.3 [3.5]; range, 0-15), commonly used measures of overall physical health in older adults. Generalized estimating equation models estimated physical function across 3 assessments over 5.4 years of follow up as a function of CES-D scores at baseline. Adjusting for age, sex, race, and education, each 1-point higher CES-D score was associated with a 0.34-point lower absolute level of physical performance (p < .0001), but there was no evidence of a CES-D by time interaction (p = .84), indicating that depressive symptoms at baseline were not associated with greater physical performance decline over time. In secondary analyses, with CES-D scores modeled in 4 categories, overall physical performance showed a graded, inverse association across CES-D categories (p's < .0001). However, we observed no threshold effect for depressive symptoms in relation to change in physical performance. Compared with the referent group (CES-D = 0), the 2 middle CES-D categories (CES-D = 1 or 2-3) evidenced some decline in physical performance over time, but the highest CES-D group (CES-D > or =4) showed no significant physical decline over time (p = .89). We observed a strong cross-sectional association between depressive symptoms and overall physical performance. Physical function declined over time, yet depressive symptoms did not consistently contribute to greater decline over an average of 5.4 years of follow up among older adults. Findings highlight the importance of longitudinal models in understanding the relation between depressive symptomatology and physical health.

A Detailed Analysis of Task Performance With and Without Computer Monitoring

Participants (N = 115) performed a computerized task under 3 conditions: no supervision, direct human supervision, and computer monitoring. Differences in performance across groups was evaluated using summary performance measures and detailed analyses of group performance over time. There was a statistically significant difference in performance quality but not performance quantity between the groups. The nonmonitored and computer-monitored groups had higher quality of performance compared with the direct human supervision group. Performance varied when examined in detail at different points in time during the experimental task. Together the results suggest that direct human supervision motivated participants but that participants in the other 2 groups were more sensitive to varying task demands.

Predictors of recovery in activities of daily living among disabled older Mexican Americans.

Although functional ability decreases with age and is associated with poor health outcomes, decline in functional ability is reversible. The aim of this study is to describe the rate of recovery of functional ability and to identify factors associated with it over a two-year period among older Mexican Americans. 245 functionally disabled older Mexican Americans were included in a two-year prospective cohort study. Data are from the Hispanic Established Population for the Epidemiological Study of the Elderly (H-EPESE), a population-based study of non-institutionalized Mexican Americans in the South-western United States. Activities of daily living (ADL), lower body mobility (tandem balance, eight-foot walk, and repeated chair stands), depressive symptomatology, body mass index, and self-reported medical conditions were obtained. Over a two-year period, of the 245 subjects at baseline who reported functional disability in at least 1 of 7 ADLs, 83 totally recovered their ADL ability, 108 remained disabled, 36 died, and 18 were lost to follow-up. Factors significantly associated with recovery included younger age (65-74) (OR 2.18, 95% CI 1.08-4.42), higher summary performance measure of lower body function (OR 1.19, 95% CI 1.05-1.34), few depressive symptoms (OR 2.84, 95% CI 1.39-5.78), and a BMI > or = 30 Kg/m2 (OR 3.08, 95% CI 1.17-8.07). Higher numbers of ADL limitations at baseline were associated with lower odds of ADL recovery. Two-year recovery from ADL disability among older Mexican Americans was high (33.9%). Factors independently associated with recovery include younger age, few depressive symptoms, good lower body function, and high BMI.