Composite Performance Measures Research Articles

Composite Metric for Benchmarking Site Performance in Transcatheter Aortic Valve Replacement: Results From the STS/ACC TVT Registry.

Transcatheter aortic valve replacement (TAVR) is a transformative therapy for aortic stenosis. Despite rapid improvements in technology and techniques, serious complications remain relatively common and are not well described by single outcome measures. The purpose of this study was to determine whether there is site-level variation in TAVR outcomes in the United States using a novel 30-day composite measure. We performed a retrospective cohort study using data from the Society of Thoracic Surgeons/American College of Cardiology Transcatheter Valve Therapies Registry to develop a novel ranked composite performance measure that incorporates mortality and serious complications. The selection and rank order of the complications for the composite was determined by their adjusted association with 1-year outcomes. Sites with risk-adjusted outcomes significantly more or less frequent than the national average based on a 95% probability interval were classified as performing worse or better than expected. The development cohort consisted of 52 561 patients who underwent TAVR between January 1, 2015, and December 31, 2017. Based on associations with 1-year risk-adjusted mortality and health status, we identified 4 periprocedural complications to include in the composite risk model in addition to mortality. Ranked empirically according to severity, these included stroke, major, life-threatening or disabling bleeding, stage III acute kidney injury, and moderate or severe perivalvular regurgitation. Based on these ranked outcomes, we found that there was significant site-level variation in quality of care in TAVR in the United States. Overall, better than expected site performance was observed in 25/301 (8%) sites, performance as expected was observed in 242/301 sites (80%), and worse than expected performance was observed in 34/301 (11%) sites. Thirty-day mortality; stroke; major, life-threatening, or disabling bleeding; and moderate or severe perivalvular leak were each substantially more common in sites with worse than expected performance as compared with other sites. There was good aggregate reliability of the model. There are substantial variations in the quality of TAVR care received in the United States and 11% of sites were identified as providing care below the average level of performance. Further study is necessary to determine structural, process-related, and technical factors associated with high- and low-performing sites.

STS Adult Cardiac Surgery Database: 2021 Update on Outcomes, Quality, and Research

The Society of Thoracic Surgeons Adult Cardiac Surgery Database is the most mature and comprehensive cardiac surgery database. It has been the foundation for quality measurement and improvement activities in cardiac surgery, facilitated the generation of accurate risk adjusted performance benchmarks and serves as a platform for novel research. Recent enhancements have added to the database's functionality, ease of use, and value to multiple stakeholders. This report is the sixth in a series of annual reports that provide updated volumes, outcomes, database-related developments, quality improvement initiatives, and research summaries using the Adult Cardiac Surgery Database in the past year.

Adaptively Allocating Constraint-Handling Techniques for Constrained Multi-objective Optimization Problems

For solving constrained multi-objective optimization problems (CMOPs), an effective constraint-handling technique (CHT) is of great importance. Recently, many CHTs have been proposed for solving CMOPs. However, no single CHT can outperform all kinds of CMOPs. This paper proposes an algorithm, namely, ACHT-M2M, which adaptively allocates the existing CHTs in an M2M framework for solving CMOPs. To be more specific, a CMOP is first decomposed into several constrained multi-objective optimization subproblems by ACHT-M2M. Each subproblem has a subpopulation in a subregion. CHT for each subregion is adaptively allocated according to a proposed composite performance measure. Population for the next generation is selected from subregions by selection operators with different CHTs and the obtained nondominated feasible solutions in each generation are used to update a predefined archive. ACHT-M2M assembles the advantages of different CHTs and makes them cooperate with each other. The proposed ACHT-M2M is finally compared with the other 12 representative algorithms on benchmark CMOPs and the experimental results further confirm the effectiveness of ACHT-M2M for solving CMOPs.

Variation between general practitioners in type 2 diabetes processes of care

AimsTo explore variation in general practitioners’ (GPs’) performance of six recommended procedures in type 2 diabetes patients <75 years without cardiovascular disease. MethodsCross-sectional study of quality of diabetes care in Norway based on electronic health records from 2014. GPs (clustered in practices) were divided in quintiles based on a composite measure of performance of six processes of care. We fitted a multilevel partial ordinal regression model to identify GP factors associated with being in quintiles with better performance. ResultsWe identified 6015 type 2 diabetes patients from 275 GPs in 77 practices. The GPs performed on average 63.4% of the procedures; on average 46% in the poorest quintile to 81% in the best quintile with a larger range in individual GPs. After adjustments, use of a structured follow-up form was associated with GPs being in upper three quintiles (OR 12.4 (95% CI 2.37–65.1). Routines for reminders were associated with being in a better quintile (OR 2.6 (1.37–4.92). GPs’ age >60 years and heavier workload were associated with poorer performance. ConclusionWe found large variations in GPs’ performance of processes of care. Factors reflecting structure and workload were strongly associated with performance.

Measuring Performance: Ranking State Success Over Two Decades in India

This paper proposes new composite measures of relative and dynamic state performance to improve governance and delivery of public programs in developing countries with a federal structure. We rank the performance of 19 major Indian states on two large development programs launched by the Indian government over the last two decades using publicly available data. Although we find volatility in performance over time, there exists a positive correlation between measures of state capacity, development and accountability with program outcomes. Our findings have important implications for both the design and implementation of public service programs of such large scale.

Human Capital and Environmental Sustainability

Human capital has been shown as an important determinant of environmental performance at the firm and province levels. We try to identify the impact of human capital on environmental performance at the country level. To address potential endogeneity and measurement errors regarding human capital, we use the primary school enrollment rate a century ago as an instrument for today’s human capital stock. The instrumental variable estimation, based on a sample of 72 countries, shows that human capital has a statistically significant effect on the Environmental Performance Index, a composite measure of national environmental performance, primarily through improving biodiversity, habitat, and sustainable agriculture, all of which are the indicators related to environmental sustainability. These findings from cross-country analysis are broadly consistent with those of firm-level or province-level studies, emphasizing the role of human capital in enhancing the awareness of and compliance with environmental regulations, thus leading to an improvement in environmental performance.

Symptom Level Associations Between Attention-Deficit Hyperactivity Disorder and School Performance.

Attention-deficit hyperactivity disorder (ADHD) is associated with reduced school performance. To determine which ADHD symptoms and subtypes have the strongest association, we used type and frequency of symptoms on the 2014 National Survey of the Diagnosis and Treatment of ADHD and Tourette Syndrome (NS-DATA) to create symptom scores for inattention and hyperactivity-impulsivity and define subtypes (ADHD-Inattentive [ADHD-I], ADHD-Hyperactive-Impulsive, ADHD-Combined [ADHD-C]). Regression methods were used to examine associations between symptoms and subtype and a composite measure of school performance. Children with ADHD-C and ADHD-I had higher adjusted odds of having reduced overall school performance (ADHD-C = 5.8, 95% confidence interval [CI] = 3.1-10.9; ADHD-I = 5.5, 95% CI = 3.1-10.1) compared with children without ADHD. All inattentive symptoms were significantly related to reduced school performance in reading, writing, and handwriting, while 6 of 9 symptoms were significantly associated in mathematics. Children with ADHD-I were significantly more likely than children with other ADHD subtypes to receive a school-based Individualized Education Program or 504 Plan. ADHD-I symptoms may be broadly linked to reduced school performance.

Association Between HEDIS Performance and Primary Care Physician Age, Group Affiliation, Training, and Participation in ACA Exchanges.

There are a limited number of studies investigating the relationship between primary care physician (PCP) characteristics and the quality of care they deliver. To examine the association between PCP performance and physician age, solo versus group affiliation, training, and participation in California's Affordable Care Act (ACA) exchange. Observational study of 2013-2014 data from Healthcare Effectiveness Data and Information Set (HEDIS) measures and select physician characteristics. PCPs in California HMO and PPO practices (n = 5053) with part of their patient panel covered by a large commercial health insurance company. Hemoglobin A1c testing; medical attention nephropathy; appropriate treatment hypertension (ACE/ARB); breast cancer screening; proportion days covered by statins; monitoring ACE/ARBs; monitoring diuretics. A composite performance measure also was constructed. For the average 35- versus 75-year-old PCP, regression-adjusted mean composite relative performance scores were at the 60th versus 47th percentile (89% vs. 86% composite absolute HEDIS scores; p < .001). For group versus solo PCPs, scores were at the 55th versus 50th percentiles (88% vs. 87% composite absolute HEDIS scores; p < .001). The effect of age on performance was greater for group versus solo PCPs. There was no association between scores and participation in ACA exchanges. The associations between population-based care performance measures and PCP age, solo versus group affiliation, training, and participation in ACA exchanges, while statistically significant in some cases, were small. Understanding how to help older PCPs excel equally well in group practice compared with younger PCPs may be a fruitful avenue of future research.

Assessing various sensorimotor and cognitive functions in people with epilepsy is feasible with robotics

BackgroundEpilepsy is a common neurological disorder characterized by recurrent seizures, along with comorbid cognitive and psychosocial impairment. Current gold standards of assessment can quantify cognitive and motor performance, but may not capture all subtleties of behavior. Here, we study the feasibility of assessing various upper limb sensorimotor and cognition functions in people with epilepsy using the Kinarm robotic assessment system. We quantify performance across multiple behavioral domains and additionally consider the possible effects of epilepsy subtype and medication. MethodsWe recruited individuals with a variety of epilepsy subtypes. Participants performed 8 behavioral tasks that tested motor, cognitive, and sensory domains. We collected data on the same tasks from a group of control participants that had no known neurological impairments. We quantified performance using Task Scores, which provide a composite measure of overall performance on a given task and are adjusted for age, sex, and handedness. ResultsWe collected data from 46 individuals with epilepsy and 92 control participants. The assessment was well-tolerated, with no adverse events recorded. Cognitive tasks testing spatial working memory, executive function, and motor response inhibition were the most frequently impaired in the epilepsy cohort, with 33/46 (72%) being outside the normal range on at least one of these tasks. Additionally, 29/46 (63%) were impaired on at least one task testing primarily motor skill, and 14/46 (30%) were impaired on a proprioceptive sensory task. People with either focal epilepsy or generalized epilepsy performed significantly worse on both motor and cognitive tasks than control participants after correcting for multiple comparisons. There were no statistical differences between generalized and focal epilepsy groups on Task Scores. Finally, individuals taking topiramate trended toward having worse performance on a spatial working memory task than other individuals with epilepsy who were not taking topiramate. ConclusionsKinarm robotic assessment is feasible in individuals with epilepsy and is well-tolerated. Our robotic paradigm can detect impairments in various sensorimotor and cognitive functions across the population with epilepsy. Future studies will explore the role of epilepsy subtype and medications.

Increasing perceived hand size improves motor performance in individuals with stroke: a home-based training study.

BackgroundIncreasing perceived hand size with magnifying lenses improves tactile discrimination and induces changes in action performance. We previously demonstrated that motor skills (tested with grip force, finger tapping, and a reach to grasp tasks) improved when actions were performed with magnified compared to normal vision; twenty-eight percent of 25 participants with stroke exhibited significant improvement on a composite measure of motor performance with magnification as compared to a session without magnification.MethodsTo investigate the potential implications of magnification of vision for motor rehabilitation, we recruited individuals with stroke from the original cohort who exhibited an improvement of at least 10% in grip force and/or finger tapping for a home training protocol. Six individuals with stroke completed a two-week home-based training program in which they performed a range of activities while looking at their hand magnified. Motor skills were measured before, immediately after, and two weeks after the training.ResultsFive of the six participants showed an improvement on motor tasks when tested after the training. In two participants the improvement was evident immediately after the training and persisted in time, while it occurred at two-weeks post-training in the other participants. These results suggest that the magnification of vision is a potential tool for the rehabilitation of post-stroke motor deficits.

Dual vs. Single Tasking During Circular Walking: What Better Reflects Progression in Parkinson's Disease?

Background and Aim: Reliable, valid and sensitive measures of dual-task-associated impairments in patients with Parkinson's disease (PD) may reveal progressive deficits unnoticed under single-task walking. The aim of this study was to quantitatively identify markers of progressive gait deficits in idiopathic PD while walking over a circular trajectory condition in single-task walking and in different dual-task conditions: (1) circular walking while checking boxes on a paper sheet as fast as possible and (2) circular walking while performing subtraction of 7 as fast as possible. In addition, we aimed to study the added value of dual-tasking assessment over single (circular) walking task assessment in the study of PD progression.Methods: The assessments were performed every 6 months over a (up to) 5 years period for 22 patients in early-stage PD, 27 patients in middle-stage PD and 25 healthy controls (HC). Longitudinal changes of 27 gait features extracted from accelerometry were compared between PD groups and HCs using generalized estimating equations analysis, accounting for gait speed, age, and levodopa medication state confounders when required. In addition, dual-task-interference with gait and cognitive performance was assessed, as well as their combination.Results: The results support the validity and robustness of some of the gait features already identified in our previous work as progression markers of the disease in single-task circular walking. However, fewer gait features from dual-task than from single-task assessments were identified as markers of progression in PD. Moreover, we did not clearly identify progressive worsening of dual-task-interference in patients with PD, although some group differences between early and middle stages of PD vs. the control group were observed for dual-task interference with the gait task and with the concurrent tasks.Conclusions: Overall, the results showed that dual-tasking did not have added value in the study of PD progression from circular gait assessments. Our analyses suggest that, while single-task walking might be sensitive enough, dual-tasking may introduce additional (error) variance to the data and may represent complex composite measures of cognitive and motor performance.

Pharmacy characteristics correlating to performance in a community pharmacy network

ObjectiveTo assess the correlation between pharmacy characteristics and a broad set of performance measures used to support a community pharmacy network. MethodsBaseline characteristics regarding demographics, services provided, technology, and staffing were collected via a 68-item survey for 123 pharmacies participating in the North Carolina community pharmacy enhanced services network. Performance metric data were collected, and scores were calculated for each pharmacy. Outcome measures for this study comprised of 4 adherence measures, a risk-adjusted hospitalization measure, a risk-adjusted emergency department visit measure, a total cost of medical care measure, and a composite pharmacy performance measure. Generalized estimating equations (GEE) were used to create multivariable statistical models measuring the correlation between pharmacy characteristics and performance measures. ResultsAfter inclusion criteria were applied, 115 pharmacies remained in the analysis. These pharmacies were primarily single and multiple independent pharmacies, at 36.5% and 59.1%, respectively. Five characteristics were significantly associated with 3 measures, and none were associated with 4 or more. Having pharmacists in non dispensing roles was positively associated with total score, diabetes adherence, and chronic medication adherence. Home visits were positively associated with hypertension adherence, ED visit rate, and total score. Offering a smoking cessation program was positively associated with chronic medication, hypertension, and statin adherence. Offering free home delivery was positively associated with diabetes adherence but negatively associated with total medical spending and ED visit rate. Using dispensing automation was negatively associated with adherence to chronic medications, renin-angiotensin system antagonists, and statins. ConclusionNo pharmacy characteristics were associated with a majority of performance measures chosen. Additional research is needed to identify structural variables that can be used as minimum participation criteria for high-performing pharmacy networks.

Composite Measure of Industrial Performance for Cross-country Analysis

A country’s industrial performance can be assessed using a number of statistical indicators reflecting the level, growth and structure of industrial activities. However, policymakers prefer a single composite measure over a set of indicators to obtain an overall picture of their country’s relative performance compared to that of other countries. The paper describes UNIDO’s experience of constructing such a measure, namely the Competitive Industrial Performance (CIP) index, which considers countries’ productive capacity, intensity of industrialization and impact on the world market as major components of industrial performance. The compilation of the CIP index, however, poses two major challenges. First, the composite measure for a broad international comparison can only be constructed when data are available for all indicators and if the computation methodology is uniform across the countries. Second, as the final result is a normalized index used for country ranking, any uncertainty in one of the indicators may result in a shift of a given country in the overall ranking. The primary data source for the compilation of the CIP index is UNIDO’s database based on an underlying quality assurance framework for international comparability. To address the second problem, a sensitivity analysis is performed. The analysis reveals a substantially high correlation between the default CIP rankings and the alternatives, indicating that the composite measures are robust in a comparative assessment of countries’ industrial performance.

Framework for assessing pharmacy value.

Excess spending and poor quality in the US healthcare system has led to proliferation of performance-based payment models. These models have the potential to enhance value by creating a meritocratic system whereby providers delivering the best patient care are rewarded, while providers failing to provide such care are given incentives to improve. However, early experience suggests that unless these systems are appropriately designed, payments can be withheld from high performers, bonuses paid to low performers, and health disparities can be worsened. Performance-based payments are new to community pharmacies, and opportunity exists to strengthen pharmacy value measurement and potentially avoid problems observed with other performance-based payment models. This article describes the process by which a framework to assess community pharmacy value was developed, then applies the framework to produce a draft composite pharmacy performance measure. The pharmacy value framework addresses potential shortcomings of existing community pharmacy performance measures through four key principles: 1) theory-based quality and spending measures, 2) scoring which accounts for measure reliability, 3) full risk-adjustment, and 4) a value matrix to identify high and low value pharmacies. Based on these principles, a draft community pharmacy composite performance measure was developed, and was successful in dividing community pharmacies into high, typical, and low value categories. By using this framework to develop future composite measures, payers may find closer alignment between performance-based payments and actual pharmacy performance. This early work is intended to encourage further research into the establishment of a scientifically firm foundation for pharmacy performance measurement. More testing is needed to determine reliability, validity, and comparative superiority of any composite measure derived from this framework before it is used to support performance-based pharmacy payment models.

Active Versus Passive Investing: Evidence From The 2009-2017 Market

The paper explores the research question: During the March 1, 2009 to December 31, 2017 time period, which investment management style, active or passive, produced the better risk-adjusted performance?The study tested nine hypotheses, derived from the above research question for the period.The Sharpe composite portfolio performance measure, that combines risk and return into a single value, was used to measure, analyze, and rank risk-adjusted performance.The study, comprised of 9 statistical tests, found that on a risk-adjusted basis that the active indices(proxies for active management) Sharpe ratios did not significantly exceed the passive indices (proxies for passive management) Sharpe ratios for the period tested.

School effects on students' engagement with school: Academic performance moderates the effect of school support for learning on students' engagement

The aim of the present cross-sectional study was to investigate how school support-for-learning, from peers and teachers, influences the relationship between prior academic performance and an indicator of cognitive engagement (students' future aspirations and perceptions of schoolwork). A sample of 4406 adolescent students from 68 schools in Portugal completed the self-report Student Engagement Instrument as a measure of future aspirations and perceptions of schoolwork, and perceived support for learning. We obtained students' previous year exam grades in Maths and Portuguese from school records to form a composite measure of academic performance. We used Hierarchical Linear Modelling (HLM) to assess whether school-level support-for-learning moderates the student-level effect of performance on engagement. Higher achieving students reported higher scores for our indicator of cognitive engagement than lower achieving students. This association was weaker in schools with high support-for-learning than those with low support-for-learning. Altogether, our results demonstrate the importance of supportive school environments in buffering the negative effect of poor performance on engagement and promoting educational equity.

Profiling Surgeon Performance for Breast Cancer Lumpectomy by Composite Measurement of Reoperations, Cosmetic Outcomes, and Patient Preferences.

Patients want information to search for destination of care for breast-conserving surgery (BCS). To inform patients wanting a lumpectomy, we aimed to develop a pilot project that communicated composite quality measure (QM) results using a '4-star' rating system. Two patient-centered QMs were included in the model-reoperation rate (ROR) and cosmetic outcome (COSM). A prospective database was reviewed for stage 0-3 patients undergoing initial lumpectomy by three surgeons from 2010 to 2015. Self-reported COSM was assessed by survey. Multivariate analyses were used to test for interactions between surgeon and other variables known to influence RORs and COSMs. Models of surgeon profiling were developed that summed the ROR and COSM performance scores, then reported results using a Centers for Medicare and Medicaid Services (CMS) star-type system. Functionality for a patient to 'weight' the importance of the ratio of ROR:COSM before profiling was introduced. The unadjusted ROR for stage 1-3 patients for three surgeons was 9.5, 13.0, and 16.3%, respectively (p=0.179) [overall rate 10.4% (38/366)]. After risk adjustment, differences between surgeons were observed for RORs, but not COSMs. Overall, patients reported excellent, good, fair, and poor COSMs of 55, 30, 11 and 4%, respectively. Composite star scores reflected differences in performance by surgeon, which could increase, or even disappear, dependent on the patient's weighting of the ROR:COSM ratio. Composite measures of performance can be developed that allow patients to input their weighted preferences and values into surgeon profiling before they consider a destination of care for BCS.

Predicting performance of junior doctors: Association of workplace based assessment with demographic characteristics, emotional intelligence, selection scores, and undergraduate academic performance

Introduction: Predicting workplace performance of junior doctors from before entry or during medical school is difficult and has limited available evidence. This study explored the association between selected predictor variables and workplace based performance in junior doctors during their first postgraduate year.Methods: Two cohorts of medical students (n = 200) from one university in Western Australia participated in the longitudinal study. Pearson correlation coefficients and multivariate analyses utilizing linear regression were used to assess the relationships between performance on the Junior Doctor Assessment Tool (JDAT) and its sub-components with demographic characteristics, selection scores for medical school entry, emotional intelligence, and undergraduate academic performance.Results: Grade Point Average (GPA) at the completion of undergraduate studies had the most significant association with better performance on the overall JDAT and each subscale. Increased age was a negative predictor for junior doctor performance on the Clinical management subscale and understanding emotion was a predictor for the JDAT Communication subscale. Secondary school performance measured by Tertiary Entry Rank on entry to medical school score predicted GPA but not junior doctor performance.Discussion: The GPA as a composite measure of ability and performance in medical school is associated with junior doctor assessment scores. Using this variable to identify students at risk of difficulty could assist planning for appropriate supervision, support, and training for medical graduates transitioning to the workplace.

ECAS A-B-C: alternate forms of the Edinburgh Cognitive and Behavioural ALS Screen

Background: The Edinburgh Cognitive and Behavioural ALS Screen (ECAS) is a short assessment by which neuropsychological symptoms can be detected and quantified in people with ALS. To avoid potential practice effects with repeated administration, here we present alternative versions of the ECAS suitable for measuring change over time. Objective: To develop two alternate versions of the ECAS: ECAS-B and ECAS-C. Method: One hundred and forty-nine healthy adult participants were recruited. Thirty participants completed a pilot study in developing the alternate versions. Two groups of 40 participants were administered the ECAS-B or ECAS-C and compared to published data of the original ECAS (ECAS-A) to determine equivalence. An additional 39 participants were administered the ECAS consecutively, either repeating the original version (ECAS-A-A-A) serially or the different versions (ECAS-A-B-C) to determine potential practice effects. Recordings of assessments were scored by a second researcher to determine inter-rater reliability. Results: No significant differences were found between versions (A, B, C) of the composite performance measures of ALS Specific, ALS Non-Specific, and ECAS Total scores. Repeated serial administration of ECAS-A (A-A-A) produced some practice effects for composite scores, whereas no such effects were found when alternate versions were administered serially (A-B-C). Exceptionally high intra-class correlations were found for all three versions of the ECAS suggesting a high degree of rater agreement. Conclusion: The newly developed alternate forms of the ECAS are both highly equitable to the original ECAS-A and enable avoidance of practice effects, thus supporting their use in measuring cognition and behaviour over time.

The antecedents and impact of industry consultants on the future performance of new banks

PurposeThe purpose of this paper is to examine the antecedents and performance outcomes when startup firms in the US banking industry hire industry consultants.Design/methodology/approachThis study uses a sample of prospective startup banks that applied for a new bank charter application in Florida between 1996 and 2005. Logistic regression, ordinary least squares or ordered logistic regression models were used to test hypotheses.FindingsAnalysis suggests complexity and regulatory change are factors in a founder’s decision to hire a consultant. Consultants have a positive impact on firm financial performance but not on a composite multifactor measure of performance. Additional analyses suggest the effectiveness of consulting assistance hinges on specific attributes of the consulting firm, but cumulative consulting experience is not one of these attributes.Research limitations/implicationsThis study focuses on the impact of consultants on new venture performance in a single industry using archival data. Additional research is likely needed to test the generalizability of the findings in other research contexts and examine motives beyond the financial ones investigated in this study.Practical implicationsResults suggest that hiring a consultant at startup can satisfy financial stakeholders, but, in a regulated industry, hiring a consultant at startup does not improve a composite, multifactor measure of performance that is important to industry regulators. When deciding whether to commit scarce resources to hiring consultants, founding teams should be clear which external stakeholder and which measure of performance they are seeking to improve.Originality/valueWhile the business advisory role of informal players such as family and friends and more formal players such as board members and federal, state or local governments have been well documented, little attention has been paid to the contributions of industry consultants in startup firms. These overlooked intermediaries play an important role in the successful launch of a new firm. This study examines when and why such advisors might create value for new firms.